How can I get a specifc value/string from a website

Solar2D General Questions/Discussion
#1

So lets say I want my app to get and store a value from a webpage.

For instance this page:
https://en.wikipedia.org/wiki/Quigley_(film)

I want to get the running time string:

How can I get it?

#2

You typically require server side API that you can connect to and make requests to. Wikipedia probably has its own API, but I am not sure what it gives access to, to whom and how.

#3

I read this:

https://forums.coronalabs.com/topic/71495-read-text-on-webpage/

Maybe I will try this one out.

#4

Just a heads up. You need to be careful about how you access websites and how you handle the data that you receive.

If you want something specific from a Wikipedia page, for instance, then you are probably searching for some exact string. If something changes with the site, your app may break. If the user somehow inputs a wrong page, your app may break, etc.

There is also the issue of how you will let the user know of what you are doing. If you are making an app just for yourself, then this doesn’t matter, but if you are making an app that you plan to release to the public, then you don’t just want to download entire pages in order to parse them for one or two pieces of data. It’d be wasteful and take much longer than needed, as well as consume more battery than needed (not to mention uncivilized countries without unlimited data connections). You’d also need to explain in your privacy policy that you are taking the user to all sorts of websites to download entire web pages, etc.

#5

MediaWiki, and thus Wikipedia, have an API - https://www.mediawiki.org/wiki/API:Main_page

#6

I haven’t tried implementing that to my project so I will give that a whirl.

Thanks a bunch!

#7

You typically require server side API that you can connect to and make requests to. Wikipedia probably has its own API, but I am not sure what it gives access to, to whom and how.

#8

I read this:

https://forums.coronalabs.com/topic/71495-read-text-on-webpage/

Maybe I will try this one out.

#9

Just a heads up. You need to be careful about how you access websites and how you handle the data that you receive.

If you want something specific from a Wikipedia page, for instance, then you are probably searching for some exact string. If something changes with the site, your app may break. If the user somehow inputs a wrong page, your app may break, etc.

There is also the issue of how you will let the user know of what you are doing. If you are making an app just for yourself, then this doesn’t matter, but if you are making an app that you plan to release to the public, then you don’t just want to download entire pages in order to parse them for one or two pieces of data. It’d be wasteful and take much longer than needed, as well as consume more battery than needed (not to mention uncivilized countries without unlimited data connections). You’d also need to explain in your privacy policy that you are taking the user to all sorts of websites to download entire web pages, etc.

#10

MediaWiki, and thus Wikipedia, have an API - https://www.mediawiki.org/wiki/API:Main_page

#11

I haven’t tried implementing that to my project so I will give that a whirl.

Thanks a bunch!