Parse RSS files?

Solar2D General Questions/Discussion
#1

How do I parse an RSS file so that I can just print the titles or something like that?
Does it exist a fully working example on how to do this? [import]uid: 24111 topic_id: 19980 reply_id: 319980[/import]

#2

Hi,

look at this link for an RSS Parser and read the comments!

http://developer.anscamobile.com/code/rss-parser

greetz,

Jürgen [import]uid: 19590 topic_id: 19980 reply_id: 77791[/import]

#3

I downloaded the files from github but couldn’t get them working :confused: [import]uid: 24111 topic_id: 19980 reply_id: 77801[/import]

#4

That application is highly dependent on the Corona UI Widgets if you’re trying to just build the app straight up. After I published that project, Ansca updated the widget library breaking the app. I have a new version that I’m about to post back that gets all of the tableView stuff working again . . . but . . .

A new update to the widgets is coming in the next couple of daily builds and all of the widget stuff is changing again.

However, you should be able to take screen1.lua and see how I’m using rss.lua and create your own. There have been minor changes to rss.lua which you will probably want. There was also some changes to all of the network.download() processing in screen one which is valuable moving forward.

All the support code around rss.lua isn’t too functional, but rss.lua should be reasonably functional.
[import]uid: 19626 topic_id: 19980 reply_id: 77838[/import]

#5

In the application that you made it seems that there are multiple screens and all that. All i want is one simple screen that just prints the title and everything in a structured way to the corona terminal. So that I can make it look the way I want it to look later.

Do you know any good way I can fix that? [import]uid: 24111 topic_id: 19980 reply_id: 78039[/import]

#6

Have a look at

http://monkeybin.no/blog/archives/2011/04/26/creating-an-rss-and-json-driven-app-with-corona-sdk/ [import]uid: 100222 topic_id: 19980 reply_id: 78044[/import]

#7

@jeff.wesson

I tested it and it works with that specific rss but I can’t get anscamobile (as an example) rss to work :confused:

View_News.lua:

[code]
module(…, package.seeall)

function new()
local g = display.newGroup()

local callback = function(rss)
local syndication = SyndicationFeed:new(“NRK Feed”)
syndication:parseFeed(rss)

local items = syndication:getItems()
for i=1, #items do
local item = items[i]
local itemGroup = display.newGroup()
local title = display.newText(item.title, 10, 10, “Helvetica”, 16)
itemGroup:insert(title)
itemGroup.y = 10 + (40 * (i-1))
g:insert(itemGroup)
end
end

UrlLoader:new(“http://feeds.feedburner.com/AnscaMobileBlog?format=xml”, callback)

return g
end
[/code] [import]uid: 24111 topic_id: 19980 reply_id: 78065[/import]

#8

You have to download the RSS file to your sandbox, put it in system.TemporaryDirectory or system.DocumentsDirectory (and as of the latest build you should be able to use system.CachesDirectory – the right place…)

Once you download it, you simply call the rss parser and it should return you a nice table of stories.

Now not all RSS feeds are the same sadly. I built mine against Wordpress based feeds. It should process the Corona SDK blog just fine. There are a ton of namespaces that add fields and I don’t parse them all. But I should get the core ones.

If you find fields that you think I should add let me know!
[import]uid: 19626 topic_id: 19980 reply_id: 78068[/import]

#9

What if I want the user to be able to choose a rss site? How do I put files in the the system folders ( or are those folders on the computer or the iOS device? [import]uid: 24111 topic_id: 19980 reply_id: 78076[/import]

#10

First of all, I’m reworking the RSS parser to return more information. In its current incarnation, it only returns the items and no info about the feed itself. I’m putting in some support for a few name spaces, but not all of them.

You can prompt the reader for a URL if you like. The rss parser itself doesn’t do any downloading for you. It expects to find a file in one of your sandbox folders. You tell the parser the name of the file and the folder it’s in.

My sample code has an example of how to download a feed with it, but you have to make that work for you.
[import]uid: 19626 topic_id: 19980 reply_id: 78081[/import]

#11

I’ve gotten rid of all of the widget fluff and instead of this being an actually working UI project, its now a very simplified example.

https://github.com/robmiracle/Corona-SDK-RSS-Reader

There is a main.lua with an example of how to download a feed and call the rss parser, plus the minimum support files for it.

The new rss.parser returns the full feed, not just an array of items. I didn’t need the other stuff for my table views, but to make this more general, I figured some people might want the other bits.

It does not support any name spaces at this time beyond the one Dubln Core: creator field and the Content name space, but only on individual items and not the feed as a whole.

I will probably add some limited support for Media and iTunes namme spaces at some point.
[import]uid: 19626 topic_id: 19980 reply_id: 78088[/import]

#12

I managed to view a rss file the way I wanted with the new files :slight_smile:
But I had to download the rss and put it in the folder :confused:
I used this rss: http://www.thesixthaxis.com/feed/

This is my main.lua file which is the only file that i changed things in:

[code]
local rss = require(“rss”)
local print_r = require(“utility”).print_r

local feedURL = “http://blog.anscamobile.com/feed/”
local feedName = “index.rss”
local baseDir = system.CachesDirectory

function displayFeed(feedName, feedURL)

–print(“entering displayFeed”, feedName, feedURL)

local function processRSSFeed(file, path)
– print(“Parsing the feed”)
local story = {}
local feed = rss.feed(file, path)
– print_r(feed)
local stories = feed.items
native.setActivityIndicator( false )
– print("Num stories: " … #stories)
– print("Got “, #stories, " stories”)
–print_r(stories)
mytext = {}
for i=1, #stories do
mytext[#mytext+1] = display.newText(stories[i].title, 50,50 + (i*50), system.contentFont, 25)
end
end

local function onAlertComplete( event )
return true
end

local networkListener = function( event )
utility.print_r(event)
if ( event.isError ) then
local alert = native.showAlert( “RSS”, “Feed temporarily unavaialble.”,
{ “OK” }, onAlertComplete )
else
– print(“calling processRSSFeed because the feed is avaialble”)
processRSSFeed(feedName, baseDir)
end
return true
end

function MyNetworkReachabilityListener(event)
–native.showAlert(“network”, “is reachable”, {“OK”})
– print( “isReachable”, event.isReachable )
network.setStatusListener( “blog.anscamobile.com”, nil )
if event.isReachable then
– download the latest file
native.setActivityIndicator( true )
–native.showAlert(“network”, “downloading”, {“OK”})
network.download(feedURL, “GET”, networkListener, feedName, baseDir)
else
– print(“not reachable”)
–native.showAlert(“network”, “using cached copy”, {“OK”})
– look for an existing copy
local path = system.pathForFile(feedName, baseDir)
local fh, errStr = io.open( path, “r” )
if fh then
io.close(fh)
– print(“calling processRSSfeed because the network isn’t reachable”)
processRSSFeed(feedName, baseDir)
else
local alert = native.showAlert( “RSS”, “Feed temporarily unavaialble.”,
{ “OK” }, onAlertComplete )
end
end
return true
end

if network.canDetectNetworkStatusChanges then
network.setStatusListener( “blog.anscamobile.com”, MyNetworkReachabilityListener )
else
native.showAlert(“network”, “not supported”, {“OK”})
– print(“network reachability not supported on this platform”)
end
end

displayFeed(feedName, feedURL)

[/code] [import]uid: 24111 topic_id: 19980 reply_id: 78094[/import]

#13

Now the only thing I want is to be able to just grab the latest one from the internet instead of download it on the computer and put it in the Application Support folder :slight_smile: [import]uid: 24111 topic_id: 19980 reply_id: 78096[/import]

#14

On line 4, put in the URL for your feed.

On lines 47 and 72, change the address from blogs.anscamoble.com to your site (so it will check to make sure your site is up, not Ansca’s)

And it should download the file for you. That’s what’s happening in line 52.
[import]uid: 19626 topic_id: 19980 reply_id: 78103[/import]

#15

Now I tried this, but it just says "Feed temporarily unavaialble:

[code]
local rss = require(“rss”)
local print_r = require(“utility”).print_r

local feedURL = “http://www.thesixthaxis.com/feed/”
local feedName = “index.rss”
local baseDir = system.CachesDirectory

function displayFeed(feedName, feedURL)

–print(“entering displayFeed”, feedName, feedURL)

local function processRSSFeed(file, path)
– print(“Parsing the feed”)
local story = {}
local feed = rss.feed(file, path)
– print_r(feed)
local stories = feed.items
native.setActivityIndicator( false )
– print("Num stories: " … #stories)
– print("Got “, #stories, " stories”)
–print_r(stories)
mytext = {}
for i=1, #stories do
mytext[#mytext+1] = display.newText(stories[i].title, 50,50 + (i*50), system.contentFont, 25)
end
end

local function onAlertComplete( event )
return true
end

local networkListener = function( event )
utility.print_r(event)
if ( event.isError ) then
local alert = native.showAlert( “RSS”, “Feed temporarily unavaialble.”,
{ “OK” }, onAlertComplete )
else
– print(“calling processRSSFeed because the feed is avaialble”)
processRSSFeed(feedName, baseDir)
end
return true
end

function MyNetworkReachabilityListener(event)
–native.showAlert(“network”, “is reachable”, {“OK”})
– print( “isReachable”, event.isReachable )
network.setStatusListener( “http://www.thesixthaxis.com”, nil )
if event.isReachable then
– download the latest file
native.setActivityIndicator( true )
–native.showAlert(“network”, “downloading”, {“OK”})
network.download(feedURL, “GET”, networkListener, feedName, baseDir)
else
– print(“not reachable”)
–native.showAlert(“network”, “using cached copy”, {“OK”})
– look for an existing copy
local path = system.pathForFile(feedName, baseDir)
local fh, errStr = io.open( path, “r” )
if fh then
io.close(fh)
– print(“calling processRSSfeed because the network isn’t reachable”)
processRSSFeed(feedName, baseDir)
else
local alert = native.showAlert( “RSS”, “Feed temporarily unavaialble.”,
{ “OK” }, onAlertComplete )
end
end
return true
end

if network.canDetectNetworkStatusChanges then
network.setStatusListener( “http://www.thesixthaxis.com”, MyNetworkReachabilityListener )
else
native.showAlert(“network”, “not supported”, {“OK”})
– print(“network reachability not supported on this platform”)
end
end

displayFeed(feedName, feedURL)

[/code] [import]uid: 24111 topic_id: 19980 reply_id: 78116[/import]

#16

Well that can be for one of two reasons…

At line 48, event.isReachable is false which is likely the cause.

On line’s 47 and 72, you include “http://” at the beginning of the line. These calls are NOT URL’s but machine DNS names. Leave the http:// off and try it again.

The other place that will cause you that error is on line 34 if network.download failed to download the file.
[import]uid: 19626 topic_id: 19980 reply_id: 78121[/import]

#17

Now I did this:

local rss = require("rss")  
local print\_r = require("utility").print\_r  
  
local feedURL = "www.thesixthaxis.com/feed/"  
local feedName = "index.rss"  
local baseDir = system.CachesDirectory  
  
function displayFeed(feedName, feedURL)  
  
--print("entering displayFeed", feedName, feedURL)  
  
 local function processRSSFeed(file, path)  
 -- print("Parsing the feed")  
 local story = {}  
 local feed = rss.feed(file, path)  
 -- print\_r(feed)  
 local stories = feed.items  
 native.setActivityIndicator( false )  
 -- print("Num stories: " .. #stories)  
 -- print("Got ", #stories, " stories")  
 --print\_r(stories)  
 mytext = {}  
 for i=1, #stories do  
 mytext[#mytext+1] = display.newText(stories[i].title, 50,50 + (i\*50), system.contentFont, 25)  
 end  
 end  
  
 local function onAlertComplete( event )  
 return true  
 end  
  
 local networkListener = function( event )  
 utility.print\_r(event)  
 if ( event.isError ) then  
 local alert = native.showAlert( "RSS", "Feed temporarily unavaialble.",   
 { "OK" }, onAlertComplete )  
 else  
 -- print("calling processRSSFeed because the feed is avaialble")  
 processRSSFeed(feedName, baseDir)  
 end  
 return true  
 end  
  
 function MyNetworkReachabilityListener(event)  
 native.showAlert("network", "is reachable", {"OK"})  
 print( "isReachable", event.isReachable )  
 network.setStatusListener( "www.thesixthaxis.com", nil )  
 if event.isReachable then  
 -- download the latest file  
 native.setActivityIndicator( true )  
 --native.showAlert("network", "downloading", {"OK"})  
 network.download(feedURL, "GET", networkListener, feedName, baseDir)  
 else  
 -- print("not reachable")  
 --native.showAlert("network", "using cached copy", {"OK"})  
 -- look for an existing copy  
 local path = system.pathForFile(feedName, baseDir)  
 local fh, errStr = io.open( path, "r" )  
 if fh then  
 io.close(fh)  
 -- print("calling processRSSfeed because the network isn't reachable")  
 processRSSFeed(feedName, baseDir)  
 else  
 local alert = native.showAlert( "RSS", "Feed temporarily unavaialble.",   
 { "OK" }, onAlertComplete )  
 end  
 end  
 return true  
 end  
  
 if network.canDetectNetworkStatusChanges then  
 network.setStatusListener( "www.thesixthaxis.com", MyNetworkReachabilityListener )  
 else  
 native.showAlert("network", "not supported", {"OK"})  
 -- print("network reachability not supported on this platform")  
 end  
end  
  
displayFeed(feedName, feedURL)

And the terminal says this:

isReachable true  
table: 0x757f360 {  
 [isError] =\> true  
 [name] =\> "networkRequest"  
 [response] =\> "unsupported URL"  
}

But if I change line 47 and 72 to network.setStatusListener( “www.thesixthaxis.com/feed”, MyNetworkReachabilityListener )

Then the terminal says this:

isReachable false [import]uid: 24111 topic_id: 19980 reply_id: 78124[/import]

#18

There is a difference between a URL and a server name.

The network.setStatusListener needs a server name, i.e.

network.setStatusListener( “www.thesixthaxis.com”, MyNetworkReachabilityListener )

is correct.

URL’s need to have an “http://” in front of it (or ftp: or mailto: depending on the resource you’re trying to access).

Thus line 4 needs to be:

local feedURL = “http://www.thesixthaxis.com/feed/”

You took the “http://” off of line for and that became an invalid URL.

[import]uid: 19626 topic_id: 19980 reply_id: 78133[/import]

#19

Still got the same problem though :confused:

local rss = require("rss")  
local print\_r = require("utility").print\_r  
  
local feedURL = "http://www.thesixthaxis.com/feed/"  
local feedName = "index.rss"  
local baseDir = system.CachesDirectory  
  
function displayFeed(feedName, feedURL)  
  
--print("entering displayFeed", feedName, feedURL)  
  
 local function processRSSFeed(file, path)  
 -- print("Parsing the feed")  
 local story = {}  
 local feed = rss.feed(file, path)  
 -- print\_r(feed)  
 local stories = feed.items  
 native.setActivityIndicator( false )  
 -- print("Num stories: " .. #stories)  
 -- print("Got ", #stories, " stories")  
 --print\_r(stories)  
 mytext = {}  
 for i=1, #stories do  
 mytext[#mytext+1] = display.newText(stories[i].title, 50,50 + (i\*50), system.contentFont, 25)  
 end  
 end  
  
 local function onAlertComplete( event )  
 return true  
 end  
  
 local networkListener = function( event )  
 utility.print\_r(event)  
 if ( event.isError ) then  
 local alert = native.showAlert( "RSS", "Feed temporarily unavaialble.",   
 { "OK" }, onAlertComplete )  
 else  
 -- print("calling processRSSFeed because the feed is avaialble")  
 processRSSFeed(feedName, baseDir)  
 end  
 return true  
 end  
  
 function MyNetworkReachabilityListener(event)  
 --native.showAlert("network", "is reachable", {"OK"})  
 print( "isReachable", event.isReachable )  
 network.setStatusListener( "www.thesixthaxis.com", nil )  
 if event.isReachable then  
 -- download the latest file  
 native.setActivityIndicator( true )  
 --native.showAlert("network", "downloading", {"OK"})  
 network.download(feedURL, "GET", networkListener, feedName, baseDir)  
 else  
 -- print("not reachable")  
 --native.showAlert("network", "using cached copy", {"OK"})  
 -- look for an existing copy  
 local path = system.pathForFile(feedName, baseDir)  
 local fh, errStr = io.open( path, "r" )  
 if fh then  
 io.close(fh)  
 -- print("calling processRSSfeed because the network isn't reachable")  
 processRSSFeed(feedName, baseDir)  
 else  
 local alert = native.showAlert( "RSS", "Feed temporarily unavaialble.",   
 { "OK" }, onAlertComplete )  
 end  
 end  
 return true  
 end  
  
 if network.canDetectNetworkStatusChanges then  
 network.setStatusListener( "www.thesixthaxis.com", MyNetworkReachabilityListener )  
 else  
 native.showAlert("network", "not supported", {"OK"})  
 -- print("network reachability not supported on this platform")  
 end  
end  
  
displayFeed(feedName, feedURL)

It says:

[code]
isReachable true
table: 0x554dc0 {
[isError] => false
[name] => “networkRequest”
[response] => “/Users/Oskar/Library/Application Support/Corona Simulator/robmiracle-Corona-SDK-RSS-Reader-81dbd9d-9612EC2C053FB2F4DDC1E5FB45A737D9/Documents/index.rss”
}

/Users/Oskar/Library/Application Support/Corona Simulator/robmiracle-Corona-SDK-RSS-Reader-81dbd9d-9612EC2C053FB2F4DDC1E5FB45A737D9/Caches/index.rss: No such file or directory
Runtime error
…ds/robmiracle-Corona-SDK-RSS-Reader-81dbd9d/main.lua:17: attempt to index local ‘feed’ (a nil value)
stack traceback:
[C]: ?
…ds/robmiracle-Corona-SDK-RSS-Reader-81dbd9d/main.lua:17: in function ‘processRSSFeed’
…ds/robmiracle-Corona-SDK-RSS-Reader-81dbd9d/main.lua:39: in function <…ds>
[/code] [import]uid: 24111 topic_id: 19980 reply_id: 78135[/import] </…ds>

#20

Okay, your running into a bug that was just fixed in today’s daily build.

Change line 6 to:

local baseDir = system.DocumentsDirectory [import]uid: 19626 topic_id: 19980 reply_id: 78140[/import]