Sqlite - db:nrows - ignores ORDER BY clause in SQL Statement

Hello,

Regardless of how I construct the underlying table (use/do not use primary key, auto increment etc), when using db:nrows with an sql statement that includes an ORDER BY clause, the result set is sorted in ascending order of id and not in the order suggested by the ORDER BY clause.

Some sample code below:

-- include sqlite  
 require "sqlite3"  
 -- open test.db (if test.db doesn't exist it will be created)  
 local path = system.pathForFile("test.db", system.DocumentsDirectory)  
 db = sqlite3.open( path )  
  
 -- start fresh each time for testing  
 local dropTable = [[DROP TABLE highscores;]]  
 db:exec( dropTable )  
  
 -- version 1 primary key, auto increment  
 -- following test cases fail to order results as spec in sql  
 local tableSetup = [[CREATE TABLE IF NOT EXISTS highscores ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "multiplier" NUMERIC, "mode" TEXT, "score" NUMERIC);]]  
 --local tableSetup = [[CREATE TABLE IF NOT EXISTS highscores ("id" INTEGER, "multiplier" NUMERIC, "mode" TEXT, "score" NUMERIC);]]  
 --local tableSetup = [[CREATE TABLE IF NOT EXISTS highscores ("id" INTEGER PRIMARY KEY NOT NULL, "multiplier" NUMERIC, "mode" TEXT, "score" NUMERIC);]]  
 db:exec( tableSetup )  
  
 -- add some records  
 local multiplier = 2  
 local mode = "timed"  
 local score = 10000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 2  
 local mode = "timed"  
 local score = 20000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 2  
 local mode = "timed"  
 local score = 30000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 2  
 local mode = "timed"  
 local score = 40000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
  
 local multiplier = 4  
 local mode = "timed"  
 local score = 10000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 4  
 local mode = "timed"  
 local score = 20000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 4  
 local mode = "timed"  
 local score = 30000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 local multiplier = 4  
 local mode = "timed"  
 local score = 40000  
 local sql =[[INSERT INTO highscores VALUES (NULL, ']]..multiplier..[[',']]..mode..[[',']]..score..[[');]]  
 db:exec( sql )  
  
 -- print the sqlite version to the terminal  
 print( "version " .. sqlite3.version() )  
  
 -- rather than output results in 'id' order, we are attempting to output the three top scores in order  
 -- of highest to lowest. we attempt this with the following sql  
 -- unfortunately the order by clause is ignored or perhaps the db:nrows call can only operate in order of id  
 -- which may require you to apply your own sort function to the results array?  
 for row in db:nrows("SELECT \* FROM highscores WHERE multiplier = 2 ORDER BY score DESC LIMIT 1,3") do  
 local text = row.id.." "..row.multiplier.." "..row.mode.." "..row.score  
 local t = display.newText(text, 20, 30 \* row.id, null, 16)  
 t:setTextColor(255,0,255)  
 end  
  
 -- close the db on event applicationExit  
 local function onSystemEvent( event )  
 if( event.type == "applicationExit" ) then   
 db:close()  
 end  
 end  
  
 -- setup the system listener to catch applicationExit  
 Runtime:addEventListener( "system", onSystemEvent )  

I found several older unresolved posts along similar lines.

The basic select and even limit clauses work, alas the ORDER BY clause appears to get bumped.

Any assistance greatly appreciated.

Kind regards, Andrew [import]uid: 12361 topic_id: 6607 reply_id: 306607[/import]

but you’re positioning the text by a multiple of row.id!! which will never change

the order returned is right but row 2 will aways be at 2 * 30, and row 3 will always be at 3 * 30.

try taking your WHERE clause out and you’ll see what happens

try this instead

[lua]local rowcount=0

for row in db:nrows(“SELECT * FROM highscores WHERE multiplier = 2 ORDER BY score DESC LIMIT 1,3”) do
local text = row.id…" “…row.multiplier…” “…row.mode…” "…row.score

local t = display.newText(text, 20, 30 * rowcount, null, 16)
t:setTextColor(255,0,255)
end

rowcount=rowcount+1

end[/lua]
[import]uid: 6645 topic_id: 6607 reply_id: 23153[/import]

Thank you for your reply jmp909,

I didn’t have a problem with the output itself.

row.id does in fact change with each iteration of the for (loop) and displays fine.

My issue was that it appeared that when the SQL executed the result array was not sorted in the manner specified in the ‘ORDER BY’ portion of the SELECT statement. That appears a more subtle issue to debug as I have examples where it works and others where it appears not to like the one I posted. I wonder for instance if there are differences between a sqlite db created in Corona (Lua) code to one created using the Sqlite plug-in for Firefox (I am beginning to think there may be).

The easy fix (if you wanted to reverse the order of displayed records) based on say ‘score’ was simply to display them in reverse order either using a negative factor of row.id or reduce the Y position by some fixed factor (LastY - 15).

I will do some more testing today on the SQL execution issue but when I recreated the db again last night it appears to be working as expected, so whilst a bit wary I am happy.

Thanks again, Andrew [import]uid: 12361 topic_id: 6607 reply_id: 23348[/import]

hi, you’ve missed my point here. the database is returning the order correctly but then you’re reordering the list on screen based on row.id

“row.id” isn’t the order returned from the query, it’s the order you entered them into the database

also SQL LIMITs starts at 0 not 1.

so you’re example query returns your third, second and first entries in that order. then it takes the row.id for each one and multiples that by 30 to get the y value.

so your first result returned from your query is {id=3 , multiplier=2, score=30000) and this gets placed at y=3*30 = 90

then the second result returned from your query is {id=2, multiplier=2, score=20000} and this gets placed at y=2*30 = 60

etc

so visually your order is wrong, even though the results table from your query was returned correctly. try using print(text) in your loop… you will see your console output is in the correct order. you’re just reordering them onscreen incorrectly…

from console: 3 2 timed 30000 2 2 timed 20000 1 2 timed 10000
also like i said change your LIMIT to 0,3 not 1,3 and you will find you get your top score of 40000 appearing in your results…

from console, using LIMIT 0,3: 4 2 timed 40000 3 2 timed 30000 2 2 timed 20000
regards
j

[import]uid: 6645 topic_id: 6607 reply_id: 23396[/import]

Thanks jmp909,

I really appreciate your help, and for the misunderstanding, my reply wasn’t as clear as it could have been this morning.

The example I posted was a bit of a late night mashup based on the sqlite example on the ansca mobile demos http://developer.anscamobile.com/content/sqlite for sqlite which uses row.ids in that manner - and I (like you) noticed several problems when I reviewed the code the next morning. I only used row.id in respect of that particular example, therefore I knew the ids were in the order of the data as entered. In the real world as I said I am using an offset like (LastY +/- 15) based on the physical direction required for the display of text objects, not constrained by the use or misuse of the row.id.

…and yes, I am using LIMIT 0,3 or 0,5 in my code, to produces a table of high scores (apologies for the typo).

The SQL is functioning well now. All part of the fun of transitioning between coding languages.

I think we can close the book on this one thanks J.

Kind regards, Andrew [import]uid: 12361 topic_id: 6607 reply_id: 23427[/import]