SQLite slow?

I’m running a basic database program in Corona SDK, and everything works. However it takes my code 5-10 seconds just to insert 100 rows of data (only 4 columns each) into the SqLite database. However if I run similar code in Java, it takes less than half a second. Am I doing something wrong with my code or is this just the way Corona SDK is?

Here is the code (Sorry i didn’t know how to work the bbcode tag):

require “sqlite3”

–Generate players
local players = {};
for i = 1, 100 do
     local name;
     local contact;
     local power;
     local r = math.random(1,2);

     if (r == 1 ) then
     name = “Joe”
else
     name = “Hakeem”
end

     contact = math.random(10,100);
     power = math.random(10,100);

     players[#players+1] = {name = name, contact = contact, power = power};
end

–Create player database
local path = system.pathForFile(“data.db”, system.DocumentsDirectory)
db = sqlite3.open( path )

db:exec[[
CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT, contact INTEGER, power INTEGER);
]]

print( "version " … sqlite3.version() )

for i = 1, #players do
     local insertQuery = [[INSERT INTO test VALUES]] … [[(NULL,]] … [["]] … players[i].name… [["]]
     … [[,]] … players[i].contact … [[,]] … players[i].power … [[);]]
     print(insertQuery);
     db:exec(insertQuery);
end

–Handle the applicationExit event to close the db
local function onSystemEvent( event )
     if( event.type == “applicationExit” ) then
          db:close()
     end
end
Runtime:addEventListener( “system”, onSystemEvent )
 

How are you measuring the time?

Rob

I’m not actually using any sort of timer to measure the time. My times are estimates. Java definitely took < 1 second and Corona took > 5 seconds to insert the same information. If you run the code, how long it takes will be apparent.

Try the following.  It should speed it up significantly.  (Take a look at this link for more details: http://www.sqlite.org/faq.html#q19.)

db:exec("BEGIN TRANSACTION;") for i = 1, #players do local insertQuery = [[INSERT INTO test VALUES]] .. [[(NULL,]] .. [["]] .. players[i].name.. [["]] .. [[,]] .. players[i].contact .. [[,]] .. players[i].power .. [[);]] print(insertQuery); db:exec(insertQuery); end db:exec("END TRANSACTION;")
  • Andrew
1 Like

I just ran your code putting in select:

print( system.getTimer() )

during the run.  It took around 477ms to run in the Simulator on my MacBook Pro.  If you are judging it based on print statements, keep in mind that there is IO buffering going on with the prints, so they can be delayed showing up.

Rob

Even removing the print statements, it still took about 9.266 s (timer) to run the code. Andrew’s solution worked wonders and cut down the time to .162 seconds. I also looked back on my Java code and I saw that I used a commit method to create my database, which explains why it was much faster.

atrizhong: 9 seconds in simulator? Like Rob, it took a fraction of second for me. Strange.

Corona SDK’s simulator uses the sqlite3 installed with the OS on Macs.  I don’t know about Windows.  If it’s like iOS vs. Android, we use the built in on iOS and include it on Android.

It’s possible that could be an issue.  It’s also possible that your local sqlite3 is not working correctly.  I would suggest running the test by building for a device and seeing it there. 

Rob

I ran the same code on my Android phone (without Andrew’s speed boost) and it took 6.8 seconds to populate the database, which is surprising given that my windows computer is (supposed to be) significantly faster than my phone. I couldn’t test it on an iPhone because I don’t have a Mac to build the application.

I don’t know what to make of this.

Interesting, on my Nexus 7, it took 4.2 seconds.  I also noticed that the splash screen didn’t go away either.

I refactored it to run in a timer, after 250ms and that was about 3.5 seconds.  Using AukStudio’s change, it dropped to 0.6 seconds to do the whole thing.

Rob

How are you measuring the time?

Rob

I’m not actually using any sort of timer to measure the time. My times are estimates. Java definitely took < 1 second and Corona took > 5 seconds to insert the same information. If you run the code, how long it takes will be apparent.

Try the following.  It should speed it up significantly.  (Take a look at this link for more details: http://www.sqlite.org/faq.html#q19.)

db:exec("BEGIN TRANSACTION;") for i = 1, #players do local insertQuery = [[INSERT INTO test VALUES]] .. [[(NULL,]] .. [["]] .. players[i].name.. [["]] .. [[,]] .. players[i].contact .. [[,]] .. players[i].power .. [[);]] print(insertQuery); db:exec(insertQuery); end db:exec("END TRANSACTION;")
  • Andrew

I just ran your code putting in select:

print( system.getTimer() )

during the run.  It took around 477ms to run in the Simulator on my MacBook Pro.  If you are judging it based on print statements, keep in mind that there is IO buffering going on with the prints, so they can be delayed showing up.

Rob

Even removing the print statements, it still took about 9.266 s (timer) to run the code. Andrew’s solution worked wonders and cut down the time to .162 seconds. I also looked back on my Java code and I saw that I used a commit method to create my database, which explains why it was much faster.

atrizhong: 9 seconds in simulator? Like Rob, it took a fraction of second for me. Strange.

Corona SDK’s simulator uses the sqlite3 installed with the OS on Macs.  I don’t know about Windows.  If it’s like iOS vs. Android, we use the built in on iOS and include it on Android.

It’s possible that could be an issue.  It’s also possible that your local sqlite3 is not working correctly.  I would suggest running the test by building for a device and seeing it there. 

Rob

I ran the same code on my Android phone (without Andrew’s speed boost) and it took 6.8 seconds to populate the database, which is surprising given that my windows computer is (supposed to be) significantly faster than my phone. I couldn’t test it on an iPhone because I don’t have a Mac to build the application.

I don’t know what to make of this.

Interesting, on my Nexus 7, it took 4.2 seconds.  I also noticed that the splash screen didn’t go away either.

I refactored it to run in a timer, after 250ms and that was about 3.5 seconds.  Using AukStudio’s change, it dropped to 0.6 seconds to do the whole thing.

Rob