Which one is faster?

Solar2D General Questions/Discussion
#1

Hi All,

I have some local datas. For Example i have 5000 records like below:

Id=1

Name =“xxx”

Surname =“yyy”

Age = 21

Id=2

Name =“zzz”

Surname =“ttt”

Age = 43

etc…

I have to Id.

I think two different methods to access this data.

First Method:

I will define a table in a diffrent lua file and i will storage all data like below:

SampleData={}

SampleData[1].Name=“xxx”

SampleData[1].Surname=“yyy”

SampleData[1].Age=21 

SampleData[2].Name=“zzz”

SampleData[2].Surname=“ttt”

SampleData[2].Age=43 

etc…

I will Define into a lua file about 5000 records.

Second Method:

I will storage the data into a  local SQLlite database and execute a query to  access this data like :

" SELECT * FROM SampleData WHERE Id=1"

Which method is faster? Which method is more accurate?

Thanks in advance…

#2

For me I would put this in SQL without any thought of the alternative. Its just the logical thing to do. You can select records from it etc, you will probably not display the whole thing at once right?

If for some reason its too slow in your use case, then think about alternatives like defining it all in a Lua file, but I find that quite extreme in my opinion and not worth considderation unless the SQL way can not be accomodated at all. 5000 records is not that much when it comes to a database.

Sorry, I did not really answer your question though :slight_smile:

#3

Thank you very much for your answer jonjonsson.

If a need to search by name or surname absolutely i could access to data from sql but i always access to data by Id and i would know to Id :slight_smile:

Is there any guys to have idea about this?

For example: Rob, Brent ??  :)  :slight_smile:

#4

To me the decision is based on this:

Do you need to access all the data at once.  In other words are you going to put all 5000 records in a tableView?  Do you need to do any data analysis that will look at all the data?  For instance, if you are building a mapping system and need to find all records in a 20 mile radius from the user’s current location?

In this case, you need all the data in memory all the time.  A table is the right way to go.

If you only need a few records at a time (give me all records that have an age between 18 and 32), then a database is the right way to go.  Your in memory foot print will be smaller and you won’t have to loop over too may records etc.  You could do the mapping system this way, if your database held distances from a fixed point (i.e. give me everyone with a distance of 20mi or less).  But because people move, and if you have to get the lat/long and calculate the distance from where you are, then you need all records in memory.

Does that help?

Rob

#5

have you profiled both under actual usage conditions to see if it even matters at all?? (i must assume not, since you don’t already know the answer) that’s the “right” way to do it.

>> Which method is faster? Which method is more accurate?

table[N] is as optimized as it gets for native Lua, (assuming both the table reference and reference to variable containing N are both as local as possible), but will consume a lot of memory. (again: profile it!)

neither, no “inaccuracy” is inherent in either method.

>>but i always access to data by Id

which implies single-record access by key. suggest you re-read Jon and Rob’s answers which are both good – just consider them within context of your _actual usage_. for example, if you’re doing ONE single-record access every ten seconds or so, then I doubt you’d ever notice the difference between either. on the other hand, if you’re doing 1000 single-record accesses and need to respond real-time, then table[N] will be your only viable option - no way you’re doing 1000 individual sql calls in real-time.

fwiw, hth

#6

Rob and davebollinger, thank you so much for your answers.

I dont need to all data at the same time. First i will genarete 10 random number between 1 - 5000 and i will use only ID’s of this 10 numbers. 

After yours answers i think to true way is access to lua file directly. 

My only hesitation is access to big text file (.lua file) may force the memory (.lua file should be 200- 300 K) 

Thanks again to your answers.

#7

Another factor to consider could be how often the data gets updated. If I am understanding your plan, it sounds like you are considering putting your 5000 records in your lua file and then build your app and deploy it. This will mean when you have some new records or when those people get one year older etc you will have to send out a new update to the App Store (slow and inefficient). Using an Sqlite db might could mean (with proper app setup) you never have to update your app to cater for data changes. Consider how you could download data updates off the net etc. Good luck with your project. 

#8

For me I would put this in SQL without any thought of the alternative. Its just the logical thing to do. You can select records from it etc, you will probably not display the whole thing at once right?

If for some reason its too slow in your use case, then think about alternatives like defining it all in a Lua file, but I find that quite extreme in my opinion and not worth considderation unless the SQL way can not be accomodated at all. 5000 records is not that much when it comes to a database.

Sorry, I did not really answer your question though :slight_smile:

#9

Thank you very much for your answer jonjonsson.

If a need to search by name or surname absolutely i could access to data from sql but i always access to data by Id and i would know to Id :slight_smile:

Is there any guys to have idea about this?

For example: Rob, Brent ??  :)  :slight_smile:

#10

To me the decision is based on this:

Do you need to access all the data at once.  In other words are you going to put all 5000 records in a tableView?  Do you need to do any data analysis that will look at all the data?  For instance, if you are building a mapping system and need to find all records in a 20 mile radius from the user’s current location?

In this case, you need all the data in memory all the time.  A table is the right way to go.

If you only need a few records at a time (give me all records that have an age between 18 and 32), then a database is the right way to go.  Your in memory foot print will be smaller and you won’t have to loop over too may records etc.  You could do the mapping system this way, if your database held distances from a fixed point (i.e. give me everyone with a distance of 20mi or less).  But because people move, and if you have to get the lat/long and calculate the distance from where you are, then you need all records in memory.

Does that help?

Rob

#11

have you profiled both under actual usage conditions to see if it even matters at all?? (i must assume not, since you don’t already know the answer) that’s the “right” way to do it.

>> Which method is faster? Which method is more accurate?

table[N] is as optimized as it gets for native Lua, (assuming both the table reference and reference to variable containing N are both as local as possible), but will consume a lot of memory. (again: profile it!)

neither, no “inaccuracy” is inherent in either method.

>>but i always access to data by Id

which implies single-record access by key. suggest you re-read Jon and Rob’s answers which are both good – just consider them within context of your _actual usage_. for example, if you’re doing ONE single-record access every ten seconds or so, then I doubt you’d ever notice the difference between either. on the other hand, if you’re doing 1000 single-record accesses and need to respond real-time, then table[N] will be your only viable option - no way you’re doing 1000 individual sql calls in real-time.

fwiw, hth

#12

Rob and davebollinger, thank you so much for your answers.

I dont need to all data at the same time. First i will genarete 10 random number between 1 - 5000 and i will use only ID’s of this 10 numbers. 

After yours answers i think to true way is access to lua file directly. 

My only hesitation is access to big text file (.lua file) may force the memory (.lua file should be 200- 300 K) 

Thanks again to your answers.

#13

Another factor to consider could be how often the data gets updated. If I am understanding your plan, it sounds like you are considering putting your 5000 records in your lua file and then build your app and deploy it. This will mean when you have some new records or when those people get one year older etc you will have to send out a new update to the App Store (slow and inefficient). Using an Sqlite db might could mean (with proper app setup) you never have to update your app to cater for data changes. Consider how you could download data updates off the net etc. Good luck with your project.