TableView with thousands/millions of rows

Solar2D Widgets
#1

Hello,

I need to create a TableView with many thousands of row elements.  On iOS, this is easily accomplished because the TableView asks the TableViewDelegate how many rows are in the table, and then only asks the delegate for the rows that it needs as they scroll onscreen.

Is there any way to create a table with thousands or even millions of rows, without having to do “tableView:insertRow{}” inside a giant for loop?  The documentation seems to indicate that there is no way to do this, because I need to call insertRow to tell the height and params for each row to the TableView

-Ed

#2

We only render the on screen rows (and perhaps a few on either side).  You should be able to have a large number of rows without too much issue.  You’re constrained by memory more than anything.  But tableViews are a **User** Interface object.  Are users really going to scroll through thousands and millions of rows?

The insertRow method isn’t to put rows on the screen but to populate the table and we handle which rows are visible.

Rob

#3

Hi Rob, thanks for the reply.  Yes, I understand that the insertRow function only populates the table rather than, i.e., calling the onRowRender.

You make a good point that the TableView is a user interface object, and I probably would not expect my users to actually scroll through thousands of entries on my table.  However, I am averse to arbitrarily limiting the number of items that the user can access in my table.  For example, my table is a list of stars in my database, in reverse order of their brightness.  Even though I don’t expect the user to scroll down to the 85,000th star, having a brightness cutoff seems a bit arbitrary.  Perhaps it is necessary to do so, though.

Thanks again for the reply.

#4

You might have other problems beside tableView constraints. Suggest loading your data into SQLite on your PC/Mac and seeing how large the actual file gets. Opening a db of this size and manipulating it in memory might be just too much to handle to for the mobile os.

#5

Hi ksan,

Thanks for the suggestion.  I have actually already written and published this app for iOS, (it is called StarMap 3D+).  I am just thinking about how I will port it to Corona for Android and Windows Phone support.

Right now my app contains a database of ~107k stars, and the star catalog is fully scrollable from beginning to end.  The star data is straight C++ objects in a vector for this purpose.  I don’t use SQL or any database software for data storage, because the app needs maximum random-access performance during OpenGL rendering.

Aside from my case though, maybe there might be other situations where this kind of “lazy loading” of large amounts of row data would be advantageous.  For example, if you had an app that accessed online census data, or some other ordered data that the app doesn’t necessarily even have in memory.  Maybe the user could search for an item, and be brought to the general area around the item, if the order of the surrounding items provided some kind of context to the user, or something.

#6

Wow! I looked at your app on App Store. It is very very impressive. So if I get this right it was written in Objective-C right? It will be interesting to see how your port compares in performance & capabilities. Best of luck!

#7

Just to re-itterate, I don’t believe there is a limit on the number of rows outside of whatever memory usage and performance limits the device puts on you.

If the inserting of the rows is the issue, perhaps a scrollView would be an option or taking the open source version and altering it to fit your needs.

#8

Thanks, ksan.  Yes, it was written in Objective-C++ (a fusion of Objective-C and C++).  Porting it to Corona is going to be a huge task, but I think well worth it in the end.

Rob Miracle:  I will look into the open source version, and see if I can work with that later.  At this point, I’m not even sure if inserting the rows will be a big performance problem.  I’ll need to run some tests on that.

#9

We only render the on screen rows (and perhaps a few on either side).  You should be able to have a large number of rows without too much issue.  You’re constrained by memory more than anything.  But tableViews are a **User** Interface object.  Are users really going to scroll through thousands and millions of rows?

The insertRow method isn’t to put rows on the screen but to populate the table and we handle which rows are visible.

Rob

#10

Hi Rob, thanks for the reply.  Yes, I understand that the insertRow function only populates the table rather than, i.e., calling the onRowRender.

You make a good point that the TableView is a user interface object, and I probably would not expect my users to actually scroll through thousands of entries on my table.  However, I am averse to arbitrarily limiting the number of items that the user can access in my table.  For example, my table is a list of stars in my database, in reverse order of their brightness.  Even though I don’t expect the user to scroll down to the 85,000th star, having a brightness cutoff seems a bit arbitrary.  Perhaps it is necessary to do so, though.

Thanks again for the reply.

#11

You might have other problems beside tableView constraints. Suggest loading your data into SQLite on your PC/Mac and seeing how large the actual file gets. Opening a db of this size and manipulating it in memory might be just too much to handle to for the mobile os.

#12

Hi ksan,

Thanks for the suggestion.  I have actually already written and published this app for iOS, (it is called StarMap 3D+).  I am just thinking about how I will port it to Corona for Android and Windows Phone support.

Right now my app contains a database of ~107k stars, and the star catalog is fully scrollable from beginning to end.  The star data is straight C++ objects in a vector for this purpose.  I don’t use SQL or any database software for data storage, because the app needs maximum random-access performance during OpenGL rendering.

Aside from my case though, maybe there might be other situations where this kind of “lazy loading” of large amounts of row data would be advantageous.  For example, if you had an app that accessed online census data, or some other ordered data that the app doesn’t necessarily even have in memory.  Maybe the user could search for an item, and be brought to the general area around the item, if the order of the surrounding items provided some kind of context to the user, or something.

#13

Wow! I looked at your app on App Store. It is very very impressive. So if I get this right it was written in Objective-C right? It will be interesting to see how your port compares in performance & capabilities. Best of luck!

#14

Just to re-itterate, I don’t believe there is a limit on the number of rows outside of whatever memory usage and performance limits the device puts on you.

If the inserting of the rows is the issue, perhaps a scrollView would be an option or taking the open source version and altering it to fit your needs.

#15

Thanks, ksan.  Yes, it was written in Objective-C++ (a fusion of Objective-C and C++).  Porting it to Corona is going to be a huge task, but I think well worth it in the end.

Rob Miracle:  I will look into the open source version, and see if I can work with that later.  At this point, I’m not even sure if inserting the rows will be a big performance problem.  I’ll need to run some tests on that.