Huge ScrollView sluggish with huge number of objects and groups

Solar2D Advanced Graphics
#1

I’m trying to get a handle on ScrollView performance, and why I’m seeing performance deterioration displaying huge content. 

My app currently runs Landscape only, and displays ~30pt Hebrew text with diacritical marks.

The texts are displayed in vertically moving scrollview, based on primo.cerar’s widgetsFix code discussed and found in the http://forums.coronalabs.com/topic/42131-in-g2-scrollview-is-not-that-smooth/page-2.

On an iPhone 5, the total text sizes range from 3 screen heights to 60 screen heights. Every word has from one to three diacritical marks. Previous versions of my app displayed each word as its own group object in black characters so that I can detect touch events on the word and to create background highlight rectangles and text color changes that dynamically change color to show the word that currently being referenced and its properties.

In order to draw the diacritical marks in various colors to emphasize special aspects, I’m now drawing partial words and each diacritical mark as its own object. To further stress the system, in some cases, I’m drawing each mark five times to create an inset effect.  It does look great on screen, but is seriously hogging performance (as could be expected).

The number of objects ending up on the screen can range from a couple of hundred objects and their related words to a maximum of nearly 20,000 multicolored objects for displaying 2700 words.  

At some number of objects, seemingly depending on the specific iOS device and its processing power, the scroll view’s behavior begins to deteriorate severely:

Touch the screen, then slowly move finger up, then release: scrollview does not move until I release. 

With the scrollview from primo.cerar’s widgetsFix code scrolling a huge numbers of objects, flicking does not work at all. Detection of flicking is handled in the frame event handler. It appears that at some large numbers of objects or greater being moved, the frame event handler never (or rarely) gets triggered. Changing frame rate from 30 to 60 fps seems to not make a difference with large texts, and could make it work worse for medium size texts, if the problem is that the system has too much to do between frames.

I did get flicking to work in a barely acceptable manner by moving some of the flick-detecting code into the touch handler. I haven’t found a way to get slow scrolling to follow my finger. Putting a Print in the touch handler shows me that _view.y is being set during the entire time my finger is moving; the rendering of the move doesn’t happen until I stop moving my finger. With small texts, everything works smoothly.

Programmatic ScrollTo (using TransitionTo) works without any perceptible delays.

It really seems that it’s taking the system too long to calculate what to do with all those objects to be able to smoothly render those appearing in the window of the screen. 

My Mac and PC software does the same type of display using a huge canvas, but it seems the canvas “equivalent” here, Snapshots, can’t be instantiated large enough to take my entire text. 

I’ve considered adding and removing text rows like in TableView widget, but my text rows and their borders are not simple rectangles, so the code to manage this won’t be simple, and I’m not sure I can keep up with rapid flicks. This may ultimately be the way to handle this problem.

I’m wondering if there’s a way to pre-render (on the fly) and aggregate my multicolored text object groups so that the scrollview has way fewer objects to manage. I do need to detect touch on words, but can calculate what’s being touched from location. I do need to dynamically alter the text and background color of highlighted words perhaps once every 1/2 second, but that involves changing the word that was highlighted and then the word that will be highlighted; the other 2700 words remain static.

Thoughts on this tough subject will be greatly appreciated. 

-Tom

#2

Hi Tom,

So just out of curiosity, using snapshots (display.newSnapshot()) won’t work because those snapshots would be too large and not render properly within the constraints of OpenGL? Could you combine several snapshots in a series to reduce the overall number of objects within the scrollView to far less than what you have now?

Brent

#3

This sounds a lot like a memory problem.  Corona SDK display.newText() objects are images.  We take your text, call the OS’s text rendering tools and get back an image that we can display in OpenGL texture space.  OpenGL doesn’t have a UI like a normal app would have.  It’s all textures.   Texture memory is allocated in power of two values.  That is a 36 pixel high image by 36 pixel wide (perhaps a typical single character), is going to be 64x64x4 bytes of memory or 16kbytes.  20,000 of those would work out to over 327 megabytes of memory.  No single app is going to perform well using that much texture memory.  Even moving around 20,000 objects takes time too

Rob

#4

Thanks Brent and Rob for your comments. 

Brent, can I, in fact, create multiple snapshots containing the content I need (e.g. the equivalent of ~60 iPhone 5 screens) as you seem to be suggesting, keeping them all available offscreen, to be dragged in place as the user scrolls through them, or will I quickly run out of texture memory?

If texture memory is an issue, would I have to re-render the snapshots on demand (like the listview widget does, but on a larger scale) to achieve the effect of seemless scrolling?

-Tom

#5

Tom, 

Here’s an idea building on Brent’s guidance. You could try to use tableView instead of scrollView. Imagine creating rows at screenwidth and height and placing one of your snapshots into each row. tableView widget keeps in memory what is visible plus a couple rows ahead in the scroll direction. This way you wouldn’t have to worry about whether you can afford to keep 60 screens worth of snapshots in memory and tableView would do the heavy lifting for you. Just make your rows clear with no lines in between and it will look almost the same as a scrollView. 

Hope this helps. Regards,

Kerem

#6

Kerem,

Thanks for the suggestion. I don’t think it’s a viable option, as I’ve got an option of displaying two columns, one with Hebrew text on the right, one with English translation on the left. Each Hebrew verse spans one or more lines, and verses can start and end in the middle of lines. The English translation therefore appears as paragraphs for each verse to the left of the Hebrew, starting as close as possible to the Y position of the start of the Hebrew verse. There’s no practical way to keep the formatting and keep lines separable as would be needed to use a tableview.

I’m thinking of a more freeform tableview implementation, that renders directly onto (and removes directly from) the desired position of a scrollview.

Or, creating a snapshot for each word or maybe for each verse and placing them all onto a scrollview. That would radically decrease the number objects from a max of around 20,000 objects for all the glyphes to 2000 word snapshots or perhaps 200 verse snapshots being moved around on the scrollview.

-Tom

#7

Hi Tom, 

I can visualise the challenge. Each verse and English translation is likely to be of different height and therefore you will need to adjust the row heights accordingly. I think it can work and might be less effort on your part compared to the other options but of course its your call. Will be curious to hear how to make it work in the end. Please let us know once you figure something out. 

All the best with your project. Regards,

Kerem

#8

More like: the numbers are large and in right column:

11111 1111 11111 1111

11 1111 111 1111 1111

11 111 11111 1111 111

2222 2 2222 222 11 11

22 22222 22222 222 22

The letters (a translates 1, etc.) in smaller letters in the left column, the b’s beginning to the right of the fourth line of numbers, where verse 2 begins:

aaaaa aaa aaaa aaaa      

aaa aaaa aaaa aaa a      

aa aaaa aaaaaa aa a

aaaa aa.

bbbbb bb bbb bb bbb bbb

bb bbbb bbbb bbbb.      

=====================================

I’m really leaning towards using several hundred small snapshots, one for each verse. e.g. the 1’s above would be a snapshot, located above the sentence beginning with the 2’s. Just not sure the devices’ memories will bear that load.

-Tom

#9

Hi Tom,

So just out of curiosity, using snapshots (display.newSnapshot()) won’t work because those snapshots would be too large and not render properly within the constraints of OpenGL? Could you combine several snapshots in a series to reduce the overall number of objects within the scrollView to far less than what you have now?

Brent

#10

This sounds a lot like a memory problem.  Corona SDK display.newText() objects are images.  We take your text, call the OS’s text rendering tools and get back an image that we can display in OpenGL texture space.  OpenGL doesn’t have a UI like a normal app would have.  It’s all textures.   Texture memory is allocated in power of two values.  That is a 36 pixel high image by 36 pixel wide (perhaps a typical single character), is going to be 64x64x4 bytes of memory or 16kbytes.  20,000 of those would work out to over 327 megabytes of memory.  No single app is going to perform well using that much texture memory.  Even moving around 20,000 objects takes time too

Rob

#11

Thanks Brent and Rob for your comments. 

Brent, can I, in fact, create multiple snapshots containing the content I need (e.g. the equivalent of ~60 iPhone 5 screens) as you seem to be suggesting, keeping them all available offscreen, to be dragged in place as the user scrolls through them, or will I quickly run out of texture memory?

If texture memory is an issue, would I have to re-render the snapshots on demand (like the listview widget does, but on a larger scale) to achieve the effect of seemless scrolling?

-Tom

#12

Tom, 

Here’s an idea building on Brent’s guidance. You could try to use tableView instead of scrollView. Imagine creating rows at screenwidth and height and placing one of your snapshots into each row. tableView widget keeps in memory what is visible plus a couple rows ahead in the scroll direction. This way you wouldn’t have to worry about whether you can afford to keep 60 screens worth of snapshots in memory and tableView would do the heavy lifting for you. Just make your rows clear with no lines in between and it will look almost the same as a scrollView. 

Hope this helps. Regards,

Kerem

#13

Kerem,

Thanks for the suggestion. I don’t think it’s a viable option, as I’ve got an option of displaying two columns, one with Hebrew text on the right, one with English translation on the left. Each Hebrew verse spans one or more lines, and verses can start and end in the middle of lines. The English translation therefore appears as paragraphs for each verse to the left of the Hebrew, starting as close as possible to the Y position of the start of the Hebrew verse. There’s no practical way to keep the formatting and keep lines separable as would be needed to use a tableview.

I’m thinking of a more freeform tableview implementation, that renders directly onto (and removes directly from) the desired position of a scrollview.

Or, creating a snapshot for each word or maybe for each verse and placing them all onto a scrollview. That would radically decrease the number objects from a max of around 20,000 objects for all the glyphes to 2000 word snapshots or perhaps 200 verse snapshots being moved around on the scrollview.

-Tom

#14

Hi Tom, 

I can visualise the challenge. Each verse and English translation is likely to be of different height and therefore you will need to adjust the row heights accordingly. I think it can work and might be less effort on your part compared to the other options but of course its your call. Will be curious to hear how to make it work in the end. Please let us know once you figure something out. 

All the best with your project. Regards,

Kerem

#15

More like: the numbers are large and in right column:

11111 1111 11111 1111

11 1111 111 1111 1111

11 111 11111 1111 111

2222 2 2222 222 11 11

22 22222 22222 222 22

The letters (a translates 1, etc.) in smaller letters in the left column, the b’s beginning to the right of the fourth line of numbers, where verse 2 begins:

aaaaa aaa aaaa aaaa      

aaa aaaa aaaa aaa a      

aa aaaa aaaaaa aa a

aaaa aa.

bbbbb bb bbb bb bbb bbb

bb bbbb bbbb bbbb.      

=====================================

I’m really leaning towards using several hundred small snapshots, one for each verse. e.g. the 1’s above would be a snapshot, located above the sentence beginning with the 2’s. Just not sure the devices’ memories will bear that load.

-Tom