Badly Needed: newObject = display.flattened (objectOrGroupNameToBeFlattened)

[troylyndon]

My game builds a multi-screen, scrollable board that can be as high as 10 screens.

On my board, I have numerous tiles, letters, letter outlines and letter drop-shadows - this is necessary to make it look really great, and it does. So, if each tile takes 50 display objects to look nice, the largest boards could have as many as 2500-5000 display objects.

Even if games don’t scroll, if they desire to use the Corona API to display complex imagery, this feature is long-overdue and desperately needed.

As a result, on most Android devices (except the latest Samsung or equivalents), performance is bad. Not just sorta bad. Really bad. The scrolling moves like molasses.

To solve this, I have attempted, even with Daily Build 2850, to use display.capture (only grabs from the screen), display.save (only saves objects the size of the screen) and display.newSnapshot (can’t get it to work at all), but I’m under the impression it is also limited to the size of the device’s screen resolution.

I don’t have time to go through my actual tests here, nor do you likely want to read it. I’d just like to know, has  anyone successfully flattened a group that is larger than the screen size, and when displaying the new flattened group, been able to keep it in its original resolution, size and location? Just a very simple need that I’m sure many scrollable games would have in order to improve performance.

If not, would Corona please consider creating a simple new function to flatten a group?

newObject = display.flattened (objectOrGroupNameToBeFlattened)

Ideally, this new object would retain the exact size, resolution and location as the group.

Further, it would be nice to have the ability to attach the object to a display group (as with other objects) and more importantly, the opportunity to have Corona remove and nil-out the object or group being flattened. That would be cool and useful. Wouldn’t it?

[Develephant]

Hi,

Not sure if you already have, but I would put that in the feature request tracker here http://feedback.coronalabs.com/forums/188732-corona-sdk-feature-requests-feedback

This doesn’t solve your issue directly, but any complex graphics in apps are generally created externally and loaded in.

Cheers.

[torbenratzlaff]

Hey,

I’m not sure, if the feature you suggest would solve your problem. You’re are trying to move around lot’s of display object and for the feature you suggest it would take way to much texture memory to run better.

So I think the solution lies somewhere else.

1. Only move the objects, that are neccesary. If you move a group, all objects inside the group will be updated, regardles if they are on screen or not. You could try to scroll, by only moving the objects that have to be moved in order to be seen on screen.

2. Reuse (or delete and recreate) objects as you scroll. I suppose you use many similar objects inside the list. For example, if you like 100 times the same graphic, like a button or icon, you could reuse them dynamicaly instead of creating 100 of them.

In short, you would use similar techniques like the ones used for optimizing tiles gameworld.

[davebollinger]

each of your “tiles” have the 50 or so layers each, correct?  so it’s the complexity of the tiles that’s limiting, correct?  if each individual tile were “flattened” would the rest of your display performance be ok?  (that is, you aren’t ALSO limited by the number of tiles, even if flattened, are you?)

if so, you could probably pre-render your individual tile “groups” to texture

https://docs.coronalabs.com/daily/api/library/graphics/newTexture.html

(assuming those 50 layers aren’t “dynamic”, in which case this whole approach falls apart, though snapshots might work instead if they’re only “occassionally dynamic”)

the rest of your display would still remain tile-based, but the tiles themselves would be much “simpler”

[roaminggamer]

Yeah, I was going to suggest using a snapshot for the non-dynamic components of your scene/level.  No need to keep multiple static objects around if you can simply replace them with a snapshot.

[troylyndon]

Thanks develophant. I will add the feature request.

Thanks roaminggamer for the suggestion to use display.snapshot to work for my non-unique empty tile background images. This will not work for my tiles that are unique from level to level.

Thanks davebollinger for the suggestion to use graphics.newTexture, but yes, these display objects are dynamic for each level. Once setup, they don’t change until the next level.

Thanks torbenratzlaff for the suggestion to build and scroll the images as they are needed. I might try this, but the challenge remains that more CPU time would be required during scrolling than at setup. So what I gain in fewer objects to move, I still lose (in performance) building them on-the-fly.

So you can see, having the display.flattened feature is the only sure thing that will increase performance because instead of moving 2500-5000 objects, it will be moving only one large image.

[rob]

Actually isn’t this what display.newSnapshot() does?  https://docs.coronalabs.com/api/library/display/newSnapshot.html

Rob

[troylyndon]

Hi Rob,

The following code is a modified version of the ‘fish’ example for display.newSnapshot.

As you will see, it works fine. However, if you take out the comment lines at the bottom (which removes the original display objects), you will see that this results in the newSnapshot disappearing.

1. So, what makes the newSnapshot any different from a container in which groups and objects can be placed into, since their source objects must remain?

2. I tried to attach a touch listener to it, but it didn’t take. Any reason why?

   display.setStatusBar(display.HiddenStatusBar) W=display.contentWidth H=display.contentHeight math.randomseed( 0 ) local grp=display.newGroup() local fish={} for i=1,25 do fish[i] = display.newImage( “spirit.png” ) fish[i]:translate( math.random( -200, 200 ), math.random( -900, 900 ) ) grp:insert(fish[i]) end local snapshot = display.newSnapshot( W*.5, H ) snapshot.group:insert( grp ) snapshot:translate( W*.5,H*.5 ) – Center snapshot snapshot:invalidate() – Invalidate snapshot --for i=1,#fish do – display.remove(fish[i]) – fish[i]=nil --end --display.remove(grp) --grp=nil

[davebollinger]

that’s the difference between snapshots and canvases:

use a canvas if it’s a one-time render, because you can dispose of the constituent parts afterwards and retain the texture as is gaining the performance benefit of “flattening” the display, flip side is you’ll have no convenient way to rebuild it other than from scratch

use a snapshot if you wish to retain the constituent parts, potentially changing them later, invalidating and generating a new texture, still gaining all the performance benefits of a pre-rendered texture, but flip side is you have to hang on to all those pieces maybe wasting memory if its dynamic nature isn’t truly needed/exploited

sounds like what you really want is a canvas (see my prior reply)

[troylyndon]

Hi Dave,

You wrote, “sounds like what you really want is a canvas (see my prior reply)” - I don’t see your ‘prior reply’ - am I missing something?

[troylyndon]

Okay. Can you please help me with this more in-depth code example which I created based upon the recommendations here? I’ve commented it clearly so anyone can help me debug it and understand how to accomplish the desired result – which is:

1. consolidate numerous objects from a group into a flattened image, reducing memory consumption of the objects as when they were independent;

2. flatten a group which when all objects are displayed spans beyond the viewing area of one screen

3. display the memory consumption in Corona through each step to clearly understand what is happening

   – initialize a few things display.setStatusBar(display.HiddenStatusBar) display.setDefault( “anchorX”, 0 ) display.setDefault( “anchorY”, 0 ) W=display.contentWidth H=display.contentHeight math.randomseed( 0 ) --======================================================================================= function showMemoryInfo() local m=math.floor(collectgarbage( “count” )) / 1024 — with / 1024, makes it into MB local t=math.floor(system.getInfo(“textureMemoryUsed”) / 1024) / 1024 – into MB local mu=m+t local ttlMem_str = string.format(“total = %.3f MB”, mu) local memUsage_str = string.format(“memory usage = %.3f MB”, m) local texUsage_str = string.format(“textures = %.3f MB”, t) local ts="\n “…memUsage_str…”, “…texUsage_str…”, "…ttlMem_str if G_dm~=nil then display.remove(G_dm) G_dm=nil end G_dm=display.newText(ts,0,0,native.systemFont,display.viewableContentHeight*.025) return ts end --======================================================================================= print("1: Before anything: "…showMemoryInfo()) local objGroup=display.newGroup() local obj={} local i for i=1,25 do obj[i] = display.newImage( “spirit.png” ) – you can put any small image file here obj[i]:translate( math.random(-W*.1,W*.1), math.random( -H,H ) ) – column in screen center objGroup:insert(obj[i]) end print("2: After creating 25 display objects: "…showMemoryInfo()) – local snapshot = display.newSnapshot( W*.5, H*2 ) snapshot:translate( W*.25,0 ) – Center snapshot print("3: After creating newSnapshot: "…showMemoryInfo()) snapshot.canvas:insert( objGroup ) print("4: After inserting objGroup into snapshot.canvas: "…showMemoryInfo()) snapshot:invalidate(“canvas”) – Invalidate snapshot print(“5: After shapshot:invalidate(‘canvas’)”…showMemoryInfo()) local function scroll(event) if event.y<H*.5 then snapshot.y=snapshot.y-10 – any idea why this doesn’t work? else – I’ve trid snapshot.y, snapshot.group.y and snapshot.canvas.y and snapshot.y=snapshot.y+10 – none of them work end end snapshot.canvas:addEventListener(“touch”,scroll) – see comments above and help me make this functional --for i=1,#obj do – get ready to remove original objects – display.remove(obj[i]) – this section remove the objects in the canvas – obj[i]=nil – so they disappear from view --end – comment this section out to see the objects again --display.remove(objGroup) --grp=nil print(“6: After removal of original objects”…showMemoryInfo())

[troylyndon]

still no answer to this one?

[mjram]

Try with:

[lua]

snapshot:addEventListener(“touch”, scroll)

[/lua]

Regards

[troylyndon]

I have updated the code which proves that creating a ‘canvas’ does NOT work because when you attempt to remove the original source objects from the canvas (new texture), it disappears from the new texture - making it no more valuable than a glorified display group - in other words, it DOES NOT create a new texture based upon the elements drawn - but it instead, simply ‘references’ the original source objects from within a newly defined canvas.

We still have a desparate need to be able to ‘flatten’ such a ‘canvas’ so that it becomes it’s own single image so that when the original source images are removed, they do not affect the new flattened canvas.

Notice the last lines of the code which remove the original source objects is commented out. You can run it WITH and WITHOUT removing the original source images to see for yourself that the statements above are true.

-- initialize a few things display.setStatusBar(display.HiddenStatusBar) display.setDefault( "anchorX", 0 ) display.setDefault( "anchorY", 0 ) local W=display.contentWidth local H=display.contentHeight local h=H\*2 print(W, H) math.randomseed( os.time() ) --======================================================================================= ------------------------------------------------- -- code routines ------------------------------------------------- local function funcButton(e) local obj = e.target -- Key pressed -- if(e.phase == "began") then display.getCurrentStage():setFocus(obj) obj.\_isFocus = true obj.oldX = obj.x obj.oldY = obj.y elseif(e.phase == "moved") then obj.x = obj.oldX + (e.x - e.xStart) obj.y = obj.oldY + (e.y - e.yStart) -- Key released -- elseif(e.phase == "ended" or e.phase == "cancelled") then if obj.\_isFocus then obj.\_isFocus = false display.getCurrentStage():setFocus(nil) end -- if obj.\_isFocus then end -- touch began, moved, ended, cancelled return true end --======================================================================================= local function showMemoryInfo() local m=math.floor(collectgarbage("count")) / 1024 --- with / 1024, makes it into MB local t=math.floor(system.getInfo("textureMemoryUsed") / 1024) / 1024 -- into MB local mu=m+t local ttlMem\_str = string.format("total = %.3f MB", mu) local memUsage\_str = string.format("memory usage = %.3f MB", m) local texUsage\_str = string.format("textures = %.3f MB", t) local ts="\n "..memUsage\_str..", "..texUsage\_str..", "..ttlMem\_str if G\_dm~=nil then display.remove(G\_dm) G\_dm=nil end G\_dm=display.newText(ts,0,0,native.systemFont,display.viewableContentHeight\*.025) return ts end --======================================================================================= print("1: Before anything:"..showMemoryInfo()) local obj={} for i=1,150 do obj[i] = display.newText("T",0,0,native.systemFontBold,H\*0.25) obj[i].fill = {math.random(0, 255)/255, math.random(0, 255)/255, math.random(0, 255)/255} obj[i].x=math.random(W)-(W\*.5) obj[i].y=math.random(h)-H end print("2: After creating display objects:"..showMemoryInfo()) local canvas = graphics.newTexture( { type="canvas", width=W, height=h} ) local rect = display.newImageRect( canvas.filename, -- "filename" property required canvas.baseDir, -- "baseDir" property required W,h) print("3: After creating newTexture:"..showMemoryInfo()) -- -- Create display object with texture as contents for i = 1, #obj do canvas:draw(obj[i]) end print("4: After drawing objects onto canvas:"..showMemoryInfo()) -- canvas:invalidate() rect:addEventListener("touch", funcButton) timer.performWithDelay(100, function() print("5: After canvas:invalidate('canvas')"..showMemoryInfo()) end) --for i = 1, #obj do -- obj[i]:removeSelf() -- obj[i] = nil --end --timer.performWithDelay(100, function() print("6: After removal of original objects"..showMemoryInfo()) end)

[troylyndon]

Flattened at last! But I need help with one final step.

I’m working on a newer version of this code (newer than the previous version posted above) that actually saves the canvas (display.save) with isFullResolution=true; and then reloads it with display.newImage. It works! However, it takes several seconds to save a large image the size of 10 or so screens.

Does anyone know of a way to perform the save to memory (instead of to the Documents.Directory) to avoid the long delay of saving and loading?

[torbenratzlaff]

It’s pretty simple, you have to wait a frame, before you can remove the drawn objects.

As the docmentation states, the new objects are added to the canvas on the next rendering process.

The way you do it iis like creating a display object an removing it immediatly. So obviously there’s nothing to see on the next rendered frame.

Just use a timer at the end and you are fine:

timer.performWithDelay(10, function() for i = 1, #obj do obj[i]:removeSelf() obj[i] = nil end end)

Greetings

Torben

[troylyndon]

Great point, Torben. So, the following code works to create the flattened image, but it takes soooooo long to “invalidate” or perhaps the better word is “render” that it’s impractical (half a second for just 1500 objects). Any ideas on how to speed it up?

lowercase “h” = H*3, which means that our canvas is 3 times the height of the screen

display.setStatusBar(display.HiddenStatusBar) display.setDefault( "anchorX", 0 ) display.setDefault( "anchorY", 0 ) local W=display.contentWidth local H=display.contentHeight local h=H\*3 print("Resolution of Canvas:"..W.." x "..H) math.randomseed( os.time() ) local obj={} --======================================================================================= -- code routines --======================================================================================= local function funcButton(e) -- allow the user to scroll to see the multi-screen high image local obj = e.target -- Key pressed -- if(e.phase == "began") then display.getCurrentStage():setFocus(obj) obj.\_isFocus = true obj.oldX = obj.x obj.oldY = obj.y elseif(e.phase == "moved") then obj.x = obj.oldX + (e.x - e.xStart) obj.y = obj.oldY + (e.y - e.yStart) -- Key released -- elseif(e.phase == "ended" or e.phase == "cancelled") then if obj.\_isFocus then obj.\_isFocus = false display.getCurrentStage():setFocus(nil) end -- if obj.\_isFocus then end -- touch began, moved, ended, cancelled return true end --======================================================================================= local function showMemoryInfo() print("-------------------------") print("Time: "..(system.getTimer()/1000).." seconds") local m=math.floor(collectgarbage("count")) / 1024 --- with / 1024, makes it into MB local t=math.floor(system.getInfo("textureMemoryUsed") / 1024) / 1024 -- into MB local mu=m+t local ttlMem\_str = string.format("total = %.3f MB", mu) local memUsage\_str = string.format("memory usage = %.3f MB", m) local texUsage\_str = string.format("textures = %.3f MB", t) local ts="\n "..memUsage\_str..", "..texUsage\_str..", "..ttlMem\_str if G\_dm~=nil then display.remove(G\_dm) G\_dm=nil end G\_dm=display.newText(ts,0,0,native.systemFont,display.viewableContentHeight\*.025) return ts end --======================================================================================= local function afterInvalidate() print("5: After canvas:invalidate('canvas')"..showMemoryInfo()) for i = 1, #obj do     obj[i]:removeSelf()     obj[i] = nil end timer.performWithDelay(100, function() print("6: After removal of original objects"..showMemoryInfo()) end) end --======================================================================================= -- 5th version MAIN CODE --======================================================================================= print("1: Before anything:"..showMemoryInfo()) for i=1,150 do obj[i] = display.newText("T",0,0,native.systemFontBold,H\*0.25) obj[i].fill = {math.random(0, 255)/255, math.random(0, 255)/255, math.random(0, 255)/255} obj[i].x=math.random(W)-(W\*.5) obj[i].y=math.random(h)-H end -- now draw an X in the upper-left, upper-right, lower-left and lower-right of the canvas local x=-W\*.5; local y=-h\*.5 obj[#obj+1]=display.newText("X",x,y,native.systemFontBold,H\*0.25) obj[#obj+1]=display.newText("X",x,y+h-obj[#obj].height,native.systemFontBold,H\*0.25) obj[#obj+1]=display.newText("X",x+W-obj[#obj-1].width,y,native.systemFontBold,H\*0.25) obj[#obj+1]=display.newText("X",x+W-obj[#obj-2].width,y+h-obj[#obj-2].height,native.systemFontBold,H\*0.25) print("2: After creating display objects:"..showMemoryInfo()) local canvas = graphics.newTexture( { type="canvas", width=W, height=h} ) local rect = display.newImageRect( canvas.filename, -- "filename" property required canvas.baseDir, -- "baseDir" property required W,h) rect:addEventListener("touch", funcButton) print("3: After creating newTexture:"..showMemoryInfo()) -- Create display object with texture as contents for i = 1, #obj do canvas:draw(obj[i]) end print("4: After drawing objects onto canvas:"..showMemoryInfo()) canvas:invalidate() timer.performWithDelay(100, afterInvalidate) --delay a tiny bit to allow Corona to render

[torbenratzlaff]

Don’t get me wrong, but what did you exspect?

I mean, you are drawing like 1500 objects, this is a huge amount of image data to process (render).

I’m also a bit confused, I thought that’s what you wanted. Gaining runtime performance in exchange for loading time (the computing has to happen somewhere after all).

My advice is the following.

1. Use bitmap fonts to increase the creation and changes of you text. (maybe you could do the same for the shadows ?)

2. Only create objects, that are on screen. For every screen position estimate which objects are seen and which not.

3. Do not remove objects, instead store and reuse them. (e.g. you got a button next to every row, just relabel it, change it’s listener and use it in another row)

I used similar techniques for tile based games (also for endless runners) and it works great. It’s a bit tricky to get the setup rght, but it’s worth it most of the times.

[troylyndon]

Hi torbenratzlaff, excellent feedback. I have a couple more questions…

1. Can you clarify how you would use bitmap fonts? It use TTF (TrueType) fonts in the game now. Are you suggesting I create my own images for each letter in the alphabet?

2. your comment about only creating objects that appear on screen is appealing. But how do you ‘store’ them and then later ‘reuse’ them without requiring the setup/render time?

[torbenratzlaff]

1. The most simple way would be using ponyfont, which has been released some weeks ago here in the forums (https://forums.coronalabs.com/topic/62490-ponyfont-modern-bitmap-font-support-for-corona-sdk/) There you should find everything you need to know, including how to convert your ttf files into a bitmap font.

2. Well, after all you got a big table which cotains the data of all the objects. Let’s keep it simple here and say you got a line of text and a button each row. So the data for each line is for example:

   { text = “hello”, buttonListener = listenerFunction, }

Also you would need a way to estimate, which lines are in view. You do this by using simple spacial hashing (like in collisiondetection). This is a rather simple version, you would need to implement your own code to updating the scroller position and stuff.

The important thing is not to remove the texts and button. Instead just reuse them for other lines by changing their text and listener.

local lineHeight = 100 --pixel height of each line local displayHeight = 1200 --actual display height local scrollPosition = displayHeight\*0.5 --the position of the screen center in relation to the scrollable area local lastLinesInView = {0, 0} --the range of lines, that is currently in view local activeLines = {} --here are all visible lines listed local linesData = { --this the list with the data for each line as mentioned above [1] = {text = "line1", butonListener = listener1}, [2] = {text = "line2", butonListener = listener2}, [3] = {text = "line3", butonListener = listener3}, } local function updateScroller(newScrollPosition) scrollPosition = newScrollPosition local lastLinesInViewStart = lastLinesInView[1] local lastLinesInViewEnd = lastLinesInView[2] local newLinesInViewStart = math.floor((newScrollPosition - displayHeight\*0.5) / lineHeight) local newLinesInViewEnd = math.ceil((newScrollPosition - displayHeight\*0.5) / lineHeight) lastLinesInView[1] = newLinesInViewStart lastLinesInView[2] = newLinesInViewEnd --remove not needed lines for i=lastLinesInViewStart, lastLinesInViewEnd do if i \< newLinesInViewStart or i \> newLinesInViewEnd then local line = activeLines[i] if line then local text = line [1] local button = line[2] --disable text and button here activeLines[i] = nil end end end for i=newLinesInViewStart, newLinesInViewEnd do --create and position lines local lineData = linesData[i] if lineData then --check if data exists for this line local line = line = activeLines[i] local lineScollY = i\*lineHeight --the original position of the line inside the scroller local lineY = scrollPosition - lineScollY + displayHeight\*0.5 if not line then --line is not already in view and has to be created local lineText = lineData.text local buttonListener = lineData.buttonListener local text --create text object and set text text.y = lineY local button --create button object and add listener button.y = lineY activeLines[i] = {text, button} else --the line already existst and just needs to be repositioned line[1].y = lineY line[2].y = lineY end end end end