Why does texture memory jump?

Audio and Graphics
#1

Hello,

I have a question about the logic behind texture memory.

Here is my code:

local rect = display.newImageRect("rect.png", display.actualContentWidth, display.actualContentHeight) rect.x = display.contentCenterX rect.y = display.contentCenterY local function printMemUsage() local memUsed = (collectgarbage("count")) / 1000 local texUsed = system.getInfo( "textureMemoryUsed" ) / 1000000 print("\n---------MEMORY USAGE INFORMATION---------") print("System Memory Used:", string.format("%.03f", memUsed), "Mb") print("Texture Memory Used:", string.format("%.03f", texUsed), "Mb") print("------------------------------------------\n") return true end -- Only load memory monitor if running in simulator if (system.getInfo("environment") == "simulator") then --Runtime:addEventListener( "enterFrame", printMemUsage) timer.performWithDelay( 1000, printMemUsage, -1 ) end

“rect” is a square created in Inkscape, sized 2048x1536. Fill set to black.

The image file size is 15.6KB. This is the only image and code loaded into this project, as this is out of the main.lua file of a new blank project.

My terminal readout is this:

 ---------MEMORY USAGE INFORMATION--------- System Memory Used: 0.272 Mb Texture Memory Used: 12.583 Mb ------------------------------------------

My question:

Why is there such a huge jump in memory used? Right now for every image I am using, I create it in the size I want it to be on screen. For example, on the iPad if I want a background I create the image as 2048x1536 and load it as newImageRect at that size. If I want a menu at a certain size I create it at that size. If this is incorrect please tell me. 

If it is, would I go about by reducing the dimensions in the graphic program, then increasing it again in Corona? Wouldn’t this cause the image to lose resolution?

This issue came to my attention as I discovered my current project utilized over 700MB of texture memory - creating a _slight _issue for me.

I already read this and it didn’t really answer my question

Thank you very much!

#2

You’re confusing file size with memory size. JPEG and PNG both compress image data down to reduce the on-disk size of the image. Let’s look at how an image is constructed.

RGB images that Corona uses consist of four color channels: Red, Green, Blue and Alpha Transparency. Each channel is 2 dimensional array of pixel values for that color that range from 0 to 255. 0 meaning no color, 255 meaning full color.  In your case, your rect is 2048x1536x4 8 bit integers of values. The image data alone takes up 12,582,912 bytes of memory. The image file also has several Kbytes of meta data that it also stores. If you were to save that image as a TIFF file, it would be 13mb on disk.

PNG uses a compression method where it finds a run of color values that are the same and it represents those color values as two bytes or so. With a solid black image, PNG can compress it down to virtually nothing. The 15Kb of data is mostly meta data. As your image becomes more complex the less it’s able to compress and the larger the file size would be.

So the on-disk size is meaningless to the amount of memory the image will take up.

So is 12mb of texture memory a lot? For a background image? Not really. For all your graphics? Most certainly. I had a client once who gave me full screen animations even though the action took up maybe half the screen, the rest was just transparency. To run the animation was taking well over 100mb of memory. I trimmed the images down to where it was tight around the object and drastically reduced the memory usage.

2048x1536 is an iPad Retina image size. If your users are on non-Retina iPads (and this includes the host of 600x1024 android tablets) or all the phones that are in the 640x1140 range, You’re using way more texture memory than those devices can show.  Retina iPads have the memory to handle the larger textures, but the older ones do not.

This is why Corona SDK uses dynamic images. You provide two or three sizes of images. Build your app to use the smallest images and Corona will say “Oh, high res devices, I need to use the higher quality images”. This way older devices can run on smaller images that will work better on their lower memory footprint devices.

Just to give you an idea of how much the memory difference is: On a non-Retina iPad, that same rect.png if it were 1024x768 would be a mere 3,145,728 bytes:  3.1mb vs. 12.5mb. 1/4th of the memory used.

At a minimum you should be providing 1x sized images based on 1024x768 (rect.png) and 2x images at 2048x1536 (rect@2x.png). Then I would need to see your config.lua to advise further.

Rob

#3

Hi Rob,

Thank you very much for that detailed explanation, it was exactly what I was looking for. It makes sense, I figured there was a correlation there. 

As my only target device is the iPad retina, I haven’t messed with dynamic scaling on this project.

For future reference, it is always best to use an image that takes full advantage of the screen of the device used, correct?

And then manage the texture memory required by loading/unloading the images as required.

Thanks again

#4

Another question to add to this, as it is really confusing me now.

Utilizing composer, I assume that all images should be loaded and event listeners added in scene:create(). If I move everything to scene:show then all of the images with listeners attached get attached twice and that’s no good.

When should they be unloaded then? If I load all of my scenes, texture memory increases to around 700MB and the app crashes on my Gen 3 iPad. My lack of understanding is showing.

Moving forward I assume I need to call scene:destroy to every scene when I go to the next one? The app is a agriculture management game, and is very texture/menu image reliant. 

#5

"For future reference, it is always best to use an image that takes full advantage of the screen of the device used, correct?"

I’m not sure I follow what you mean. Should your app use the full screen? Yes. Apple will reject apps that have black letterbox bars. If you mean should I use images that fit the screen the best for whatever device the user is on? I would say most likely Yes. You need to understand the trade offs.

On an iPad Mini and non-Retina iPads, you’re loading a 12mb background image that will be scaled down to fit the screen. There has to be some processing to scale that 2048x1536 down to 1024x768. You’re using 4X the memory for that image (even scaled, its 12mb), but you only had to include one image in your app bundle. You don’t want the app bundle growing too large either.  Generally speaking lower resolution devices have less memory and have slower CPU/GPU combos which means you’re taxing those devices a bit more.

"And then manage the texture memory required by loading/unloading the images as required."

Again it’s about balance. It takes time and computing power to load an image, destroy it and let garbage collection work. You want to keep your memory lean. Generally speaking if you’re going to be using the same image over and over, perhaps its more efficient to hide it and show it again when needed rather than dump it and reload it. But if the image is big enough, and you’re not changing it often, dumping and reloading might make more sense (like backgrounds).

"Utilizing composer, I assume that all images should be loaded and event listeners added in scene:create(). If I move everything to scene:show then all of the images with listeners attached get attached twice and that’s no good."

You should create your objects in scene:create() with the exception of things that spawn during the use of the scene (i.e. spawning more bad guys in a game level). Those spawns need to happen as the game rules dictate and should not be created in advance. For touch and tap listeners, its okay to do those in scene:create().

Any other listeners like “enterFrame”, “collision”, etc. should be started in scene:show()'s “did” phase and stopped in scene:hide()'s “will” phase, unless they are object related, that is you put an enterFrame listener on a spawned object like a bullet, in which case when the bullet is created, it gets it’s enterFrame listener. When the bullet is destroyed, it removes it’s own enterFrame listener.

The reason things get attached twice is both scene:show() and scene:hide() get called twice. For scene:show() it gets called just before the scene starts transitioning on screen and it gets called a second time after it’s completely on screen.  The event table has a member variable called phase (event.phase). That will contain the string “will” or “did”. If you get the “will” phase, its starting to transition on. If you get the "did"phase it’s already on screen and ready to start. This is what I was referring to above.  Your scene:show() (and hide) have to have an if statement in it that tests for the phase or things execute twice.  For scene:hide() “will” is just before the previous scene starts its off-screen transition, and “did” is after it’s fully hidden.

"When should they be unloaded then? If I load all of my scenes, texture memory increases to around 700MB and the app crashes on my Gen 3 iPad. My lack of understanding is showing."

Composer wants to cache scenes to improve efficiency. But that can lead to large memory usage. There is a flag that can be set that will cause Composer to dump its “view” (i.e. sceneGroup, scene.view, self.view) on scene change. This is perhaps the easiest way to let Composer clean up after itself. You can set the composer.recycleOnSceneChange variable to true and when you change scenes it will clean up the previous one automatically. But be aware, it’s just killing the view group and any children in it. It does not completely un-require the scene. Where this is a problem is if you have executed code in the main chunk of the scene:

local composer = require("composer") local scene = composer.newScene() -------------------- local lives = 3 local someRandomNumber = math.random(10) \* 100 -- code in the main chunk that runs, not defined functions --------------------- function scene:create( event )     local sceneGroup = self.view     ... end ...

Using the scene recycler will not reinitialize lives to 3 if it has changed, nor will it re-execute the formula to  generate the random number. You would be responsible for resetting those values in scene:show()'s “will” phase.  See:

https://docs.coronalabs.com/api/library/composer/recycleOnSceneChange.html

The other option is to fully dump the scene using composer.removeScene( “scenename” ). This is my personal preference. It’s harsh since the scene has to be completely re-created but it’s better than bogging down the device because memory usage is too high. Personally I call composer.removeScene(“scenename”) just before I call composer.gotoScene(“scenename”). But in my case it’s not because of memory concerns it’s to make it so code in the main chunk gets re-executed. If your memory is the concern, you should in the scene you just changed to’s “did” phase:

local lastScene = composer.getSceneName( "previous" ) composer.removeScene( lastScene )

that will clean up the last scene you were in and reset it for re-entry later.

"Moving forward I assume I need to call scene:destroy to every scene when I go to the next one? The app is a agriculture management game, and is very texture/menu image reliant."

You never call scene:destroy() yourself. scene:destroy() gets called when composer.removeScene() gets called. Its a place to clean up things that composer doesn’t manage that you perhaps loaded during that scene such as audio.

One final point, when you can, you should use Image Sheets (https://docs.coronalabs.com/api/library/graphics/newImageSheet.html). This lets you combine multiple smaller images into a single larger image. Which lets you load the one large image once and then use less texture memory over all. There are of course some limits (imageSheet sizes are best if they don’t get bigger than 2048x2048). They are more efficient when you use a tool like TexturePacker to manage the optimal placement of images.

Rob

#6

Hi Rob,

Thank you again for answering all of my questions. I’ve learned to become much more efficient with image management and cut my texture memory usage by two-thirds.

I really appreciate you taking the time to answer me, makes life _far _less stressful! 

#7

You’re confusing file size with memory size. JPEG and PNG both compress image data down to reduce the on-disk size of the image. Let’s look at how an image is constructed.

RGB images that Corona uses consist of four color channels: Red, Green, Blue and Alpha Transparency. Each channel is 2 dimensional array of pixel values for that color that range from 0 to 255. 0 meaning no color, 255 meaning full color.  In your case, your rect is 2048x1536x4 8 bit integers of values. The image data alone takes up 12,582,912 bytes of memory. The image file also has several Kbytes of meta data that it also stores. If you were to save that image as a TIFF file, it would be 13mb on disk.

PNG uses a compression method where it finds a run of color values that are the same and it represents those color values as two bytes or so. With a solid black image, PNG can compress it down to virtually nothing. The 15Kb of data is mostly meta data. As your image becomes more complex the less it’s able to compress and the larger the file size would be.

So the on-disk size is meaningless to the amount of memory the image will take up.

So is 12mb of texture memory a lot? For a background image? Not really. For all your graphics? Most certainly. I had a client once who gave me full screen animations even though the action took up maybe half the screen, the rest was just transparency. To run the animation was taking well over 100mb of memory. I trimmed the images down to where it was tight around the object and drastically reduced the memory usage.

2048x1536 is an iPad Retina image size. If your users are on non-Retina iPads (and this includes the host of 600x1024 android tablets) or all the phones that are in the 640x1140 range, You’re using way more texture memory than those devices can show.  Retina iPads have the memory to handle the larger textures, but the older ones do not.

This is why Corona SDK uses dynamic images. You provide two or three sizes of images. Build your app to use the smallest images and Corona will say “Oh, high res devices, I need to use the higher quality images”. This way older devices can run on smaller images that will work better on their lower memory footprint devices.

Just to give you an idea of how much the memory difference is: On a non-Retina iPad, that same rect.png if it were 1024x768 would be a mere 3,145,728 bytes:  3.1mb vs. 12.5mb. 1/4th of the memory used.

At a minimum you should be providing 1x sized images based on 1024x768 (rect.png) and 2x images at 2048x1536 (rect@2x.png). Then I would need to see your config.lua to advise further.

Rob

#8

Hi Rob,

Thank you very much for that detailed explanation, it was exactly what I was looking for. It makes sense, I figured there was a correlation there. 

As my only target device is the iPad retina, I haven’t messed with dynamic scaling on this project.

For future reference, it is always best to use an image that takes full advantage of the screen of the device used, correct?

And then manage the texture memory required by loading/unloading the images as required.

Thanks again

#9

Another question to add to this, as it is really confusing me now.

Utilizing composer, I assume that all images should be loaded and event listeners added in scene:create(). If I move everything to scene:show then all of the images with listeners attached get attached twice and that’s no good.

When should they be unloaded then? If I load all of my scenes, texture memory increases to around 700MB and the app crashes on my Gen 3 iPad. My lack of understanding is showing.

Moving forward I assume I need to call scene:destroy to every scene when I go to the next one? The app is a agriculture management game, and is very texture/menu image reliant. 

#10

"For future reference, it is always best to use an image that takes full advantage of the screen of the device used, correct?"

I’m not sure I follow what you mean. Should your app use the full screen? Yes. Apple will reject apps that have black letterbox bars. If you mean should I use images that fit the screen the best for whatever device the user is on? I would say most likely Yes. You need to understand the trade offs.

On an iPad Mini and non-Retina iPads, you’re loading a 12mb background image that will be scaled down to fit the screen. There has to be some processing to scale that 2048x1536 down to 1024x768. You’re using 4X the memory for that image (even scaled, its 12mb), but you only had to include one image in your app bundle. You don’t want the app bundle growing too large either.  Generally speaking lower resolution devices have less memory and have slower CPU/GPU combos which means you’re taxing those devices a bit more.

"And then manage the texture memory required by loading/unloading the images as required."

Again it’s about balance. It takes time and computing power to load an image, destroy it and let garbage collection work. You want to keep your memory lean. Generally speaking if you’re going to be using the same image over and over, perhaps its more efficient to hide it and show it again when needed rather than dump it and reload it. But if the image is big enough, and you’re not changing it often, dumping and reloading might make more sense (like backgrounds).

"Utilizing composer, I assume that all images should be loaded and event listeners added in scene:create(). If I move everything to scene:show then all of the images with listeners attached get attached twice and that’s no good."

You should create your objects in scene:create() with the exception of things that spawn during the use of the scene (i.e. spawning more bad guys in a game level). Those spawns need to happen as the game rules dictate and should not be created in advance. For touch and tap listeners, its okay to do those in scene:create().

Any other listeners like “enterFrame”, “collision”, etc. should be started in scene:show()'s “did” phase and stopped in scene:hide()'s “will” phase, unless they are object related, that is you put an enterFrame listener on a spawned object like a bullet, in which case when the bullet is created, it gets it’s enterFrame listener. When the bullet is destroyed, it removes it’s own enterFrame listener.

The reason things get attached twice is both scene:show() and scene:hide() get called twice. For scene:show() it gets called just before the scene starts transitioning on screen and it gets called a second time after it’s completely on screen.  The event table has a member variable called phase (event.phase). That will contain the string “will” or “did”. If you get the “will” phase, its starting to transition on. If you get the "did"phase it’s already on screen and ready to start. This is what I was referring to above.  Your scene:show() (and hide) have to have an if statement in it that tests for the phase or things execute twice.  For scene:hide() “will” is just before the previous scene starts its off-screen transition, and “did” is after it’s fully hidden.

"When should they be unloaded then? If I load all of my scenes, texture memory increases to around 700MB and the app crashes on my Gen 3 iPad. My lack of understanding is showing."

Composer wants to cache scenes to improve efficiency. But that can lead to large memory usage. There is a flag that can be set that will cause Composer to dump its “view” (i.e. sceneGroup, scene.view, self.view) on scene change. This is perhaps the easiest way to let Composer clean up after itself. You can set the composer.recycleOnSceneChange variable to true and when you change scenes it will clean up the previous one automatically. But be aware, it’s just killing the view group and any children in it. It does not completely un-require the scene. Where this is a problem is if you have executed code in the main chunk of the scene:

local composer = require("composer") local scene = composer.newScene() -------------------- local lives = 3 local someRandomNumber = math.random(10) \* 100 -- code in the main chunk that runs, not defined functions --------------------- function scene:create( event )     local sceneGroup = self.view     ... end ...

Using the scene recycler will not reinitialize lives to 3 if it has changed, nor will it re-execute the formula to  generate the random number. You would be responsible for resetting those values in scene:show()'s “will” phase.  See:

https://docs.coronalabs.com/api/library/composer/recycleOnSceneChange.html

The other option is to fully dump the scene using composer.removeScene( “scenename” ). This is my personal preference. It’s harsh since the scene has to be completely re-created but it’s better than bogging down the device because memory usage is too high. Personally I call composer.removeScene(“scenename”) just before I call composer.gotoScene(“scenename”). But in my case it’s not because of memory concerns it’s to make it so code in the main chunk gets re-executed. If your memory is the concern, you should in the scene you just changed to’s “did” phase:

local lastScene = composer.getSceneName( "previous" ) composer.removeScene( lastScene )

that will clean up the last scene you were in and reset it for re-entry later.

"Moving forward I assume I need to call scene:destroy to every scene when I go to the next one? The app is a agriculture management game, and is very texture/menu image reliant."

You never call scene:destroy() yourself. scene:destroy() gets called when composer.removeScene() gets called. Its a place to clean up things that composer doesn’t manage that you perhaps loaded during that scene such as audio.

One final point, when you can, you should use Image Sheets (https://docs.coronalabs.com/api/library/graphics/newImageSheet.html). This lets you combine multiple smaller images into a single larger image. Which lets you load the one large image once and then use less texture memory over all. There are of course some limits (imageSheet sizes are best if they don’t get bigger than 2048x2048). They are more efficient when you use a tool like TexturePacker to manage the optimal placement of images.

Rob

#11

Hi Rob,

Thank you again for answering all of my questions. I’ve learned to become much more efficient with image management and cut my texture memory usage by two-thirds.

I really appreciate you taking the time to answer me, makes life _far _less stressful!