Increasing memory usage after display objects are removed

joshuacalebbryan · August 20, 2019, 7:36pm

Hello all,

My issue put simply is this : I create hundreds of display objects, and, upon removing them, my application’s memory does not reduce as it should when the objects are removed and garbage collection ( collectgarbage( “collect” ) ) is called.

Display object information setup:

local image\_sheet\_file = "test\_tile\_set.png" local options = { &nbsp; &nbsp; width = 32, &nbsp; &nbsp; height = 32, &nbsp; &nbsp; numFrames = 4, &nbsp; &nbsp; sheetContentWidth = 64, &nbsp; &nbsp; sheetContentHeight = 64, } sheet = graphics.newImageSheet( image\_sheet\_file, options )

Table storage for display objects:

object\_table = {}

Generation of display objects:

local function generate\_object(image\_sheet\_index, postion) local obj = display.newImage( sheet,&nbsp; image\_sheet\_index,&nbsp; postion.x, postion.y ) obj.id = #object\_table function obj.remv\_func() obj:removeSelf() obj = nil end return obj end

I call this function several times depending on the need, sometimes ~50 times:

object\_table[#object\_table+1] = {} object\_table[#object\_table].displayed\_object = generate\_object(image\_sheet\_index, postion) --position is a table containing the x and y coordinates --image\_sheet\_index is what I use to determine which frame from sheet to use

Removal of display objects:

object\_table[id].displayed\_object.remv\_func() object\_table[id] = nil --the "id" is relative to the object needing to be removed

I am concerned that I am not removing each display object properly in order for garbage collection to eliminate their memory. I am calling collectgarbage( “collect” ) manually and displaying collectgarbage( “count” ) before and after upon a key press for testing.

From what I understand the function “obj.remv_func()” should first remove the object from the display ( obj:removeSelf() ) and then nil it ( obj = nil ) which makes it eligible for garbage collection. Please let me know if my understanding is incorrect.

XeduR · August 20, 2019, 9:15pm

You are on the right track.

The display object has to first be removed, as you are doing. However, I don’t think that you should setup a separate function for removing the object.

In Lua, tables remain in memory for as long as a reference to them exists somewhere in the code. In your create generate_object function, when you return obj from the function, the function loses the reference to obj and now the only reference to it, i.e. the table, is in object_table[#object_table].displayed_object. You can call remv_func() as many times as you want, but you only set the reference to obj nil inside the function. If you call the function again, there will be the same reference again and nothing changes. (Well, if you use :removeSelf(), then the game would crash as the display object would have been removed on the first try.)

I would recommend generally using display.remove() to remove display objects. But, even then all you need is:

display.remove( object\_table[id].displayed\_object ) object\_table[id] = nil

Now, there can be several reasons as to why your memory usage isn’t going down after deleting them. Most likely culprit is that you haven’t actually done anything to the object_table itself. Every time you increase then length of the table by adding new values to it, you increase the memory that the table consumes. The only way to bring down the memory usage is to set the main table to nil. However, if you are going to be using the table repeatedly, then it is best to keep it as is instead of constantly removing and recreating it.

Here’s an example of what I mean. You can see the memory usage grow once the table is populated. Setting all entries to nil does nothing for memory usage, but finally setting the table to nil frees up all the memory.

local function checkMemory() collectgarbage( "collect" ) local memUsage\_str = string.format( "MEMORY = %.3f KB", collectgarbage( "count" ) ) print( memUsage\_str, "TEXTURE = "..(system.getInfo("textureMemoryUsed") / (1024 \* 1024) ) ) end timer.performWithDelay( 500,checkMemory, 12) print( "start" ) local t = {} local iterations = 10000 timer.performWithDelay( 1500, function() for i = 1, iterations do t[i] = i end print( "entries added" ) end ) timer.performWithDelay( 3000, function() for i = 1, iterations do t[i] = i end print( "entries set to nil" ) end ) timer.performWithDelay( 4500, function() t = nil print("t = nil") end )

joshuacalebbryan · August 20, 2019, 10:58pm

XeduR @Spyric , thank you for the reply. I did not fully represent how my code is structured, which I did to simply things and to avoid sharing my ideas ( those Russian spies are everywhere :ph34r: ).

I have a tile map file made with Tiled, which my program reads and creates a table-matrix based on the rows and columns of tiles:

Tile\_Map\_Loader.tile\_map = {} local function map\_data\_parse\_line(line\_id, line) --I am passing file lines to this function local x\_pos = 1 for i = 1, #line do local c = line:sub(i,i) if c == ',' then x\_pos = x\_pos + 1 else if Tile\_Map\_Loader.tile\_map[line\_id] == nil then Tile\_Map\_Loader.tile\_map[line\_id] = {} end Tile\_Map\_Loader.tile\_map[line\_id][x\_pos] = {} Tile\_Map\_Loader.tile\_map[line\_id][x\_pos].image\_sheet\_index = tonumber(c) end end end

Basically, this would take the following example file lines:
0,0,1,0,0

0,0,0,2,0

0,0,0,0,0

and cause print( Tile_Map_Loader.tile_map[1][3].image_sheet_index) to return 1 or print( Tile_Map_Loader.tile_map[2][4].image_sheet_index) to return 2.

When I generate my tiles as noted in my first post>generate_object(image_sheet_index, position**)**, I would call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = generate\_tile(image\_sheet\_index, position)

When I remove the tiles, I call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile.remv\_func() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

I say all of this because I think the problem you proposed:

“Every time you increase then length of the table by adding new values to it, you increase the memory that the table consumes. The only way to bring down the memory usage is to set the main table to nil”

seems to not be the problem, because I really don’t have a table increasing in size as far as it’s indexes go; the indexes are already created. However, I am creating and eventually deleting a new attribute ( displayed_tile ) of the table ( Tile_Map_Loader.tile_map[y][x] ) which could translate to what you are saying.

You then proposed:

display.remove( object\_table[id].displayed\_object ) object\_table[id] = nil

I have tried this, but with object:removeSelf():

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile:removeSelf() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

The problem with that is I think the actual display object stored in “Tile_Map_Loader.tile_map[y][x].displayed_tile” is not being set to nil, and think that because there is nothing else I can see that would cause the memory to keep accumulating.

Looking forward to reading any thoughts

joshuacalebbryan · August 20, 2019, 11:08pm

Well, XeduR @Spyric , you may have provided the answer as my tests are showing positive results.

Where I call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile.remv\_func() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

I have added the code:

local image\_sheet\_index = Tile\_Map\_Loader.tile\_map[y][x].image\_sheet\_index&nbsp; Tile\_Map\_Loader.tile\_map[y][x] = nil Tile\_Map\_Loader.tile\_map[y][x] = {} Tile\_Map\_Loader.tile\_map[y][x].image\_sheet\_index = image\_sheet\_index

I did this because what you said about table memory increasing made me suspect that Tile_Map_Loader.tile_map[y][x] was increasing in size every time I redefined the displayed_tile attribute, even though I set it to nil.

XeduR · August 21, 2019, 3:59am

Well, it is good that you benefited from that. However, as you seem to be at a very early stage in your project, I want to give you two pieces of advice.

Don’t optimise yet. In fact, you may not need to optimise at all. Too early optimisation is a headache that won’t stop and you don’t even know if what you are doing is necessary (or has any positive effect). In fact, too early optimisation will most likely only slow down your development.
I don’t know how serious you were, but you really don’t need to worry about someone stealing your idea. Mobile games, and ideas in general, are all about execution. Calling ideas a dime a dozen is overstating their worth, and besides, you are working with Tiled, like thousands of games before you. The point when others will become interested in your idea is when it has already transcended from an idea to a product or a service, and most likely even then only after it becomes financially successful.

XeduR · August 20, 2019, 9:15pm

You are on the right track.

The display object has to first be removed, as you are doing. However, I don’t think that you should setup a separate function for removing the object.

In Lua, tables remain in memory for as long as a reference to them exists somewhere in the code. In your create generate_object function, when you return obj from the function, the function loses the reference to obj and now the only reference to it, i.e. the table, is in object_table[#object_table].displayed_object. You can call remv_func() as many times as you want, but you only set the reference to obj nil inside the function. If you call the function again, there will be the same reference again and nothing changes. (Well, if you use :removeSelf(), then the game would crash as the display object would have been removed on the first try.)

I would recommend generally using display.remove() to remove display objects. But, even then all you need is:

display.remove( object\_table[id].displayed\_object ) object\_table[id] = nil

Now, there can be several reasons as to why your memory usage isn’t going down after deleting them. Most likely culprit is that you haven’t actually done anything to the object_table itself. Every time you increase then length of the table by adding new values to it, you increase the memory that the table consumes. The only way to bring down the memory usage is to set the main table to nil. However, if you are going to be using the table repeatedly, then it is best to keep it as is instead of constantly removing and recreating it.

Here’s an example of what I mean. You can see the memory usage grow once the table is populated. Setting all entries to nil does nothing for memory usage, but finally setting the table to nil frees up all the memory.

local function checkMemory() collectgarbage( "collect" ) local memUsage\_str = string.format( "MEMORY = %.3f KB", collectgarbage( "count" ) ) print( memUsage\_str, "TEXTURE = "..(system.getInfo("textureMemoryUsed") / (1024 \* 1024) ) ) end timer.performWithDelay( 500,checkMemory, 12) print( "start" ) local t = {} local iterations = 10000 timer.performWithDelay( 1500, function() for i = 1, iterations do t[i] = i end print( "entries added" ) end ) timer.performWithDelay( 3000, function() for i = 1, iterations do t[i] = i end print( "entries set to nil" ) end ) timer.performWithDelay( 4500, function() t = nil print("t = nil") end )

joshuacalebbryan · August 20, 2019, 10:58pm

XeduR @Spyric , thank you for the reply. I did not fully represent how my code is structured, which I did to simply things and to avoid sharing my ideas ( those Russian spies are everywhere :ph34r: ).

I have a tile map file made with Tiled, which my program reads and creates a table-matrix based on the rows and columns of tiles:

Tile\_Map\_Loader.tile\_map = {} local function map\_data\_parse\_line(line\_id, line) --I am passing file lines to this function local x\_pos = 1 for i = 1, #line do local c = line:sub(i,i) if c == ',' then x\_pos = x\_pos + 1 else if Tile\_Map\_Loader.tile\_map[line\_id] == nil then Tile\_Map\_Loader.tile\_map[line\_id] = {} end Tile\_Map\_Loader.tile\_map[line\_id][x\_pos] = {} Tile\_Map\_Loader.tile\_map[line\_id][x\_pos].image\_sheet\_index = tonumber(c) end end end

Basically, this would take the following example file lines:
0,0,1,0,0

0,0,0,2,0

0,0,0,0,0

and cause print( Tile_Map_Loader.tile_map[1][3].image_sheet_index) to return 1 or print( Tile_Map_Loader.tile_map[2][4].image_sheet_index) to return 2.

When I generate my tiles as noted in my first post>generate_object(image_sheet_index, position**)**, I would call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = generate\_tile(image\_sheet\_index, position)

When I remove the tiles, I call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile.remv\_func() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

I say all of this because I think the problem you proposed:

“Every time you increase then length of the table by adding new values to it, you increase the memory that the table consumes. The only way to bring down the memory usage is to set the main table to nil”

seems to not be the problem, because I really don’t have a table increasing in size as far as it’s indexes go; the indexes are already created. However, I am creating and eventually deleting a new attribute ( displayed_tile ) of the table ( Tile_Map_Loader.tile_map[y][x] ) which could translate to what you are saying.

You then proposed:

display.remove( object\_table[id].displayed\_object ) object\_table[id] = nil

I have tried this, but with object:removeSelf():

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile:removeSelf() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

The problem with that is I think the actual display object stored in “Tile_Map_Loader.tile_map[y][x].displayed_tile” is not being set to nil, and think that because there is nothing else I can see that would cause the memory to keep accumulating.

Looking forward to reading any thoughts

joshuacalebbryan · August 20, 2019, 11:08pm

Well, XeduR @Spyric , you may have provided the answer as my tests are showing positive results.

Where I call:

Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile.remv\_func() Tile\_Map\_Loader.tile\_map[y][x].displayed\_tile = nil

I have added the code:

local image\_sheet\_index = Tile\_Map\_Loader.tile\_map[y][x].image\_sheet\_index&nbsp; Tile\_Map\_Loader.tile\_map[y][x] = nil Tile\_Map\_Loader.tile\_map[y][x] = {} Tile\_Map\_Loader.tile\_map[y][x].image\_sheet\_index = image\_sheet\_index

I did this because what you said about table memory increasing made me suspect that Tile_Map_Loader.tile_map[y][x] was increasing in size every time I redefined the displayed_tile attribute, even though I set it to nil.

XeduR · August 21, 2019, 3:59am

Well, it is good that you benefited from that. However, as you seem to be at a very early stage in your project, I want to give you two pieces of advice.

Don’t optimise yet. In fact, you may not need to optimise at all. Too early optimisation is a headache that won’t stop and you don’t even know if what you are doing is necessary (or has any positive effect). In fact, too early optimisation will most likely only slow down your development.
I don’t know how serious you were, but you really don’t need to worry about someone stealing your idea. Mobile games, and ideas in general, are all about execution. Calling ideas a dime a dozen is overstating their worth, and besides, you are working with Tiled, like thousands of games before you. The point when others will become interested in your idea is when it has already transcended from an idea to a product or a service, and most likely even then only after it becomes financially successful.