is there a memory leak in network.download?

[sieler2]

I’ve spent hours trying to track down a small textureMemory leak, and it seems to occur sporadically when network.download calls my event handler for the download, and sporadically when it leaves my event handler.

(Yes, via google and the forum search I’ve read many pages of tips and problem reports and solutions for Corona and Lua memory leaks … most of the Corona ones seem to be related to creating GUI objects and then losing track of them, which I’m not doing.)

My testing is pointing to network.download

My code “ticks” every 2 seconds.  On a tick, it calls network.download to start a download, and then schedules another tick in 2 seconds.  The last thing it does as it exits is it checks the usage of textureMemory.

The first thing my download event handler (passed to network.download) does is check the usage of textureMemory … and I sometimes  see it has increased by 240 or more bytes (when it’s more, it’s always a multiple of 240!).  (The event handler is shown below, after the console output.)

The last thing my download event handler does is check the usage of textureMemory … I never see an increase (i.e., my event handler isn’t increasing textureMemory use inside itself).

The first thing my tick routine does is check textureMemory … and I sometimes 

see it has increased by 240 or more bytes (when it’s more, it’s always a multiple of 240!).

(BTW, my ‘tick’ routine does not request a new network.download until the prior one has finished.)

Neither scenes nor Director are in use, and absolutely no calls of the form “…new<something>” are being done in my handler.  I also commented out the only GUI-related call, a call setting “image.fill”, and the leak still occurs.

I.e., I’m reasonably sure (thanks to many calls to checking textureMemory use) that the leak occurs

sporadically after the timer event handler finishes (my “tick”) and before I get to the first line of my network.download event handler, and sporadically again when I leave it (and before I get to my ‘tick’).

By “sometimes” or “sporadically” I mean somewhere between once every seven calls to once every 500 calls … it seems to be quite unpredictable (except near the start, see below).

After 20,000 network downloads, I’ve leaked 2 MB of textureMemory, which is about one 480 byte

leak per 400 calls, on average.

Any ideas how I can further narrow down what’s happening?

(E.g., are the internals of network.download available for inspection?)

I’m also hoping that the multiple of 240 might be meaningful to someone.

I do see  one pattern … I get 7 files download, then memory leak of 1440 bytes, 10 files downloaded, then memory leak of 240 bytes, 10 files downloaded, then memory leak or 240 bytes, then several hundred (?) downloaded with no leak.

The pattern is that I’ve seen that sequence of 7/10/10 and 1440/240/240 five runs in a row on the simulator.

(I haven’t run the thousands of files test recently … it had more than 3 leaks … I’ll be running it overnight.)

Simulator version 2017.3184 (2017.12.8), running on Mac OS X Sierra.

Leak also occurs roughly similarly on an Android (Galaxy S7), not tested on an iOS device.

thanks,

Stan

Console log:

Feb 11 11:18:15.917 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:15.954 network event: ended, is err: false

Feb 11 11:18:18.461 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:18.513 network event: ended, is err: false

Feb 11 11:18:20.827 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:20.879 network event: ended, is err: false

Feb 11 11:18:23.184 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:23.232 network event: ended, is err: false

Feb 11 11:18:25.307 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:25.366 network event: ended, is err: false

Feb 11 11:18:27.337 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:27.394 network event: ended, is err: false

Feb 11 11:18:29.574 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:29.628 network event: ended, is err: false

                    Texture mem usage increased: 1,520,452 (7) delta = 1,440 : end of tick -> start on_download_event

Feb 11 11:18:31.922 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:31.978 network event: ended, is err: false

Feb 11 11:18:34.258 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:34.296 network event: ended, is err: false

Feb 11 11:18:36.343 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:36.380 network event: ended, is err: false

Feb 11 11:18:38.845 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:38.882 network event: ended, is err: false

Feb 11 11:18:40.913 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:40.978 network event: ended, is err: false

Feb 11 11:18:43.325 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:43.361 network event: ended, is err: false

Feb 11 11:18:45.690 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:45.751 network event: ended, is err: false

Feb 11 11:18:48.186 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:48.251 network event: ended, is err: false

Feb 11 11:18:50.787 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:50.823 network event: ended, is err: false

Feb 11 11:18:53.133 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:53.169 network event: ended, is err: false

                    Texture mem usage increased: 1,520,692 (8) delta = 240 : end of tick -> start on_download_event

Feb 11 11:18:55.536 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:18:55.605 network event: ended, is err: false

Feb 11 11:18:57.911 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:18:57.948 network event: ended, is err: false

Feb 11 11:19:00.328 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:19:00.365 network event: ended, is err: false

Feb 11 11:19:02.985 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:19:03.055 network event: ended, is err: false

Feb 11 11:19:05.241 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:19:05.279 network event: ended, is err: false

Feb 11 11:19:07.347 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:19:07.386 network event: ended, is err: false

Feb 11 11:19:09.680 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:19:09.718 network event: ended, is err: false

Feb 11 11:19:11.984 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:19:12.051 network event: ended, is err: false

Feb 11 11:19:14.511 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:19:14.571 network event: ended, is err: false

Feb 11 11:19:16.999 Load from net: http://www.sieler.com/tahoe.jpg

Feb 11 11:19:17.035 network event: ended, is err: false

Feb 11 11:19:17.036 Texture mem usage increased: 1,520,932 (9) delta = 240 : end of tick -> start on_download_event

Feb 11 11:19:19.137 Load from net: http://www.sieler.com/tahoe2.jpg

Feb 11 11:19:19.173 network event: ended, is err: false

Sample on_download_event routine (passed as listener to network.download):

local temppath = "" local download\_dir = system.TemporaryDirectory local download\_path = system.pathForFile ("tahoe.jpg", dir) -- example -- ... ----------------------------------------------------------------- function on\_download\_event (event) local temppath, rslt, s print ("network event: " .. event.phase .. ", is err: " .. fmt\_tf (event.isError)) get\_tmem ("start on\_download\_event") -- sometimes shows growth in textureMemory since end of tick if event.isError then native.requestExit() -- not happening during my testing end if event.phase == "ended" then download\_handle = nil temppath = system.pathForFile ("temp", download\_dir) rslt, s = os.rename (temppath, download\_path) downloading = false end event = nil get\_tmem ("end on\_download\_event") -- never shows growth in textureMemory end -- on\_download\_event

get_tmem gets tmem = system.getInfo (“textureMemoryUsed”)

and then compares it to the prior value.  If an increase is noticed, we print the “increased” message and increment the number of increases seen (the small value in the parenthesis).

(I expect a few increases from startup up until the point where I’ve looped once or twice, of course.)

[sieler2]

BTW, this app is intended to run for several months at a time, which is why small memory leaks over time matter.

[sieler2]

If I run it overnight, here’s the pattern of textureMemory increases, relative to the number of download requests:

after    7 files, increase of 1,440 : end of tick -> start on_download_event

after   10 files, increase of   240 : end of tick -> start on_download_event

after   10 files, increase of   240 : end of tick -> start on_download_event

after   70 files, increase of 1,200 : end of tick -> start on_download_event

after  100 files, increase of   480 : end of tick -> start on_download_event

after  800 files, increase of 2,160 : end of tick -> start on_download_event

after 1000 files, increase of   480 : end of tick -> start on_download_event

after 8000 files, increase of 1,440 : end of tick -> start on_download_event

which sure looks interesting!

Each increase was just prior to network.download invoking my listener.

BTW, ordinary Lua memory (as reported by collectgarbage) did not leak.

Stan

[roaminggamer]

I suggest making a very basic and yet fully functional standalone example  that demonstrates this issue.  Then zip it up and provide in in a post below.

(Click ‘more reply options’ button next to ‘POST’ button for uploading controls.)

That way others here can both duplicate the issue and perhaps help debug the cause of the problem.

In the worst case, you’ll then be ready to submit a bug which requires a standalone example anyways.

[sieler2]

Ok, shrunk it down, duplicated it.

But, after a lot of testing, it looks like the problem is likely only in the simulator.

(Testing on an Android showed no memory leaks after 1,500 downloads in the app;

in the simulator, it’s after 10, 100, and 200 downloads.)

And, I narrowed it down to an apparent interaction between a newText object and network.download.

If I update the text in the text object:    t_misc = “some message I just built”

then I get the leak.  (No, it’s not in the building of the message … (a) I’m careful, (b) it would be a Lua mem leak, not texture mem : )

If I don’t update the text, the leak disappears.

If I set a debugging option to skip the actual download, the leak also disappears.

In main.lua, I noted the line of the complicit call (about line 250 or so), and documented a few variables that make debugging easier / quicker / more verbose.

In this test, I’m doing a few less updates of text objects than I did in my original app, and the texture memory leak is a bit slower.

Still, if anyone wants to look, it’s attached (thanks for the suggestion, @roaminggamer!)

Stan

[anon63346430]

If it is only in sim I wouldn’t lose any sleep personally.

[sieler2]

> if only in sim

Although generally true, I find it convenient to do most initial testing via the simulator.  

In this case, I was trying to avoid egregious use of network bandwidth on the real server by using

a combo of the simulator and the Mac’s local web server

I come from an OS background where it would be absurdly easy for me to say "we’re losing textureMemory?  Let’s

put a debug breakpoint at the textureMemory allocator and see who’s calling it".   Several minutes later, and I’d know

the answer.   Sadly, that’s not possible on any of the “modern” platforms I use.  

The point being: I hoped someone would say "hey, texture memory leaks are common enough that Corona Labs instrumented the

simulator (and the iOS and Android SDKs) to log all calls to textureMemory" (#1) but no one volunteered that kind of info.

Stan

#1. *I*'d do that if I controlled them! 

(But, that’s the benefit of nearly 50 years of programming experience

[anon63346430]

Sim is buggy at best (recently)… device builds are much better.  Not saying this is excusable at all but it is what it is.  Maybe report it as a bug - send a simple use case to support and they will hopefully action it.

[sieler2]

BTW, this app is intended to run for several months at a time, which is why small memory leaks over time matter.

[sieler2]

If I run it overnight, here’s the pattern of textureMemory increases, relative to the number of download requests:

after    7 files, increase of 1,440 : end of tick -> start on_download_event

after   10 files, increase of   240 : end of tick -> start on_download_event

after   10 files, increase of   240 : end of tick -> start on_download_event

after   70 files, increase of 1,200 : end of tick -> start on_download_event

after  100 files, increase of   480 : end of tick -> start on_download_event

after  800 files, increase of 2,160 : end of tick -> start on_download_event

after 1000 files, increase of   480 : end of tick -> start on_download_event

after 8000 files, increase of 1,440 : end of tick -> start on_download_event

which sure looks interesting!

Each increase was just prior to network.download invoking my listener.

BTW, ordinary Lua memory (as reported by collectgarbage) did not leak.

Stan

[roaminggamer]

I suggest making a very basic and yet fully functional standalone example  that demonstrates this issue.  Then zip it up and provide in in a post below.

(Click ‘more reply options’ button next to ‘POST’ button for uploading controls.)

That way others here can both duplicate the issue and perhaps help debug the cause of the problem.

In the worst case, you’ll then be ready to submit a bug which requires a standalone example anyways.

[sieler2]

Ok, shrunk it down, duplicated it.

But, after a lot of testing, it looks like the problem is likely only in the simulator.

(Testing on an Android showed no memory leaks after 1,500 downloads in the app;

in the simulator, it’s after 10, 100, and 200 downloads.)

And, I narrowed it down to an apparent interaction between a newText object and network.download.

If I update the text in the text object:    t_misc = “some message I just built”

then I get the leak.  (No, it’s not in the building of the message … (a) I’m careful, (b) it would be a Lua mem leak, not texture mem : )

If I don’t update the text, the leak disappears.

If I set a debugging option to skip the actual download, the leak also disappears.

In main.lua, I noted the line of the complicit call (about line 250 or so), and documented a few variables that make debugging easier / quicker / more verbose.

In this test, I’m doing a few less updates of text objects than I did in my original app, and the texture memory leak is a bit slower.

Still, if anyone wants to look, it’s attached (thanks for the suggestion, @roaminggamer!)

Stan

[anon63346430]

If it is only in sim I wouldn’t lose any sleep personally.

[sieler2]

> if only in sim

Although generally true, I find it convenient to do most initial testing via the simulator.  

In this case, I was trying to avoid egregious use of network bandwidth on the real server by using

a combo of the simulator and the Mac’s local web server

I come from an OS background where it would be absurdly easy for me to say "we’re losing textureMemory?  Let’s

put a debug breakpoint at the textureMemory allocator and see who’s calling it".   Several minutes later, and I’d know

the answer.   Sadly, that’s not possible on any of the “modern” platforms I use.  

The point being: I hoped someone would say "hey, texture memory leaks are common enough that Corona Labs instrumented the

simulator (and the iOS and Android SDKs) to log all calls to textureMemory" (#1) but no one volunteered that kind of info.

Stan

#1. *I*'d do that if I controlled them! 

(But, that’s the benefit of nearly 50 years of programming experience

[anon63346430]

Sim is buggy at best (recently)… device builds are much better.  Not saying this is excusable at all but it is what it is.  Maybe report it as a bug - send a simple use case to support and they will hopefully action it.