BitmapPaint blocks Corona's main thread?

joagre · March 29, 2018, 11:16pm

Hi,

I use a BitmapPaint (https://docs.coronalabs.com/api/type/BitmapPaint/index.html) to fill Rects with images from disk. This operation seems to block the Corona main thread causing a stutter in my scroll window.

I create Rects like this:

frames[i] = display.newRect(parent, width / 2, height / 2, width, height) frames[i].strokeWidth = 1 frames[i]:setStrokeColor(0.3, 0.3, 0.3) frames[i]:setFillColor(0, 0, 0)

and later on fill them (when the user scroll them onto the screen) with an image:

frames[k].fill = { &nbsp; type = "image", &nbsp; filename = listOfFilenames[l], &nbsp; baseDir = system.CachesDirectory }

I can not pre-load all images beforehand. Too many of them.

Any ideas on how I can solve this? Is my observation wrong? Is the blocking expected?

Thanks

/Joakim

roaminggamer · March 30, 2018, 12:56am

how big are the images?

problem on device and/or simulator

in any case tell us more about the problem hardware

In all likelihood the problem is the load. i.e. read from ‘disk’ and get texture into texture memory. This takes time for big images or multiple images in the same period.

joagre · March 30, 2018, 7:11am

The images are 20-800kB in size (average size ~150kB) and may be as many as ~50 in bad cases. The problem occurs both on my Android device (Nexus 5X with 16GB) and in the simulator.

I have reconsiderad when it comes to the blocking. I now also think that it is heavy load I see and maybe the total amount of texture memory plays a part as well.

I have revamped the app so that it releases texture/image memory as soon as the image is off-screen (with padding). This made things better. Things would be even better if I let the devices upload small/middle sized thumbnails of each image attachment. Will do that in a later release.

Thanks. I’m happy. Nothing is like a nights sleep.

Cheers

/Joakim

RedBeach · March 30, 2018, 7:44am

I assume that you are going thru a loop in that frames table and filling the rects with the image, right? Try adding a timer (a small one, like 10ms) between each frame so you allow the system to continue working instead of getting blocked until all images were loaded.

joagre · March 30, 2018, 8:50am

This improved the situtation. At least it looks like that.

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; timer.performWithDelay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; math.random(10, 100), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; function() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; frame.fill = { &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; type = "image", &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; filename = options.filename, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDirectory &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; removeProgress() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end)

The random time is perhaps counter productive?

Thanks!

/Joakim

roaminggamer · March 30, 2018, 3:44pm

What are the pixel dimensions?

Caution: Releasing textures unnecessarily may also bite you.

roaminggamer · March 30, 2018, 3:58pm

Also, Please run this command and tell us what prints in the debug log on your device:

print( system.getInfo("maxTextureSize"))

If you don’t want to look in the log do this:

Copy this: https://github.com/roaminggamer/RG_FreeStuff/raw/master/AskEd/askEdStarter.zip
Add this code to main.lua:

local max = system.getInfo(“maxTextureSize”) display.newText( max, display.contentCenterX, display.contentCenterY + 200 )
Build and run it on your device…

There should be a number on the screen.

What is it?

PS - Remember, run it on the device.

roaminggamer · March 30, 2018, 4:02pm

Note: I wouldn’t make that timer random. I would increment the value each time through the loop.

i.e. If you have 10 images to display:

local delay = 0 ... loop starts here delay = delay + 30 timer.performWithDelay( delay, function() frame.fill = { type = "image", filename = options.filename, baseDir = options.baseDirectory } removeProgress() end)

Random times (especially that short) will just stack up all the draws between 2 or 3 frames.

joagre · April 3, 2018, 1:05am

Hi,
I have added 50ms delay between each fill operation in the loop which fills the empty Rects. In the demo links below I fill each Rect as soon as it enters the screen and “unfill” it, i.e. release the texture memory as soon as it leaves the screen.

With BitmapPaint fill activated:
https://drive.google.com/open?id=1_HzU2a-mybEP_SCFP7NxoliauRcn_TxM

With BitmapPaint fill disabled:
https://drive.google.com/open?id=1w40A74DVvkFq7KOl39bSZ4fM52o6N4tO

The only difference in the code base is:

spock:Corona jocke$ svn diff widgets/attachmentslistwidget.lua =================================================================== --- widgets/attachmentslistwidget.lua&nbsp;&nbsp; &nbsp;(revision 870) +++ widgets/attachmentslistwidget.lua&nbsp;&nbsp; &nbsp;(working copy) @@ -201,12 +202,15 @@ &nbsp;&nbsp;&nbsp;&nbsp; elseif options.filename ~= nil and options.baseDir ~= nil then &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; removeProgress() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if options.delay == nil then +--[[&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; frame.fill = { &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; type = "image", &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; filename = options.filename, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDir &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } +--]] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; else +--[[&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; timer.performWithDelay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; options.delay, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; function() @@ -216,6 +220,7 @@ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDir &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end) +--]]

I also check the memory usage with the following snippet:

local function checkMem() &nbsp; collectgarbage("collect") &nbsp; local memoryUsed = collectgarbage("count") &nbsp; local maxTextureSize = system.getInfo("maxTextureSize") &nbsp; local textureMemoryUsed = system.getInfo("textureMemoryUsed") / 1048576 &nbsp; print("====") &nbsp; print("System Memory:", string.format("%.00f", memoryUsed) .. " kB") &nbsp; print("Max Texture Size:", maxTextureSize) &nbsp; print("Texture Memory:", string.format("%.03f", textureMemoryUsed) .. " MB") end timer.performWithDelay(4000, checkMem, -1)

And with BitmapPaint fill activated it gave me:

==== System Memory: 2308 kB Max Texture Size: 16384 Texture Memory:&nbsp;&nbsp; &nbsp;60.225 MB

The images I use are these (sorry for the theme, got bored):

-rw-r--r--@ 1 jocke&nbsp; staff&nbsp;&nbsp; 61661 27 Apr&nbsp; 2017 lenin.jpeg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 284992 27 Apr&nbsp; 2017 lenin.png -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 519158 27 Apr&nbsp; 2017 lenin2.jpeg lrwxr-xr-x&nbsp; 1 jocke&nbsp; staff&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 34 23 Maj&nbsp; 2017 snowden.jpg -\> ../../../../../docroot/snowden.jpg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 732648 24 Apr&nbsp; 2017 sovjet.jpg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 822548 21 Apr&nbsp; 2017 spiderman.png spock:images jocke$ ls -l ../../../../../docroot/snowden.jpg -rw-r--r--&nbsp; 1 jocke&nbsp; staff&nbsp; 28404 13 Feb&nbsp; 2017 ../../../../../docroot/snowden.jpg

In summary: The filling operations seem to choke the main thread.

I might add that I do not use https://docs.coronalabs.com/api/library/widget/newScrollView.html but instead use my own scroll module in order to be able fine tune the look and feel down to the last pixel/transition/timing. It works nicely but maybe this is the culprit here? Maybe Corona’s ScrollView does magic stuff to offload the main thread, i.e. using posix threads or whatnot? I do nothing magic in my scroll implementation: https://drive.google.com/open?id=1i08xdV79RI8k90EOoURKoSYRFOUl51GM

Can you see something broken in all of this? Sigh.

Cheers
/Joakim

joagre · April 3, 2018, 2:23am

Hmm, I Guess I have to live with this. Filling will take its time.

rob · April 3, 2018, 3:12pm

Our scrollView doesn’t offload anything to another thread.

Rob

roaminggamer · March 30, 2018, 12:56am

how big are the images?

problem on device and/or simulator

in any case tell us more about the problem hardware

In all likelihood the problem is the load. i.e. read from ‘disk’ and get texture into texture memory. This takes time for big images or multiple images in the same period.

joagre · March 30, 2018, 7:11am

The images are 20-800kB in size (average size ~150kB) and may be as many as ~50 in bad cases. The problem occurs both on my Android device (Nexus 5X with 16GB) and in the simulator.

I have reconsiderad when it comes to the blocking. I now also think that it is heavy load I see and maybe the total amount of texture memory plays a part as well.

I have revamped the app so that it releases texture/image memory as soon as the image is off-screen (with padding). This made things better. Things would be even better if I let the devices upload small/middle sized thumbnails of each image attachment. Will do that in a later release.

Thanks. I’m happy. Nothing is like a nights sleep.

Cheers

/Joakim

RedBeach · March 30, 2018, 7:44am

I assume that you are going thru a loop in that frames table and filling the rects with the image, right? Try adding a timer (a small one, like 10ms) between each frame so you allow the system to continue working instead of getting blocked until all images were loaded.

joagre · March 30, 2018, 8:50am

This improved the situtation. At least it looks like that.

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; timer.performWithDelay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; math.random(10, 100), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; function() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; frame.fill = { &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; type = "image", &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; filename = options.filename, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDirectory &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; removeProgress() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end)

The random time is perhaps counter productive?

Thanks!

/Joakim

roaminggamer · March 30, 2018, 3:44pm

What are the pixel dimensions?

Caution: Releasing textures unnecessarily may also bite you.

roaminggamer · March 30, 2018, 3:58pm

Also, Please run this command and tell us what prints in the debug log on your device:

print( system.getInfo("maxTextureSize"))

If you don’t want to look in the log do this:

Copy this: https://github.com/roaminggamer/RG_FreeStuff/raw/master/AskEd/askEdStarter.zip
Add this code to main.lua:

local max = system.getInfo(“maxTextureSize”) display.newText( max, display.contentCenterX, display.contentCenterY + 200 )
Build and run it on your device…

There should be a number on the screen.

What is it?

PS - Remember, run it on the device.

roaminggamer · March 30, 2018, 4:02pm

Note: I wouldn’t make that timer random. I would increment the value each time through the loop.

i.e. If you have 10 images to display:

local delay = 0 ... loop starts here delay = delay + 30 timer.performWithDelay( delay, function() frame.fill = { type = "image", filename = options.filename, baseDir = options.baseDirectory } removeProgress() end)

Random times (especially that short) will just stack up all the draws between 2 or 3 frames.

joagre · April 3, 2018, 1:05am

Hi,
I have added 50ms delay between each fill operation in the loop which fills the empty Rects. In the demo links below I fill each Rect as soon as it enters the screen and “unfill” it, i.e. release the texture memory as soon as it leaves the screen.

With BitmapPaint fill activated:
https://drive.google.com/open?id=1_HzU2a-mybEP_SCFP7NxoliauRcn_TxM

With BitmapPaint fill disabled:
https://drive.google.com/open?id=1w40A74DVvkFq7KOl39bSZ4fM52o6N4tO

The only difference in the code base is:

spock:Corona jocke$ svn diff widgets/attachmentslistwidget.lua =================================================================== --- widgets/attachmentslistwidget.lua&nbsp;&nbsp; &nbsp;(revision 870) +++ widgets/attachmentslistwidget.lua&nbsp;&nbsp; &nbsp;(working copy) @@ -201,12 +202,15 @@ &nbsp;&nbsp;&nbsp;&nbsp; elseif options.filename ~= nil and options.baseDir ~= nil then &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; removeProgress() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if options.delay == nil then +--[[&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; frame.fill = { &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; type = "image", &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; filename = options.filename, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDir &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } +--]] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; else +--[[&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; timer.performWithDelay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; options.delay, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; function() @@ -216,6 +220,7 @@ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; baseDir = options.baseDir &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end) +--]]

I also check the memory usage with the following snippet:

local function checkMem() &nbsp; collectgarbage("collect") &nbsp; local memoryUsed = collectgarbage("count") &nbsp; local maxTextureSize = system.getInfo("maxTextureSize") &nbsp; local textureMemoryUsed = system.getInfo("textureMemoryUsed") / 1048576 &nbsp; print("====") &nbsp; print("System Memory:", string.format("%.00f", memoryUsed) .. " kB") &nbsp; print("Max Texture Size:", maxTextureSize) &nbsp; print("Texture Memory:", string.format("%.03f", textureMemoryUsed) .. " MB") end timer.performWithDelay(4000, checkMem, -1)

And with BitmapPaint fill activated it gave me:

==== System Memory: 2308 kB Max Texture Size: 16384 Texture Memory:&nbsp;&nbsp; &nbsp;60.225 MB

The images I use are these (sorry for the theme, got bored):

-rw-r--r--@ 1 jocke&nbsp; staff&nbsp;&nbsp; 61661 27 Apr&nbsp; 2017 lenin.jpeg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 284992 27 Apr&nbsp; 2017 lenin.png -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 519158 27 Apr&nbsp; 2017 lenin2.jpeg lrwxr-xr-x&nbsp; 1 jocke&nbsp; staff&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 34 23 Maj&nbsp; 2017 snowden.jpg -\> ../../../../../docroot/snowden.jpg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 732648 24 Apr&nbsp; 2017 sovjet.jpg -rw-r--r--@ 1 jocke&nbsp; staff&nbsp; 822548 21 Apr&nbsp; 2017 spiderman.png spock:images jocke$ ls -l ../../../../../docroot/snowden.jpg -rw-r--r--&nbsp; 1 jocke&nbsp; staff&nbsp; 28404 13 Feb&nbsp; 2017 ../../../../../docroot/snowden.jpg

In summary: The filling operations seem to choke the main thread.

I might add that I do not use https://docs.coronalabs.com/api/library/widget/newScrollView.html but instead use my own scroll module in order to be able fine tune the look and feel down to the last pixel/transition/timing. It works nicely but maybe this is the culprit here? Maybe Corona’s ScrollView does magic stuff to offload the main thread, i.e. using posix threads or whatnot? I do nothing magic in my scroll implementation: https://drive.google.com/open?id=1i08xdV79RI8k90EOoURKoSYRFOUl51GM

Can you see something broken in all of this? Sigh.

Cheers
/Joakim

joagre · April 3, 2018, 2:23am

Hmm, I Guess I have to live with this. Filling will take its time.