How to sample the color of a pixel of an image

[luantiang]

Hello all,

Is there a way to sample the color, in particular, the alpha, of a pixel of an image (PNG format), similar to,

display.colorSample( x, y, listener )?

The use case for this, is that most of my scene images (PNG) loaded via either “newImage” or “newImageRect” have many “empty” or transparent or alpha = 0, pixels. There are many overlapping smaller images (as touched players) these bigger scene images (PNG).

I do not want any “touch” or “tap” to be active on these bigger scene images, when the user touches on the “empty” or transparent part of the bigger scene images. So I would like to sample the “touch” point so to disable any “touch” action, when I detect the alpha of the pixel of the bigger scene image is 0.

My current work-around to bring the bigger scene image to “toBack()” has not worked well, as there could have multiple scene images at the same time, so the “touch” and/or “tap” has been quite confusing, when transparent part of image is touched, and the wrong image is action upon. Also, there could be many other smaller images could be behind, as they are loaded and invoked at later time.

I searched the internet, many image functions requires binding to some C libraries (like FFI and etc), but I found that Love2D has an interesting API,

r, g, b, a = ImageData:getPixel( x, y )

at https://love2d.org/wiki/ImageData:getPixel. This seems to do the work. Is there something similar for Corona?

Or is there a display object property similar to object.isHitTestable? that I could specify that only non-transparent pixels of a image (PNG) is hit testable?

Thank you.

Regards, Luan

[StarCrunch]

Hi Luan.

I have something like ImageData:getPixel() in my Bytemap plugin (docs). Basically usage would be along these lines:

local Bytemap = require("plugin.Bytemap") local bm = Bytemap.loadTexture{ filename = "MyImage.png", is\_non\_external = true } local w, h = bm.width, bm.height local alpha = bm:GetBytes{ format = "alpha" } -- alternatively can grab single pixel here local function GetAlpha (x, y) local index = (y - 1) \* w + x return alpha:byte(index) end local a = GetAlpha(2, 3) -- etc.

You’d want to generalize it a bit, of course.

The docs still list loadTexture() as WIP , but that’s just an oversight and I’ll try to fix it soon. Also, for Android note the AssetReader plugin also mentioned early on in the above docs.

[luantiang]

Hello StarCrunch,

Many thanks for your prompt response and help. I have done a quick try with “Bytemap”. Here are some quick findings. I am not sure if it is something I have mis-understood, or if they are errors. Below are tested using Corona Simulator on Windows 10.

(1) Empty string of length 0

format = “alpha”

format = “blue”

format = “green”

All three cases above always produce a empty string of length 0, in function “getBytes( opts )”, in both situation, where x1, y1, x2, y2 are specified, and where these are not specified. They produces same results even with “get_info = true”.

(2) Run Time Error

format = “red”, get_info = true,

format = “mask”, get_info = true,

format = “rgb”, get_info = true,

format = “rgba”, get_info = true,

All above produces the below error,

18:35:29.914  ERROR: Runtime error
18:35:29.914  attempt to concatenate a table value
18:35:29.914  stack traceback:
18:35:29.914      [C]: in function ‘GetBytes’
18:35:29.914      D:\Corona LUA\My Codes\codes collections v01\image bytemap v03\main.lua:31: in main chunk
 

(3) Good To Go

format = “mask”

format = “rgb”

format = “rgba”

format = “red”

All above four cases works well, without the “get_info = true”.

Question:

What would be more optimal and/or efficient approach, in terms of execution speed, texture memory and other consideration?

(a) do only once, invokes “local bm = Bytemap.loadTexture( config )” when I load an image using “newImage” or “newImageRect”, “bm” would stay throughout the duration, and there could be more as more images get loaded, incurring more and more texture memory, until the image is removed, or

(b) repeatedly invokes “local bm = Bytemap.loadTexture( config )”, in my “touch” event, once processing finished, set “bm = nil”,

would this save on texture memory? but it would incur more processing time in loading the texture, right?

Once again, many thanks for your help and guidance.

Regards, Luan

[StarCrunch]

Argh, I guess it’s still WIP after all.  :) I must have stepped on something when I added it and broken some other things. I’ll try to look into this soon.

As you probably saw in the docs, a bytemap can be in one of two modes: external, where it’s usable by display objects as a source of image data, like a canvas, and non-external, where it’s merely a data container. In the external case you’ll need to call Bytemap:releaseSelf() when you’re done with it.

In either case you will have data both in system RAM as well as on the GPU. This is sort of unavoidable since you want the data handy to test. But once you call GetBytes() you will have it in the form of a Lua string (and 1/4 the size, with only alpha), so you could indeed just throw away the bytemap itself. In external mode you might also try the Deallocate() method, although I haven’t given it as much attention as it needs; on Android you’d want to forgo this, though, as the data is needed to repopulate the image after a suspend.

I haven’t benchmarked speed relative to what newImage() and friends do, but suppose loading will be slightly slower. The purpose is slightly different, namely to manage the pixel bytes, so this wasn’t an immediate concern.

[StarCrunch]

Well, this was fairly easy to find and fix. The Windows binary should be updated soon. I’ll try to do the rest tonight.

EDIT : Just pushed the others too, plus updated the docs.

[luantiang]

Thanks for the advice. I would keep this in mind. Just re-run my quick test. All are good to go!

Many thanks for this plug-in.

Cheers!

Regards, Luan

PS> To de-allocate the texture, byteMap:releaseSelf() did the trick. byteMap:Deallocate() did not seem to effect it. This is tested with Simulator on Windows 10, using image on system.ResourcesDirectory as well as image on system.DocumentsDirectory.

[StarCrunch]

Hi Luan.

I have something like ImageData:getPixel() in my Bytemap plugin (docs). Basically usage would be along these lines:

local Bytemap = require("plugin.Bytemap") local bm = Bytemap.loadTexture{ filename = "MyImage.png", is\_non\_external = true } local w, h = bm.width, bm.height local alpha = bm:GetBytes{ format = "alpha" } -- alternatively can grab single pixel here local function GetAlpha (x, y) local index = (y - 1) \* w + x return alpha:byte(index) end local a = GetAlpha(2, 3) -- etc.

You’d want to generalize it a bit, of course.

The docs still list loadTexture() as WIP , but that’s just an oversight and I’ll try to fix it soon. Also, for Android note the AssetReader plugin also mentioned early on in the above docs.

[luantiang]

Hello StarCrunch,

Many thanks for your prompt response and help. I have done a quick try with “Bytemap”. Here are some quick findings. I am not sure if it is something I have mis-understood, or if they are errors. Below are tested using Corona Simulator on Windows 10.

(1) Empty string of length 0

format = “alpha”

format = “blue”

format = “green”

All three cases above always produce a empty string of length 0, in function “getBytes( opts )”, in both situation, where x1, y1, x2, y2 are specified, and where these are not specified. They produces same results even with “get_info = true”.

(2) Run Time Error

format = “red”, get_info = true,

format = “mask”, get_info = true,

format = “rgb”, get_info = true,

format = “rgba”, get_info = true,

All above produces the below error,

18:35:29.914  ERROR: Runtime error
18:35:29.914  attempt to concatenate a table value
18:35:29.914  stack traceback:
18:35:29.914      [C]: in function ‘GetBytes’
18:35:29.914      D:\Corona LUA\My Codes\codes collections v01\image bytemap v03\main.lua:31: in main chunk
 

(3) Good To Go

format = “mask”

format = “rgb”

format = “rgba”

format = “red”

All above four cases works well, without the “get_info = true”.

Question:

What would be more optimal and/or efficient approach, in terms of execution speed, texture memory and other consideration?

(a) do only once, invokes “local bm = Bytemap.loadTexture( config )” when I load an image using “newImage” or “newImageRect”, “bm” would stay throughout the duration, and there could be more as more images get loaded, incurring more and more texture memory, until the image is removed, or

(b) repeatedly invokes “local bm = Bytemap.loadTexture( config )”, in my “touch” event, once processing finished, set “bm = nil”,

would this save on texture memory? but it would incur more processing time in loading the texture, right?

Once again, many thanks for your help and guidance.

Regards, Luan

[StarCrunch]

Argh, I guess it’s still WIP after all.  :) I must have stepped on something when I added it and broken some other things. I’ll try to look into this soon.

As you probably saw in the docs, a bytemap can be in one of two modes: external, where it’s usable by display objects as a source of image data, like a canvas, and non-external, where it’s merely a data container. In the external case you’ll need to call Bytemap:releaseSelf() when you’re done with it.

In either case you will have data both in system RAM as well as on the GPU. This is sort of unavoidable since you want the data handy to test. But once you call GetBytes() you will have it in the form of a Lua string (and 1/4 the size, with only alpha), so you could indeed just throw away the bytemap itself. In external mode you might also try the Deallocate() method, although I haven’t given it as much attention as it needs; on Android you’d want to forgo this, though, as the data is needed to repopulate the image after a suspend.

I haven’t benchmarked speed relative to what newImage() and friends do, but suppose loading will be slightly slower. The purpose is slightly different, namely to manage the pixel bytes, so this wasn’t an immediate concern.

[StarCrunch]

Well, this was fairly easy to find and fix. The Windows binary should be updated soon. I’ll try to do the rest tonight.

EDIT : Just pushed the others too, plus updated the docs.

[luantiang]

Thanks for the advice. I would keep this in mind. Just re-run my quick test. All are good to go!

Many thanks for this plug-in.

Cheers!

Regards, Luan

PS> To de-allocate the texture, byteMap:releaseSelf() did the trick. byteMap:Deallocate() did not seem to effect it. This is tested with Simulator on Windows 10, using image on system.ResourcesDirectory as well as image on system.DocumentsDirectory.

[david.ciaudo]

I’ve played a bit with your plugin StarCrunch and it seems that GetBytes method returns incorrect values. For example for 255,0,0 (red pixel) it returns 254,0,0. It seems there a minus 1 somewhere in the rgb values. Also I did the test with a 1,1,1 pixel (almost full black) and it returns 0,0,0

local Bytemap = require("plugin.Bytemap")
local bm = Bytemap.loadTexture{filename = "gfx/tex.png", is_non_external = true}
local w, h = bm.width, bm.height
local data = bm:GetBytes{format = "rgb"}
for i=1,h do
    for j=1,w do
        print("line " .. i, "column " .. j, data:byte((i-1)*w*3+(j-1)*3+1), data:byte((i-1)*w*3+(j-1)*3+2), data:byte((i-1)*w*3+(j-1)*3+3))
    end
end

[StarCrunch]

Ah, nuts.

I think it’s the premultiplied alpha, in particular this line: https://github.com/ggcrunchy/solar2d-plugins/blob/master/Bytemap/shared/plugin.ByteMap.cpp#L594 (the shift is a division by 256, effectively, so ever-so-slightly off).

I’ll try to fix that in the not-too-distant future.

[david.ciaudo]

Ahah no problem, sorry to bring this topic to life ^^
I was just looking for a way to make a level editor using an image under solar2D.