Normal Map lighting effects with image sheets

Solar2D Advanced Graphics
#1

To follow up on another (now closed) topic normal-map-filters-help-needed, I have new observations on the matter:

In the below image, the top row has individually loaded images, the second row - frames from an image sheet (2 images, sheet size 512x256). It is clear that point light position in texture space (0 to 1) is applied to the whole sheet texture, not the used frame. As I read in other posts, it has to do with OpenGL design / limitation.

Is my assumption correct that it is not possible to use image sheets with normal maps for lighting effects and I should stop wasting time trying?

Second question is when I use directional light with the same settings as above, I get a weird “flip” effect, when the light seem to roll over the center of the image at the end of the loop.

However, when I comment out the following code (effectively disabling normalization of the coordinate values to 0–1 range from original -1.414 – 1.414 range), the light animation looks correct:

-- posX = Remap(posX, -1,1,0,1) -- posY = Remap(posY, -1,1,0,1)

This makes me think that this statement from the docsis false :

“The origin and the growth direction of each axis is the same as described for pointLightPos”

and, in fact, dirLightDirection origin is at the center of the image, as opposed to top left corner.

I’d appreciate if anyone can shed some light on this for me (no pun intended)

Here is the function I use to calculate light position:

 -- LIGHT TIMER FUNCTION -- calculate rotation local lightTimer = timer.performWithDelay( 100, function() -- inline function repeats 10x/sec for i = 1, #testPlates do local testPlate = testPlates[i] -- testPlate.rotation = testPlate.rotation + 4 -- local rot = math.mod(testPlate.rotation,360) local rot = system.getTimer() / 10 local rotrad = math.rad(rot) -- normalizaton local Remap = function(value, from1, to1, from2, to2) return (value - from1) / (to1 - from1) \* (to2 - from2) + from2; end -- how far is light from the center local origX = 1 local origY = 1 -- standard rotation formula local posX = origY \* math.sin(rotrad) + origX \* math.cos(rotrad) local posY = origY \* math.cos(rotrad) - origX \* math.sin(rotrad) print("posX:"..posX) -- posX = Remap(posX, -1,1,0,1) -- posY = Remap(posY, -1,1,0,1) testPlate.fill.effect.dirLightDirection = { posX, posY, .1} -- x, y, z values for the Normal Map ; applied to each object -- testPlate.fill.effect.pointLightPos = { posX, posY, .2 } -- x, y, z values for the Normal Map ; applied to each object end -- end for loop end, 0) -- end inline function and set repeat to infinite -- LIGHT TIMER FUNCTION END

The test project is attached as well.

#2

Hi @andrei18.

Thanks for the report. I’d love to get Engineering to investigate this, but they are working on critical, time sensitive issues at the moment. I think your best bet, since you have a sample project is to use the “Report a Bug” link at the top of the page. 

But before you do, could your Remap function be generating divide by 0 issues? I would think that would generate a runtime error, but it’s possible to disable them.

Rob

#3

@andrei18  Correct, image sheets cannot be used with composite effects.  There is an open request for that on the Corona feature request site, please add your vote

#4

@Rob: Interesting note on division by 0, I got that code from the web and didn’t really study it. Just ran some tests to simulate this - with from1 and to1 being equal , no errors - getting  -1.#INF as the result. But surely, there is no real life case where source range will be 0 length, right?

When you suggested to file a bug report, you meant the first or the second issue I described? the first being paint effect looking at the whole image sheet texture, the second - directional light coords.

@sporkfin: Thanks for providing a closure for me :frowning: . I did add my vote to the request.

#5

Probably won’t help, but I got around this issue by scaling a fill of a rect to show only the frame I want, instead of using the image as a sheet. The FX then apply just fine.

#6

I was thinking the bug report would be for the 1st issue since it needs some research. But it might not hurt to submit the 2nd one as well.

I tried tracing through you Remap function and I can’t see how it would generate a div by zero error, but that certainly feels like what’s going on.

Rob

#7

@horacebury  That’s a cool workaround.  I’ll have to give it try.  It would really cut down on my texture load!

#8

@andrei18 A direction doesn’t have an origin. It’s a vector, a difference of two points. As the docs say, the principal directions (horizontal, vertical, away/toward the screen) are the same as with points, which are basically the origin plus a direction.

What might be useful is a note such as “The natural way to calculate the direction is by subtracting the point {.5, .5, 0} from pointLightPos.”

On the GPU side of things you have texture coordinates typically in the 0-1 range, rather than pixels as with image sheets. These are provided to your shader, and it’s pretty easy to do a remap like your own and turn them back into vectors and so on.

With an image sheet, though, you suddenly can’t rely on this nice 0-1 zone, and somehow the effect needs to know what you actually have. The way I’ve achieved this in some of my own effects is by supplying the center and dimensions of the object in question (either actual position / size or, more robust but less flexible, the texture region) and remap from there. Unfortunately this costs you those shader inputs (assuming you have any to spare!) and the trouble that goes with it. I imagine concerns like this are why the various effects don’t support it.

Can you use snapshots or canvases?

#9

@StarCrunch: Wow, thanks for this in-depth explanation, I’ll have to take some time to digest it.

I am not sure I want to go down the snapshot or canvases route, since I can do away with a series of individual images.

Would using snapshots/canvases have advantage performance-wise over single images?

@Rob: with all this info, it doesn’t seem like a bug, besides, this has been brought up multiple times in the past on this forum and is mentioned in docs under limitations in compositePainit section, so I will hold off with a bug report submission.

On a side note, I was experimenting a bit and added filename parameter to fill options like:

type = "composite", paint1 = {type = "image", sheet = imageSheetColor, frame = 2, filename="texture.png"}, paint2 = {type = "image", sheet = imageSheetNormal, frame = 2} }

The fill effectively reads the dimensions from image specified in by filename parameter, but loads texture from image sheet. It didn’t help since it just scaled down my image sheet on x to fit, but I thought I’d share. Maybe someone can use this for an alternative scaling method or some other workaround.

#10

@andei18  I’m curious, what are the dimensions of the full image sheet and each frame?

#11

@sporkfin: frame 256x256 sheet 512x256 (2 frames - horizontal)

#12

Can you try running it again on a 512x512 sheet (4 frames)?

#13

Hi @andrei18.

Thanks for the report. I’d love to get Engineering to investigate this, but they are working on critical, time sensitive issues at the moment. I think your best bet, since you have a sample project is to use the “Report a Bug” link at the top of the page. 

But before you do, could your Remap function be generating divide by 0 issues? I would think that would generate a runtime error, but it’s possible to disable them.

Rob

#14

@andrei18  Correct, image sheets cannot be used with composite effects.  There is an open request for that on the Corona feature request site, please add your vote

#15

@Rob: Interesting note on division by 0, I got that code from the web and didn’t really study it. Just ran some tests to simulate this - with from1 and to1 being equal , no errors - getting  -1.#INF as the result. But surely, there is no real life case where source range will be 0 length, right?

When you suggested to file a bug report, you meant the first or the second issue I described? the first being paint effect looking at the whole image sheet texture, the second - directional light coords.

@sporkfin: Thanks for providing a closure for me :frowning: . I did add my vote to the request.

#16

Probably won’t help, but I got around this issue by scaling a fill of a rect to show only the frame I want, instead of using the image as a sheet. The FX then apply just fine.

#17

I was thinking the bug report would be for the 1st issue since it needs some research. But it might not hurt to submit the 2nd one as well.

I tried tracing through you Remap function and I can’t see how it would generate a div by zero error, but that certainly feels like what’s going on.

Rob

#18

@horacebury  That’s a cool workaround.  I’ll have to give it try.  It would really cut down on my texture load!

#19

@andrei18 A direction doesn’t have an origin. It’s a vector, a difference of two points. As the docs say, the principal directions (horizontal, vertical, away/toward the screen) are the same as with points, which are basically the origin plus a direction.

What might be useful is a note such as “The natural way to calculate the direction is by subtracting the point {.5, .5, 0} from pointLightPos.”

On the GPU side of things you have texture coordinates typically in the 0-1 range, rather than pixels as with image sheets. These are provided to your shader, and it’s pretty easy to do a remap like your own and turn them back into vectors and so on.

With an image sheet, though, you suddenly can’t rely on this nice 0-1 zone, and somehow the effect needs to know what you actually have. The way I’ve achieved this in some of my own effects is by supplying the center and dimensions of the object in question (either actual position / size or, more robust but less flexible, the texture region) and remap from there. Unfortunately this costs you those shader inputs (assuming you have any to spare!) and the trouble that goes with it. I imagine concerns like this are why the various effects don’t support it.

Can you use snapshots or canvases?

#20

@StarCrunch: Wow, thanks for this in-depth explanation, I’ll have to take some time to digest it.

I am not sure I want to go down the snapshot or canvases route, since I can do away with a series of individual images.

Would using snapshots/canvases have advantage performance-wise over single images?

@Rob: with all this info, it doesn’t seem like a bug, besides, this has been brought up multiple times in the past on this forum and is mentioned in docs under limitations in compositePainit section, so I will hold off with a bug report submission.

On a side note, I was experimenting a bit and added filename parameter to fill options like:

type = "composite", paint1 = {type = "image", sheet = imageSheetColor, frame = 2, filename="texture.png"}, paint2 = {type = "image", sheet = imageSheetNormal, frame = 2} }

The fill effectively reads the dimensions from image specified in by filename parameter, but loads texture from image sheet. It didn’t help since it just scaled down my image sheet on x to fit, but I thought I’d share. Maybe someone can use this for an alternative scaling method or some other workaround.