Texture Wrapping/Fills do not work with ImageSheets (Example Code attached)

Hello All,

Sorry if this is a repost, I couldn’t tell if the first when went through to moderation. If so, please delete this topic.

I am experiencing an issue using TextureWrapping to tile a frame from an ImageSheet where the frame is not actually wrapped, but instead the bitmap behind the ImageSheet is wrapped instead.

The attached file “screenshot.png” is an example of the problem. In the attached project, “Texture.png” is 64x128 where the top 64x64 section (frame 1) is red with a white “T” and the bottom 64x64 section (frame 2) is blue with a white “O” , the imagesheet is defined:

display.setDefault("textureWrapX", "repeat") display.setDefault("textureWrapY", "repeat") local fileName = "Texture.png" local imageSheetOptions = { frames = { { x = 0, y = 0, width = 64, height = 64 }, { x = 0, y = 64, width = 64, height = 64 }, } } local imageSheet = graphics.newImageSheet(fileName, imageSheetOptions)

The rectangle is defined:

local rect = display.newImageRect("Texture.png", 64, 128) -- alternatively: display.newRect(0, 0, 64, 128) rect.x = 200 rect.y = 200 rect.width = 64 rect.height = 128 rect.fill = { type = "image", sheet = imageSheet, frame = 1 } rect.fill.scaleY = 0.5

WIth this code I would expect to see two of the red squares, but instead I see the red and blue square.

Can someone point me where I’m going wrong with the above code?

Albert

Hello CoronaLabs Employees,

I submitted a bug report for this issue but never received an email back with the bug report ticket number.

I think this is a bug, but it would be nice to at least have a solid answer that this is simply to complicated for Corona so that I can develop with another engine.

Albert

For reference, the new bug report case # is 42343

Hi Albert,

Thanks for filing the report. Let me check into it and do a little testing on our side…

Take care,

Brent

@ apucciani

The texture wrapping triggers when you step outside the texture, not out of a frame, so the result is to be expected, unfortunately. I don’t know if there’s any real sane way to handle this in general, though I’d like to be wrong. I hadn’t even considered this case until your post.

I can think of a couple ways to do this with custom effects. One would be two-pass, where the first pass just copies the frame into a sanitized rect and then the second applies scaling / scrolling. I have a one-pass idea too, but it requires baking the frame info into the shader, so unclean and not very reusable.  :slight_smile:

I might give those a go later tonight. I already hacked on your example a bit and found what won’t work.  :D These are good corner cases to discover early!

(Image sheet-aware effects seem to require some mechanism for getting the uv values, in order to choose one as a reference and work in a “local” coordinate system. Not sure what the best such API would be, yet.)

@StarCrunch

Thanks for the reply. I’m basically figuring the same thing. My work around is to simply create many display objects but it’s not exactly ideal.

The only reason I assumed this feature would work are because of the comments/replies in the official blog post announcing this feature (https://coronalabs.com/blog/2013/11/07/tutorial-repeating-fills-in-graphics-2-0/) and the documentation to the ImageSheet API (https://docs.coronalabs.com/api/type/ImageSheetPaint/index.html) which makes explicit reference to the scaleX/scaleY/x/y properties, though I now know it’s because they are inherited. In my opinion these properties should throw an error if they are set, otherwise the results they provide are not the functionality that is expected.

One thing that I’ve noticed is that the CoronaSDK API does not seem to make a consistent attempt at error handling, a lot of cases that should throw an error (such as setting a fill with an invalid/misconfigured paint, or setting an ImageRect with an invalid image filename) do not throw an error, but simply silently fail and leave you with a nil object.

I digress, if the functionality cannot be provided because the UV coordinates would be hard to match to the underlying quad, I’ll file a feature request. From the performance perspective it would be better for the engine to handle allocating all of the vertices and offsetting the UV coordinates to meet the display object’s settings rather than having 20-30 display objects in the rendering pipeline, at least the vertices can all be passed to the GPU in a single call to reduce overhead.

Thanks again.

Hi.

Well, I played around with some stuff, and here’s what I’ve got so far:

display.setDefault("textureWrapX", "repeat") display.setDefault("textureWrapY", "repeat") local fileName = "Texture.png" local imageSheetOptions = { frames = { { x = 0, y = 0, width = 64, height = 64 }, { x = 0, y = 64, width = 64, height = 64 }, } } local imageSheet = graphics.newImageSheet(fileName, imageSheetOptions) local rect = display.newImageRect("Texture.png", 64, 128) rect.x = 200 rect.y = 200 rect.width = 64 rect.height = 128 rect.fill = { type = "image", sheet = imageSheet, frame = 1 } --rect.fill.scaleY = 0.5 local W, H = 64, 128 local frames = {} local nframes = #imageSheetOptions.frames for i, frame in ipairs(imageSheetOptions.frames) do local x, y, w, h = frame.x / W, frame.y / H, frame.width / W, frame.height / H frames[#frames + 1] = ("\n\t\t\tFrames[%i] = vec4(%f, %f, %f, %f);"):format(i - 1, x, y, w, h) end do local kernel = { category = "filter", name = "copy" } kernel.vertexData = { { name = "frame", default = 1, min = 1, max = nframes, index = 0 } } kernel.vertex = [[varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_UV vec2 pos) { P\_UV vec4 Frames]] .. ("[%i];"):format(nframes) .. [[]] .. table.concat(frames, "") .. [[P\_UV vec4 frame = Frames[int(CoronaVertexUserData.x) - 1]; uv\_rel = frame.xy + frame.zw \* CoronaTexCoord; return pos; } ]] kernel.fragment = [[varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, uv\_rel)); }]] graphics.defineEffect(kernel) end do local kernel = { category = "filter", name = "scale" } kernel.vertexData = { { name = "x", default = 0, min = -65535, max = 65535, index = 0 }, { name = "y", default = 0, min = -65535, max = 65535, index = 1 }, { name = "scaleX", default = 1, min = 0, max = 10, index = 2 }, { name = "scaleY", default = 1, min = 0, max = 10, index = 3 } } kernel.vertex = [[varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { uv\_rel = step(CoronaVertexUserData.xy, pos) / CoronaVertexUserData.zw; return pos; }]] kernel.fragment = [[varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, fract(uv\_rel))); }]] graphics.defineEffect(kernel) end do -- Kernel -- local kernel = { category = "filter", name = "copy\_scale" } kernel.graph = { nodes = { copy = { effect = "filter.custom.copy", input1 = "paint1" }, scale = { effect = "filter.custom.scale", input1 = "copy" }, }, output = "scale" } graphics.defineEffect(kernel) end rect.fill.effect = "filter.custom.copy\_scale" rect.fill.effect.scale.x = rect.x rect.fill.effect.scale.y = rect.y rect.fill.effect.scale.scaleY = .5 timer.performWithDelay(1500, function(event) rect.fill.effect.copy.frame = (event.count - 1) % 2 + 1 end, 0)

There are some filtering issues going on, but that’s standard image sheet stuff.

You can print out the kernel.vertex (and others, of course) if you want to see what the final code looks like.

The multi-pass behavior is strange… the first effect seems to get full-image texture coordinates, while the second uses image sheet-style ones. If not for this, hard-coding the texture coordinates wouldn’t have been necessary. It seems like a Corona bug, but maybe I’m overlooking something.

The two passes were only really to avoid that hard-coding, so they don’t really serve much purpose. I’ll see about tightening this up into one go.

Unfortunately, without some nicer way to grab the uv-rects (my intuition is leaning toward some API like graphics.getFrameRect(image_sheet, frame), that returns the raw [0-1] values in some form, as a table or four values, for instance), some of those calculations done early on seem to be necessary. Strictly speaking you don’t need to bake them into the shader like I did: just get them on the Lua side and then pass them in as four inputs, instead of the frame, and then the shader would even be reusable. (Harder to do that in one pass, though.)

Also, there might be a way to effect this more cleanly with snapshots, say with a shader like the scale one above, sans the need for relative texture coordinates. I gave that a try last night without any luck, but then it was already 2 AM and I probably wasn’t at my best.  :slight_smile:

Flattened into one pass:

display.setDefault("textureWrapX", "repeat") display.setDefault("textureWrapY", "repeat") local fileName = "Texture.png" local imageSheetOptions = { frames = { { x = 0, y = 0, width = 64, height = 64 }, { x = 0, y = 64, width = 64, height = 64 }, } } local imageSheet = graphics.newImageSheet(fileName, imageSheetOptions) local rect = display.newImageRect("Texture.png", 64, 128) rect.x = 200 rect.y = 200 rect.width = 64 rect.height = 128 rect.fill = { type = "image", sheet = imageSheet, frame = 1 } --rect.fill.scaleY = 0.5 local W, H = 64, 128 local frames = {} local nframes = #imageSheetOptions.frames for i, frame in ipairs(imageSheetOptions.frames) do local x, y, w, h = frame.x / W, frame.y / H, frame.width / W, frame.height / H frames[#frames + 1] = ("\n\t\t\tFrames[%i] = vec4(%f, %f, %f, %f);"):format(i - 1, x, y, w, h) end do local kernel = { category = "filter", name = "scale" } kernel.vertexData = { { name = "frame", default = 1, min = 1, max = nframes, index = 0 }, { name = "scaleX", default = 1, min = 0, max = 10, index = 1 }, { name = "scaleY", default = 1, min = 0, max = 10, index = 2 } } kernel.vertex = [[varying P\_UV vec4 frame; varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { P\_UV vec4 Frames]] .. ("[%i];"):format(nframes) .. [[]] .. table.concat(frames, "") .. [[frame = Frames[int(CoronaVertexUserData.x) - 1]; uv\_rel = (CoronaTexCoord - frame.xy) / (frame.zw \* CoronaVertexUserData.yz); return pos; } ]] kernel.fragment = [[varying P\_UV vec4 frame; varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, frame.xy + frame.zw \* fract(uv\_rel))); }]] graphics.defineEffect(kernel) end rect.fill.effect = "filter.custom.scale" rect.fill.effect.scaleY = .5 timer.performWithDelay(1500, function(event) rect.fill.effect.frame = (event.count - 1) % 2 + 1 end, 0)

Again, this must be a per-image sheet shader, unfortunately. I do have a way I could evaluate the frame rect in Lua and feed in those values instead, which would remove that requirement, but if this wasn’t already confusing, it certainly would be then!  :) Also, that still needs a lot of cross-platform testing. (If interested, let me know!)

Yes, as @starcrunch points out, texture repeat modes apply to the entire texture. You cannot repeat a frame of the image sheet (aka texture atlas) using OpenGL’s built-in texture repeat modes:

http://stackoverflow.com/questions/662107/how-to-use-gl-repeat-to-repeat-only-a-selection-of-a-texture-atlas-opengl

Also, FYI, with the shader approach, you’d have to find a way that avoids for-loops to get consistent performance results across a variety of devices.

@StarCrunch

Thanks for the Shader! I can’t wait to get this home tonight and pick this apart, you really put a lot of effort into this. I’m assuming you don’t work directly on Corona, so this is even better that you spent so much time to help me.

I’ll post back here later with my results.

Albert

Hello CoronaLabs Employees,

I submitted a bug report for this issue but never received an email back with the bug report ticket number.

I think this is a bug, but it would be nice to at least have a solid answer that this is simply to complicated for Corona so that I can develop with another engine.

Albert

For reference, the new bug report case # is 42343

Hi Albert,

Thanks for filing the report. Let me check into it and do a little testing on our side…

Take care,

Brent

@ apucciani

The texture wrapping triggers when you step outside the texture, not out of a frame, so the result is to be expected, unfortunately. I don’t know if there’s any real sane way to handle this in general, though I’d like to be wrong. I hadn’t even considered this case until your post.

I can think of a couple ways to do this with custom effects. One would be two-pass, where the first pass just copies the frame into a sanitized rect and then the second applies scaling / scrolling. I have a one-pass idea too, but it requires baking the frame info into the shader, so unclean and not very reusable.  :slight_smile:

I might give those a go later tonight. I already hacked on your example a bit and found what won’t work.  :D These are good corner cases to discover early!

(Image sheet-aware effects seem to require some mechanism for getting the uv values, in order to choose one as a reference and work in a “local” coordinate system. Not sure what the best such API would be, yet.)

@StarCrunch

Thanks for the reply. I’m basically figuring the same thing. My work around is to simply create many display objects but it’s not exactly ideal.

The only reason I assumed this feature would work are because of the comments/replies in the official blog post announcing this feature (https://coronalabs.com/blog/2013/11/07/tutorial-repeating-fills-in-graphics-2-0/) and the documentation to the ImageSheet API (https://docs.coronalabs.com/api/type/ImageSheetPaint/index.html) which makes explicit reference to the scaleX/scaleY/x/y properties, though I now know it’s because they are inherited. In my opinion these properties should throw an error if they are set, otherwise the results they provide are not the functionality that is expected.

One thing that I’ve noticed is that the CoronaSDK API does not seem to make a consistent attempt at error handling, a lot of cases that should throw an error (such as setting a fill with an invalid/misconfigured paint, or setting an ImageRect with an invalid image filename) do not throw an error, but simply silently fail and leave you with a nil object.

I digress, if the functionality cannot be provided because the UV coordinates would be hard to match to the underlying quad, I’ll file a feature request. From the performance perspective it would be better for the engine to handle allocating all of the vertices and offsetting the UV coordinates to meet the display object’s settings rather than having 20-30 display objects in the rendering pipeline, at least the vertices can all be passed to the GPU in a single call to reduce overhead.

Thanks again.

Hi.

Well, I played around with some stuff, and here’s what I’ve got so far:

display.setDefault("textureWrapX", "repeat") display.setDefault("textureWrapY", "repeat") local fileName = "Texture.png" local imageSheetOptions = { frames = { { x = 0, y = 0, width = 64, height = 64 }, { x = 0, y = 64, width = 64, height = 64 }, } } local imageSheet = graphics.newImageSheet(fileName, imageSheetOptions) local rect = display.newImageRect("Texture.png", 64, 128) rect.x = 200 rect.y = 200 rect.width = 64 rect.height = 128 rect.fill = { type = "image", sheet = imageSheet, frame = 1 } --rect.fill.scaleY = 0.5 local W, H = 64, 128 local frames = {} local nframes = #imageSheetOptions.frames for i, frame in ipairs(imageSheetOptions.frames) do local x, y, w, h = frame.x / W, frame.y / H, frame.width / W, frame.height / H frames[#frames + 1] = ("\n\t\t\tFrames[%i] = vec4(%f, %f, %f, %f);"):format(i - 1, x, y, w, h) end do local kernel = { category = "filter", name = "copy" } kernel.vertexData = { { name = "frame", default = 1, min = 1, max = nframes, index = 0 } } kernel.vertex = [[varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_UV vec2 pos) { P\_UV vec4 Frames]] .. ("[%i];"):format(nframes) .. [[]] .. table.concat(frames, "") .. [[P\_UV vec4 frame = Frames[int(CoronaVertexUserData.x) - 1]; uv\_rel = frame.xy + frame.zw \* CoronaTexCoord; return pos; } ]] kernel.fragment = [[varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, uv\_rel)); }]] graphics.defineEffect(kernel) end do local kernel = { category = "filter", name = "scale" } kernel.vertexData = { { name = "x", default = 0, min = -65535, max = 65535, index = 0 }, { name = "y", default = 0, min = -65535, max = 65535, index = 1 }, { name = "scaleX", default = 1, min = 0, max = 10, index = 2 }, { name = "scaleY", default = 1, min = 0, max = 10, index = 3 } } kernel.vertex = [[varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { uv\_rel = step(CoronaVertexUserData.xy, pos) / CoronaVertexUserData.zw; return pos; }]] kernel.fragment = [[varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, fract(uv\_rel))); }]] graphics.defineEffect(kernel) end do -- Kernel -- local kernel = { category = "filter", name = "copy\_scale" } kernel.graph = { nodes = { copy = { effect = "filter.custom.copy", input1 = "paint1" }, scale = { effect = "filter.custom.scale", input1 = "copy" }, }, output = "scale" } graphics.defineEffect(kernel) end rect.fill.effect = "filter.custom.copy\_scale" rect.fill.effect.scale.x = rect.x rect.fill.effect.scale.y = rect.y rect.fill.effect.scale.scaleY = .5 timer.performWithDelay(1500, function(event) rect.fill.effect.copy.frame = (event.count - 1) % 2 + 1 end, 0)

There are some filtering issues going on, but that’s standard image sheet stuff.

You can print out the kernel.vertex (and others, of course) if you want to see what the final code looks like.

The multi-pass behavior is strange… the first effect seems to get full-image texture coordinates, while the second uses image sheet-style ones. If not for this, hard-coding the texture coordinates wouldn’t have been necessary. It seems like a Corona bug, but maybe I’m overlooking something.

The two passes were only really to avoid that hard-coding, so they don’t really serve much purpose. I’ll see about tightening this up into one go.

Unfortunately, without some nicer way to grab the uv-rects (my intuition is leaning toward some API like graphics.getFrameRect(image_sheet, frame), that returns the raw [0-1] values in some form, as a table or four values, for instance), some of those calculations done early on seem to be necessary. Strictly speaking you don’t need to bake them into the shader like I did: just get them on the Lua side and then pass them in as four inputs, instead of the frame, and then the shader would even be reusable. (Harder to do that in one pass, though.)

Also, there might be a way to effect this more cleanly with snapshots, say with a shader like the scale one above, sans the need for relative texture coordinates. I gave that a try last night without any luck, but then it was already 2 AM and I probably wasn’t at my best.  :slight_smile:

Flattened into one pass:

display.setDefault("textureWrapX", "repeat") display.setDefault("textureWrapY", "repeat") local fileName = "Texture.png" local imageSheetOptions = { frames = { { x = 0, y = 0, width = 64, height = 64 }, { x = 0, y = 64, width = 64, height = 64 }, } } local imageSheet = graphics.newImageSheet(fileName, imageSheetOptions) local rect = display.newImageRect("Texture.png", 64, 128) rect.x = 200 rect.y = 200 rect.width = 64 rect.height = 128 rect.fill = { type = "image", sheet = imageSheet, frame = 1 } --rect.fill.scaleY = 0.5 local W, H = 64, 128 local frames = {} local nframes = #imageSheetOptions.frames for i, frame in ipairs(imageSheetOptions.frames) do local x, y, w, h = frame.x / W, frame.y / H, frame.width / W, frame.height / H frames[#frames + 1] = ("\n\t\t\tFrames[%i] = vec4(%f, %f, %f, %f);"):format(i - 1, x, y, w, h) end do local kernel = { category = "filter", name = "scale" } kernel.vertexData = { { name = "frame", default = 1, min = 1, max = nframes, index = 0 }, { name = "scaleX", default = 1, min = 0, max = 10, index = 1 }, { name = "scaleY", default = 1, min = 0, max = 10, index = 2 } } kernel.vertex = [[varying P\_UV vec4 frame; varying P\_UV vec2 uv\_rel; P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { P\_UV vec4 Frames]] .. ("[%i];"):format(nframes) .. [[]] .. table.concat(frames, "") .. [[frame = Frames[int(CoronaVertexUserData.x) - 1]; uv\_rel = (CoronaTexCoord - frame.xy) / (frame.zw \* CoronaVertexUserData.yz); return pos; } ]] kernel.fragment = [[varying P\_UV vec4 frame; varying P\_UV vec2 uv\_rel; P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { return CoronaColorScale(texture2D(CoronaSampler0, frame.xy + frame.zw \* fract(uv\_rel))); }]] graphics.defineEffect(kernel) end rect.fill.effect = "filter.custom.scale" rect.fill.effect.scaleY = .5 timer.performWithDelay(1500, function(event) rect.fill.effect.frame = (event.count - 1) % 2 + 1 end, 0)

Again, this must be a per-image sheet shader, unfortunately. I do have a way I could evaluate the frame rect in Lua and feed in those values instead, which would remove that requirement, but if this wasn’t already confusing, it certainly would be then!  :) Also, that still needs a lot of cross-platform testing. (If interested, let me know!)

Yes, as @starcrunch points out, texture repeat modes apply to the entire texture. You cannot repeat a frame of the image sheet (aka texture atlas) using OpenGL’s built-in texture repeat modes:

http://stackoverflow.com/questions/662107/how-to-use-gl-repeat-to-repeat-only-a-selection-of-a-texture-atlas-opengl

Also, FYI, with the shader approach, you’d have to find a way that avoids for-loops to get consistent performance results across a variety of devices.

@StarCrunch

Thanks for the Shader! I can’t wait to get this home tonight and pick this apart, you really put a lot of effort into this. I’m assuming you don’t work directly on Corona, so this is even better that you spent so much time to help me.

I’ll post back here later with my results.

Albert