snapshot as paint input?

Solar2D Advanced Graphics
#41

Hey guys! I really appreciated your feedback on TextureResourceCanvas. I have small request - I added several features to it, can you provide your feedback for them? They aren’t documented yet, because I’m not sure if they’re final. So, give it a try:

  1. basically, new feature is anchorX and anchorY properties of TextureResourceCanvas. With them you can change origin of Canvas’ viewport / pan

  2. width and height for Canvases (only) are now modifiable. It can be used to change size of Canvas’ viewport / zoom.

I attached updated Canvas Demo example. I would be very grateful for your feedback.

#42

Hi, I was also trying to know if possible to convert a snapshot into a sprite. There is any update in this subject? a year later? 

Thanks.

#43

@vlads, I tried two different approaches to this but without success. Perhaps I am misunderstanding something that you can help me with. I am using Build 2866.

I took the existing Flashlight demo that comes in the Samples folder of Corona SDK and made the following tweaks:

I added this texture:

local tex = graphics.newTexture( { type="canvas", width=200, height=200 } ) local blackRect = display.newRect(0,0,200,200) blackRect:setFillColor(0) tex:draw( blackRect ) -- Create a circle and draw/render it to the texture local glow = display.newCircle( 0, 0, 90 ) glow.fill.effect = "generator.radialGradient" glow.fill.effect.color1 = { 1, 1, 1, 1 } glow.fill.effect.color2 = { 0, 0, 0, 0 } glow.fill.effect.center\_and\_radiuses = { 0.5, 0.5, 0.3, 0.9 } glow.fill.effect.aspectRatio = 1 tex:draw( glow ) -- Schedule texture objects to be rendered to texture before next frame tex:invalidate()

Then I tried each of the following methods in an attempt to set the mask from the texture. (Besides this, I also tried to create a Mask directly from the canvas file but that blows up as previously indicated StarCrunch.)

local mask = display.newRect(0, 0, 200, 200) mask.fill = { type="image", filename=tex.filename, baseDir = tex.baseDir} image:setMask( mask )

local mask = display.newImageRect(tex.filename, tex.baseDir, 200, 200) image:setMask( mask )

In both these cases, the “mask” is simply displayed at the top left corner of the game.

Below is my full main.lua. The resources from the Flashlight sample app are required to make it run.

-- Abstract: Flashlight sample project -- Demonstrates masking an image. The flashlight skews depending -- on location from center. -- -- Version: 1.0 (September 1, 2010) -- -- Sample code is MIT licensed, see https://www.coronalabs.com/links/code/license -- Copyright (C) 2010 Corona Labs Inc. All Rights Reserved. -- -- Supports Graphics 2.0 ------------------------------------------------------------ display.setStatusBar( display.HiddenStatusBar ) local centerX = display.contentCenterX local centerY = display.contentCenterY -- Black background local bkg = display.newRect( centerX, centerY, 768, 1024 ) bkg:setFillColor( 0 ) -- Image to be masked local image = display.newImageRect( "image.png", 768, 1024 ) image:translate( centerX, centerY ) --Texture for mask?? local tex = graphics.newTexture( { type="canvas", width=200, height=200 } ) local blackRect = display.newRect(0,0,200,200) blackRect:setFillColor(0) tex:draw( blackRect ) -- Create a circle and draw/render it to the texture local glow = display.newCircle( 0, 0, 90 ) glow.fill.effect = "generator.radialGradient" glow.fill.effect.color1 = { 1, 1, 1, 1 } glow.fill.effect.color2 = { 0, 0, 0, 0 } glow.fill.effect.center\_and\_radiuses = { 0.5, 0.5, 0.3, 0.9 } glow.fill.effect.aspectRatio = 1 tex:draw( glow ) -- Schedule texture objects to be rendered to texture before next frame tex:invalidate() -- Masks local mask = graphics.newMask( "circlemask.png" ) --image:setMask( mask ) local mask2 = display.newRect(0, 0, 200, 200) mask2.fill = { type="image", filename=tex.filename, baseDir = tex.baseDir} image:setMask( mask2 ) -- Create display object with texture as contents local mask3 = display.newImageRect(tex.filename, tex.baseDir, 200, 200) mask3.x = display.contentCenterX mask3.y = display.contentCenterY --image:setMask( mask3 ) local radiusMax = math.sqrt( centerX\*centerX + centerY\*centerY ) function onTouch( event ) local t = event.target local phase = event.phase if "began" == phase then display.getCurrentStage():setFocus( t ) -- Spurious events can be sent to the target, e.g. the user presses -- elsewhere on the screen and then moves the finger over the target. -- To prevent this, we add this flag. Only when it's true will "move" -- events be sent to the target. t.isFocus = true -- Store initial position t.x0 = event.x - t.maskX t.y0 = event.y - t.maskY elseif t.isFocus then if "moved" == phase then -- Make object move (we subtract t.x0,t.y0 so that moves are -- relative to initial grab point, rather than object "snapping"). local maskX = event.x - t.x0 local maskY = event.y - t.y0 t.maskX = maskX t.maskY = maskY -- Stretch the flashlight as it moves further away -- from the screen's center local radius = math.sqrt( maskX\*maskX + maskY\*maskY ) local scaleDelta = radius/radiusMax t.maskScaleX = 1 + scaleDelta t.maskScaleY = 1 + 0.2 \* scaleDelta -- Rotate it appropriately about screen center local rotation = math.deg( math.atan2( maskY, maskX ) ) t.maskRotation = rotation elseif "ended" == phase or "cancelled" == phase then display.getCurrentStage():setFocus( nil ) t.isFocus = false end end return true end image:addEventListener( "touch", onTouch ) -- By default, the mask will limit the touch region to areas that lie inside -- both the mask and the image being masked. We can override this by setting the -- isHitTestMasked property to false, so the touch region lies inside the entire image. image.isHitTestMasked = false -- Display instructions local labelFont = native.systemFont local myLabel = display.newText( "Touch circle to move flashlight", centerX, 200, labelFont, 34 ) myLabel:setFillColor( 1, 1, 1, 180/255 )
#44

I’m not sure what are you trying to do…

Setting canvases as masks wouldn’t work for now.

#45

@vlads I am trying to dynamically create a mask by using the texture canvas. My sample code as-is does not work as I want. (I want the flashlight effect.) If you comment out “image:setMask( mask2 )” and uncomment “image:setMask( mask )”, the flashlight works just like in the original flashlight demo since it using the png. What I want is for the canvas texture to be used (“mask2” or “mask3”). Your prior comment says “But you can set mask to objects using it as a fill without a problem.” but I must not be doing that correctly.

#46

As I wrote, you can not use canvases as mask images for now.

#47

Bummer. Okay. So what did you mean by “But you can set mask to objects using it as a fill without a problem.”?

#48

you can not use canvas in graphics.newMask(filename=canvas.filename, baseDir=canvas.baseDir). This wouldn’t work

#49

Hey Vlads, I just wanted to express my gratitude for these new features! Being able to draw to a texture is incredible! I had been using snapshots to develop a game that uses entirely procedurally generated graphics. I was having performance problems though so I had to implement object caching. Once this new feature came out, I converted my code to use textures and discovered that my objects were created so quickly that I no longer needed object caching! Texture canvases rock!

#50

Any news on this?

#51

What are you looking for specifically?

Rob

#52

What Walter said above.

I need to be able to create a snap shot and then use that snapshot as a fill or in a composite filter as the 2nd paint channel.

The only way to do that now I believe would be to save and load the image each frame which would kill the framerate.

#53

Hello, everyone!
So here is a new feature. Give it a spin, we would like to get your feedback on it.
Half a year ago we introduced texture apis. Building on that API here is modifiable texture. It resembles Snapshot object, but it is quite different, even if APIs may seem similar.
 
New feature is TextureResourceCanvas type. It is in-memory texture, you can render stuff into it, and than assign it to other objects. Note that you can assign it to several different objects.
It is a subject to manual texture management, so beware of memory leaks. If you don’t use it anymore, you have to release it. After you release it, you’re loosing pointer Lua handle to it, but Display Objects using it would not become invalid.
 
I will put some code here to explain how it works.
 
Creating with newTexture:
[lua]
local canvasTexture = graphics.newTexture( {
     type = “canvas”
     , width = 128
     , height = 128
     , pixelWidth = 256
     , pixelHeight = 256
} )
[/lua]
Here width and height here is dimensions of canvas where you would draw your objects, basically dimensions of visible world in canvas.
Pixel width/height - size of underlying texture. If omitted, this values would be selected to correspond to pixel dimensions of display object with width/height. You can set it to like 32 for cool pixely effects (with proper scaling modes), or just some other number, for example to save memory etc.
 
After it is created you can assign it to the fill to other objects. Use “canvasTexture.filename” and “canvasTexture.baseDir” pretty much anywhere where those parameters are expected. For example, in composite paint or creating image rect:
[lua]
local circle = display.newCircle( display.contentCenterX, display.contentHeight*3/4, w/2 )
circle.fill = {
    type=“composite”,
    paint1={ type=“image”, filename=“corona.png” },
    paint2={ type=“image”, filename=canvasTexture.filename, baseDir=canvasTexture.baseDir } – magic here!
}
circle.fill.effect = “composite.phoenix”
 

local rect = display.newImageRect(
    canvasTexture.filename,  – “filename” property required
    canvasTexture.baseDir,   – “baseDir” property required
    display.contentWidth,
    display.contentHeight
)

[/lua]
 
To draw to canvas, you would use method canvasTexture:draw(). This would put your display objects off the screen and into internal queue. You have to manually update the texture with canvasTexture:invalidate().

This call would schedule to render objects in internal queue before next frame, and after rendering them it would move them to cache.
You can also set custom background colour. This colour would fill the texture when it is cleared.
[lua]
canvasTexture:draw( someCircle )
canvasTexture:draw( someRect )

canvasTexture:setBackground( 0,0,0,1 )

canvasTexture:invalidate()
[/lua]

Note that (0; 0) point is in the centre of the canvas.
 
Thing to keep in mind is that when app is put in background, Android will make all GPU textures invalid, so you would have to redraw your canvas resource. Read documentation on how to do it.
 
Reading documentation is good in any way. It describes more edge cases and methods:
 
newTexture documentation <-- read me before using
TextureResourceCanvas documentation <-- me too

Canvas texture resources has a lot of fun applications, but also some limitations. Like nested textures can work weirdly, or native objects would not work at all.

Also, there is no possibility to add setPixel/getPixel. Textures are stored on GPUs and do not provide direct buffer access.

 
Also, I rewrote snapshot paint example, to use canvas. It has fire in the middle to demonstrate manual updates. It is attached to this post. Also, some other random example with star in it and composite paint.

#54

Hi Vlads

Thanks for this. Great stuff, I will be testing it to destruction shortly.

Matt

#55

So far everything seems fine. The texture seems to be sized a little different to just using a normal image as I needed to alter some settings in the shaders to get everything to line up again.

I will dig deep into the code, see if anything jumps out as being different or if its just my code.

Anyway I’m processing 2 x 1024 canvases, then sending them to 2 different shaders, each a composite paint at the moment ( but that might not be needed now ).

After that I take a snapshot of the result for additional processing on screen.

So far, testing on device I’m not seeing any different as to when I’m just using 1024 sized images.

Works as far back as an iPod Touch 5, which has the chip below the A7, and I can get 60fps no problem using optimised settings in the shaders.

So, all in all, this is perfect for what I need. It really opens up Corona to a lot of new techniques.

#56

@ vlads Seems to blow up with masks.  :( Sprites seem okay, in my limited test. (Mixing it into the second example wasn’t the best way to test.  :stuck_out_tongue: )

Anyhow, I’ve got a ton of (image-based) use cases, so I’ll report if anything turns up.

#57

I should had mentioned. Yes, for technical reasons you can not use TextureResourceCanvas as a mask (at this point). But you can set mask to objects using it as a fill without a problem.

#58

Picking some low-hanging fruit from among my ideas, here’s a thing to paint “time” into a texture:

-- UV map thing -- -- Permission is hereby granted, free of charge, to any person obtaining -- a copy of this software and associated documentation files (the -- "Software"), to deal in the Software without restriction, including -- without limitation the rights to use, copy, modify, merge, publish, -- distribute, sublicense, and/or sell copies of the Software, and to -- permit persons to whom the Software is furnished to do so, subject to -- the following conditions: -- -- The above copyright notice and this permission notice shall be -- included in all copies or substantial portions of the Software. -- -- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, -- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -- IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY -- CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, -- TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE -- SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -- -- [MIT license: http://www.opensource.org/licenses/mit-license.php] -- local CW, CH = display.contentWidth, display.contentHeight local tex = graphics.newTexture{ type = "canvas", width = CW, height = CH } tex:setBackground(0, 0, 0, 1) tex:invalidate() local img = display.newImageRect(tex.filename, tex.baseDir, CW, CH) img:translate( display.contentCenterX, display.contentCenterY ) local previousX, previousY local threshold = 2 local thresholdSq = threshold \* threshold local time = 0 local function draw (x, y) if time \< 256 then local o = display.newImage("brush.png", x, y) o:setFillColor(time / 256, 0, 0) tex:draw(o) tex:invalidate("cache") previousX, previousY, time = x, y, time + 1 end end local function listener (event) local x, y = event.x - img.x, event.y - img.y if event.phase == "began" then draw(x, y) elseif event.phase == "moved" then local dx = x - previousX local dy = y - previousY local deltaSq = dx^2 + dy^2 if deltaSq \> thresholdSq then draw(x, y) end end end Runtime:addEventListener("touch", listener) -- do local kernel = { category = "composite", group = "uv\_map", name = "basic" } kernel.vertexData = { { index = 0, name = "t", min = 0, max = 1, default = 0 } } kernel.fragment = [[P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { P\_COLOR float r = texture2D(CoronaSampler1, uv).r; if (CoronaVertexUserData.x \>= r) return vec4(0.); // check time vs. uv return CoronaColorScale(texture2D(CoronaSampler0, uv)); }]] graphics.defineEffect(kernel) end local start = display.newCircle(CW \* .6, CH \* .7, 55) start:setFillColor(1, 0, 0) start:addEventListener("touch", function(event) if event.phase == "ended" then local r = display.newRect(display.contentCenterX, display.contentCenterY, CW, CH) r.fill = { type = "composite", paint1 = { type = "image", filename = "Image1.jpg" }, -- your image here paint2 = { type = "image", filename = tex.filename, baseDir = tex.baseDir } } r.fill.effect = "composite.uv\_map.basic" img.isVisible, event.target.isVisible = false, false transition.to(r.fill.effect, { t = 1, time = 9000 }) -- can get reverse effect going 1 -\> 0, of course end return true end)

It takes an image of your choice and the brush from the CanvasPaint example (probably should have just used a circle, since the brush has its own alpha and you get a peculiar tapering fade).

When you run it, drag around on the screen to paint in time values, which gradually increase from 0 to 1, going into the red channel. Once it hits 1 it stops painting.

Click the red circle when you’re ready and the effect’s time threshold will transition from 0 to 1, unveiling pixels with the right time values. You can adjust the “// check time vs. uv” line to change the behavior.

This is a pretty boring proof of concept. The idea can get a lot fancier, though.

#59

@ vlads  I’m attempting something quite a bit more ambitious this time, which involves storing vertices in a texture.

I’ve tried to use vertex texture fetch. GLSL sources suggest this should “Just Work” provided the textures were already bound.

So far I’ve done something like this, in the fragment kernel:

CoronaSampler0 += 8.; // Shader fails to compile, but sampler name shows up in log

to suss out the sampler name. Then, after fixing that, in the vertex kernel:

uniform sampler2D u\_FillSampler0; P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { // ... stuff P\_POSITION vec2 new\_pos = texture2DLod(u\_FillSampler0, uv, 0.0).rg; return new\_pos; // what our work gave us }

This kind of thing compiles and runs, but I seem to be consistently getting “black”, suggesting the sampler is unbound.

Are there complications in Corona’s surrounding boilerplate, or am I just missing something obvious here?

(This is on Windows and I do have a way to detect that vertex texture units are available beforehand, although it’s pretty clumsy: basically, a dummy pixel with an auxiliary shader that branches on  gl_MaxVertexTextureImageUnits , followed by display.colorSample() and such.)

Also, might legitimate VTF support be a possibility? I think this would bring a lot of power in situations similar to my own.

#60

I don’t see where you can bound this sampler. We’re bounding textures to CoronaSampler0 (also to CoronaSampler0 if you use composite paint). Try using CoronaSampler0 instead of u_FillSampler0, and don’t declare it, it’s already declared & bound.

EDIT: I see it, it is in our sample. Well… basically CoronaSampler# is a synonym to u_FillSampler#. Use that instead and don’t declare it.