[RESOLVED] Why snapshot:append() and not snapshot.clearAtRender?

[rakoonic2]

From the latest daily build docs:

‘Also, adds snapshot:append( object ) to enable canvas-style drawing to the snapshot texture. This is useful to draw onto the snapshot without clearing the contents of the texture or the need to call snapshot:invalidate().’

Can I ask what is the logic behind adding this functionality like this, and breaking one of the standard concepts of snapshots (or 2 rather - adding children to the .group property like any other sort of container, and using :invalidate() as the means to trigger the redraw), as opposed to simply not clearing the colour and alpha channels when a redraw happens (which I proposed as a property of a snapshot like this:

snapshot.clearAtRender = true / false

which could also be combined with:

snapshot.clearColor = { r, g, b, a }

for more flexibility, without complicating an already complicated feature?

I have to say, I’m very surprised and more than a little confused how this (append) was deemed to be the solution to the ‘paint package’ problem.

[ingemar]

+1

[walter]

Well, there’s a reason we didn’t document this yet

As for clearAtRender, we investigated that, but if you dig deeper, you run into several problems:

1. When you draw on top of the texture (no clear), you need to know which objects to draw. Internally, we track these in a separate list, which is what snapshot:append() goes to. They later get merged into the snapshot’s group once they’ve been rendered (this merge is to resolve problem #2).

2. When a GL context changes, you need to be able to redraw everything correctly, so eventually all objects have to be stored somewhere. It also lets you easily create undo/redo.

An alternative might be to expose the internal list as a separate group-like property (open to suggestions on names?), but this could get confusing if this group had separate anchor, x,y, properties, so you’d have to disallow them.

[rakoonic2]

In answer to point 1, you’d simply draw whatever was in the snapshot group.

If you are doing a paint program there is really no problem with the programmer clearing the group each frame and adding only what he wants to append - to my mind it is certainly the logical way of doing things.

Point 2 - aaaah ok now I get you This does raise a further question though - does this mean that snapshots will always retain a full draw list of all objects that have been passed to them? IE I draw a few 1000 lines using append() - they are all in memory?

I don’t think the redo/undo is useful at all really - if a programmer wants that functionality in his program then he can easily program it in (eg instead of updating the snapshot with each line, he maintains the last 100 or so as live objects so he can undo/redo between them at will).

But you have certainly made me think.

The only ‘solution’ that would work with my way, and it may be do-able but you’ve probably already looked into it, would be upon loss of the canvas, to physically write the snapshot contents to the device to let you restore them automatically upon resume or whatever (for when apps go to sleep etc). I know this could eat up memory very quickly (don’t wanna think how big an uncompressed 2048x2048 image would be) but could it be a viable alternative?

PS We need the ability to save and load snapshot images too - that way if needs be I could write my own version of the above, but obviously it is just a generally useful feature to have and I’m sure it is on the cards. I only mention it here as it might present a work-around for the above issues.

[walter]

In answer to point 1, you’d simply draw whatever was in the snapshot group.

If you are doing a paint program there is really no problem with the programmer clearing the group each frame and adding only what he wants to append - to my mind it is certainly the logical way of doing things.

That still suffers from the problem. It’s a subtle point. You have to consider the base case, and then the case where you draw on top of the existing texture.

Consider the base case where there is one translucent rectangle that’s already drawn in the existing texture. Then you want to draw a circle on top. You want to draw the circle on top of the existing texture by itself. If you draw what was in the snapshot group, you have to render the rectangle and then the circle, which is not what you want.

My point is having a property that specifies the mode doesn’t help you. You still need some separate list to keep track of the “deltas”, which then get drawn on top of the existing texture on the next render pass.

As for saving an image. That’s not viable. glReadPixels is too slow. Tracking the objects is way faster, and solves a bunch of the other issues around #2. And as long as you avoid calling invalidate(), you are effectively in a non-clear/accumulation mode.

So that’s why we did snapshot:append(). I’m open to changing this to track the deltas in a secondary group property (snapshot.canvasGroup or snapshot.accumulationGroup or snapshot.overlayGroup?), but it would have to be subject to limitations so that the secondary group transform/anchor matches the properties of the normal snapshot group.

[walter]

I do see a use case where you want to draw on top of the existing texture, but when you call invalidate(), you want the original texture to be restored, so subsequent accumulations are “forgotten”. Effectively, you want to temporarily draw stuff on top but then clear that. So we can add a property for that.

In contrast, in the current implementation, if you call invalidate(), it redraws everything, the original base texture and subsequent objects you draw on top.

[rakoonic2]

I think there’s a misunderstanding here. If all you do when you update a snapshot via :invalidate() is draw into the area without clearing it first, then you’ll be painting on top which is what most people want from this feature.

In your last post you say:

but when you call invalidate(), you want the original texture to be restored, so subsequent accumulations are “forgotten”.

This is not what people want when thinking of a paint package though.

The way I see it there are only really 2 direct ways of using snapshots - it either clears the canvas and draws everything in its group, or it doesn’t clear the canvas and draws everything in its group on top of the previous snapshot state - hence why the append thing stumps me a little (although your technical points are of course entirely valid).

Since I’ve already got a means of making snapshots accumulate how I want (via the double buffering) I’m not unduly worried, but I *am* concerned that what you say in a few places means we will never get a way of saving a snapshot to a file (although I’m hoping I misunderstood and you were referring to saving / loading when the app context changes). I do accept that as it stands that I am not going to be able to handle a proper art program with an infinite number of lines easily (although I would in this case refer back to my idea of my code keeping the last X 100 elements live or whatever, and whenever you update the snapshot I could save it then, as some sort of draft format for example).

If a general save of a snapshot image takes a few seconds, I think most of us could live with it in that context as this wouldn’t be a real-time action - IE a function we can call to do that - I am not referring to saving and loading now when you swap apps or something People are going to be screaming for this function, whether it is slow or not, as it lets you make all kinds of photo editing apps and what-not.

Also, I’d be very curious regarding your thoughts on what I said about making snapshots image factories as opposed to an actual image type thing themselves - look at the third post here: http://forums.coronalabs.com/topic/40590-snapshot-as-paint-input/

I mention it now because I think if it is something you’d consider doing, it needs to be planned out ASAP - it wouldn’t change much from our point of view but I imagine it might involve some serious changes under the hood. Which is never a strong selling point I admit, but I think awesome though snapshots are, they will ultimately fall well short of their potential if you can’t clone or re-use the image multiple times in real-time.

[walter]

I think we’re largely on the same page, but we’re talking about several use cases simultaneously, so it’s getting hard to track. Let me send you an e-mail so we can get on same page about all the use cases, as it’s gone beyond the “why is it called snapshot:append()?”

[rakoonic2]

You should probably mark this thread as resolved, because although you are still hammering out the specifics of this feature, I certainly have the answer to this thread’s title’s question

[walter]

FYI. We’ve changed append() so that you interact with a ‘canvas’ property.

I’ve posted some code here to show a cool effect of a trailing particle/brush effect:

http://forums.coronalabs.com/topic/40930-redrawing-snapshots-anyway-to-not-clear-them-each-frame-or-to-partially-clear-with-alpha/?p=212929

[ingemar]

+1

[walter]

Well, there’s a reason we didn’t document this yet

As for clearAtRender, we investigated that, but if you dig deeper, you run into several problems:

1. When you draw on top of the texture (no clear), you need to know which objects to draw. Internally, we track these in a separate list, which is what snapshot:append() goes to. They later get merged into the snapshot’s group once they’ve been rendered (this merge is to resolve problem #2).

2. When a GL context changes, you need to be able to redraw everything correctly, so eventually all objects have to be stored somewhere. It also lets you easily create undo/redo.

An alternative might be to expose the internal list as a separate group-like property (open to suggestions on names?), but this could get confusing if this group had separate anchor, x,y, properties, so you’d have to disallow them.

[rakoonic2]

In answer to point 1, you’d simply draw whatever was in the snapshot group.

If you are doing a paint program there is really no problem with the programmer clearing the group each frame and adding only what he wants to append - to my mind it is certainly the logical way of doing things.

Point 2 - aaaah ok now I get you This does raise a further question though - does this mean that snapshots will always retain a full draw list of all objects that have been passed to them? IE I draw a few 1000 lines using append() - they are all in memory?

I don’t think the redo/undo is useful at all really - if a programmer wants that functionality in his program then he can easily program it in (eg instead of updating the snapshot with each line, he maintains the last 100 or so as live objects so he can undo/redo between them at will).

But you have certainly made me think.

The only ‘solution’ that would work with my way, and it may be do-able but you’ve probably already looked into it, would be upon loss of the canvas, to physically write the snapshot contents to the device to let you restore them automatically upon resume or whatever (for when apps go to sleep etc). I know this could eat up memory very quickly (don’t wanna think how big an uncompressed 2048x2048 image would be) but could it be a viable alternative?

PS We need the ability to save and load snapshot images too - that way if needs be I could write my own version of the above, but obviously it is just a generally useful feature to have and I’m sure it is on the cards. I only mention it here as it might present a work-around for the above issues.

[walter]

In answer to point 1, you’d simply draw whatever was in the snapshot group.

If you are doing a paint program there is really no problem with the programmer clearing the group each frame and adding only what he wants to append - to my mind it is certainly the logical way of doing things.

That still suffers from the problem. It’s a subtle point. You have to consider the base case, and then the case where you draw on top of the existing texture.

Consider the base case where there is one translucent rectangle that’s already drawn in the existing texture. Then you want to draw a circle on top. You want to draw the circle on top of the existing texture by itself. If you draw what was in the snapshot group, you have to render the rectangle and then the circle, which is not what you want.

My point is having a property that specifies the mode doesn’t help you. You still need some separate list to keep track of the “deltas”, which then get drawn on top of the existing texture on the next render pass.

As for saving an image. That’s not viable. glReadPixels is too slow. Tracking the objects is way faster, and solves a bunch of the other issues around #2. And as long as you avoid calling invalidate(), you are effectively in a non-clear/accumulation mode.

So that’s why we did snapshot:append(). I’m open to changing this to track the deltas in a secondary group property (snapshot.canvasGroup or snapshot.accumulationGroup or snapshot.overlayGroup?), but it would have to be subject to limitations so that the secondary group transform/anchor matches the properties of the normal snapshot group.

[walter]

I do see a use case where you want to draw on top of the existing texture, but when you call invalidate(), you want the original texture to be restored, so subsequent accumulations are “forgotten”. Effectively, you want to temporarily draw stuff on top but then clear that. So we can add a property for that.

In contrast, in the current implementation, if you call invalidate(), it redraws everything, the original base texture and subsequent objects you draw on top.

[rakoonic2]

I think there’s a misunderstanding here. If all you do when you update a snapshot via :invalidate() is draw into the area without clearing it first, then you’ll be painting on top which is what most people want from this feature.

In your last post you say:

but when you call invalidate(), you want the original texture to be restored, so subsequent accumulations are “forgotten”.

This is not what people want when thinking of a paint package though.

The way I see it there are only really 2 direct ways of using snapshots - it either clears the canvas and draws everything in its group, or it doesn’t clear the canvas and draws everything in its group on top of the previous snapshot state - hence why the append thing stumps me a little (although your technical points are of course entirely valid).

Since I’ve already got a means of making snapshots accumulate how I want (via the double buffering) I’m not unduly worried, but I *am* concerned that what you say in a few places means we will never get a way of saving a snapshot to a file (although I’m hoping I misunderstood and you were referring to saving / loading when the app context changes). I do accept that as it stands that I am not going to be able to handle a proper art program with an infinite number of lines easily (although I would in this case refer back to my idea of my code keeping the last X 100 elements live or whatever, and whenever you update the snapshot I could save it then, as some sort of draft format for example).

If a general save of a snapshot image takes a few seconds, I think most of us could live with it in that context as this wouldn’t be a real-time action - IE a function we can call to do that - I am not referring to saving and loading now when you swap apps or something People are going to be screaming for this function, whether it is slow or not, as it lets you make all kinds of photo editing apps and what-not.

Also, I’d be very curious regarding your thoughts on what I said about making snapshots image factories as opposed to an actual image type thing themselves - look at the third post here: http://forums.coronalabs.com/topic/40590-snapshot-as-paint-input/

I mention it now because I think if it is something you’d consider doing, it needs to be planned out ASAP - it wouldn’t change much from our point of view but I imagine it might involve some serious changes under the hood. Which is never a strong selling point I admit, but I think awesome though snapshots are, they will ultimately fall well short of their potential if you can’t clone or re-use the image multiple times in real-time.

[walter]

I think we’re largely on the same page, but we’re talking about several use cases simultaneously, so it’s getting hard to track. Let me send you an e-mail so we can get on same page about all the use cases, as it’s gone beyond the “why is it called snapshot:append()?”

[rakoonic2]

You should probably mark this thread as resolved, because although you are still hammering out the specifics of this feature, I certainly have the answer to this thread’s title’s question

[walter]

FYI. We’ve changed append() so that you interact with a ‘canvas’ property.

I’ve posted some code here to show a cool effect of a trailing particle/brush effect:

http://forums.coronalabs.com/topic/40930-redrawing-snapshots-anyway-to-not-clear-them-each-frame-or-to-partially-clear-with-alpha/?p=212929