Sanity checks regarding removeSelf() and lingering properties

Solar2D General Questions/Discussion
#1

Hi.

I’d just like to get clarification on a few points which I haven’t seen explicitly spelled out in the docs.

(1 - I’ve seen checks for object validity such as:

if object.removeSelf ~= nil then -- make sure display object hasn't been removed

Can that be relied upon?

I’d always been checking against parent , but that seems to have become less robust: snapshots’ groups, for instance, have no parent yet may still be alive and kicking. This hasn’t caused me any problems (yet?), but obviously isn’t quite right.[1]

(2 - Does display.remove () do any such check? The docs only mention the case where the argument is nil , so I’ve always hewed to that.

(3 - In the simulator at least, it seems that although the  removeSelf and parent properties are  nil  following a call like  removeSelf (), others such as  x and y  can still be looked up, not becoming inaccessible until a subsequent frame.

Are there (or could there be) any guarantees about this? If it does indeed work now–and I’m not just getting lucky–might it break down the road?[2]

In particular, I’m wondering about  “finalize” listeners, one of the more obvious places to grab an arbitrary object’s final state. Event dispatch seems to occur after the actual remove ( removeSelf and parent are already gone, anyway), so I’m concerned that querying the remaining properties might be hit-or-miss.

If it turns out there’s nothing to worry about, I could tidy up more than a few paranoid bits of code.  :slight_smile:

Thanks!

[1] - 

That said, it _was _fine for quite a long time (despite relying on something of an educated guess), until the then-new G2.0 features undermined certain assumptions. It’s not inconceivable that something similar could happen with  removeSelf , say if the method was outright removed from objects like the stage or the aforementioned snapshot groups, where its only real purpose would be to service the check in question.

Incidentally, I could see this being a common enough need to warrant a  display.isValid () predicate in Corona itself, even if it’s something dead simple–like the code above–under the hood. (The name comes courtesy of such a function available in  roaminggamer’s SSK.)

[2] -

If I’m not mistaken, the display object properties were indeed wiped immediately, not so very long ago. But I imagine the newer behavior is meant to improve SDK stability, which suggests it might be here to stay.

#2

Generally I do:   if displayObject and displayObject.removeSelf then  as my test.  I believe that display.remove() is just a wrapper that does that check.

Rob

#3

in my experience, it’s a bit like the “browser sniffing” debate in web development…

the presence (or absense) of .removeSelf doesn’t really/truly/fully/100% indicate that an object “is valid”.  rather, it indicates only that the object “is still valid enough” to still be present in the display (scene graph) and so a call to remove it from the scene graph is still considered valid. (even if it had already been removed previously during this same frame, and won’t be completely “gone” until next frame - the call itself is still “valid”)  fe, try this immediate mode loop:

for i = 1,100 do   if (obj and obj.removeSelf) then     obj.removeSelf()     print("removed it again, no problem") end end

everything that i’ve “witnessed” about Corona’s internals occurs within that frame loop processing.  that’s why you see so many “timer.performWithDelay(1,someCleanupRoutine)” -type workarounds for things that need to wait for something to “complete internal processing” before executing.

it’s undocumented, so no-one but Corona staff really knows, but imagine something like:

onCoronasInternalFrameTick:   do internal system stuff (like ticking box2d, memory management, etc, who knows/who cares?)   process system events   process device ui events   process timers   process user's frame ticks (registered enterFrame event listeners)   any internal final update of scene graph prior to rendering   .. etc

or something along those lines, who knows the exact order?  doesn’t really matter, and at any rate, if you accept any pattern along those lines, it’ll become apparent that if you “do something” in your frame loop (or in response to an event, or etc), it may not fully “clean up” until the next time the frame loop “does the internal stuff”.  the circumstantial evidence of various performWithDelay() hacks are compelling in this regard.

#4

@ Rob

I actually had some of your examples in mind, writing that up. I guess my first question boils down to: “If somebody from engineering is reading along, are they nodding their heads or frowning?”  :slight_smile:

Obviously it’s easy to wrap up the “seems to work” behavior, so it’s not so much a problem of fixing it should it break.[1] I’m just hoping to nail some stuff down. I share your belief about  display.remove (), but it wouldn’t hurt to be sure! Ditto on “finalize”.

@ davebollinger

Uff, the frame tick is another can of worms.[2][3] 

As you might have guessed, I’d like details on more than a few things. :D  Often just a “yes”, “no”, or “don’t rely on it”. It would involve getting engineers to divulge everything, though, so there are obvious practical hassles: time, frequent pestering, staying up-to-date, and so on. Oh well.

Footnotes ahoy!

[1] - I forgot about it before, but I have been tripped up by changing internals. Transitions used to expose enough state, discoverable through  pairs () and  print () and friends, so that you could figure out when a transition had finished. No longer! That broke some things.  :stuck_out_tongue:

[2] - Occasionally in my work I need to consult this, for instance to render stuff off-screen once all of an object’s bones are in position, then use the result. It tends to be a bit painful, but still feasible. Unfortunately, as you pile on more and more fanciness, it gets tangled really fast!

[3] - At some point, I was trying to write a few things using an approach like

if not HasThisBeenCalledYetThisFrame() then DoStuff() -- possibly expensive, or just needs to be idempotent in the short run end

to lazily perform once-per-frame-at-most calculations. This requires a little bookkeeping, such as a frame ID. An enterFrame listener would seem to be the natural place to update it.

If we guess right about how a frame unfolds, say that transitions and timers are processed before  enterFrame listeners, our frame comparison would work flawlessly when called from either of the former, but only sometimes from the latter. It all depends when the ID-updating listener gets called, relative to other enterFrame listeners that care about the ID. Some of them could be seeing the future! (And it does not work.)

(If addEventListener () just appends to an array or linked list, and if  removeEventListener () leaves the remaining elements in order, and if the listeners are called in the order they were added, I suppose Runtime:addEventListener () could be monkey-patched to always move a privileged “leaveFrame” listener to the end of the line. Quite a lot of unknowns, though.)

Thinking out loud here… if I’m not mistaken, the feature request page didn’t exist when I was attempting this stuff. Maybe I’ll whip up a proposal for some sort of frame ID, since I wouldn’t mind revisiting the laziness technique and that’s all it really needed.

#5

Well the documentation for display.remove() says:

This is the same as calling object:removeSelf(), but will first check if the object exists before attempting to remove.

The exact method in the code isn’t all that important, but it does check to see if the object does exist first.  Which is why Engineering added this a couple of years ago.

Rob

#6

The way it’s written, I got the impression that was just a rephrasing of the previous line, to spell out the reasoning behind the  nil check. For that matter, the example reinforces this assumption.  :slight_smile:

If it does indeed have the expected behavior, a clause like ", or if the object has previously been removed, " might clear things up.

#7

I would suggest you hit the button at the bottom of the docs that lets you provide feed back on it.  Our documentation team and then address it.

Rob

#8

re the frame tick:  fwiw and ymmv, but… for my own purposes i only ever register a single runtime enterFrame listener - i treat it more-or-less like the global system event handler.  it’s always running, but not necessarily doing much, i handle who it delegates to myself - about the only thing it always does is update a frame counter.  in short, but essentially:

onEnterFrame = function(e)   frameCounter = frameCounter + 1   if (delegate) then delegate() end end

i don’t do the “many objects each with their own enterFrame listeners” (and timers, and transitions, and…) thing – too much “black box” syndrome for my tastes.

#9

@ Rob

Good idea. Done.

@ davebollinger

That’s a good way to do it, when you have control. I wonder how things will unfold (in general, not strictly in one of our situations) once the store gets going and there’s a lot of plugin mix-and-match going on.

Are you doing anything interesting with your frame counter? I did end up posting a feature request last night, but it’s probably a bit specific to my own use case. It might have better luck with more.  :slight_smile:

#10

Following up on this.

First off, I haven’t heard back on display.remove(), nor have the daily docs changed. I don’t know if this is meaningful or not. Anyhow, that’s something of a secondary concern.

I’ve been playing around with some additional projects, to beat them into shape for desktop builds, and I got stung.  :frowning: In a couple of places I stumbled across a numChildren property that’s nil despite removeSelf still being present. In both instances, the problem code is inside a timer. For whatever reason, in these circumstances, removeSelf never disappears.

After much frustration, I’ve narrowed it down to some sort of interaction with the widget library. (I did previously notice some removeSelf shenanigans when I perused the GitHub source, so it wasn’t a complete surprise, but I don’t remember the details.) Despite it being possible to eventually have widgets in play in my usual projects, they seemed to show up late enough that the “safe” infrastructure was in place. Or something, I don’t know. Anyhow, when I imported my updated removeSelf -using submodules into one in particular, KERBLOOEY!  :frowning:

It’s reproduced in this Gist. If require(“widget”) is removed from main.lua , no issues.

If the prevailing opinion is that this a bug, I’ll file a report. Though I might work it from the “shortcoming” angle (I believe there’s such an option?) with the goal of getting an official solution. I have a couple others to file as well, anyhow.

That said, here’s my prospective workaround (run at startup), accommodating snapshots as well:

do local type = type -- Monkey-patch display.newSnapshot(). Maintain some weak parent lookups for a snapshot's -- group and canvas until the snapshot has been removed. local WeakParent = setmetatable({}, { \_\_mode = "v" }) local old\_newSnapshot = display.newSnapshot function display.newSnapshot (...) local snapshot = old\_newSnapshot(...) WeakParent[snapshot.canvas], WeakParent[snapshot.group] = snapshot, snapshot return snapshot end -- Validity predicate, with special consideration for snapshot groups and canvases local function IsValid (object) if type(object) == "table" then if object.parent ~= nil then return true else return IsValid(WeakParent[object]) end end return false end --- Detects whether the input is a display object that has not yet been removed. -- @function display.isValid -- @pobject object Display object. -- @treturn boolean Is this a valid display object? display.isValid = IsValid end

For reference, here were my snapshot tests:

local ss = display.newSnapshot(1,1) -- Test snapshot, group, canvas print("SS", IsValid(ss)) print("SG", IsValid(ss.group)) print("SC", IsValid(ss.canvas)) local sg, sc = ss.group, ss.canvas -- Make sure the components still have a strong reference if this one is nil if true then ss = nil print("SG2", IsValid(sg)) print("SC2", IsValid(sc)) -- Test validity once the snapshot is removed else ss:removeSelf() print("SG3", IsValid(sg)) print("SC3", IsValid(sc)) end
#11

I’ve not had a chance to try and get a specific answer for this. I would suggest that you assume, this is all display.remove() does:

function display.remove( object )     if object and object.removeSelf() then          object:removeSelf()     end end

and nothing more. If you don’t nil the object, the functions are likely still going to be there and still needs to be nilled out.

Rob

#12

Generally I do:   if displayObject and displayObject.removeSelf then  as my test.  I believe that display.remove() is just a wrapper that does that check.

Rob

#13

in my experience, it’s a bit like the “browser sniffing” debate in web development…

the presence (or absense) of .removeSelf doesn’t really/truly/fully/100% indicate that an object “is valid”.  rather, it indicates only that the object “is still valid enough” to still be present in the display (scene graph) and so a call to remove it from the scene graph is still considered valid. (even if it had already been removed previously during this same frame, and won’t be completely “gone” until next frame - the call itself is still “valid”)  fe, try this immediate mode loop:

for i = 1,100 do   if (obj and obj.removeSelf) then     obj.removeSelf()     print("removed it again, no problem") end end

everything that i’ve “witnessed” about Corona’s internals occurs within that frame loop processing.  that’s why you see so many “timer.performWithDelay(1,someCleanupRoutine)” -type workarounds for things that need to wait for something to “complete internal processing” before executing.

it’s undocumented, so no-one but Corona staff really knows, but imagine something like:

onCoronasInternalFrameTick:   do internal system stuff (like ticking box2d, memory management, etc, who knows/who cares?)   process system events   process device ui events   process timers   process user's frame ticks (registered enterFrame event listeners)   any internal final update of scene graph prior to rendering   .. etc

or something along those lines, who knows the exact order?  doesn’t really matter, and at any rate, if you accept any pattern along those lines, it’ll become apparent that if you “do something” in your frame loop (or in response to an event, or etc), it may not fully “clean up” until the next time the frame loop “does the internal stuff”.  the circumstantial evidence of various performWithDelay() hacks are compelling in this regard.

#14

@ Rob

I actually had some of your examples in mind, writing that up. I guess my first question boils down to: “If somebody from engineering is reading along, are they nodding their heads or frowning?”  :slight_smile:

Obviously it’s easy to wrap up the “seems to work” behavior, so it’s not so much a problem of fixing it should it break.[1] I’m just hoping to nail some stuff down. I share your belief about  display.remove (), but it wouldn’t hurt to be sure! Ditto on “finalize”.

@ davebollinger

Uff, the frame tick is another can of worms.[2][3] 

As you might have guessed, I’d like details on more than a few things. :D  Often just a “yes”, “no”, or “don’t rely on it”. It would involve getting engineers to divulge everything, though, so there are obvious practical hassles: time, frequent pestering, staying up-to-date, and so on. Oh well.

Footnotes ahoy!

[1] - I forgot about it before, but I have been tripped up by changing internals. Transitions used to expose enough state, discoverable through  pairs () and  print () and friends, so that you could figure out when a transition had finished. No longer! That broke some things.  :stuck_out_tongue:

[2] - Occasionally in my work I need to consult this, for instance to render stuff off-screen once all of an object’s bones are in position, then use the result. It tends to be a bit painful, but still feasible. Unfortunately, as you pile on more and more fanciness, it gets tangled really fast!

[3] - At some point, I was trying to write a few things using an approach like

if not HasThisBeenCalledYetThisFrame() then DoStuff() -- possibly expensive, or just needs to be idempotent in the short run end

to lazily perform once-per-frame-at-most calculations. This requires a little bookkeeping, such as a frame ID. An enterFrame listener would seem to be the natural place to update it.

If we guess right about how a frame unfolds, say that transitions and timers are processed before  enterFrame listeners, our frame comparison would work flawlessly when called from either of the former, but only sometimes from the latter. It all depends when the ID-updating listener gets called, relative to other enterFrame listeners that care about the ID. Some of them could be seeing the future! (And it does not work.)

(If addEventListener () just appends to an array or linked list, and if  removeEventListener () leaves the remaining elements in order, and if the listeners are called in the order they were added, I suppose Runtime:addEventListener () could be monkey-patched to always move a privileged “leaveFrame” listener to the end of the line. Quite a lot of unknowns, though.)

Thinking out loud here… if I’m not mistaken, the feature request page didn’t exist when I was attempting this stuff. Maybe I’ll whip up a proposal for some sort of frame ID, since I wouldn’t mind revisiting the laziness technique and that’s all it really needed.

#15

Well the documentation for display.remove() says:

This is the same as calling object:removeSelf(), but will first check if the object exists before attempting to remove.

The exact method in the code isn’t all that important, but it does check to see if the object does exist first.  Which is why Engineering added this a couple of years ago.

Rob

#16

The way it’s written, I got the impression that was just a rephrasing of the previous line, to spell out the reasoning behind the  nil check. For that matter, the example reinforces this assumption.  :slight_smile:

If it does indeed have the expected behavior, a clause like ", or if the object has previously been removed, " might clear things up.

#17

I would suggest you hit the button at the bottom of the docs that lets you provide feed back on it.  Our documentation team and then address it.

Rob

#18

re the frame tick:  fwiw and ymmv, but… for my own purposes i only ever register a single runtime enterFrame listener - i treat it more-or-less like the global system event handler.  it’s always running, but not necessarily doing much, i handle who it delegates to myself - about the only thing it always does is update a frame counter.  in short, but essentially:

onEnterFrame = function(e)   frameCounter = frameCounter + 1   if (delegate) then delegate() end end

i don’t do the “many objects each with their own enterFrame listeners” (and timers, and transitions, and…) thing – too much “black box” syndrome for my tastes.

#19

@ Rob

Good idea. Done.

@ davebollinger

That’s a good way to do it, when you have control. I wonder how things will unfold (in general, not strictly in one of our situations) once the store gets going and there’s a lot of plugin mix-and-match going on.

Are you doing anything interesting with your frame counter? I did end up posting a feature request last night, but it’s probably a bit specific to my own use case. It might have better luck with more.  :slight_smile:

#20

Following up on this.

First off, I haven’t heard back on display.remove(), nor have the daily docs changed. I don’t know if this is meaningful or not. Anyhow, that’s something of a secondary concern.

I’ve been playing around with some additional projects, to beat them into shape for desktop builds, and I got stung.  :frowning: In a couple of places I stumbled across a numChildren property that’s nil despite removeSelf still being present. In both instances, the problem code is inside a timer. For whatever reason, in these circumstances, removeSelf never disappears.

After much frustration, I’ve narrowed it down to some sort of interaction with the widget library. (I did previously notice some removeSelf shenanigans when I perused the GitHub source, so it wasn’t a complete surprise, but I don’t remember the details.) Despite it being possible to eventually have widgets in play in my usual projects, they seemed to show up late enough that the “safe” infrastructure was in place. Or something, I don’t know. Anyhow, when I imported my updated removeSelf -using submodules into one in particular, KERBLOOEY!  :frowning:

It’s reproduced in this Gist. If require(“widget”) is removed from main.lua , no issues.

If the prevailing opinion is that this a bug, I’ll file a report. Though I might work it from the “shortcoming” angle (I believe there’s such an option?) with the goal of getting an official solution. I have a couple others to file as well, anyhow.

That said, here’s my prospective workaround (run at startup), accommodating snapshots as well:

do local type = type -- Monkey-patch display.newSnapshot(). Maintain some weak parent lookups for a snapshot's -- group and canvas until the snapshot has been removed. local WeakParent = setmetatable({}, { \_\_mode = "v" }) local old\_newSnapshot = display.newSnapshot function display.newSnapshot (...) local snapshot = old\_newSnapshot(...) WeakParent[snapshot.canvas], WeakParent[snapshot.group] = snapshot, snapshot return snapshot end -- Validity predicate, with special consideration for snapshot groups and canvases local function IsValid (object) if type(object) == "table" then if object.parent ~= nil then return true else return IsValid(WeakParent[object]) end end return false end --- Detects whether the input is a display object that has not yet been removed. -- @function display.isValid -- @pobject object Display object. -- @treturn boolean Is this a valid display object? display.isValid = IsValid end

For reference, here were my snapshot tests:

local ss = display.newSnapshot(1,1) -- Test snapshot, group, canvas print("SS", IsValid(ss)) print("SG", IsValid(ss.group)) print("SC", IsValid(ss.canvas)) local sg, sc = ss.group, ss.canvas -- Make sure the components still have a strong reference if this one is nil if true then ss = nil print("SG2", IsValid(sg)) print("SC2", IsValid(sc)) -- Test validity once the snapshot is removed else ss:removeSelf() print("SG3", IsValid(sg)) print("SC3", IsValid(sc)) end