Corona/LUA Runtime - Great optimization potential?

I started working with Spine some time ago and started implementing it into my game. While writing my own library to make the use of Spine animations a lot easier I took a closer look at the Corona and LUA Spine runtime.

The strange thing is, the runtimes make no use of metatables and always create functions from scratch when a new object (e.g. skeleton, bone etc.) is created. This means a lot more memory is needed for storing those functions, especially when you’ve got more then a few animated skeletons.

Also external functions (e.g. math.abs, display.newGroup etc.) and external Spine modules (e.g. spine.utils, spine.SkeletonJson etc.) are not localized.

Does anybody know if there is a more optimized version of the runtime somewhere?

Otherwise I have to contact the Spine creators and ask for an update. (or do it myself)

The latest spine runtimes can be found on Esoteric Software’s gitHub: https://github.com/EsotericSoftware/spine-runtimes 

Yeah, that’s the version I was referring to.

Spine is entirely global as well as you can see in spine-corona/spine.lua…

Yeah, been experiencing very sluggish behaviour when using more than one skeleton - especially as new animations are being called when onComplete is triggered by a previous animation (or any other event start, onEnd, events). After continually doing this, frame rate drops right down to 2, lua memory is bulked. Upon changing the scene, the frame rate stays low and the lua memory still holds the memory it picked up from the spine animation scene. Haven’t found this on a looping animation, only one that plays a new animation after an event…

Perhaps a table somewhere is just collecting animation data and storing it globally? Any ideas?

Torbenratzlaff - be good to know how you get on. In the meantime, I’ll start bug busting further…

So far I’m done with the following:

  • adding metatables wherever possible and necessary

  • localizing global variables and function

  • localizing variables in time critical processes (e.g. loops, runtime)

  • removing unnecessary code (e.g. mesh transformation)

  • wrinting a wrapper, so skeletonData, spritesheets and others have to be only created once

  • creating a new skeleton from preexisting skeletonData with a single line of code

Now I ask myself, what to do with those changes. Is there a spine “official” in the Corona forums? Or should I make a pull request to the above mentioned GitHub project?

@kilopop

Concerning the memory leak you mentioned. Could you go a little bit more into detail how to reproduce that bug?

I’m interested in this as well. I have a WIP wrapper that lets you load a spine and “tether” it to a corona physics object AND combines all the spine events into one easy event.phase / event.data

 hero.spine = spine.new("spine/hero.json", { x=hero.x, y=hero.y, imageDir = "spine/heroParts", --imageSheet = "spine/hero.png", --imageData = "spine.hero", tetherObject = hero, tetherOffsetY = hero.height/2, mixTime = 0.25, }) hero.spine:play("idle",true)  

I’d be happy to donate the wrapper to the overall project…

BTW, I would do both a pull request and a fork of the spine runtimes…

@torbenratzlaff: Certainly, I’d be interested to see what you’ve come up with. What are the performance improvements like? I would suggest you start a post on the Esoteric Runtime board: http://esotericsoftware.com/forum/viewforum.php?f=7&sid=57e243a5c329cd004672f602078ca116 To ask the the question of the best method to get these users changes out there. In the meantime, I’d be interested in testing your new version to see if it affects the issues I’m experiencing.

To clarify the bottlenecking I’m getting. The library I’ve developed works similar to spineHelper. It uses a metatable to create the spine objects and then animation is added in a separate object oriented function M:animation. There is no problem if animations are called within the onComplete event e,g,

state.onComplete = function (trackIndex, loop) state:addAnimationByName( data ) end

The above would trigger particular animations according to what is set in the data every time the animation triggers an onComplete event. In this case the animations loop forever happily with no problems. 

But if the animation module is called multiple times then the memory mounts up and the framerate decreases. An example setup uses eventListeners to go between a scene page and the onComplete event of the Spine library:
 

-- Scene 1 -- obj[name].spine is a table with a reference to a skeleton.group -- There are 4 animations, antics1 - antics4. Each time animLoop is called, a random one plays function M:animLoop (event) local event = event local name = event.tag local rand = math.random( 1, 4 ) obj[name].spine:animation( {onComplete="animLoop", sequence="antics"..rand}) end Runtime:addEventListener("animLoop", M) -- Spine Library state.onComplete = function (trackIndex, loop) --loopCount if onComplete[trackIndex] ~= nil then Runtime:dispatchEvent( {name=onComplete[trackIndex], tag=name} ) end end

Each time the Spine module is called, it declares local variables with the skeleton, state etc that are created and associated with the object when Spine was being created.

function M:animation (params) local params = params or {} local skeleton = self.skeleton local root = self.root local stateData = self.stateData local state = self.state local name = self.name -- rest of module end

So I kinda wonder if these local variables are mounting up over time?

Any ideas at this stage would be great. I could post the library as it is so far if that helps.

How does your library handle continual animation calls @torbenratzlaff? If you were to use event listeners like this in your onComplete events would you get the same results?

@no2games - It would be great to see your wrapper. In particular how you are handling spine events.

There are a few different Spine libraries out there. It would be great to combine the best of them to create an on-going library. SpineHelper is an adequate start but only handles simple tasks.

@kilopop Mine is not overly complicated… What I tried to do was extend skeleton to contain all the animation, images, etc to keep them in one structure. I also killed all that extra math and locked the animations into 30 or 60 FPS based on what the config.lua is set at… I figure if your animations can’t keep up you have bigger issues.

Finally I wrapped all the state events into one onEvent() but it should probably be renamed to onAnimation()

So I use it like this… Create a display object…

hero = display.newObject() physics.addBody(hero)

Then I create the spine as a metatable (or object) off of that display object

hero.spine = spine.new("spine/hero.json", { x=hero.x, y=hero.y, imageDir = "spine/heroParts", tetherObject = hero, mixTime = 0.25, }) hero.spine:play("idle",true) 

Then I hide the original display object

hero.isVisible = false

I keep my spine runtimes in a package structure, so you’ll need to update the one spine = require()

Anything you can do to speed this up or improve the stability is appreciated. 

-- Project: Spine Wrapper 0.3 -- -- Date: Mar 20, 2015 -- Updated: Apr 14, 2015 local spine = require "com.esoteric.spine-corona.spine" local M = {} M.delta = 1 / display.fps function M.new(spineJson, options) -- defaults options = options or {} local speed = options.speed or 1 local tetherObject = options.tetherObject or nil local offsetX, offsetY = options.tetherOffsetX or 0, options.tetherOffsetY or 0 -- Json loading local jsonData = spine.SkeletonJson.new() jsonData.scale = options.scale or 1 local skeletonData = jsonData:readSkeletonDataFile(spineJson) -- build our skeleton object local skeleton = spine.Skeleton.new(skeletonData) skeleton.delta = M.delta -- function to load from images or texture pack / shoebox if options.imageSheet and options.imageData then skeleton.imageData = require (options.imageData) skeleton.imageSheet = graphics.newImageSheet( options.imageSheet, skeleton.imageData.sheet ) end function skeleton:createImage(attachment) if options.imageDir then return display.newImage(options.imageDir .. "/" .. attachment.name .. ".png") elseif self.imageSheet and self.imageData then local frame = self.imageData:getFrameIndex(attachment.name) return display.newImage(self.imageSheet, frame) end end skeleton.count = 0 -- dump animation data local animations = skeleton.data.animations for i = 1, #animations do print ("Animation found:", animations[i].name) end function skeleton.animationExists(name) for i = 1, #animations do if name == animations[i].name then return true end end return false end -- initial animation/pose setup skeleton:setToSetupPose() skeleton.stateData = spine.AnimationStateData.new(skeletonData) local stateData = skeleton.stateData -- set global mix duration if #animations \> 1 then for i = 1, #animations do for j = 1, #animations do if not (i==j) then stateData:setMix(animations[i].name, animations[j].name, options.mixTime or 0.5) end end end end -- setup animation queue skeleton.state = spine.AnimationState.new(stateData) local state = skeleton.state -- Events: -- let's combine these events in a more corona-like fashion if options.onEvent then local e = {} local eventFunction = options.onEvent state.onStart = function (trackIndex) e.phase = "started" e.trackIndex = trackIndex e.name = state:getCurrent(trackIndex).animation.name if options.default==nil then -- don't fire started with default animation eventFunction(skeleton,e) end end state.onEnd = function (trackIndex) e.phase = "ended" e.trackIndex = trackIndex e.name = state:getCurrent(trackIndex).animation.name eventFunction(skeleton,e) end state.onComplete = function (trackIndex, loopCount) e.phase = "completed" e.trackIndex = trackIndex e.loopCount = loopCount e.name = state:getCurrent(trackIndex).animation.name eventFunction(skeleton,e) end state.onEvent = function (trackIndex, event) e.phase = event.data.name or "unknown" e.trackIndex = trackIndex e.loopCount = loopCount e.data = event.intValue or event.floatValue or event.stringValue or "" e.name = state:getCurrent(trackIndex).animation.name eventFunction(skeleton,e) end end local function offScreen(object) local bounds = object.contentBounds local sox, soy = display.screenOriginX, display.screenOriginY if bounds.xMax \< sox then return true end if bounds.yMax \< soy then return true end if bounds.xMin \> display.actualContentWidth - sox then return true end if bounds.yMin \> display.actualContentHeight - soy then return true end return false end function skeleton.destroy() Runtime:removeEventListener("enterFrame", skeleton.enterFrame) if skeleton.group then display.remove(skeleton.group) skeleton.group = nil end skeleton.imageSheet = nil skeleton = nil end function skeleton.enterFrame() local hasGroup = skeleton and skeleton.group and skeleton.group.numChildren local state = skeleton.state -- if we loose our group then destroy if hasGroup then if tetherObject and tetherObject.x and tetherObject.y then skeleton.group.rotation = tetherObject.rotation skeleton.group.x, skeleton.group.y = tetherObject.x + offsetX, tetherObject.y + offsetY else skeleton:destroy() return false end -- offscreen if offScreen(skeleton.group) then return end -- Update the state with the delta time, apply it, and update the world transforms. state:update(skeleton.delta) state:apply(skeleton) if skeleton then skeleton:updateWorldTransform() end else state:update(M.delta) -- in case we are missing state:apply(skeleton) skeleton:destroy() return false end return true end function skeleton.init() if tetherObject then skeleton.group.rotation = tetherObject.rotation skeleton.group.x, skeleton.group.y = tetherObject.x + offsetX, tetherObject.y + offsetY end -- Update the state with the delta time, apply it, and update the world transforms. state:update(skeleton.delta) state:apply(skeleton) skeleton:updateWorldTransform() return true end -- \*\*\* simple wrapper functions \*\*\* -- what's playing function skeleton:currentAnimation(trackIndex) if not self.state then return end local currentState = self.state:getCurrent(trackIndex or 0) if not currentState then return end return currentState.animation.name end -- play an animation function skeleton:play(name, loop, trackIndex) if self.isPaused then self:resume() end if self.animationExists(name) and self:currentAnimation() ~= name then self.state:setAnimationByName(trackIndex or 0, name or self.data.animations[1].name, loop or false) else --print ("WARNING: Animation not found or already playing:",name) return false end self.isPaused = false end -- add an animation to the queue function skeleton:add(name, loop, delay, trackIndex) if self.animationExists(name) and self:currentAnimation() ~= name then self.state:addAnimationByName(trackIndex or 0, name or self.data.animations[1].name, loop or false, delay) else --print ("WARNING: Animation not found or already playing:",name) end end -- pause the animation function skeleton:pause() if not self.isPaused then --print("pausing") self.isPaused = true Runtime:removeEventListener("enterFrame", self.enterFrame) end return self.isPaused end -- resume the animation function skeleton:resume() if self.isPaused then --print ("resuming") self.isPaused = false timer.performWithDelay(M.delta, function () Runtime:addEventListener("enterFrame", self.enterFrame) end) end return self.isPaused end -- set default placement if skeleton.group then skeleton.group.x = options.x or 0 skeleton.group.y = options.y or 0 if options.anchored then skeleton.group.anchorChildren = true skeleton.group.anchorX,skeleton.group.anchorY = 0.5,0.5 end end -- kick off the first frame to load the parts skeleton.init() -- start paused skeleton.isPaused = true -- unless we choose a default animation if options.default then skeleton:play(options.default, options.loop) end return skeleton end return M

Also, it seems like the lookup table optimizations got factored out of bone positioning… I’m not advocating that lookup tables actually improve performance, but if someone wanted to test here’s a bone.lua that I put together this afternoon that uses one…

local Bone = {} local lookup = {} for i = 0, 359 do lookup[i] = math.sin( math.pi / 180 \* i ) end function Bone.new (data, skeleton, parent) if not data then error("data cannot be nil", 2) end if not skeleton then error("skeleton cannot be nil", 2) end local self = { data = data, skeleton = skeleton, parent = parent, x = 0, y = 0, rotation = 0, rotationIK = 0, scaleX = 1, scaleY = 1, flipX = false, flipY = false, m00 = 0, m01 = 0, worldX = 0, -- a b x m10 = 0, m11 = 0, worldY = 0, -- c d y worldRotation = 0, worldScaleX = 1, worldScaleY = 1, worldFlipX = false, worldFlipY = false, } function self:updateWorldTransform (flipX, flipY) local parent = self.parent if parent then self.worldX = self.x \* parent.m00 + self.y \* parent.m01 + parent.worldX self.worldY = self.x \* parent.m10 + self.y \* parent.m11 + parent.worldY if (self.data.inheritScale) then self.worldScaleX = parent.worldScaleX \* self.scaleX self.worldScaleY = parent.worldScaleY \* self.scaleY else self.worldScaleX = self.scaleX self.worldScaleY = self.scaleY end if (self.data.inheritRotation) then self.worldRotation = parent.worldRotation + self.rotationIK else self.worldRotation = self.rotationIK end self.worldFlipX = parent.worldFlipX ~= self.flipX self.worldFlipY = parent.worldFlipY ~= self.flipY else local skeletonFlipX, skeletonFlipY = self.skeleton.flipX, self.skeleton.flipY if skeletonFlipX then self.worldX = -self.x else self.worldX = self.x end if skeletonFlipY then self.worldY = -self.y else self.worldY = self.y end self.worldScaleX = self.scaleX self.worldScaleY = self.scaleY self.worldRotation = self.rotationIK self.worldFlipX = skeletonFlipX ~= self.flipX self.worldFlipY = skeletonFlipY ~= self.flipY end --local radians = math.rad(self.worldRotation) --local cos = math.cos(radians) --local sin = math.sin(radians) local angle = (self.worldRotation - self.worldRotation % 1) % 360 local cos = lookup[(angle+90) % 360] local sin = lookup[angle] if self.worldFlipX then self.m00 = -cos \* self.worldScaleX self.m01 = sin \* self.worldScaleY else self.m00 = cos \* self.worldScaleX self.m01 = -sin \* self.worldScaleY end if self.worldFlipY then self.m10 = -sin \* self.worldScaleX self.m11 = -cos \* self.worldScaleY else self.m10 = sin \* self.worldScaleX self.m11 = cos \* self.worldScaleY end end function self:setToSetupPose () local data = self.data self.x = data.x self.y = data.y self.rotation = data.rotation self.rotationIK = self.rotation self.scaleX = data.scaleX self.scaleY = data.scaleY self.flipX = data.flipX self.flipY = data.flipY end function self:worldToLocal (worldCoords) local dx = worldCoords[1] - self.worldX local dy = worldCoords[2] - self.worldY local m00 = self.m00 local m10 = self.m10 local m01 = self.m01 local m11 = self.m11 if self.worldFlipX ~= self.worldFlipY then m00 = -m00 m11 = -m11 end local invDet = 1 / (m00 \* m11 - m01 \* m10) worldCoords[1] = dx \* m00 \* invDet - dy \* m01 \* invDet worldCoords[2] = dy \* m11 \* invDet - dx \* m10 \* invDet end function self:localToWorld (localCoords) local localX = localCoords[1] local localY = localCoords[2] localCoords[1] = localX \* self.m00 + localY \* self.m01 + self.worldX localCoords[2] = localX \* self.m10 + localY \* self.m11 + self.worldY end self:setToSetupPose() return self end return Bone

@no2games - Good stuff, very condensed library. I like the way you’ve handled the events.

Only thing I can offer at first glance is skeleton:createImage is not returned dynamically resized images. If you need to, you could use:

return display.newImageRect(options.imageDir … “/” … attachment.name … “.png”, attachment.width, attachment.height)

Thanks.

I didn’t realize the attachment passed the width and height. Nice.

I’m happy to commit this code to gitHub and let everyone have at improvements. I think an “unstable” fork/branch of Spine’s LUA Runtimes would also be an interesting idea too. I’ve got a few things I wouldn’t mind changing, and I’m sure having them in GitHub will make it easier for them to merge into official releases.

Anyway, between the three of us (@no2games, @kilopop and @torbenratzlaff) I’m sure we have a fairly substantial community update.

Yeah, sounds good. My Spine library is a little more specific to our book app framework. But you could take a look through and see how it is doing anything you might be able to use.

Currently though there is the issue that calling the animation function repeatedly results in memory increases and framerate drops. This could be same for your skeleton:play no2game?. If you pass your onEvent to a function which repeatedly calls :play do you have resource issues eventually?

I’ll package up the library and post here with an example. I would greatly appreciate any feedback as to what might be causing this. Without knowing exactly how Spine holds data, I originally thought it has to do with some of Spine’s global functions.

I don’t have any issues with multiple calls to skeleton:play(). I make sure a new animation uses the same enterFrame() loop and it seems to be fine.

I only seem to have resource issues with lots of spine animations loaded. It doesn’t matter if it is the same spine or a bunch of different ones AND it seems that most of the time is spent in the enterFrame() loop doing the update(), apply() and worldTransform(). You can see in the wrapper that I check to see if a spine is off screen and basically short circuit the enterFrame() and that helps a bunch, so my assumption is that’s where things are getting bogged down.

My hope is that there’s some optimization to be done in those routines that can deliver some extra perfomance.

Anyway, I’d love to take a peek at your stuff. I haven’t even started on adding easy slot swapping (which i will need to do) and maybe there’s some things I can steal :slight_smile:

Yeah I didn’t appear to have issues with multiple calls until the calls were happening more frequently such as every 2 seconds. Certainly everything is multiplied with multiple skeletons.

Also there is no resource drain if you setup a test condition that plays animations within onComplete e.g.

state.onComplete = function (trackIndex, loop)
    state:addAnimationByName( data )
end

The drain only occurs when you call the animation function.

Anyway, here’s my Spine library, SpineTrigger, collected with an example project:

https://dl.orangedox.com/fwtujtJVvpG2Ey8npr/spineTrigger.zip

This one creates 6 dragons and sets their animations off. When they each trigger onComplete, they call a function which changes the name of the animation and sends it back into the animation function.

There is a performance function which displays framerate and luaMem. You’ll see this quickly degrade from 60fps down to 1 fps. The memory continues to balloon. So something is compounding somewhere each time the animation function is called. I would have blamed the global nature of Spine but it might be something more or less straight forward…

Steal away. I’ll steel myself for the feedback :slight_smile:

I have not enough time to share my SpineWrapper with you, as it fullfills the very specific needs of my project.

But I’d like to share my optimized corona spine runtime with you:

http://torben-ratzlaff.de/stuff/spine%20remastered%20vs001.zip

Just call the spine librabry with require(“spine.spine”)

@kilopop

Concerning the memory issues, it seems to be the fault of your wrapper.

If you call

state:setAnimation(...)

directly inside the “dragon” function the main lua, (just add the need parameters to the event)

there is no memory leak and the framerate stays at 60.

I believe the reason is the “updateState” function.

As calling timers and Runtimes in a quasi recursive function is not a good idea in most cases.

Hope I could spare some time in the future to optimize further.

@torbenratzlaff  Thanks for sharing. I’ll have a look at this today. I’m assuming you would be okay with sharing these updates on gitHub somewhere?

@kilopop, yep, what @torbenratzlaff said… You are creating a new infinite timer every time you call updateState() and only removing it once… I did a quick hack to get it working…

 if not self.removeEL then \_gal.timerStash[name] = timer.performWithDelay( 1, updateState, -1 ) self.removeEL = updateState --allow instance to keep reference to this for pause/ resume purposes --Runtime:addEventListener("enterFrame", updateState) print("Added Timer") end

Replace what you have around line 380 with this, and a new timer will only get created when removeEL doesn’t exist. FYI, I don’t know if I’m a fan of the timers so I’d go back to your enterFrame events. :slight_smile:

I use one enterframe per spine animation, but I’ve thought about optimizing the wrapper to have a single enterFrame() event so things like frame delta only need to be calculated once. That may help your performance too.

@no2games

Yeah, sure I’m ok with sharing it on github.

But I would like to do that myself in the next couple of days.

Sounds good. I’ll hold on to my revisions and submit them when you post.

Thanks!