Corona SDK Performance and Bunnymark challenge

Solar2D General Questions/Discussion
#1

Hello everyone. I’m a fan of Lua based frameworks/engines and have used a bunch, but not yet Corona SDK, which recently caught my interest. A few months ago I ran some comparisons of my favorite engines/frameworks through bunnymark to see how they all performed in this type of test. 

The defold tests are bottle necked by the max bunny limit so I decided to run the tests from my phone targeting the fps I got from the defold particlefx test because it is fixed at 15000 bunnies. I tested on my LG V20 with the chrome browser, everything kept consistent as possible and retested for the most accurate numbers.

Defold particlefx 22fps @ 15000

Defold go.animate() 23fps @ 10000

Phaser 22 fps @ 3100

Gideros (particles) 23 fps @ 7500

Pixi.js (4.7.3) 23fps @ 10500

Openfl 8.0.0 22fps @ 39000

Kha 21fps @ 50000

Then I tested two of my favorite Lua based engines natively on android:

Defold particlefx 31fps @ 15000

Defold go.animate() 31fps @ 12000

Gideros (particles) 31 fps @ 15000

I want to add Corona SDK to my bunnymark comparison on android, but did not see any bunnymark tests made for Corona SDK. I have no experience with Corona so I was wondering if anyone would be up to port or make their own Corona SDK bunnymark to rep for their game engine. Would love to see how it holds against Defold and Gideros since they are both free Lua engine alternatives. I think it’s a great way to explore engine strengths and limitations.

Here are the source projects for those of you interested in how bunnymark was programmed in it’s perspective engines/frameworks:

Defold - https://github.com/britzl/defold-bunnymark

Phaser - http://phaser.io/sandbox/Pjdxyqzl

Gideros - https://github.com/lemon07r/gideros-bunnymark

pixi.js - https://github.com/pixijs/bunny-mark

openfl - https://github.com/lemon07r/openfl-bunnymark

kha - https://github.com/lemon07r/kha-html5-bunnymark

#2

Bumping.

Is the community not all that active compared to other engines?

#3

my guess is just that few here care how fast bunnies can be drawn. (but if you care, then why not port it yourself?)

#4

Agreed. I personally don’t have the spare time to port a benchmark. I’m sure others are in the same boat. Just too busy for this.

#5

Is that the general mindset of the Corona community? A little sad to see for me cause when I did this for other engines there was a lot more interest and activity, even in kha the main developer answered questions, defold had a very active thread with well over 50 replies, gideros had multiple people interested, and so forth. Im sure it would only take a half hour to a few hours to port if I was motivated to do so but no point if nobody is interested or if the community isnt super active.

Nothing personal, I understand that some people are busy or have other important priorities. Was just disappointed not to find individuals who might be interested like I did in other communities because that’s one of my favorite parts about learning and trying other game engines.

If I do end up getting around to it I can leave my results here and how I felt about Corona coming from other engines, but I likely wont do the engine full justice like someone experienced could and might end up representing the engine in a poor way if my port isn’t made optimally. 

One of the features Corona is advertised for literally on the main page is "Speed and performance

From bleeding-fast OpenGL graphics to the lightweight scripting power of Lua, Corona is optimized for performance at every core level. With Corona, your apps are automatically compiled at build time, streamlined for performance and stability." So I was surprised that there werent really any benchmarks or anything to actually test the validity of these claims.

#6

I don’t understand the attitude in your posts. It doesn’t endear me to your cause.
 

I’m probably misreading your posts. It is very hard to get tone right.

The people here (like myself and others) who answer questions for new folks are busy trying to make a living, so we are careful with our time.

Converting a benchmark is interesting, but time consuming. I personally can’t justify the time to do it at this point.

Thanks for raising the topic, and I’d love to see it done, but I for one can’t help. Still there are a lot of smart folks here, so if you’re willing to wait a little you may be surprised and see someone post a conversion.

#7

One more thing. I don’t see an original language-/ engine-agnostic design spec for this benchmark.

Without a spec I’d have to reverse engineer one of those tests to implement this, all the time risking an apples versus oranges comparison due to my misunderstanding of the port-from source.

This is a lot of work.

Interesting, but man time consuming. :frowning:

PS - In 2013 I wrote a series of benchmarks for Corona. I’m checking to see if I still have the code and will post a link here if I do.

While is isn’t your Bunnymark, you can still run it.

#8

Found it: https://github.com/roaminggamer/RG_FreeStuff/blob/master/AskEd/2018/08/cbench.zip

This is some seriously old code. It is still valid, but it did need a little love to convert it to modern SSK and to Graphics 2.0 from Graphics 1.0.

Here is an HTML5 build (not sure how well this will run in HTML5):

https://roaminggamer.github.io/RGDocs/pages/html5_tests/tests/cbench/

#9

Well… it runs but the scroller is hosed up so you can’t see the end of the results.

#10

I really like Corona and, besides one project, all of my current inDev projects, are done using Corona as it has a lots of strengths. The unpleasant but honest truth ist, extreme performance is just not one of those - and I tried really hard before I added a second framework just for one project.

While I haven’t written any bunnymark and I don’t know the details but any serious implementation would need to be done *in* the engine itself, using f.i. particle systems (as some of the above listed implementions seem to use) which may run fully on the C++ side or even on the GPU is just nonsense and any such implementation is just plain useless. Haxe based engines like Kha of course are fine because the generated C++ is actually built from the same language you also use for the rest of your game/project.

Anyway - the benchmark seems really simple and would be a tiny project but implemented in the typical Corona way I predict you’re in for a disappointing result.

#11

Hello,
I tried to port this bunnymark test to Corona, just for the curiosity to see how it performs compared to the other engines. You can find it here: https://github.com/Melix19/corona-bunnymark

Let me know the results!

(If someone that has more experience with Corona than me finds out that the code can be improved, fell free to send a PR)

#12

I have also builded the project to html5. Here you can find it out: https://melix19.github.io

#13

Looks good, but the fps counter seems to be wrong after 1000 bunnies. It’s still running fairly smoothly but it’s reporting 3-4 fps.

#14

There is not a real fps counter on Corona so i tried different codes to port it to the project. This is the best that i found.

#15

@Melix19,

1.  You’re awesome.  Thanks for porting.

  1. I didn’t agree with your implementation of the FPS counter so used my own and added a memory meter too.

  2. I also made an optimized mover more like the code here: https://github.com/pixijs/bunny-mark/blob/master/src/Bunny.js

You can see my version of your port here: https://github.com/roaminggamer/RG_FreeStuff/tree/master/AskEd/2018/08/bunnymark

Direct download : https://github.com/roaminggamer/RG_FreeStuff/raw/master/AskEd/2018/08/bunnymark.zip

Folders:

PS - Your mover was elegant by the way.  Very nice.

PPS - I posted an HTML build of ‘modified’ here: https://roaminggamer.github.io/RGDocs/pages/html5_tests/tests/bunnymark1/index.html

#16

I’d say the performance of the HTML5 build-target (still maturing) is sub-optimal, but on other build-targets the performance is pretty good.

I suspect if you knock the FPS to 30 in config.lua you will actually see a frame rate increase at higher bunny counts. 

This is because you are halving the calls to all calculations.

#17

keep in mind that html5 is still in beta and barely 4 months old

#18

Some really cool stuff. I built an apk using the latest public release of Corona and @roaminggamer’s optimized version of bunnymark, it got 18-23 fps at 1000 bunnies. I rebuilt the Defold version of the benchmark using their latest version as well to see if anything had changed and for consistency, tested on the same phone right after roaminggamer’s version of bunnymark I still get 12k bunnies at 31 fps using their recommended method of go.animate(), particlesfx could hit up to 15k but wouldnt be a very real approach to drawing sprites. This was tested on my LG V20.

Some side observations, building the android apk was pretty hassle free like it is for defold, I had issues getting gideros to build me one some time ago, and my other observsation is apk size, I believe gideros had the largest one but I forget how big, defold’s bunnymark which comes with the options to choose from 5 different methods of drawing bunnies was only 2.88mb, corona was in the middle at 6.86mb. However from what I understand Corona has a lot more higher level constructs and such, meanwhile defold works a little closer to the metal with less overhead so I think this was to be expected. Benchmarks only tell one side of a story, so there is no better engine here, just engines that are better at different things. 

Still this seems a little slow, I think there could be some possible tweaks made to get more bunnies out of corona.

#19

image sheet/sprite might help… test appears to be a batching nightmare

#20

Yeah, batching is for sure the most required optimization here and I’m sure Defold does using a texture atlas, so at the moment the comparison is not remotely fair.

I’ll give this a try later today and try some additional optimizations, but there’s only so much you can do with such simple code. With regards to the general code of Defold I’d say they’re in general not at a lower level as Corona and the update code for the bunnies isn’t even that optimized (could use an additional local here or there) but then, it might just not have an impact in such a small piece of code calling into engine APIs that many times.

The probably biggest Defold performance advantage is it’s usage of LuaJIT - it’s just a so much faster VM even with a disabled jit compiler (as that’s not allowed on iOS and does have some issues wrt the memory manager on Android).