Recently at the Game Developers Conference in San Francisco, Fat Red Couch CTO, Terry Schussler demoed some impressive features from “A New Home for Charlie”, one of Fat Red Couch’s latest interactive children’s book.
Here’s a full transcript of Terry’s demo.
Terry Schussler: So, basically, A New Home for Charlie, built in Corona, is an app that allows you to both create art, play games with the art, and then actually enjoy an interactive storybook.
So, the first thing that we did was, we created a paint program in Corona that actually is sort of like a junior version of Photoshop. In the program, you can actually drop in different backgrounds from the scenes in the story and then you use different tools – as you can see here – to draw on top of the surface. We also have different colored pens you can use, different styles of stroke, and then we also have a whole bunch of brushes that are like the brushes that you see in Photoshop. So, I can take these and I can add different types of content.
So, another thing that we did is we added little discoverables like the ability to change the brush stroke size or the style of the drawings. So, instead of rotating the brush and stamping it, we actually just slide it along the path.
Then, we also added objects. So, this is using Corona shapes. These objects can be resized, rotated. And then, instead of using just colors, you can also use textures, you can fill the shape with texture, and then you can color tint it to get a certain kind of look.
Using the texture feature allows us to have a repeating texture inside of a dynamically sized and rotated shape and you can make multiples of these, position them around. And we created layers in the paint program so that the drawing layer is separate from the object layers.
You also have stamps. So, these are stamps that are based on character images that are in the storybook. So, this is Charlie the Chocolate Chihuahua and then we have the other dogs – Sammy, Felix, and other characters from the story. And so, these are the characters to be used to create your own scene.
Creating custom puzzles
The images can then be saved. You can save it to your Camera Roll or you can just save it in the internal gallery.
And then, the really neat thing that we let you with the image besides save it off in your Camera Roll and share with your family is that you can go into the Games section, the Play feature, and you can use the image in a jigsaw puzzle component that we built. So, this jigsaw puzzle shows different images. You can see stock images that we packed into the app or images that you’ve created in the art program. And so, here’s the image that we just created. It’s dynamically being tessellated into a puzzle. We also used Corona filter effects to create different types of looks. We even animated the pixelated filter here, for example, to create a more challenging sort of variation of the puzzle. And then, we hit Start and the image is deconstructed and laid out so you can go ahead and do that.
This app is aimed at fairly young children and so we limited the number of pieces that you can deconstruct the puzzle into about 81 or so max. But Corona’s fast enough that we’ve done 10,000-piece puzzles using the exact same code and highly perform and able to, you know, drag and drop the puzzle pieces – little tiny ones – into the puzzle grid.
We’ve built other versions of this that use more and more filter effects using a wobble filter and other types of both animated and unanimated filters and created quite a lot of fun for a jigsaw puzzle experience.
This is a pretty unique experience in a sense that kids can not only use images we provide but also be able to take advantage of the art program and build their own puzzles.
Then, the real hero part of this app is the storybook experience. I’ve been working on storybook apps for about twenty years and, you know, they traditionally have a “read to me” or “read by myself” mode. What we’ve done in here is we actually extended that concept to read, watch, and play. So, you can read the storybook or have it read to you, but you can also watch the storybook as a cartoon or a television show type of experience, and you also have the ability to play and just focus on the interactive aspects of the storybook scenes – so we skip the story, jump you directly into the interactive mode for each scene so you can play with the xylophone fence or shake the tablet and watch acorns fall out of the tree or what-have-you. So, that’s a kind of different opportunity.
Custom animations
On top of Corona, we built our own animation engine in Lua and we leveraged off-the-shelf animation tools to construct the animation sequences and then we convert to data describing those into Lua tables and run those at runtime. So, all of the animation that you see in the storybook scenes – the interactive animations that you see – all of those are actually brought in through our animation in Lua code on top of Corona SDK.
We built an animation platform with quite a variety of functionality, including variable speed frame rate. So, individual objects can run at arbitrary frame rates based on the timer mechanism on Corona rather than being locked to 30 or 60 frames per second.
For example, if we jump into the next scene, I’ll show you, if you take a look at the screen here, there’s a fast-forward feature. When you do that, you basically fast-forward the animation so it runs really fast through what would have normally been a 30-frame-per-second animation sequence, but now it’s just going hyper-speeding through the animation frames at about 900 frames per second.
So, to show you another example, if I go back to the main screen, you’ll see how it speeds up and animates the characters in really fast, so that you don’t have to watch that entire build every time you go back to the main landing page of the application. So, the first time, you’ll see it run at its normal speed. And then, programmatically, we changed the frame rate.
This allowed us to build a lot of functionality like palindromic playback of animation; we can loop it a number of times; we can do special things in terms of having one animation trigger a sound or another animation at the end of its playback; stay onscreen at the end or disappear. So, there’s a lot of really nice sequencing things that we were able to accomplish.
And, instead of having to manually figure out the placement of objects, we used – as I said – an off-the-shelf timeline editor to produce the data structures that we then brought into Corona. So, this really afforded us to build a very sophisticated, very smooth animation experience without writing a lot of code and really letting the animators produce the scenes. So, individual scenes in the storybook were able to be produced in under a day, once the artwork had been created for the elements.
Simplifying user experience
As I mentioned, there’s a lot of other really neat things that we did. So, we simplified the user experience by letting them read, watch, or play. What we’re really doing here is we’re turning on and off individual features and we gave users access to those individual controls along the settings right here. So, if I want, for example, to turn off the text display, I can do that any time I want right there. If I want to turn off the narrator sound, I can do that. So, we’re simplifying the user experience by letting them choose a modality that makes sense – read, watch, and play – but, once they get in and start familiarizing themselves with the user experience, they can tweak it to be exactly what they want.
You can see here, for example, if I turn auto fast-forward on, when I go from one scene to the next, it jumps the user directly to the interactive phase of the scene so that they can tap on objects and have characters animate in response.
I’ll take you also to you build a storybook scene selector. A user can go into a particular scene and then we added nice little animation features like that so you can actually play music on this little xylophone fence that we built.
We also took advantage of features like the shake effects so you can actually shake acorns out of the tree. We actually built one animation of an acorn dropping and then we programmatically manipulated it so that we spawned multiple copies, we’ve randomized their XY offset position, we’ve randomized the rotation effect so they don’t all land in the same spot with the same orientation. We could even change the scale randomly – some could be a little bigger than others.
We even put other objects in there. There’s balls and stuff that will come out and something like a football. It’s sort of random. So, if you shake it enough times, you might hit the motherload of a lot of stuff. And so, there’s little Easter eggs like that – again, leveraging Corona features.
Desktop publishing
Now, this app, we’ve actually been working on producing a Mac App Store version of and we’re pretty much done with that. We’re just doing some final testing on it.
The biggest issues that we had to deal with were dealing with the resize event because we wanted to allow the app to run at an arbitrary window size. So, as you know, designing traditionally for mobile in either a landscape or portrait orientation, in some rare cases, you might do both. Clearly, a storybook app is traditionally going to be in a landscape orientation and, you know, when you resize, your window could be very vertical, it could be very horizontal, it could be all over the place. So, there’s some design characteristics that you have to deal with in terms of how you lay out your elements on the screen so that you fill the screen properly and lay elements on the screen in the appropriate location.
Corona does a lot of work for us in that regard. We added some additional code to calculate positioning of objects to be relative to the edges of the screen so that the controls are always in the upper left corner or relative to the bottom of the screen. Also, to keep the scaling and the positioning of objects consistent so we weren’t changing aspect ratio on animation art, and this allows us to deliver a Mac app version. The experience is actually more enjoyable because, on a big monitor, a 21-inch or 30-inch monitor, you can really dig deep in the storybook experience.
Multilayer bitmap
All the artwork is bitmap but we leveraged Corona’s multilayer bitmap mechanism just using a simple naming convention so that we’re able to actually master our art at 4K resolution then we bring down from that to two different smaller resolution on images and then we pack those pairs into the application so that the application can dynamically fill the screen with the best quality art for their device. And then, Corona, of course, does a lot of that work for you underneath – automatically picking the right assets for you based on the device information.
That was a huge win for us because, actually, we built one animation using one representative set of assets and then let Corona do the rest of the layout work for us by just making sure we had the right assets for the right naming convention in the bundle for the app.
Charles McKeever: I have one question for you and that is are you going to be speaking at CoronaCon?
Terry: I sure hope so! I’ve submitted what I hope is an interesting panel idea. Actually, I don’t remember what it was now but I remember it was a really good one at the time and, yeah, I’m really encouraged to do that. I’ve spoken at a lot of user conferences for Macworld, Macromedia over the years and I really enjoy that so I look forward to having the opportunity again this year.
Charles: I think everybody would benefit from your knowledge and that would be a great session.
Terry: Thank you. I appreciate that.
Download “A New Home for Charlie” and learn more about Fat Red Couch’s catalog of apps.
Promote Your Apps:
Let’s Meet Face-to-Face:
Thank you for watching, listening, and following Corona Geek: