Children's Books - read along ability and interactvity

Hi there,

We publish songs, textbooks and stories for young kids.  We have been struggling for some time to get the read-along capability to happen using iBooks Author, Flash and presentation programs - without success.  It’s a simple click each word to hear the pronunciation goal we have in mind.

It has always been lurking in our minds that programming skills would be the only way to make our interactive storybook dreams come true.

I’ve been using Flash to test this out and I can get the desired interactivity when exported as a swf file or bundled with AIR.  I used Flash CC Pro and used their HTML 5 template.

Here’s what i get so far but I can’t get it to work the same on the iPad.  This is raw interactivity/functionality (no graphics or anything else yet)   http://www.stevehanlonmusic.com/CanvasTest_WordAudio.html

(Also, notice the fuzziness of the words when turned into Buttons.  Kind of strange to lose the sharpness of the words.)

So, we came back to Corona.  And we’re hoping to push aside the fear of programming and learn how to do this.

Can this be done with Corona?  Can we make interactive storybooks that have these features:

  1. Clickable single words to hear back audio files for pronunciation (like in the linked example above)

  2. Clickable buttons to hear the story read to you

  3. Clickable animations inserted as movie files integrated with still drawings

  4. Clickable music files for listening and singing along (same as #1 but just longer files)

  5. A writing journal for the kids to answer questions that the book asks them. The ability to store their answers and change them.

  6. Ability to record one’s voice reading the story back (per page) and have it stored.

It’s a tall order but it’s what we’d like to accomplish.

Can we do this with Corona?  Is an upgraded version needed?

Thanks for reading and helping us out.

  1. Clickable single words to hear back audio files for pronunciation (like in the linked example above)

   Yes, this is do-able in Corona.  The gist of it is that you’d have a event handler registered for the text-box that contains the word, then the event handler function would play the audio (there may be issues with text object handling touch events, you might want to create a box around the text and make that touch-able).

 

  1. Clickable buttons to hear the story read to you

   Yes, this is do-able.  As above, you’d probably have a “Read-it-all” button with an event handler.  If you want that handler to trigger page-turn kinds of events, you’d need to do a bit more work.

 

  1. Clickable animations inserted as movie files integrated with still drawings

   Yes, this can be done in Corona.

 

  1. Clickable music files for listening and singing along (same as #1 but just longer files)

   Yes, there isn’t any real issue with length of an audio file (you can stop and pause audio).

 

  1. A writing journal for the kids to answer questions that the book asks them. The ability to store their answers and change them.

   Yes, technically you can do this – but let’s just say text input is not Corona’s strongest point (saving the data to files is easy).  You can pop-up a keyboard and read characters into a text-box, but it isn’t very pretty (and some debugging will have to be done on devices, not on the simulator, so it can be a bit slower to develop).

 

  1. Ability to record one’s voice reading the story back (per page) and have it stored.

   Yes, this can be done, though I’ve never done it myself so I’m not sure how easy it is (though the media.* set of SDK functions make it look pretty straightforward).

(and all of this is do-able with the Basic and Starter editions of Corona, no real need for any of the higher functionality)

@rshanlon: This is probably a “Down the road”, someday, kinda thing for you… but something to keep in the back of your mind as you move ahead I think…

I’d agree with jbp1, and add on top of that, if you’re looking at doing a living book type “series” of products, in addition to developing all the bits of code that make all individual elements happen (like #1-5 above), you might want to develop a little scripting system so you can throw together entire books quickly (or download small data files that are new stories on occasions), by sort of filling in the details on data forms / in text files. (and a Corona App you develop would be the “engine” that reads the files, and tells the stories).

To do the page flipping / story telling for one part of an app that uses “pages” of similar sequences (tutorials spread around a product) I used a lua table, and lua’s ability to read in text, and decode it back into a table.

So this is a little piece of a script that puts up a background, shows a couple animations on top, and has a link to a “next” page / screen. In the case of these tutorials, one panel would be shown at a time until the user tapped somewhere, then the “next” file would be read and processed (which shows a new panel).

After the engine was done, putting together new tutorials that fit the same scheme was having an artist generate content, and then practically filling in forms. Keep in mind of course, this approach works best when you have many products that use the same “template” (You can save a LOT of time, and bugs over many products this way). On top of that, once the system is created, coders can get back to doing generic coding, and a Production Assistant or Editor can put together the stories.

However, if the pages / books functionally vary a lot, it might be more trouble than its worth. Unfortunately, experience coding helps to know when developing a Content Management System (as I’ve seen them called) might pay off, and when it’s likely to be an unending uphill battle. (And you don’t have a lot of experience making these kind of coding trade-offs).

Anyways, to give an idea of what putting page together with a system like this would look like

[lua]

{
“background”: “Atelier/TutorialBack01.png”,
“objects”:[
{“overlay”:“Atelier/Tutorial/tutorial01.png” },
{“burst”: 1, “x”:480, “y”:320 },
{“bird”: 1, “x”:714, “y”:236 },

{“text”: “Welcome to the Atelier.”, “x”: 480, “y”:860 },
{“next”:“Atelier/Tutorial/Tutorial2.txt” }
]}

[/lua]

Your main app then includes the data file reader / processor. Planning carefully, using pre-fab characters, you could even have your main app include all the stock shots of objects / characters, and then just download little data files on occasions, that contained new stories (new text data files), but re-used the art/audio already built into in the app. Anywho…

PS: A little bit of code to read in a data file like above

[lua]

– loadTable() loads in a table from a file (filename), and returns it (nil on error)
function loadTable(filename)

    local path = system.pathForFile( filename, system.DocumentsDirectory)  – Where I keepa my tables
    local file = io.open( path, “r” )
    
    if file then
         – read all contents of file into a string
         local contents = file:read( “*a” )
         local myTable = Json.decode(contents)
         io.close( file )
–         print(" – load success.")
         return myTable
    end

    print(" – load fail, no pre-existing file.")
    return nil
end

[/lua

PSS: Developing the Corona engine to read and process these scripts, display animations, text, backgrounds, go forward and back between pages/scripts… Took about 2 weeks. From there, it was all gravy as PA’s could throw pages together, and test them immediately (using the corona simulator, installed on their machines).

Thanks for the replies both of you.  With exception of the Corona not being great at text input - this does sound very promising to use this SDK.

mpappas, this is in fact what we wish to do - streamline the process leading to a Content Management System since our world and characters will be the same and overall content (graphics, animations as movie files and audio files of different lengths) will be consistent from book to book.  At least we think and hope so.

Thanks for sharing some of the code - I’m a newbie so it’s all Greek to me at this point.  But after getting through some of the Getting started area and more, we’ll see if we’re cut out for programming or we end up being the “production assistant” you mention - which sounds more reasonable but having control of our apps/books is the dream.  We’ll see.

By the way, you mention tutorials - are those tutorials located here at the corona website or elsewhere?

It’s a great (read as best on the market) fit for 2, 2.5d, and video oriented apps (with the next public release) for sure. It’s got a best in class cross graphics/audio compatibly, event system model, graphics system model, virtual screen system, etc, etc. About the only major thing to whine about is 3D at this point, and *maybe* windows phone, but it’s looking like that’s no big loss (might as well whine about blackberry, haha).

CoronaLabs tuts are at: http://coronalabs.com/resources/tutorials/getting-started-with-corona/

Some others can be found at:

http://learningcorona.com/

Best of luck, and make sure to search the forums, a lot of the step by step questions have already been answered (make sure to tackle it one step at a time *after* you map out the big picture).

Oh, great!  I didn’t know you were referencing the very tuts here at Corona.  Can’t wait to try them!

You’ve been incredibly helpful!  Actually, I was just telling another person on this forum how impressed I am at how helpful people in this forum are.  Forums can be minefields at times :ph34r: .  It my first time to hang out at a coders forum.   :slight_smile:

You might want to check out the Kwik program that runs on Corona. It’s a neat little thing set up as very usable interface specifically for book authors. You can find it in the 3rd Party tools list on this site or go directly to http://kwiksher.com and check it out there. It does the word highlighting you’re looking for and much more without programming. If you find you’re wanting something that does require programming, it allows for that too. It’s also an addon to PhotoShop and you will need a copy of PS 5 or higher. Try the Creative Cloud if you don’t have a retail copy of PS. Creative Cloud is an affordable monthly plan that gives you access to all the latest versions of most Adobe products, including PS. 

If you go the Kwik route be sure to hit the forums BEFORE purchase. The most up to date version only works with Pro and higher versions of Corona, because of the daily Corona Builds, so if you have a starter Corona version (as your forum tag indicates) you will need an older version of Kwik as we wait for some major Corona changes to hit the public release.

Thank you for the heads up about Kwik (I have Creativ Cloud so it would work for us).  I looked at a few YouTube tutorials for Kwik.  It may be a route we take but honestly I really want to get Lua down to some extent.  I just feel like I keep shortchanging myself by not getting some code under my fingers.

And like you said, Kwik still allows for code entering.  So by getting some Lua experience it may all work out for the better.  We may in fact end up with Kwik but some ability to code as well.

I’m really taking to Peach Pellen’s tuts now.  Easy to follow!  Wish us luck!  Thanks again for the Kwik suggestion!

  1. Clickable single words to hear back audio files for pronunciation (like in the linked example above)

   Yes, this is do-able in Corona.  The gist of it is that you’d have a event handler registered for the text-box that contains the word, then the event handler function would play the audio (there may be issues with text object handling touch events, you might want to create a box around the text and make that touch-able).

 

  1. Clickable buttons to hear the story read to you

   Yes, this is do-able.  As above, you’d probably have a “Read-it-all” button with an event handler.  If you want that handler to trigger page-turn kinds of events, you’d need to do a bit more work.

 

  1. Clickable animations inserted as movie files integrated with still drawings

   Yes, this can be done in Corona.

 

  1. Clickable music files for listening and singing along (same as #1 but just longer files)

   Yes, there isn’t any real issue with length of an audio file (you can stop and pause audio).

 

  1. A writing journal for the kids to answer questions that the book asks them. The ability to store their answers and change them.

   Yes, technically you can do this – but let’s just say text input is not Corona’s strongest point (saving the data to files is easy).  You can pop-up a keyboard and read characters into a text-box, but it isn’t very pretty (and some debugging will have to be done on devices, not on the simulator, so it can be a bit slower to develop).

 

  1. Ability to record one’s voice reading the story back (per page) and have it stored.

   Yes, this can be done, though I’ve never done it myself so I’m not sure how easy it is (though the media.* set of SDK functions make it look pretty straightforward).

(and all of this is do-able with the Basic and Starter editions of Corona, no real need for any of the higher functionality)

@rshanlon: This is probably a “Down the road”, someday, kinda thing for you… but something to keep in the back of your mind as you move ahead I think…

I’d agree with jbp1, and add on top of that, if you’re looking at doing a living book type “series” of products, in addition to developing all the bits of code that make all individual elements happen (like #1-5 above), you might want to develop a little scripting system so you can throw together entire books quickly (or download small data files that are new stories on occasions), by sort of filling in the details on data forms / in text files. (and a Corona App you develop would be the “engine” that reads the files, and tells the stories).

To do the page flipping / story telling for one part of an app that uses “pages” of similar sequences (tutorials spread around a product) I used a lua table, and lua’s ability to read in text, and decode it back into a table.

So this is a little piece of a script that puts up a background, shows a couple animations on top, and has a link to a “next” page / screen. In the case of these tutorials, one panel would be shown at a time until the user tapped somewhere, then the “next” file would be read and processed (which shows a new panel).

After the engine was done, putting together new tutorials that fit the same scheme was having an artist generate content, and then practically filling in forms. Keep in mind of course, this approach works best when you have many products that use the same “template” (You can save a LOT of time, and bugs over many products this way). On top of that, once the system is created, coders can get back to doing generic coding, and a Production Assistant or Editor can put together the stories.

However, if the pages / books functionally vary a lot, it might be more trouble than its worth. Unfortunately, experience coding helps to know when developing a Content Management System (as I’ve seen them called) might pay off, and when it’s likely to be an unending uphill battle. (And you don’t have a lot of experience making these kind of coding trade-offs).

Anyways, to give an idea of what putting page together with a system like this would look like

[lua]

{
“background”: “Atelier/TutorialBack01.png”,
“objects”:[
{“overlay”:“Atelier/Tutorial/tutorial01.png” },
{“burst”: 1, “x”:480, “y”:320 },
{“bird”: 1, “x”:714, “y”:236 },

{“text”: “Welcome to the Atelier.”, “x”: 480, “y”:860 },
{“next”:“Atelier/Tutorial/Tutorial2.txt” }
]}

[/lua]

Your main app then includes the data file reader / processor. Planning carefully, using pre-fab characters, you could even have your main app include all the stock shots of objects / characters, and then just download little data files on occasions, that contained new stories (new text data files), but re-used the art/audio already built into in the app. Anywho…

PS: A little bit of code to read in a data file like above

[lua]

– loadTable() loads in a table from a file (filename), and returns it (nil on error)
function loadTable(filename)

    local path = system.pathForFile( filename, system.DocumentsDirectory)  – Where I keepa my tables
    local file = io.open( path, “r” )
    
    if file then
         – read all contents of file into a string
         local contents = file:read( “*a” )
         local myTable = Json.decode(contents)
         io.close( file )
–         print(" – load success.")
         return myTable
    end

    print(" – load fail, no pre-existing file.")
    return nil
end

[/lua

PSS: Developing the Corona engine to read and process these scripts, display animations, text, backgrounds, go forward and back between pages/scripts… Took about 2 weeks. From there, it was all gravy as PA’s could throw pages together, and test them immediately (using the corona simulator, installed on their machines).

Thanks for the replies both of you.  With exception of the Corona not being great at text input - this does sound very promising to use this SDK.

mpappas, this is in fact what we wish to do - streamline the process leading to a Content Management System since our world and characters will be the same and overall content (graphics, animations as movie files and audio files of different lengths) will be consistent from book to book.  At least we think and hope so.

Thanks for sharing some of the code - I’m a newbie so it’s all Greek to me at this point.  But after getting through some of the Getting started area and more, we’ll see if we’re cut out for programming or we end up being the “production assistant” you mention - which sounds more reasonable but having control of our apps/books is the dream.  We’ll see.

By the way, you mention tutorials - are those tutorials located here at the corona website or elsewhere?

It’s a great (read as best on the market) fit for 2, 2.5d, and video oriented apps (with the next public release) for sure. It’s got a best in class cross graphics/audio compatibly, event system model, graphics system model, virtual screen system, etc, etc. About the only major thing to whine about is 3D at this point, and *maybe* windows phone, but it’s looking like that’s no big loss (might as well whine about blackberry, haha).

CoronaLabs tuts are at: http://coronalabs.com/resources/tutorials/getting-started-with-corona/

Some others can be found at:

http://learningcorona.com/

Best of luck, and make sure to search the forums, a lot of the step by step questions have already been answered (make sure to tackle it one step at a time *after* you map out the big picture).

Oh, great!  I didn’t know you were referencing the very tuts here at Corona.  Can’t wait to try them!

You’ve been incredibly helpful!  Actually, I was just telling another person on this forum how impressed I am at how helpful people in this forum are.  Forums can be minefields at times :ph34r: .  It my first time to hang out at a coders forum.   :slight_smile:

You might want to check out the Kwik program that runs on Corona. It’s a neat little thing set up as very usable interface specifically for book authors. You can find it in the 3rd Party tools list on this site or go directly to http://kwiksher.com and check it out there. It does the word highlighting you’re looking for and much more without programming. If you find you’re wanting something that does require programming, it allows for that too. It’s also an addon to PhotoShop and you will need a copy of PS 5 or higher. Try the Creative Cloud if you don’t have a retail copy of PS. Creative Cloud is an affordable monthly plan that gives you access to all the latest versions of most Adobe products, including PS. 

If you go the Kwik route be sure to hit the forums BEFORE purchase. The most up to date version only works with Pro and higher versions of Corona, because of the daily Corona Builds, so if you have a starter Corona version (as your forum tag indicates) you will need an older version of Kwik as we wait for some major Corona changes to hit the public release.

Thank you for the heads up about Kwik (I have Creativ Cloud so it would work for us).  I looked at a few YouTube tutorials for Kwik.  It may be a route we take but honestly I really want to get Lua down to some extent.  I just feel like I keep shortchanging myself by not getting some code under my fingers.

And like you said, Kwik still allows for code entering.  So by getting some Lua experience it may all work out for the better.  We may in fact end up with Kwik but some ability to code as well.

I’m really taking to Peach Pellen’s tuts now.  Easy to follow!  Wish us luck!  Thanks again for the Kwik suggestion!