Drum Toy

|
|
Yay! - I just submitted my latest app to the iTunes App Store. I was going to call it iGadd, but I thought that would be a little presumptuous. So in the end, I went for the more straight forward: Drum Toy 1.0
Drum Toy uses an admittedly weird drum machine architecture and concept. You create basic loops using Drum Toy’s Every and Offset knobs, and then feather in the secret ingredient: Probability!

www.youtube.com/v/SXsLtHXegH8

In a way, Drum Toy ‘thinks’ the way drummers do. Find the pocket, lay down the main groove and sprinkle lightly with a few tasty fills or variations here and there to keep things fresh. Drum Toy is set up to force certain beats while leaving others to just the right amount of chance. This simple mechanism yields a surprising array of personality.

Drum Toy - the world’s funkiest practice metronome. Coming soon to the iTunes App Store. |

|
|
|

[import]uid: 56133 topic_id: 13881 reply_id: 313881[/import]

Really impressive stuff :smiley:

I love seeing all the great games Corona users make but it’s awesome to see non-game apps as well; it’s great to remind everyone that we don’t just make games :wink:

Peach

PS - Nicely polished look, too! [import]uid: 52491 topic_id: 13881 reply_id: 51103[/import]

Thanks Peach. Nice of you to say.
Next version, I want to add alternate sounds and convert the channel strips to a class module so I can easily create patches with more than three sounds. [import]uid: 56133 topic_id: 13881 reply_id: 51184[/import]

Hi,

good stuff! the timing on your demo seems quite tight. are you doing anything special to keep the audio in sync other than “audio.play” on a timer?

thanks
j [import]uid: 6645 topic_id: 13881 reply_id: 52322[/import]

I’m use an a listener on enterFrame(event). I compare (system time - the last frame’s system time), and if it’s greater than the number of ticks in a sixteenth, then I play the audio. Otherwise, I prepare the material for the next sixteenth.

The problem with Corona in this regard is the max frame resolution is 60fps, so there’s a limit. I haven’t figured out a way to get every tempo, only certain ones where the sixteenth value falls within a fifteen tick range. So you’ll notice that the tempo goes from 94, 98, 102, 108…(I’m making these up, but you get the idea) I haven’t figured out a way to get better tempo resolutions.

But Corona is so fast to code in, and I wanted something I could play along with that would do this, so it was a quick and dirty hack that turned out to be very usable and useful.

I’m hoping in the future Ansca will provide a midi API so i can control other devices and perhaps tighter access to the audio buffers or clock cycles so I can improve the resolution without having to recode everything in objC.

Of course - If anyone knows any tricks that I can use to optimize the algorithm, that would also be greatly appreciated. [import]uid: 56133 topic_id: 13881 reply_id: 52525[/import]

great thanks for the reply.

i’ve discussed this sort of thing here http://developer.anscamobile.com/forum/2010/12/21/timing-trouble and I think i put in a feature request somewhere, but I don’t think it would happen for a while, especially given android audio issues

j [import]uid: 10744 topic_id: 13881 reply_id: 52530[/import]

maybe the timers have better resolution than 1/60 second!?
could you provide some promo codes?
[import]uid: 70114 topic_id: 13881 reply_id: 52532[/import]

That’s a freaking essay!

As it’s almost 6am I’m not even going to attempt reading it tonight but I did have a skim and look forward to a proper read tomorrow.

Clearly a huge amount of effort went into it, I’m very impressed :slight_smile: [import]uid: 52491 topic_id: 13881 reply_id: 85284[/import]

I have a solution to your problem!

This allows you to play in sync with any tempo-- even fractional tempos. I’ve used the technique on much less powerful hardware and it’s worked perfectly with no drift of tempo (tempo never goes out of phase to use a more technical terminology).

What you were doing “wrong” was to compare the current system time to the timestamp of the last beat. This is because the last beat will always be a little “off” for several big reasons I can go into in another post.

Instead, you should be always calculating forward, with high precision floating point math, from the system time at which the play button was originally pressed and triggering the code for the next beat when the current system time passes the value of the system time when play button was pressed PLUS tempo_in_milliseconds TIMES beat_number.

So you’ll need some variables (lua will automatically type float point variables where they are needed, so don’t worry about this):

  1. one to store when the play button was pressed, we’ll call this “time_started”.

  2. the other variable we’ll use to calculate the system time we have to wait for to play the next beat at, lets call it “next_beat”

  3. to keep track of which beat number we’re on we can introduce a third variable, “counter” that counts the beats and initialize it to “1”

4, and finally we’ll need a variable that stores the tempo in milliseconds between beats, “tempoMS”, which we’ll initialize to 600000/tempo
where tempo is whatever bpm value you wish, e.g. 97

Now after calulating tempoMS by the formula 600000/tempo what we’d have as a formula for the system time value of the any given beat is:

next_beat=time_started+(counter*tempoMS)

when that time has passed, we simply increment the counter and wait til the next beat.

Using the time_started value as our perfect anchor point (not the stamp of the last beat played which will be an imperfect anchor point), and using that to calculate forward in the way I just described your tempo will stay in phase and remain in perfect phase til the battery runs dead or the floating point math precision weakens by a tiny amount over time.

Now, the caveat with this method is that you need to check that some hiccup in the OS or program hasn’t introduced a period of time which puts us behind by more than a beat… see the comment “–integrity check” in the code sample.

Now, to illustrate what I’ve been saying, try this code out that outputs to the console… first set the config.lua to output at 60fps:

application =  
 {  
 content =  
 {  
 fps = 60,  
 },  
 }  

Now save this audio file I have on Dropbox.com, tick_44.wav, http://db.tt/5qkdbPQO to your project directory.

try this code in main.lua:

[code]local tempo=95.5 – a pretty wierd tempo, to illustrate my point :slight_smile:
local tempoMS=60000/tempo/4 – for 16ths at 95.5bpm we’ll divide by 4… for normal quarter-note beats, remove the “/4” from the end of the line

local click = audio.loadSound(“tick_44.wav”)
local timestarted=system.getTimer()
local next_beat=timestarted+tempoMS

print (“tempoMS (beat length): “…tempoMS…”\n”)

local counter=1;

local function checkBeat()
local thetime = system.getTimer()
if thetime>next_beat then
audio.play(click)
print (“only off target ticks by “…math.floor(thetime-next_beat)… “ms”) --this is the amount of milliseconds we are off the perfect system time target that will never go out of phase
print (“timestarted: “…timestarted…” thetime: “…thetime…” next_beat:”…next_beat…”\n”)

–Begin playback integrity check
if thetime>next_beat+tempoMS then – i.e. we are behind two beats because of some kind of overload
print(“OH NO! Tempo has gone out of phase due to an interruption in the runtime or perhaps the audio engine, etc”)
–playback could be halted here, too
end
–End playback integrity check

counter=counter+1
next_beat=next_beat+tempoMS
–the above line could also be expressed as next_beat=timestarted+(counter*tempoMS) to find the timing for a specific beat number, the number of which is represented here by “counter”
end
end

Runtime:addEventListener(“enterFrame”,checkBeat)

–timer.performWithDelay(1,checkBeat,0)
–this should work better in theory, but doesn’t. comment out the Runtime:… line above if you try this method over the enterFrame

[/code]
Outputs this to simulator:

[code]Windows simulator build date: Dec 9 2011 @ 14:01:29
Copyright © 2009-2011 A n s c a , I n c .
Version: 2.0.0
Build: 2011.704
tempoMS (beat length): 157.06806282723

only off target ticks by 14ms
timestarted: 78 thetime: 250 next_beat:235.06806282723

only off target ticks by 13ms
timestarted: 78 thetime: 406 next_beat:392.13612565445

only off target ticks by 12ms
timestarted: 78 thetime: 562 next_beat:549.20418848168

only off target ticks by 11ms
timestarted: 78 thetime: 718 next_beat:706.2722513089

only off target ticks by 11ms
timestarted: 78 thetime: 875 next_beat:863.34031413613

only off target ticks by 10ms
timestarted: 78 thetime: 1031 next_beat:1020.4083769634

only off target ticks by 9ms
timestarted: 78 thetime: 1187 next_beat:1177.4764397906

only off target ticks by 8ms
timestarted: 78 thetime: 1343 next_beat:1334.5445026178

only off target ticks by 8ms
timestarted: 78 thetime: 1500 next_beat:1491.612565445

only off target ticks by 7ms
timestarted: 78 thetime: 1656 next_beat:1648.6806282723

only off target ticks by 6ms
timestarted: 78 thetime: 1812 next_beat:1805.7486910995

only off target ticks by 5ms
timestarted: 78 thetime: 1968 next_beat:1962.8167539267

only off target ticks by 5ms
timestarted: 78 thetime: 2125 next_beat:2119.8848167539

only off target ticks by 4ms
timestarted: 78 thetime: 2281 next_beat:2276.9528795812

only off target ticks by 2ms
timestarted: 78 thetime: 2437 next_beat:2434.0209424084

only off target ticks by 1ms
timestarted: 78 thetime: 2593 next_beat:2591.0890052356

only off target ticks by 1ms
timestarted: 78 thetime: 2750 next_beat:2748.1570680628

only off target ticks by 0ms
timestarted: 78 thetime: 2906 next_beat:2905.2251308901

only off target ticks by 15ms
timestarted: 78 thetime: 3078 next_beat:3062.2931937173

only off target ticks by 14ms
timestarted: 78 thetime: 3234 next_beat:3219.3612565445

only off target ticks by 13ms
timestarted: 78 thetime: 3390 next_beat:3376.4293193717

only off target ticks by 13ms
timestarted: 78 thetime: 3547 next_beat:3533.497382199

only off target ticks by 12ms
timestarted: 78 thetime: 3703 next_beat:3690.5654450262

only off target ticks by 11ms
timestarted: 78 thetime: 3859 next_beat:3847.6335078534
[/code]
and so on… there really isn’t a problem with any tempo from 120.0 bpm to wierd tempos like 95.5 bpm… or even 134.563 bpm.

And the reference beat time value, next_beat at which we trigger the next beat will never go out of phase with the pulse of the ideal tempo any more than the processor hardware and floating precision math of corona will allow.

Each discrete beat could be off from this perfect target ticks value by 16.7 milliseconds (1000 ticks per second/60fps) or less. This is still an issue. Carlos: Can Corona please be given the ability to trigger audio and perhaps some simple display events at a much higher granularity? Especially when not much else is going on?

In spite of a minor delay of each beat from approx 0 to 16.7 milliseconds, with this method at least the tempo pulse itself from which each beat is triggered will be rock solid. Always!

Hope this helps!

I look forward to your reply/replies! [import]uid: 106887 topic_id: 13881 reply_id: 84062[/import]

Thanks Peach,

See my additional comments here on this related thread by another developer wanting to make a music app. http://developer.anscamobile.com/forum/2012/01/14/fastestmost-accurate-way-trigger-sounds-drum-sequencer#comment-85249

I hope something can be done by Ansca to help all of us who are making music-based apps to improve the granularity at which we can trigger audio events.

Another solution would be to enable us to write to the audio buffer and schedule the “mix-in” of sound samples at a specific sample number along a continuous audio stream of 44100 samples a second. That would be awesome. Can it be done?

Thanks for replying to my thread Peach!

Cheers to all at Ansca!

Gary [import]uid: 106887 topic_id: 13881 reply_id: 85323[/import]

Great stuff. Thanks Gary. [import]uid: 29093 topic_id: 13881 reply_id: 85661[/import]

I’m curious if you have any specific recommendations or performance (Instruments) traces that would point to areas we could reasonably optimize.

The kind of precision timing you ask for is very hard to do in general, let alone be cross-platform. Typically, for very precise audio timing, it would often be suggested that on iOS or Mac, you deal directly in C and the lowest level Core Audio APIs because even Objective-C would be too slow/unreliable.

In Corona, on Mac/iOS, the event system is fairly tied to Cocoa/Obj-C (as with just about all Mac/iOS apps). So even though your app might not be doing anything, Cocoa may actually be doing things with the event loop which still take minor, but additional amounts of time. And because some services live on separate threads, there is also thread switching going on behind the scenes which also leads to unpredictable times. And because you are going through Lua, that is yet another layer which will incur minor, but additional overhead.

Apple does provide some low-level callbacks on special high priority threads for things that need very precise timings, but there are a lot of limitations and rules about using these. And because these do not live on the main thread, there are additional complications/restrictions as well. In Corona, we always make Lua callback on the same thread. This protects you from concurrency problems which can be very complicated. And requiring us to lock the Lua VM on every operation would also incur more overhead.

That is just the Apple side. On Android, I would say you are SOL for now.
As for your for 17ms margin of error you show above. Here’s one idea (which might be bad so use carefully). When you enter your callback with 17ms or less time to your next beat, create a busy loop that blocks execution until you get closer to your timed event, e.g.

local system\_getTimer = system.getTimer -- make a local variable for faster access since we are in a time critical loop  
busy\_wait = true  
while busy\_wait do  
 local new\_time = system\_getTimer()  
 if new\_time \> ready\_time then  
 busy\_wait=false  
 end  
end  

This is generally bad because you are blocking the main loop and preventing the app and OS from doing useful things (and Apple’s watchdog process will kill your app if you block for too long). However, for very short amount of times, this may acceptable and give you slightly more timing precision.

[import]uid: 7563 topic_id: 13881 reply_id: 85700[/import]

Ewing,

A whole-hearted thank you for this response!

I have to say I did tinker with that very idea of hogging cycles in a busy loop but didn’t because I thought it would do more harm than good. Still may try it and let you know how it goes.

That said it would really be nice if performWithDelay could trigger non-display-updating functions at a higher frequency than 16 ms… down to every ms or at least every 4ms would help a lot. 10ms would be a nice compromise too! Can we make this happen? It is really odd that performWithDelay takes the argument of 1ms, but only triggers at 60fps/16ms.

Cheers,

Gary
[import]uid: 106887 topic_id: 13881 reply_id: 85720[/import]

I’ve been chatting with Gary in part about my music/rhythm game “Beat Boxes” and whether I learned anything from making that game that could help with some of the precision audio timings discussed here. In fact, I measured out all of the “beats” in my game exactly as Gary described in his essay of a post, always calculating forward from the start of the track instead of from the previous beat, which would quickly get out of sync if you deviated from the subset of possible beat timings that robby used.

In the case of working within the 60FPS limitations of Corona, I think it should be possible in theory to create multiple sound files with short periods of silence in front of them, ranging from 0-15ms, and then choose the correct sound file based on the calculated delay before the next beat should sound. This would theoretically give you perfect ms granularity without any changes to the SDK from Corona (though it’s a kind of hacky workaround, and of course it would be nice if we could access real ms-precise timings directly in Corona).

Gary suggested that my idea could be taken a step further (and save a lot of time building different sound files in the process!) by having a single sound file with 16ms of silence in the front of the file, and then just “seek” to the right part of the track based on how many ms it would be until the next beat should sound. Then you still only need one sound file, and you can be sure that the sound plays exactly when it should, even if a frame refresh doesn’t occur in that exact moment.

A huge word of caution to anyone dealing with precise timings like this though: the iPhone screen actually refreshes every 16.6666667ms, giving it a precise 60FPS update cycle. The Corona simulator, on the other hand (at least on my system) actually updates frames every 16ms exactly, which is close to but not exactly 60FPS (if you’re running at 30FPS the same caveat applies, except the numbers are 33.33333ms and 32ms). So make sure you’re testing everything on your device, you’ll save a lot of potential headache. (Trust me, I’ve been there).
[import]uid: 14598 topic_id: 13881 reply_id: 86716[/import]

Well done, great creative problem solving guys.

Can I ask though, did you try the single padded track + seek method discussed above? Seems a lot better than having 17 different versions of the same sound so I’m guessing you tried and it failed.
[import]uid: 29093 topic_id: 13881 reply_id: 86836[/import]

Thanks, spideri!

And thanks again to Alan.

And, thanks to Ancsa’s staff for showing an interest in this thread.

At some point down the road I will definitely test the seek solution I proposed. But, I’m sticking with the multiple files method for now because it worked beautifully at a reasonable cost of 23KB for the 16 extra “tick” audio files and bears the advantages of requiring fewer lines of code and no extra function calls to complete (e.g. seek and channel-related) before the file plays.

It’s also a simpler solution (doesn’t require a black box seek function to work as expected at a high level of granularity, reliably across multiple devices).

I will experiment with seek at a later date, but right now I have much more need to getting back to refining the user interface.

That said, I would certainly welcome data from anyone testing the accuracy of the seek method vs the multiple audio files method. Here is an archive of my 17 tick audio files http://db.tt/GSCANdVc

Cheers!

Gary [import]uid: 106887 topic_id: 13881 reply_id: 86898[/import]

Yes!! It works! It totally smooths out the audio***. Now, I’ve got a pulse going at virtually any tempo, including 97.5 or any other tempo… I ended up using 17 different files for my “tick” sound: one with no delay, and 16 other files with from 1 to 16 milliseconds delay.

I loaded them up into a table variable, and I find the index number i want to play (which corresponds to the number of milliseconds I need padded) with this line in my checkbeat function which I call every enterframe.

local index=math.max(0,16-(math.floor(thetime-next_beat)))

where next_beat is the time in float milliseconds I’m waiting for the system time (stored in thetime) to pass.

I actually pad each beat so each beat sounds as close to exactly 20ms late past the target time that I can. The result is that all my audio is now way more evenly spaced. It’s amazing how much difference eliminating (really compensating for) 16 or less milliseconds of error makes…

Works great. And I can finally concentrate on rest of my UI work for my app.

Awesome. I love the feeling of support here in the Corona forums. There are a lot of inspired and creative minds here!

Cheers,

Gary

***UPDATE… see my second post from Feb 14. [import]uid: 106887 topic_id: 13881 reply_id: 86764[/import]

Thanks for sharing your results. That is very impressive. It’s wonderful to see that you were able to one-up a native app using Corona.

I would be interested in hearing your seek results if you do them.
[import]uid: 7563 topic_id: 13881 reply_id: 87142[/import]

***Update:

My musician’s ear could hear it was smoother. But I just did a test in an audio editor using recorded output from my old app version and the new version with the different offset tick wav files.

The result is that in the old version each tick plays up to 17 milliseconds early or late from the perfect pulse. In the new version, the tick plays up to 10.5 milliseconds early/late.

This is a 38% improvement, which I am very happy with.

I guess what this means is that for various reasons, probably code execution speed differences and/or latency in the interface to the audio engine itself accounts for latency beyond any control.

The main and most important part still is that the tempo pulse against which each beat is triggered is rock solid thanks to the tempo pulse calculation algorithm I presented in my “essay” in my initial post to this thread.

UPDATE #2: In fact, the tempo pulse for my metronome matches and might exceed that of the most accurate metronome I’ve seen for any mobile platform thus far, which I respectfully won’t name here, because though the beats are super-evenly spaced it does drift very slightly-- it drifted 6 ms after 3mins 27secs.

I was amused that a recently released and highly publicized metronome on the app store made an audacious and false claim in their app description that “No other metronomes on the app store surpass our accuracy.” Well, using a waveform editor to examine the output, I noticed that after 3 minutes at 97BPM (a tempo value that requires float precision, sub-sample accuracy), the tempo pulse of this well publicized metronome app had drifted out by 22 milliseconds, while the pulse of the former app was at least less off, and my app in development though with a jittery beat trigger, has a pulse that should remain spot-on while the other apps drift gradually away from the ideal pulse.

What’s confusing is that both of these other apps are native apps that write directly to the audio buffer using very low level audio APIs. Yet they are both still off (even though one is more ‘on’ than the other, as mentioned). Could it be that the the os+hardware combination that processes the audio buffer in the iPhones, is not perfect timing-wise!? Crazy… if this is the case, then samples in the audio buffer on my iPhone 3GS can’t be relied on to reconcile against the system clock. Integrity checks against the system clock might be needed to maintain accuracy!

That just goes to show that using the low-level audio API doesn’t mean their app is going to be the most accurate. Careful and insightful programming is going to be more important.

This probably applies to many other areas in debates over using low level frameworks versus corona: it often amounts to how intelligent the business logic and problem solving in your analysis and implementation is, and not so much how low level you go, or for that matter, what framework or SDK you use!

Cheers! [import]uid: 106887 topic_id: 13881 reply_id: 86927[/import]

Hi Gary,

What an exciting read! Thanks so much for posting your progress! I will definitely try the seek method and report back. Corona Drum Sequencer ahoy!
Cheers
Alex [import]uid: 66618 topic_id: 13881 reply_id: 93414[/import]