Out of curiosity what Windows audio Apis is Corona using and / or available to?
On WP8, we’re using DirectX’s “XAudio2” audio system. This is Microsoft’s top of the line audio system that they recommend for gaming. XAudio2 supports all of the features needed by Corona’s “audio” Lua library, such as streaming audio, setting the volume/gain on individual channels, controlling the pitch of each channel, etc.
We’ve written our own OpenAL library from scratch that bridges the cross-platform OpenAL C APIs to Microsoft’s XAudio2 system. It works a lot like how Apple’s open source OpenAL library works, which also bridges OpenAL’s cross-platform C APIs to Apple’s proprietary AudioUnit library. Now, we did this so that Corona developers can use *most* of our hidden OpenAL Lua APIs, such as things like “pitch”. Note that I say most because our OpenAL implementation doesn’t support positional sound, which is a corner we cut to get Corona for WP8 into everyone’s hands as soon as possible.
Thats cool. Ive been testing out the Xaudio2 system and it did look the best one. I am planning on making a microphone input monitor and fingers crossed some thing that might allow submixing or routing audio. I think that I will need WASAPI. Would this work in conjunction. I would ideally like to monitor whatever the audio is playing (not the microphone but whats generated by the phone/app as a mix possibly or select channels) .
DO you think I will need to make something from scratch or is there a way I can modify the current Xaudio audio system Corona is using?
WASAPI can definitely do this. In fact, I think it’s your best option for microphone “audio capture”. I’ve played with it briefly in the past and it involves you creating a C++/CX “Windows Runtime Component” library project in Visual Studio. A pure native C/C++ library won’t be able to access the WinRT APIs you need to implement this. At least that’s how I remember it.
If you intend to play/output audio via WASAPI too, then I wonder if you might end up conflicting with Corona’s usage of XAudio2. I say this because I’m pretty sure XAudio2 is a layer on top of WASAPI, because XAudio2’s job is to mix audio channels to a single stream and then output that 1 stream to WASAPI. If you intend to play audio on your end as well, outside of Corona, then perhaps the safest solution is to create another instance of the XAudio2 engine, which I believe Microsoft supports (I haven’t tried it). I would assume Microsoft mixes the audio between all XAudio2 engines to a single stream and outputs it to WASAPI at the end. I’m completely guessing here, but it seems to make sense in my mind.
Does this help?
Yeah kind of. I had read about creating a Runtime Component LIbrary. Ive seen a few examples. Some dont work. Its really hard finding documentation over it and how to access what to record from. Ill give it a go though. I think everything does finally go through WASAPI so if I Can find away of recording from that then that would be a big step. Im jumping in the deep end here but at least I have a chance to try. If you have played around with it in the past then If you could PM me any info that would be great. Im sure if it can be done Ill find a away. Im just learning how to make my own classes and include them so its an interesting challenge. Ive also got a few audio examples working (non corona) so getting there. Ideally Ill make my own SDK I can reuse with amazing audio functions.
Yeah, I think WASAPI might be your only option if you want to analyze the audio data being captured from the microphone. I had a quick look at the “AudioVideoCaptureDevice” class that is made available to C#, but you can’t get raw PCM audio data out of it, which does you no good.
I’ve noticed that Nokia has a solid example on how to do this. Their example has has 2 projects, a C# library and a C++/CX library demonstrating how to communicate between the 2 and working with WASAPI. It also has an example on how to work with XNA, but I’m pretty sure that’s deprecated now.
Ive been looking at Nokia developer section and it is very good. I think I will have more joy from them than MS.
On WP8, we’re using DirectX’s “XAudio2” audio system. This is Microsoft’s top of the line audio system that they recommend for gaming. XAudio2 supports all of the features needed by Corona’s “audio” Lua library, such as streaming audio, setting the volume/gain on individual channels, controlling the pitch of each channel, etc.
We’ve written our own OpenAL library from scratch that bridges the cross-platform OpenAL C APIs to Microsoft’s XAudio2 system. It works a lot like how Apple’s open source OpenAL library works, which also bridges OpenAL’s cross-platform C APIs to Apple’s proprietary AudioUnit library. Now, we did this so that Corona developers can use *most* of our hidden OpenAL Lua APIs, such as things like “pitch”. Note that I say most because our OpenAL implementation doesn’t support positional sound, which is a corner we cut to get Corona for WP8 into everyone’s hands as soon as possible.
Thats cool. Ive been testing out the Xaudio2 system and it did look the best one. I am planning on making a microphone input monitor and fingers crossed some thing that might allow submixing or routing audio. I think that I will need WASAPI. Would this work in conjunction. I would ideally like to monitor whatever the audio is playing (not the microphone but whats generated by the phone/app as a mix possibly or select channels) .
DO you think I will need to make something from scratch or is there a way I can modify the current Xaudio audio system Corona is using?
WASAPI can definitely do this. In fact, I think it’s your best option for microphone “audio capture”. I’ve played with it briefly in the past and it involves you creating a C++/CX “Windows Runtime Component” library project in Visual Studio. A pure native C/C++ library won’t be able to access the WinRT APIs you need to implement this. At least that’s how I remember it.
If you intend to play/output audio via WASAPI too, then I wonder if you might end up conflicting with Corona’s usage of XAudio2. I say this because I’m pretty sure XAudio2 is a layer on top of WASAPI, because XAudio2’s job is to mix audio channels to a single stream and then output that 1 stream to WASAPI. If you intend to play audio on your end as well, outside of Corona, then perhaps the safest solution is to create another instance of the XAudio2 engine, which I believe Microsoft supports (I haven’t tried it). I would assume Microsoft mixes the audio between all XAudio2 engines to a single stream and outputs it to WASAPI at the end. I’m completely guessing here, but it seems to make sense in my mind.
Does this help?
Yeah kind of. I had read about creating a Runtime Component LIbrary. Ive seen a few examples. Some dont work. Its really hard finding documentation over it and how to access what to record from. Ill give it a go though. I think everything does finally go through WASAPI so if I Can find away of recording from that then that would be a big step. Im jumping in the deep end here but at least I have a chance to try. If you have played around with it in the past then If you could PM me any info that would be great. Im sure if it can be done Ill find a away. Im just learning how to make my own classes and include them so its an interesting challenge. Ive also got a few audio examples working (non corona) so getting there. Ideally Ill make my own SDK I can reuse with amazing audio functions.
Yeah, I think WASAPI might be your only option if you want to analyze the audio data being captured from the microphone. I had a quick look at the “AudioVideoCaptureDevice” class that is made available to C#, but you can’t get raw PCM audio data out of it, which does you no good.
I’ve noticed that Nokia has a solid example on how to do this. Their example has has 2 projects, a C# library and a C++/CX library demonstrating how to communicate between the 2 and working with WASAPI. It also has an example on how to work with XNA, but I’m pretty sure that’s deprecated now.
Ive been looking at Nokia developer section and it is very good. I think I will have more joy from them than MS.