I’m wondering if anyone has had any luck with integrating image recognition software within Corona SDK and, if so, with what companies? I’m working with a museum and I’d like to be able to create an app that, when a camera is pointed at an exhibit, will redirect to a page that has more information about that exhibit. Just curious if anyone has meddled in this area.
Peter,
I don’t have any info on image recognition plugins, but I was wondering if you might not be better served using QR Code tags at the exhibits.
There is a QR scanner plugin: https://marketplace.coronalabs.com/plugin/qr-scanner.
Note: Image recognition is a computationally intensive operation that is affected by a number of factors:
- Lighting
- Facing and angle
- Distance
- Camera quality
- etc.
I don’t want to talk you out of this, but of the two, QR codes seem like they be a lot easier to do.
I agree with Ed, the QR code option would be much easier.
Ed, I recall when Corona Geek episodes were running that Steven (@StarCrunch) was having a play with the Torch Library and possible Corona integration. That would be possible “do-it-yourself” option with image recognition (but steep learning curve… NN etc!)… perhaps Steven will have more info.
BTW Ed, (@roaminggamer) I was checking out your site and got the “Error Establishing Database Connection”. I had this same thing on my own site 2 weeks ago and kept getting it every other day (when hosting on GC Compute Engine)… had to keep restarting the VM… I ended up moving over to Amazon Lightsail and haven’t looked back since!
I’m a fan of SSK… so was heading over to your site to check out the features of SSK2
Thanks for the QR recommendations, fellahs. I was actually planning on creating a demo with QR, but I’ve had a successful experience once before in another museum with an app that had image recognition (not one that I created) for the art exhibits and I was curious if anyone on here had tried it out themselves. I’m planning on using Lerg’s QR plugin, which seems fairly straightforward.
@Peter - Good luck with the project whatever path you take. If you do find some cool image recognition leads, be sure to link/share them here.
@rshumac - re: site - Yeah. My site is running a very old copy of Ubuntu and crashes every 26…36 hours now. I am going to start fresh. I will probably hand transfer some data, but mostly this is going to be an overhaul of the site. I’ll bu updating to https at the same time.
Meanwhile, I keep the SSK2 docs on gitHub: https://roaminggamer.github.io/RGDocs/pages/SSK2/
No worries @Peter, and good luck also.
@roaminggamer, cool I’ll check out the SSK2 docs at gitHub, thanks!
I’ve definitely been happy with Amazon Lightsail so far… excellent docs and cheap… $5/mth (for relatively low traffic site like mine). Seems more stable and cheaper than what I’ve experienced with Compute Engine and Digital Ocean when hosting a WP site.
@Peter Rich
OpenCV is well-recognized for computer vision tasks. Its API is huge, so binding / maintaining the whole thing would be asking a lot, to say nothing of the binary size if one did so, but maybe there are reasonable subsets that could be wrapped up for common use. There are a few posts here and there where somebody was building something atop it. I believe this uses it, for instance, and maybe this plugin does too, or something similar.
This is a good free book on neural networks, which uses character recognition as a motivating example.
Regarding what I said on the show, it was actually ArrayFire where I’d been devoting most of my attention, though I know I at least mentioned Torch in connection with luaffifb, which I’ve been meaning to finally submit one of these days. (I still need to sit down and grind out some tests and then make examples.) All of these many array / tensor SDKs also have huge APIs. :D That said, with neural networks the most common use case is probably going to be using them, rather than training them.
Unfortunately, my efforts got rather derailed since, on Windows where I do most of my development, ArrayFire’s a 64-bit library. Or rather an ecosystem: it ties together a lot of external libraries, so I got scared off of trying to build a 32-bit version. :) I did have some sporadic success piping commands to a second process, but this was nowhere near stable enough to do serious work. (And good IPC is another rabbit hole altogether!)
Every now and then I try to do some work on a texture synthesis project (something along these lines). I made a pure Lua version a few years back that can make an infinitely tileable image, but is rather slow. I’ve been gradually moving the number-crunching bits to C++, including some enhancements that were prohibitive before. Many of my recent plugins are steps along the way.
It’s been quite a hunt trying to find full-featured yet reasonably portable building blocks, especially with permissive licenses. I probably have some sort of Eigen binding coming (possibly with per-system BLAS), as well as something for FFTs or signal processing more generally (here is where the search has been especially difficult). I also have a homegrown array / tensor of sorts meant to interoperate with these, though I’m not terribly satisfied with it right now. I’ve picked up a few more hobby horses that also require this sort of machinery, so my aim has been generality.
Err… reading over the last few paragraphs, I’m not sure they stayed on topic. The project probably counts as “do-it-yourself” computer vision, so I’ll call it good and leave them. 
Hey @StarCrunch, thanks for all the excellent references! ArrayFire looks very interesting (and speedy based on those benchmarks). When I was looking through the ArrayFire docs… I just had a flashback… do you remember Numerical Recipes in C? I came across it first back in 2004 when I was mucking around with EKFs in university when faced with some damn near singular covariance matrices… hence having to look in SVDs and the like. Nothing too serious though I was simply using Matlab/Simulink with some C Mex functions
That Michael Nielsen book looks good, I’ll definitely check it out ;) . I had a play with Tensorflow earlier in the year, so might have to give it another whirl!
I do indeed know NR. (Unfortunately, it’s one of the trouble spots when it comes to permissive licenses.) Actually, while doing some of what I mentioned above, I consulted the relevant chapter while writing some FFT code. Unsurprisingly, this wasn’t very fast in pure Lua.
For that matter, I dabbled in SVDs too. :) (Rank decomposition.) There’s a lot of ongoing research here! More than ever with all the new tensor libraries, I suppose. I even tried to read some of it, eventually picking up a book for some better grounding. Didn’t follow up much with SVDs, but have implemented a few other things from that, though it always comes back around to speed. And even if the math itself were fast, all these operations will punish you to no end if you don’t have an amenable memory layout, and it doesn’t help that they tend to grow quickly in size. 
Good luck with Tensorflow! (On topic.)
If this is any indication, there’s definitely some interest in getting something going. The poster’s last words also seem to corroborate that “the most common use case is probably going to be using them, rather than training them”.
@StarCrunch, I agree… definitely using them… is the real use case. I just added my vote to that feature request. I was curious about the training part to understand the math behind it and thought it would be kind of cool to create your own little deep learning helper.
@ScottHarrison, nice stuff !.. keen to see it available in the marketplace
@rschumac (or anybody else whose interest was piqued)
Over the course of the last few days I’ve started the aforementioned Eigen library: VERY early docs
I think by and large this supersedes my “homegrown array / tensor of sorts”, which I have mixed feelings about considering how much time I sank into that.
Eigen’s a templated beast, so it takes ages to compile, and that’s even with most data types disabled to speed it up). Almost inevitably, the binding code is macro- and template-heavy as well. It’s also been interesting trying to work around some of the mismatches between C++ operators and Lua, e.g. not being able to assign to return values. Quite a few auxiliary types (mappings, views) remain in limbo until I suss some of that out; I ran into this with ArrayFire too, never really coming up with anything.
The various solvers seem to work, so I might throw LU or QR (I hope I get bonus points for overloading already-obscure acronyms in the same thread) decomposition at a morphing test I have, and maybe inverse kinematics would be worth a try too. Further suggestions welcome!
@StarCrunch, great stuff! Looking forward to seeing it in the marketplace to have a play with it. I vote for IK… mainly because it just set off a (not terrible) memory of the DH method in a robotics class from the early 2000’s… but I could possibly think of use cases for a game also.
Peter,
I don’t have any info on image recognition plugins, but I was wondering if you might not be better served using QR Code tags at the exhibits.
There is a QR scanner plugin: https://marketplace.coronalabs.com/plugin/qr-scanner.
Note: Image recognition is a computationally intensive operation that is affected by a number of factors:
- Lighting
- Facing and angle
- Distance
- Camera quality
- etc.
I don’t want to talk you out of this, but of the two, QR codes seem like they be a lot easier to do.
I agree with Ed, the QR code option would be much easier.
Ed, I recall when Corona Geek episodes were running that Steven (@StarCrunch) was having a play with the Torch Library and possible Corona integration. That would be possible “do-it-yourself” option with image recognition (but steep learning curve… NN etc!)… perhaps Steven will have more info.
BTW Ed, (@roaminggamer) I was checking out your site and got the “Error Establishing Database Connection”. I had this same thing on my own site 2 weeks ago and kept getting it every other day (when hosting on GC Compute Engine)… had to keep restarting the VM… I ended up moving over to Amazon Lightsail and haven’t looked back since!
I’m a fan of SSK… so was heading over to your site to check out the features of SSK2
Thanks for the QR recommendations, fellahs. I was actually planning on creating a demo with QR, but I’ve had a successful experience once before in another museum with an app that had image recognition (not one that I created) for the art exhibits and I was curious if anyone on here had tried it out themselves. I’m planning on using Lerg’s QR plugin, which seems fairly straightforward.
@Peter - Good luck with the project whatever path you take. If you do find some cool image recognition leads, be sure to link/share them here.
@rshumac - re: site - Yeah. My site is running a very old copy of Ubuntu and crashes every 26…36 hours now. I am going to start fresh. I will probably hand transfer some data, but mostly this is going to be an overhaul of the site. I’ll bu updating to https at the same time.
Meanwhile, I keep the SSK2 docs on gitHub: https://roaminggamer.github.io/RGDocs/pages/SSK2/
No worries @Peter, and good luck also.
@roaminggamer, cool I’ll check out the SSK2 docs at gitHub, thanks!
I’ve definitely been happy with Amazon Lightsail so far… excellent docs and cheap… $5/mth (for relatively low traffic site like mine). Seems more stable and cheaper than what I’ve experienced with Compute Engine and Digital Ocean when hosting a WP site.
@Peter Rich
OpenCV is well-recognized for computer vision tasks. Its API is huge, so binding / maintaining the whole thing would be asking a lot, to say nothing of the binary size if one did so, but maybe there are reasonable subsets that could be wrapped up for common use. There are a few posts here and there where somebody was building something atop it. I believe this uses it, for instance, and maybe this plugin does too, or something similar.
This is a good free book on neural networks, which uses character recognition as a motivating example.
Regarding what I said on the show, it was actually ArrayFire where I’d been devoting most of my attention, though I know I at least mentioned Torch in connection with luaffifb, which I’ve been meaning to finally submit one of these days. (I still need to sit down and grind out some tests and then make examples.) All of these many array / tensor SDKs also have huge APIs. :D That said, with neural networks the most common use case is probably going to be using them, rather than training them.
Unfortunately, my efforts got rather derailed since, on Windows where I do most of my development, ArrayFire’s a 64-bit library. Or rather an ecosystem: it ties together a lot of external libraries, so I got scared off of trying to build a 32-bit version. :) I did have some sporadic success piping commands to a second process, but this was nowhere near stable enough to do serious work. (And good IPC is another rabbit hole altogether!)
Every now and then I try to do some work on a texture synthesis project (something along these lines). I made a pure Lua version a few years back that can make an infinitely tileable image, but is rather slow. I’ve been gradually moving the number-crunching bits to C++, including some enhancements that were prohibitive before. Many of my recent plugins are steps along the way.
It’s been quite a hunt trying to find full-featured yet reasonably portable building blocks, especially with permissive licenses. I probably have some sort of Eigen binding coming (possibly with per-system BLAS), as well as something for FFTs or signal processing more generally (here is where the search has been especially difficult). I also have a homegrown array / tensor of sorts meant to interoperate with these, though I’m not terribly satisfied with it right now. I’ve picked up a few more hobby horses that also require this sort of machinery, so my aim has been generality.
Err… reading over the last few paragraphs, I’m not sure they stayed on topic. The project probably counts as “do-it-yourself” computer vision, so I’ll call it good and leave them.
Hey @StarCrunch, thanks for all the excellent references! ArrayFire looks very interesting (and speedy based on those benchmarks). When I was looking through the ArrayFire docs… I just had a flashback… do you remember Numerical Recipes in C? I came across it first back in 2004 when I was mucking around with EKFs in university when faced with some damn near singular covariance matrices… hence having to look in SVDs and the like. Nothing too serious though I was simply using Matlab/Simulink with some C Mex functions
That Michael Nielsen book looks good, I’ll definitely check it out ;) . I had a play with Tensorflow earlier in the year, so might have to give it another whirl!