Augmented Reality apps - possible in Corona?

Solar2D General Questions/Discussion
#1

Broadly this is already possible to add an image overlay to the camera feed.

But would it be possible to create content that reacts to the environment? Obviously not 3d models but 2d images that position and skew to where the device camera is.  This is very easy to achieve in Vuforia for instance using image targets (real world image to provide something for the device camera to track an image as though it is pinned to the image target).

Realistically, is this something that could be developed using Corona? Either from the ground up or via a Vuforia plugin?

Interested in any and all feedback or examples.

#2
  1. General Augmented Reality - Yes some apps that would be considered AR can be made w/ Corona.  I wrote a game for a client last year that cared where in the world and what physical features were nearby to make game decisions and present you with ‘encounters’.  (Game not released unfortunately as he got sidetracked by a bigger opportunity which I’m also coding for him.)

Note: I can’t give any more details that the above general statement till I get the OK from client or it is finally released.

  1. AR the way you described it (a very specific application of the concept).  No. I don’t think you could do that exact kind of app in Corona.  At least now without doing some Enterprise coding and using some sophisticated image analysis code, and likely other bits like tracking phone orientation and position.

Wasn’t Vuforia specifically designed for AR?  If so, we’re talking apples and oranges comparison.  Corona is a general SDK.  

Maybe you could integrate elements of Vuforia w/ Corona via enterprise?

Note: AR is much more than a specific usage like Pokemon Go which is what it sounds like you’re describing.  Corona can be used for a number of games and apps that would be considered AR.

PS - Some will disagree w/ my last statement which is cool, as I’ve read definitions of AR, but I tend to think any app that cares where you are in the world and adds additional data on top of that is enough to be AR.  

However, with the current Pokemon Go craze and the strong interest in various visualization technologies, most people will think that an app/game is only AR if it involves the overlaying of virtual elements onto a real-time video or camera capture.  Fair enough.

TL;DR  No I don’t think you can make a Pokemon Go style game in Corona Pro, but you might be able to w/ Enterprise.

#3

Thanks roaminggamer.

Yes, Vuforia is specifically designed for AR. So instead of having to create something from the ground up in Corona, would a Vuforia plugin be possible in Corona? Like it is with Unity: http://www.marcofolio.net/other/introduction_into_augmented_reality_with_vuforia.html

#4

I don’t write native plugins any more.  However, I imagine if I did and could get my hands on the vuforia libs and docs in a format compatible / build-able with Corona Enterprise, then the answer would be yes.

Sounds like you need to find a Enterprise developer.

#5

Great, that’s encouraging. I will post something over in the Corona Jobs forum at some stage. For now, it would be interesting to hear if any Enterprise developers have achieved this to date or any other thoughts about Vuforia with Corona. It would be fantastic to avoid having to dive into Unity for this and maintain our Corona framework.

#6

Corona cannot render 2.5, or so called 3D models?

As far as I can understand, that is required for game like Pokemon Go.

Furthermore, how to handle 2.5 (3D) collisions?

#7

Rendering - Correct, Corona is 2D only.  You can however implement a variant of projection styles on your own using math, properly constructed artwork and a solid understanding of the projection style you’re using.

Collisions - In most cases, you’ll need to use math tricks, rules-of-thumb based in the projection style’s layout, and a throrough understanding of Box 2D collision beyond the basic collision filter rules.  In some cases, you’ll have to roll-your-own COLDET and Response system.

Probably not a satisfying answering, but… an answer. :expressionless:

#8

I am not so skilled programmer as you Ed, so if I decide to move over to 3D, it is easier to go with Unity  :smiley:

Thanks.

#9

Hey, if you’re going to actually make a 3D game I’d go with a 3D engine/SDK too.  However, I will say, 2.5D ain’t 3D, so you have to decide which way to go:

One MTE usage:

https://www.youtube.com/watch?v=kXKHlhMDlfg

#10

Note: I think we’ve strayed a bit here… so my apologies to all, but I did want to finish answering the ‘appended’ questions about projections and Corona with samples (above).

#11

Thanks Ed for all clarifications.

2.5D is good enough (like Zig-Zag and Crossy Road).

I think 3D is too much for indie (you need a team and graphic designer I think).

Is there any 2.5D relatively successful/known game made in Corona so far?

#12

It seems that a lot of this conversation has focused on 3D imaging concerns.  What about simple, flat (2D) overlays on the camera at a given location?  Imagine I had people collecting “stickers” at specific geographic locations using their GPS and camera (with the overlay in the camera).  How difficult would that be in CoronaSDK?

Also, how precise would the GPS be overall?  And how does that translate to indoors?  Thanks for any clarification you might provide.

#13

Corona SDK can capture input from the video camera and use it to fill a Corona display object (newRect for example) but on iOS only. We do not and have to no plans to support this on Android. It’s simply too difficult. You could display stickers on top of the camera input with little difficulty.

Now the GPS accuracy: You’re really at the mercy of the GPS system. They are as accurate as they are. It has to do with how many satellites are in view. The more sats your phone can see the more accurate it will be. You will be less accurate indoors since building block the relatively low power GPS signal and it’s harder to pick up enough, or it will fall back to using location data from WiFi routers.

Sometimes the GPS will put you with in a meter or so of reality other times it could be tens of meters off. This is the nature of GPS.

Rob

#14

@gogigoranic8

Hi.  This is my last answer re: 2.5D here because this is not really related to the original question.  If you have further questions, please start a new thread and link back to this one to supplement the question/discussion.

Is there any 2.5D relatively successful/known game made in Corona so far?

  • Successful game?  I don’t know.
  • Successful example?  See my MTE links above.

@staypuffinpc

Adding to Rob’s last statement, I’d say that it is especially the nature of civilian GPS receivers to be inaccurate sometimes. Civilian GPS receivers are still not (but steadily improving) as accurate as military receivers.

This is an interesting read that talks about some of the reasons for this: http://www.gps.gov/systems/gps/performance/accuracy/

#15

Thanks @roaminggamer.  I fear you’re right about GPS accuracy in that regard.  We’ll have to wait a few more years still, perhaps.

#16

AR doesn’t need to use GPS. Visual image targets can be used instead which is an image in the real world that the device app uses to motion track content. Here is an example of a flat 2d object but a shape that does adjust to the tilts and zooms of the device to track onto an image target. Check last part of the video: https://www.youtube.com/watch?v=GDF-D-kGhjA

As far as I can tell, Corona out of the box cannot interpret image targets in its camera view. But Vuforia can. So I am thinking the solution is using Vuforia via Corona Enterprise, correct?

So if anyone has had experience with Vuforia in Enterprise, it would be great to hear about it. I did find Walter responding in Vuforia to some specific questions so someone in Corona land is doing it: https://developer.vuforia.com/forum/ios/using-vuforia-without-eaglview

#17

Hi

I am fairly new to Coronasdk. I am intrigued about AR and want to do similar things with CoronaSDK for Android. I know that CoronaSDK can do it for IOS. I can’t afford the enterprise version since I am just playing around with coding and most of the projects I do is free.

I read in Rob’s post somewhere here that we can put a sticker on top of the camera. This would be so cool too. I can think of so many things that can be done with this AR feature (if only available on Andriod as well).

Anyone has done this sticker on top of camera?

Thanks

Rgds

TH

#18

Hi All

So I’m not the best coder in the world, but I managed to make a little PokemonGo AR clone in Corona in less than 100 lines of code.
You can see the video here https://www.youtube.com/watch?v=SRVgbjwYeGo

You can download and use the code(however you like) here - https://bitbucket.org/sskenth/coronaopensource.git

Hope that helps you on your journey with Augmented Reality and CoronaSDK.

Good luck, and if I can answer any specific question on it, let me know

-Sat

P.S. can anyone tell me how to embed a video into a forum post, as you can see above I clearly dont know how to do it ahaha [SOLVED - Thanks Rob]

#19

Nice tutorial.

Have you tried just putting in the URL? It might be smart enough to detect youtube links.

#20

It couldn’t have been any easier and I still got it wrong, I feel so silly right now  :unsure: 

Thanks Rob