I am working on a project that requires some MASSIVE data processing and I began to wonder how much data processing Corona can handle (meaning just programming in Lua instead of using custom java or c/c++ code with Enterprise; I’m a broke college student who can’t afford Enterprise at the moment!). What I would like to know is how Corona performs in data processing when compared to an app written specifically for Android in straight Java considering the layers that Corona has to go through to get to the OS (Project -> Native Extension -> Corona engine -> OS).
*StarCrunch flies in through the window and crushes the coffeetable with a refrigerator-sized Lua reference document.*
Dang, my clicking habits are too predictable.
If you’re willing to submit to the store, it sounds like you can still get a trial license of Enterprise, though I don’t know if you would find that a good fit.
Without knowing your particular needs, this post summarizes the directions I’ve considered for a couple of expensive things (fleshed out a little post further down). Maybe that would give you some ideas. What are you doing, anyhow?
Thanks for the info StarCrunch. Alex, I see your point. Google is my friend. 
I am working on some very detailed world generation (including biomes, cities, countries, etc.) for a medieval rpg/sim/do-whatever-you-want-to game alongside of things like dynamic NPC’s and simulating major events worldwide, so lots of loops and conditionals inside of enter frame event checking. I plan on pipelining things that are outside the player’s immediate area of influence, which should help for large scale events and things like that. However, I am a bit leery about how NPC decision making and dialogue generation (which will be much more in depth than the simple trade, quest, or hire), world generation and other labor intensive actions which will have to be completed side by side will perform.
Oh man, we were just talking about how only like 4 people are going to get my ridiculous inside joke reference!
No offense meant, my friend. I was just saying that StarCrunch is basically a walking compendium of Lua knowledge, and as soon as I saw the question, I knew he was going to bust through the wall like Superman to give a fantastic answer.
Wow. I took that the wrong way. :)
Okay, so my interest is piqued. 
Also, I immediately thought of Dwarf Fortress.
If you had Enterprise, would you be throwing anything at it library-wise or just trying to stay closer to the metal? (Or getting really close…)
The Coronium approach mentioned in that linked post is actually seems like a great fit, since this sounds like an up-front thing, followed by some lower-cost, now-and-then maintenance. Coronium is using LuaJIT. You might have to do a little code gardening, but once you get something it likes, LuaJIT will straight-out scream like the proverbial bat out of hell on huge processing tasks.
What sort of techniques were you looking at using? When you mentioned college and massive data processing in the same post, my mind went toward completely different applications.
I would love to stay as low level as possible (RenderScript actually sounds great!). Coronium also sounds like an interesting option, so I’ll keep it in mind, although I would like for my game to be able to run offline. I have heard GREAT things about LuaJIT, so I might look into offline LuaJIT implementation as well.
You mentioned you noticed that I mentioned college and data processing; actually I am a Sophomore in Mechanical Engineering. I’m just minoring in Computer Science, so at this point I have had zero college level computer classes and all I know comes from this little (obsessive) hobby I have. That being said, I’m still searching for different techniques to consider. I would like to use Simplex noise for heightmap generation and then maybe a downhill walk for rivers, streams and lakes, but I’m still looking for a mobile-friendly city generation technique. Any recommendations or resources I could delve into?
Bam, it looks like you would have to use Tiled (http://www.mapeditor.org/) for massive map generations. You can create 2d/isometric maps. There is also a plugin to import to corona
Thanks for the tip undecode! Unfortunately, tiled is built mostly for manual map creation; I would like to have my worlds generated at runtime.
Okay, I’m about to flood you with links and rabbit holes.
First off, I would be remiss in not pointing out Alex’s own Segreta, which must have some interesting dungeon generation war stories.
Unfortunately, Dyson isn’t active around here any more, but he used Perlin noise to generate some stuff in his MTE, e.g. here was one of his responses to a question of mine. I see some other info elsewhere in that sub-forum, but some of those topics are huge and the posts themselves are walls-of-words, so not a quick and easy search. 
I am definitely familiar with simplex noise! I have a Corona-compatible version in Lua here, an earlier version of which appeared on the old Code Exchange. The module dependency can be replaced with the Bit module; it’s general-purpose enough that I don’t want to introduce a hard Corona dependency, is all. I have also been adapting some noise code for use in shaders as well (very little of this is my own code, so see the attributions inside).
If you go the LuaJIT route, the Lua module actually began life there. Unfortunately, I haven’t had much excuse to play with it recently (ditto LuaJIT more generally), so my most up-to-date code (plus commentary) is actually found here; I should probably report some of that back to Dr. Gustavson, one of these days! (I also have C# code, but never got it production-ready, really.) Also, if I remember correctly, Dr. Gustavson did like to hear about simplex noise “in the field”.
I forget what got me looking into simplex noise in the first place. Near as I can tell I must have done further research after reading posts like this and this. (Excellent, excellent dev blog. It’s too bad he’s been quiet, lately.) Sooner or later I mean to pick up this book as well, which may also have mentioned it.
I do know, however, that he tipped me off to easily seedable (but still high-quality) random number generators like those of Marsaglia (current code here), which I imagine you could also incorporate into your toolkit. I believe roaminggamer uses another technique as well, based on one of the hashes in the crypto library. (Also, if you’re willing to backport it to Corona’s Lua 5.1, Dirk Laurie posted an implementation of ISAAC on the Lua mailing list, a while ago: ISAAC in Lua)
Food for thought: Generating Complex Procedural Terrains Using The GPU (Some of the required features might be available on more recent devices via GLES extensions. Unfortunately those tend to require a little native-side assist.)
There were some interesting ideas here: Herringbone Wang Tiles (I am almost certain this is what led me to my own current expensive problem, with the Fast Fourier Transforms. In at least one of those articles, Sean mentions colored corners, which I have partially implemented. I was doing the texture synthesis, but the authors describe Poisson disk applications as well. I need to do some restructuring for the texture refinement passes, but have a functional first pass. Unfortunately I got totally railroaded looking into some of the computation-heavy stuff for non-random texture synthesis. )
Err, and I never did give a “how to make a city” link. Alas, I do not know one!
Anyhow, I hope something in that was useful.
EDITS : Cleared a couple things up, fixed links
StarCrunch, thanks a million! There is so much information for me to go through here, I’m going to be on a code high for the next three days straight! I’ll sift through and work something out. I’m moving in two weeks, so it’ll be a while, but when I get something up and running, I’ll post it for all to see on the WIP board.
I want my meager and insubstantial contribution to this discourse to be more than a terrible joke told poorly!
FWIW I forked Ronald Yonaba’s Jumper library (he’s a genius, and his lib is great) which provides very efficient pathfinding functionality. I realize this is so so so super-lowlevel compared to what StarCrunch was dropping, but maybe it’ll come in handy?
https://github.com/pancinteractive/Jumper
Also, Caleb created an engine called Dusk which is now the only actively supported tile management system for Corona. I haven’t done much with it, but I imagine it would be useful for PGC.
No problem, glad to help. I’ll be interested to see the results!
Also, I completely forgot to mention coroutines, which might help you time-slice your generation process. See for example the section on long-running operations here.
*StarCrunch flies in through the window and crushes the coffeetable with a refrigerator-sized Lua reference document.*
Dang, my clicking habits are too predictable.
If you’re willing to submit to the store, it sounds like you can still get a trial license of Enterprise, though I don’t know if you would find that a good fit.
Without knowing your particular needs, this post summarizes the directions I’ve considered for a couple of expensive things (fleshed out a little post further down). Maybe that would give you some ideas. What are you doing, anyhow?
Thanks for the info StarCrunch. Alex, I see your point. Google is my friend.
I am working on some very detailed world generation (including biomes, cities, countries, etc.) for a medieval rpg/sim/do-whatever-you-want-to game alongside of things like dynamic NPC’s and simulating major events worldwide, so lots of loops and conditionals inside of enter frame event checking. I plan on pipelining things that are outside the player’s immediate area of influence, which should help for large scale events and things like that. However, I am a bit leery about how NPC decision making and dialogue generation (which will be much more in depth than the simple trade, quest, or hire), world generation and other labor intensive actions which will have to be completed side by side will perform.
Oh man, we were just talking about how only like 4 people are going to get my ridiculous inside joke reference!
No offense meant, my friend. I was just saying that StarCrunch is basically a walking compendium of Lua knowledge, and as soon as I saw the question, I knew he was going to bust through the wall like Superman to give a fantastic answer.
Wow. I took that the wrong way. :)
Okay, so my interest is piqued.
Also, I immediately thought of Dwarf Fortress.
If you had Enterprise, would you be throwing anything at it library-wise or just trying to stay closer to the metal? (Or getting really close…)
The Coronium approach mentioned in that linked post is actually seems like a great fit, since this sounds like an up-front thing, followed by some lower-cost, now-and-then maintenance. Coronium is using LuaJIT. You might have to do a little code gardening, but once you get something it likes, LuaJIT will straight-out scream like the proverbial bat out of hell on huge processing tasks.
What sort of techniques were you looking at using? When you mentioned college and massive data processing in the same post, my mind went toward completely different applications.