Using Neural Networks for AI in a tactical game

[rune7]

Hi,

I’m building a multiplayer board game for 2-8 people. Part of the game includes the option to add additional/ replacing players which will be played by the device (AI). I’ve already constructed a basic AI which perform nicely but looking to expand the options to additional AIs which will perform better as player skill increase.

For that purpose I thought to design and train AIs based on neural networks (specifically LSTM or RNN variations). There are very good tools today in Lua to build, train and test such networks (such as Torch).

Has anyone been dealing with such and can share some thoughts or lend an advise?

Also, If anyone familiar with building plug-ins is interested in porting one of the existing Lua libs to work over Corona, please contact me (I do not use the Enterprise edition) 

[StarCrunch]

Hi.

Are you hoping to do training during the game too, or will that be a strictly offline process, say on the desktop in the simulator? Or even a second, independent program?

On the plugin-ish side of things, I basically spent my December getting a binding going for the ArrayFire numerics library. I’m now trying to finish the plumbing in order to use it with Corona on Win32 desktop. Pretty close now. 

I ported whatever examples I could, with the thinking that this would flush out bugs or shortcomings in my binding. With a couple exceptions, I’m now down to just the image processing and machine learning stuff, the latter giving some example of what you might be after. So, it’s close, but not yet there. (The big hold-up involves porting C+±isms like

labels\_all(lidx) = labels\_all(lidx) + 1;

which doesn’t look especially hard, but is sort of an all-or-nothing, multi-hour affair.  That said, hewing to the examples and the interface for their convenience layer is just a pragmatic decision. Barring bugs, the majority of the underlying ArrayFire C API is already bound, if one wants to explore that route.)

No Lua docs yet, unfortunately.

I have no idea how ready this is for OSX, but welcome anybody to try! Ditto for mobile.

Let me know if that sounds remotely on track with what you have in mind.

[rune7]

Hi StarCrunch,

Seems ArrayFire is a GPU enabler which may be useful but I believe that Torch (which is lua based) already relies on similar engine (actually there are at least two options there I think).

I would have liked to let the algo train on each device as well (so it grows as the player advances in skill). I don’t know how feasible this is, but there are already ios and android ports to Torch, so people seem to think its possible. If not, I will simply use it to train offline (not under Corona, but using its pure Lua version). Still, I also do not know how to port the already trained state machine (to simply output moves based on its trained params). It might mean porting most of it anyhow.

what are you planning to use yours for?

[StarCrunch]

My take on it is that Torch focuses on neural networks in particular. There are a few of these numerical libraries, but they’re all huge, so it takes a while to study them.  Incidentally, the ArrayFire developers seem keen on getting the libraries to interoperate.

With the little I know, I’m going to guess the xor_model.net here is an already-trained network, which suggests they just get passed around as binary blobs. Maybe it’s even a portable format.

If your application requires a connection anyway, being multiplayer and all, this might even be a good Coronium candidate, especially with Torch running atop LuaJIT. I’ve been leaning at targeting this myself, once the ArrayFire binding is further along.

My own use cases are image processing / computer vision, rather than neural networks. I have some stuff similar to this which was possible to shoehorn into shaders, but has ended up being rather inflexible. And then some texture builders (basically, the patch-based stuff from Kwatra mentioned at that link). There are quite a few more things I want to play with, though. 

[rune7]

I will have a look at xor_model. I’m using Coronium GS which is no longer supported by its developer, so unless I can port its engine to Coronium IO, I would not put more effort into it. I’d have selected another back-end solution if I knew that beforehand.

[StarCrunch]

@ rune7

Not sure you’re still looking into this, but somewhat related… I’ve got this FFI library loading from Corona in OS X (only fails one of the tests, so calling it good for now). If I understand it right, Facebook basically forked that (the earlier maintainer abandoned it after other work took over) for Torch specifically, so it’s even pretty appropriate.  :) I’ve done the LUA51 install from here and when time allows will try to mimic one of their REPLs to test if everything connects. There’s another million or so projects here and I might see how a few of them work as well.

Anyhow, it’s imperfect (I don’t want to figure out the iOS / Win32 / Android details, so I’m basically at the authors’ mercy for support), but if things generally work I’ll see about submitting it as an OS X plugin.

[rune7]

Hi StarCrunch,

Thanks for the tip. I will definitely look into that as its one of my main interests these days. I will not port it to corona however, since the relevant app it was meant for did not get the traction I was hoping for on store so no point in extending the AI at this stage. Perhaps later this year when I have some time. Anyway, thanks for this info.

[StarCrunch]

Hi.

Are you hoping to do training during the game too, or will that be a strictly offline process, say on the desktop in the simulator? Or even a second, independent program?

On the plugin-ish side of things, I basically spent my December getting a binding going for the ArrayFire numerics library. I’m now trying to finish the plumbing in order to use it with Corona on Win32 desktop. Pretty close now. 

I ported whatever examples I could, with the thinking that this would flush out bugs or shortcomings in my binding. With a couple exceptions, I’m now down to just the image processing and machine learning stuff, the latter giving some example of what you might be after. So, it’s close, but not yet there. (The big hold-up involves porting C+±isms like

labels\_all(lidx) = labels\_all(lidx) + 1;

which doesn’t look especially hard, but is sort of an all-or-nothing, multi-hour affair.  That said, hewing to the examples and the interface for their convenience layer is just a pragmatic decision. Barring bugs, the majority of the underlying ArrayFire C API is already bound, if one wants to explore that route.)

No Lua docs yet, unfortunately.

I have no idea how ready this is for OSX, but welcome anybody to try! Ditto for mobile.

Let me know if that sounds remotely on track with what you have in mind.

[rune7]

Hi StarCrunch,

Seems ArrayFire is a GPU enabler which may be useful but I believe that Torch (which is lua based) already relies on similar engine (actually there are at least two options there I think).

I would have liked to let the algo train on each device as well (so it grows as the player advances in skill). I don’t know how feasible this is, but there are already ios and android ports to Torch, so people seem to think its possible. If not, I will simply use it to train offline (not under Corona, but using its pure Lua version). Still, I also do not know how to port the already trained state machine (to simply output moves based on its trained params). It might mean porting most of it anyhow.

what are you planning to use yours for?

[StarCrunch]

My take on it is that Torch focuses on neural networks in particular. There are a few of these numerical libraries, but they’re all huge, so it takes a while to study them.  Incidentally, the ArrayFire developers seem keen on getting the libraries to interoperate.

With the little I know, I’m going to guess the xor_model.net here is an already-trained network, which suggests they just get passed around as binary blobs. Maybe it’s even a portable format.

If your application requires a connection anyway, being multiplayer and all, this might even be a good Coronium candidate, especially with Torch running atop LuaJIT. I’ve been leaning at targeting this myself, once the ArrayFire binding is further along.

My own use cases are image processing / computer vision, rather than neural networks. I have some stuff similar to this which was possible to shoehorn into shaders, but has ended up being rather inflexible. And then some texture builders (basically, the patch-based stuff from Kwatra mentioned at that link). There are quite a few more things I want to play with, though. 

[rune7]

I will have a look at xor_model. I’m using Coronium GS which is no longer supported by its developer, so unless I can port its engine to Coronium IO, I would not put more effort into it. I’d have selected another back-end solution if I knew that beforehand.

[StarCrunch]

@ rune7

Not sure you’re still looking into this, but somewhat related… I’ve got this FFI library loading from Corona in OS X (only fails one of the tests, so calling it good for now). If I understand it right, Facebook basically forked that (the earlier maintainer abandoned it after other work took over) for Torch specifically, so it’s even pretty appropriate.  :) I’ve done the LUA51 install from here and when time allows will try to mimic one of their REPLs to test if everything connects. There’s another million or so projects here and I might see how a few of them work as well.

Anyhow, it’s imperfect (I don’t want to figure out the iOS / Win32 / Android details, so I’m basically at the authors’ mercy for support), but if things generally work I’ll see about submitting it as an OS X plugin.

[rune7]

Hi StarCrunch,

Thanks for the tip. I will definitely look into that as its one of my main interests these days. I will not port it to corona however, since the relevant app it was meant for did not get the traction I was hoping for on store so no point in extending the AI at this stage. Perhaps later this year when I have some time. Anyway, thanks for this info.