LuaJIT bridge plugin

Solar2D Lua Language
#1

For one reason or another I recently mentioned an FFI library or something similar on Discord, and it got me reconsidering a “bridge” idea I’ve often had.

So I sort of locked myself away and now have something like that, for LuaJIT 2.1.

I’ve since been given the actual plugin repository, but haven’t populated it yet; in the meantime, you can test with something like:

local URL = "https://github.com/ggcrunchy/solar2d-plugins/raw/master/DevSnapshots/plugin.luajitbridge/%s/data.tgz"

local Platforms = {}

for _, name in ipairs{ "win32-sim", "mac-sim", "android", "iphone", "iphone-sim" } do
  Platforms[name] = { url = URL:format(name) }
end

settings =
{
	plugins = {
		["plugin.luajitbridge"] = {
                     publisherId = "com.solartools",
                     supportedPlatforms = Platforms
               }
        }
}

(f_wang on Discord has gone to town with it already.)

This has been my own “get it working” test:

local bridge = require("plugin.luajitbridge")

local state = bridge.NewState()

local b, c = state:Do([=[
  local a, t, g = ...
  local ffi = require("ffi")
  local C = ffi.C

  ffi.cdef[[
    void CoronaLog (const char * fmt, ...);
  ]]
  
  C.CoronaLog("1")
  g.print("DUCK")

  return a + 18, t.k
]=], 3, { k = 4 }, _G)

state:Do[[
  collectgarbage()
]]

Here, you create a LuaJIT state object, and then can call top-level functions there with Do(). Arguments may be passed to it, as seen above with 3, { k = 4 }, _G. They’re received on the other end through ..., proxied if they’re neither a primitive—nil, number, boolean—nor a string.

Similarly, you can return values back to Solar, also proxied as necessary, including functions that you can call from there. Obviously, anything proxied will incur a little friction.

In the above, Solar’s global table is passed in and used to access print(). Another table is accessed and one of its values returned, and some math is done on a pair of numbers.

The CoronaLog() bit references a built-in print()-alike exposed by Solar, using LuaJIT’s FFI. Functions like this are listed in the .h files here and are partially documented here.

Many of those functions were used in the examples for the Tiny C Compiler plugin, as it also makes them available to itself. (See also the “some stuff has landed in 3713” link in that post.)

(I haven’t played with stuff like this, but it looks like the two could be used together.)

KNOWN ISSUES

It still needs a decent amount of testing, and also iOS at least.

The proxying will inevitably be imperfect, and I’m sure it will fall down under certain things. :joy:

The package.path and package.cpath are just the defaults, so require() is pretty flaky. There are some quirks here…

Lua and LuaJIT use different bytecode formats. Right now, for builds, Solar’s packaging will run every Lua file through luac and stuff the result in the archive, but these will be unusable by LuaJIT. Certain built-in libraries are also affected—even in the simulator—and this is additionally why Do() takes a string rather than letting you pass a function() ... end on the spot.

(With luadec or similar some of this might be addressable, but it’s iffy and it’d probably clobber line information, for debugging; not doing debug stripping is itself another possibility, but drastic and affects the whole program.)

There is actually some build-time scripting I’d like to add to Solar itself (I have a half-finished WIP, actually), that would be useful for this plus some other ideas.

A dynamic library that uses the Lua API is also a problem, since Solar’s Lua already exposes the lua_* symbols, which have the same name in LuaJIT. To avoid the clash, I opted to do a search and replace of LuaJIT’s names (lua_insert() → l2j_insert() et al.), although maybe something with symbol hiding on ELF platforms, or a proxy DLL on Windows, would work, down the road.

Anyhow, typical already-built plugins that use the Lua API would balk. I would expect libraries that don’t touch it to work better. (Per the Lua point above, they might still need an assist to get archived.)

I’ll post the code when I decide where to put it.

The LuaJIT-specific parts are mostly in a single file. It’s possible a lot of this work could be applied toward a Lua 5.5 and / or Luau “bridge” too, though I don’t have any plans.

2 Likes