Modulous development

Solar2D Lua Language
#1

Hopefully this is the right board for this topic?

I’m building a library to load in as a framework within other projects, but the number of functions this library provides is growing fairly rapidly so maintaining them all within a single code file is becoming difficult. The functions themselves can be easily split up in to groups so I’m wanting to split into multiple .lua files and have the main library file load them all up.

I believe the require function is intended for this, but it seems this requires loading the required file in to a variable and then referencing it’s functions via that variable, which would mean each file becomes it’s own library? This isn’t really what I’m after.

The end result I’m wanting is that my library can be loaded and used like this:

[lua]local myLib = require “plugin.mylibrary”

myLib.functionOne()

myLib.functionTwo()[/lua]

But for mylibrary.lua to actually have functionOne() and functionTwo() in separate files. Additionally these functions need to be able to access variables defined within the scope of mylibrary.lua.

Currently, mylibrary.lua is like this:

[lua]local Library = require “CoronaLibrary”

local lib = Library:new{ name=‘mylibrary’, publisherId=‘foo.bah’ }

local variableOne = “foo bah”

lib.functionOne = function()

  return variableOne

end

lib.functionTwo = function()

  return variableOne

end

return lib[/lua]

Does this make sense, and is it possible?

#2

Hi,

Just as a general question, do, or can, your library functions fall into functional categories? This is generally how a large library is structured.

For example:

local mylib = require("mylib") local res = mylib.category\_one.do\_something(...) local res = mylib.category\_two.do\_something(...)

Or, do they need to all be on the root mylib object?

I ask because my answer will be different depending upon which you need. The “category” style is simpler to implement, and generally preferred by the end user.

-dev

#3

Whoops sorry - totally forgot I’d posted here, ha.

The functions can be categorised logically for the sake of splitting the code into manageable files, but I’m hoping to keep them ‘root’ for actual usability.

For example, mymodule.addSomething() and mymodule.renderSomething() could go in to say, dataHandling.lua and rendering.lua, but it would make less sense to have to call mymodule.dataHandling.addSomething() and mymodule.rendering.renderSomething().

That said, I’d be interested in both of your approaches to be honest.

#4

My approach for all projects:

  1. Build a hierarchy of modules from the very top to the very bottom. At the top I always create a module called “appEngine”. This is like the director of the app. Everything else comes underneath. My main.lua file typically only creates an instance of “appEngine” and calls a function like appEngine.startMenu()

  2. As I said before, everything needs to be organised in a hierarchy. So “myMenu” is called as a local inside “appEngine”.

  3. Whenever you instance a new object, it need to receive it’s “super” object as a parameter. So when you create a menu from the appEngine module, you would say something like:

local myMenu = menuClass.new(self)

And then in menuClass you would do something like

mySuper = theValueThatWasPassedToMeReferringToAppEngine

This way myMenu knows that it’s “parent” is appEngine and can talk to it. And appEngine of course knows that myMenu is a child object, or further down in the hierarchical tree.

  1. When you know you will create a lot of similar items, create a “manager” class above it in the tree. So for instance, when you have a game with ghosts as enemies, first create a module called myGhostManager. This module has an array that holds the child object which are modules called ghostClass, for example. The ghostManager class takes care of all “ghost management”: adding new ones, removing old ones, removing all at end of level etc…

  2. If you construct things this way, EVERY object or module will be able to talk to eachother, because they can address every single item in your hierarchy by going up and down the list.

  3. This sometimes creates ugly code like self.super.super.super.myGhostManager:killAllGhosts() - meaning, go up 3 levels in the hierarchy tree (the super of the super of the super of myself, or the parent of my parent of my parent), but it works!

  4. For debugging reasons I give every module or class I write a property called self.type, with a string describing what type of object it is (an appEngine, a ghost, a ghostManager etc…). In case I get confused I just print self.super.super.super.type to the console to see what type of object I’m talking to.

Voila. Not sure I’m making a lot of sense here, but the method above is extremely powerful and flexible!

#5

Thanks Thomas,

That does make sense to me I think, yes. My concern though is that I’m building a plugin so I want that when a development uses this plugin, the code from within that development can just call myplugin.myfunction() without having to incorporate hierarchy into that call, but that internally, within myplugin, the actual code providing all of the myfunction()'s is logically separated into ‘modules’.

I can totally see how with some developments having the hierarchy open like this is useful - i.e. somePlugin.math.round() and somePlugin.graphics.circle() would be fine while somePlugin.round() and somePlugin.circle() could cause confusion as to which does what, but I think in my case I’d prefer to just be able to call somePlugin.someFunction() without having to think about there someFunction() is going to be provided from the plugins internal hierarchy…

Perhaps it’s just that I’m new to Lua and I’m thinking about it wrongly. If what I’m trying to achieve really isn’t the done thing then I guess I should just follow your approach. I’ve never been a fan of OOP’s classes hierarchy though and Lua seems to me like it has procedural roots so I’d sort of expected require() to work more like say… PHP’s require() function, where the included code is brought in and effectively runs inline with everything else rather than it being called as a standalone class.

#6

Hi Richard, It’s just tables.

You can assign references to the modules functions  as members of the main plugin table

local myModule = {} -- declare your functions as you like. -- either as module members myModule.function1 = function () ... end -- or as locals local function2 = function () ... end -- then assign them to the main module myModule.init = function (mainModule) mainModule.function1 = myModule.function1 mainModule.function2 = function2 end return myModule

and in the plugin

local plugin = {} -- require the subModule local myModule = require "myModule" -- get the module functions myModule.init(plugin) -- now you can call them directly as they are from the main module itself plugin.function1() return plugin

just keep track of local and self scopes, they might become confusing sometimes…

#7

Thanks Arteficio. If I’m understanding this correctly, it’s an ingenious method!

One thing though - say I now have the following plugin:

[lua]

local Library = require “CoronaLibrary”

local lib = Library:new{ name=‘myplugin’, publisherId=‘foo.bah’ }

local myVariable = 0

local module1 = require “plugin.myplugin.module1”

module1.init(lib)

local module2 = require “plugin.myplugin.module2”

module2.init(lib)

return lib

[/lua]

But module1.lua and module2.lua both need to read and write to myVariable… How would I reference it from within those files?

Similarly, how would module2.function1() reference module1.function2() ? Presumably lib isn’t available within their scope otherwise your init() function wouldn’t be needed?

#8

OK, I’ve reworked half of my plugin to be modulous now, thanks to the examples above, but I’m getting confused by the variable scope.

For variables that multiple modules within the plugin need to refer to, I’m making them global within the main plugin library. However this of course means that they’re open to the code which loads in my plugin too, and I don’t want that. I want them to be local within the scope of the plugin but for the plugin to be modulous.

Next, where I have modules that need to call functions within other modules, I’m currently require’ing those at the top of the caller module, as well as using the init() trick above to add them in as library calls. I think this is only working though because the functions I’m currently sharing don’t do anything very fancy but for example if the called module had an internal local variable and I was calling a function to return that value, I’m guessing this would return different values because I’m effectively creating multiple instances of that module?

#9

You’re right.
As far as i know, the way require works is different from php. Each lua source has it’s own scope so i don’t think a local variable declared outside the module can be accessed from the module itself.

Even cross referencing between the modules might be a problem: if module1 requires module2, module2 cannot require module1

I have a possible structure in mind that might be helpful but i’m on mobile now and writing code on a smartphone is something i feel a bit uncomfortable.
I’ll be back tomorrow.

#10

Greatly appreciated and very much looking forward to seeing what you come up with! I can’t decide if I’m just going about this the wrong way, as a rookie to Lua, or if this is something the language actually fails at. If we can’t split code into tidy files without losing the ability for that code to share variables then long scripts must get very difficult to manage.

#11

I’m also on mobile right now and can’t dig further into this just yet, but https://docs.coronalabs.com/tutorial/basics/globals/index.html is quite interesting actually. The method at the end suggests that if you load a module in from the top of all other modules, the variables it creates are indeed then shared. If this works then it should resolve my issue, but it’s not making much sense to me how this doesn’t just create multiple instances of those variables, one instance within each module. In fact, I’m now wondering how you would actually go about creating a second instance of a module… Say if that module was to create an NPC and you needed multiple NPC’s, my instinct would be to do this: [lua]local npc1 = require “npcModule” local npc2 = require “npcModule”[/lua] But going by this doc, it sounds like npc2.hp = 10 would result in both npc1.hp and npc2.hp setting as 10, because they’re in the same memory space? Will play with this more tomorrow.

#12

Hi Richard,

If you require a module in different places of your code, in reality it will only be loaded once. All extra times it is required, it will just be a sort of copy, or reference to the same already loaded module.

In reality this is super handy, because you could, for example, have a module that contains the game score, or settings that are needed across your app. Everywhere you require this module, you will have a single “store of values” that is the same one. In a way, it is like a “global” module wherever you require it.

So, how do you create multiple, separate instances of a single module? Here’s how:

Create a module called npcClass.lua:

local npcClass = {} npcClass.new = function(startingHealth, playerSpeed) local self = {} self.health = startingHealth self.speed = playerSpeed return self end -- npcClass.new() return npcClass = {}

In the example above you could replace “self” with “newPlayer” if that reads better for you, but “self” is more standard and more practical, trust me.

Then two create two separate instances of an npc just say - in main.lua for example, or another module:

loyal npcClass = require("npcClass") local player1 = npcClass.new(100,5) -- for health and speed local player2 = npcClass.new(90,2)

That’s it! This is very simple, but also extremly powerful.

#13

Bingo, I think it’s all clicking in to place now, thanks!

#14

You’re welcome. Let us know if you need clarification.

Honestly, I’ve programmed in many languages, and I do feel that for 99% of my programming needs, the pseudo-OOP approach of Lua is a wonderfully quick and simple way to do things.

#15

When you require a Lua module, it gets loaded, executed and it returns a table.

When you require it again you simply get a reference to the same table.

#16

Ah yes, excellent point by Arteficio: when you require a module, the code inside the module is executed, which is superhandy. And then the variable that is returned at the end (typically a table containing functions and variables) is returned.

The second time you require the module, the code is not executed anymore.

#17

Hi,

What I am going to demonstrate here is extremely abstract. If there is interest I can follow up with a more detailed blog post on some various methods of building Lua based plugins at some point, but for now this will hopefully give you something to study.

This is aimed at answering your original question. There are a number of different things to consider depending on the functionality of your specific plugin, including scope, and whether there will be any asynchronous activity within separate modules. Such as methods from the Corona network library.

When developing in Lua, I tend to lean toward composition; passing specific modules to other modules where that functionality is needed. But again, this is very dependent on what you are trying to achieve with the plugin. Some methods are better than others.

In my experience, there are three main approaches; class instances, composition, and event driven. As noted, I tend to use composition, and because of the asynchronous nature of Corona, event driven more often. As others have mentioned, there is always a central point of entry which handles the other modules accordingly.

With all of that in mind, here are the various pieces presented in an abstract way and only with your original question in mind based on the composition methodology.

So first the project tree:

plugin myplugin dataHandling.lua engine.lua rendering.lua myplugin.lua main.lua

And the files:

myplugin.lua

local Library = require "CoronaLibrary" -- Create library local lib = Library:new{ name='myplugin', publisherId='com.develephant' } return setmetatable( lib, { \_\_index = require("plugin.myplugin.engine") } )

engine.lua

--============================================================================= --== Plugin Components --============================================================================= local rendering = require("plugin.myplugin.rendering") local dataHandling = require("plugin.myplugin.dataHandling") --============================================================================= --== Main Engine --============================================================================= local engine = {} --============================================================================= --== Engine Methods --============================================================================= function engine.doSomething() --call a render method rendering.doSomething() --do some data handling dataHandling.doSomething() end function engine.doSomethingWithInput(var1, var2) --call rendering with argument local result = rendering.doRenderThing(var1) --return the result return result end function engine.renderWithDataHandling() local res = rendering.withDataHandling(dataHandling) end --============================================================================= --== Engine Export --============================================================================= return engine

rendering.lua

--============================================================================= --== Rendering Component --============================================================================= local rendering = {} function rendering.doSomething() --render something end function rendering.renderThing(var) --render a thing, return result return some\_result end function rendering.withDataHandling(dataHandling) local result = dataHandling.getSomeData() --do some rendering stuff return some\_result end return rendering

dataHandling.lua

--============================================================================= --== DataHandling Component --============================================================================= local dataHandling = {} function dataHandling.doSomething() --data handle and return result return some\_result end function dataHandling.getSomeData() --get some data return some\_result end return dataHandling

main.lua

local myplugin = require("plugin.myplugin") --These following methods live in the engine.lua myplugin.doSomething() local result = myplugin.doSomethingWithInput("hello")

This may end up leaving you with more questions than answers, but it does demonstrate a method to solve your original question in regards to plugins.

Hope this helps at least give you some initial direction.

-dev

#18

Thanks very much, everybody.

Develephant, your approach I feel is probably the most proper way to go about this and removes the possibility that two libraries might try to create the same function name, but I can foresee engine.lua filling up with functions this way, which almost defeats the purpose of making things modulous.

What I’ve ended up doing, and I’m quite pleased with it, is creating the main engine.lua as just require()'s and init()'s as per Arteficio’s approach, and then at the top of each required library file, I’m require()'ing a main globals.lua for the global variables as per the example given on https://docs.coronalabs.com/tutorial/basics/globals/index.html , plus also require()'ing any of the other liraries that this library relies on.

I’m happy with how tidy this approach is and I like that I now effectively have global variables that are only global within the scope of the plugin, and I especially like that the init() approach means I can pick and choose which functions to make available to the plugin user without having to declare them any differently for use internally. My only concern now is that I don’t think two libraries would be able to require each other without this causing an infinite loop. I suppose though, this just means I’m forced to be more logical with the structuring which is probably a good thing.

Main plugin file:

[lua]local Library = require “CoronaLibrary”

local lib = Library:new{ name=‘foobah’, publisherId=‘foo.bah’ }

local module1 = require “plugin.foobah.module1”

module1.init(lib)

local module2 = require “plugin.foobah.module2”

module2.init(lib)

return lib[/lua]

Modules:

[lua]local vars = require “plugin.foobah.globals”

local module = {}

local someLocalVariable = 1

module.someFunction = function()

    someLocalVariable = someLocalVariable + vars.someGlobalVariable

    return someLocalVariable

end

module.init = function(main)

    main.someFunction = module.someFunction

end

return module

[/lua]

And the globals:

[lua]local module = {}

module.someGlobalVariable = 1

return module[/lua]

#19

Out of interest, is this just how Lua works as a language, or is it Corona’s interpretation of Lua that means subsequent require()'s don’t create new instances?

#20

All of Corona is pure Lua, to exact specification, and the proprietary API for the Corona engine.