How to manage Lua source code files?

When my app grows, I found the number of my source code files also grows. And some files have over 3,000 lines of code.

 

They are all placed in the same directory with main.lua, so it becomes harder & harder to manage them.

 

I do use “require” and write modules. However, I am thinking how to break down one module?

 

For example, I have a module that have 10,000+ lines of code, how should I break it?

 

Should I use this?

dofile ([filename])

and break them into chunks of code and place them into some sub-folder?

 

What’s the right way to do this when the Lua application is getting bigger & bigger?

My current project consists of 57.7k lines of Lua so far.

This includes some libraries like particle candy and such, but it’s basically a lot of Lua.

My directory structure looks like this:

/

    build.settings

    config.lua

    main.lua

    … misc 1 file libs like memory profiling

    … ttfs

    … icon pngs

/audio

/images

    /sprites – sprite sheets and map files

    /fonts – sprite sheets and map files and supporting graphics for bitmap fonts

    /ui – misc ui jpgs and things that I dont want to or cant sprite

/lib – for abstract libraries and app related code

    /app – for app specific functionality

        services.lua – networking

        analytics.lua

        util.lua – misc tools, including scene history management, some popups

        /setup

            db.lua – script to setup and manipulate mobile database

            globals.lua – datastructure, constants

            overrides.lua – any changes or additions to default lua libs

    astar.lua

    strict.lua

    etc

/scenes – ui functionality

    /section1

        scenefile1.lua

        scenefile2.lua

        etc

    /section2

        scenefile1.lua

        scenefile2.lua

        etc

    …

Just the way I’ve done it.

@jack95

Thanks a lot for your reply.

What if a single lua file has many lines of code, say 10,000+ lines, what do you do with it?

For example, a storyboard scene that contains 10,000 lines of code, do you try to break it up? and how?

Short answer: No, I don’t.

My longest file is about 3.2k lines. As a general rule, when code gets overly complex, I will try to extract as many static functions as I can.

This often helps me reduce variable usage, identify problematic code, and other benefits but I believe this is a byproduct of my coding tendencies and not necessarily useful for all people. I do believe that the more granular you can make your algorithms, the better. Even if the function is only used once, it’s useful to extract what you can into more testable and maintainable chunks.

Yes, I do have many static functions, but if they are all in one single file, it is still kind of messy. 

And I am wondering what’s the best practice for this kind of problem.

Rewrite it.

You can see the result even in something as small as the Dusk Engine vs. Ceramic. Ceramic was an early version of Dusk, all in one file. When Dusk came along, it was split into multiple ones. The investment of time was certainly worth it, as now the layout of the engine is easily understood (layer building files go in the “layer” folder, runtime things go in “run”, etc). It did take a while to “break” it out, but it was worth it.

  1. Put all your single-file libraries (that are in the form of “library.lua” without a directory) into a folder named “libraries” or something like that. It’s important not to nest libraries like CBEffects or Dusk, because they all require() things within them.

  2. Create a new folder named “components” or something like that

  3. Grab your static functions (things like math calculations, that have no dependencies) and put them in a separate file, “functions.lua”, into the components folder

  4. Grab your level data files (exported from Tiled, written by yourself, whatever) and put them into a “level_data” folder

  5. Put images into an “images” directory (or “graphics”, whichever you like most)

Now your file tree should be a bit easier to understand. Now comes the trickier part, which may take some time.

  1. Create a class system

  6.1. Identify your major classes (= object types, if you don’t know any OO languages) like perhaps your player, basic enemy, level map, etc.

  6.2. For each major class, find the classes that could go “inside” them, like maybe “house object” could go in “level map” or “motorbike” could go in “player” or “sentient poisonous purple spot” could go inside “evil slug sorcerer”

  6.3. For each major class, create a new file named the class name and put it in a new folder named “components/objects”, for example, “objects/player.lua”, “objects/level_map.lua”, “objects/evil_slug_sorcerer.lua”

  6.4. As much as possible, detach each object’s creation code and put it into the class file for each object, under a method called “new”, “createObject”, or something like that

  6.5. Everything that that object  directly does should go inside the class definition and be added within the .new() function. I’ll give an example of this at the end.

  6.6. Put the minor class definitions (that you found in step 6.2) within their “parent” class file, if the result won’t be too big (otherwise, create a new class file for them). Put the code that creates them into a local function named something like “newMotorbike”, “newHouseObject”, or “newSentientPoisonousPurpleSpot”.

    6.6-b. If you had to create a new file for the minor class definition, require() it inside the “parent” class file, not your scene file

  1. Put the code that creates your level (not that which runs it, just the code that positions the ground and such) into a “load_level.lua” file inside the “components” directory. If you use something like Dusk or Lime that builds your level for you, you can skip this step

Example for this:

File tree:

[lua]

main.lua

config.lua

build.settings

maingame.lua – Your scene file

components/

  functions.lua

  load_level.lua

objects/

  evil_slug_sorcerer.lua

  player.lua

images/

  slug.png

  spot.png

  player.png

  motorbike.png

[/lua]

evil_slug_sorcerer.lua:

[lua]


–[[

Class: Evil Slug Sorcerer

Notes about the class go here in the header.

–]]


local class_evil_slug_sorcerer = {}


– New Sentient Poisonous Purple Spot


local function newSentientPoisonousPurpleSpot()

    local spot = display.newImage(“images/spot.png”)

    function spot:squish()

        print(“Sentient Poisonous Purple Spot is squishing!”)

    end

    return spot

end


– New Evil Slug Sorcerer


function class_evil_slug_sorcerer.new()

    local slug = display.newImage(“images/slug.png”)

    slug.life = 100

    slug.power = 10

    

    slug.spot = newSentientPoisonousPurpleSpot()

    ------------------------------------------------------------------------------

    – slug:sploosh()

    ------------------------------------------------------------------------------

    function slug:sploosh()

        print(“Evil Slug Sorcerer is splooshing!”)

    end

    ------------------------------------------------------------------------------

    – slug:castSpell()

    ------------------------------------------------------------------------------

    function slug:castSpell(player)

        print(“Evil Slug Sorcerer says, “Muhahaha! You are spellified!””)

        – Animate slug, etc.

        player:getDamaged(slug.power)

    end

    ------------------------------------------------------------------------------

    – slug:getDamaged()

    ------------------------------------------------------------------------------

    function slug:getDamaged(lifePoints)

        print(“Evil Slug Sorcer says, “Sniffle, sniffle.””)

        – Animate slug, etc.

        slug.life = slug.life - lifePoints

    end

    return slug

end

return class_evil_slug_sorcerer

[/lua]

class_player.lua:

[lua]


–[[

Class: Player

Notes about the class go here in the header.

–]]


local class_player = {}


– New Motorbike


local function newMotorbike()

    local motorbike = display.newImage(“images/motorbike.png”)

    function motorbike:vroom()

        print(“Motorbike is vroom-ing!”)

    end

    return motorbike

end


– New Player


function class_player.new()

    local player = display.newImage(“images/player.png”)

    player.life = 100

    player.power = 10

    player.motorbike = newMotorbike

    ------------------------------------------------------------------------------

    – player:runAround()

    ------------------------------------------------------------------------------

    function player:runAround()

        print(“Player is running around!”)

    end

    ------------------------------------------------------------------------------

    – player:attackSlug()

    ------------------------------------------------------------------------------

    function player:attackSlug(slug)

        print(“Player says, “Back, you evil thing!””)

        – Animate player, etc.

        slug:getDamaged(player.power)

    end

    ------------------------------------------------------------------------------

    – player:getDamaged()

    ------------------------------------------------------------------------------

    function player:getDamaged(lifePoints)

        print(“Player says, “Ow! Something pinched me!””)

        – Animate player, etc.

        player.life = player.life - lifePoints

    end

    return player

end

return class_player

[/lua]

Hope this helps! You might also look at this: http://en.wikipedia.org/wiki/Low-Coupling_/_High-Cohesion_pattern

  • Caleb

My current project consists of 57.7k lines of Lua so far.

This includes some libraries like particle candy and such, but it’s basically a lot of Lua.

My directory structure looks like this:

/

    build.settings

    config.lua

    main.lua

    … misc 1 file libs like memory profiling

    … ttfs

    … icon pngs

/audio

/images

    /sprites – sprite sheets and map files

    /fonts – sprite sheets and map files and supporting graphics for bitmap fonts

    /ui – misc ui jpgs and things that I dont want to or cant sprite

/lib – for abstract libraries and app related code

    /app – for app specific functionality

        services.lua – networking

        analytics.lua

        util.lua – misc tools, including scene history management, some popups

        /setup

            db.lua – script to setup and manipulate mobile database

            globals.lua – datastructure, constants

            overrides.lua – any changes or additions to default lua libs

    astar.lua

    strict.lua

    etc

/scenes – ui functionality

    /section1

        scenefile1.lua

        scenefile2.lua

        etc

    /section2

        scenefile1.lua

        scenefile2.lua

        etc

    …

Just the way I’ve done it.

@jack95

Thanks a lot for your reply.

What if a single lua file has many lines of code, say 10,000+ lines, what do you do with it?

For example, a storyboard scene that contains 10,000 lines of code, do you try to break it up? and how?

Short answer: No, I don’t.

My longest file is about 3.2k lines. As a general rule, when code gets overly complex, I will try to extract as many static functions as I can.

This often helps me reduce variable usage, identify problematic code, and other benefits but I believe this is a byproduct of my coding tendencies and not necessarily useful for all people. I do believe that the more granular you can make your algorithms, the better. Even if the function is only used once, it’s useful to extract what you can into more testable and maintainable chunks.

Yes, I do have many static functions, but if they are all in one single file, it is still kind of messy. 

And I am wondering what’s the best practice for this kind of problem.

Rewrite it.

You can see the result even in something as small as the Dusk Engine vs. Ceramic. Ceramic was an early version of Dusk, all in one file. When Dusk came along, it was split into multiple ones. The investment of time was certainly worth it, as now the layout of the engine is easily understood (layer building files go in the “layer” folder, runtime things go in “run”, etc). It did take a while to “break” it out, but it was worth it.

  1. Put all your single-file libraries (that are in the form of “library.lua” without a directory) into a folder named “libraries” or something like that. It’s important not to nest libraries like CBEffects or Dusk, because they all require() things within them.

  2. Create a new folder named “components” or something like that

  3. Grab your static functions (things like math calculations, that have no dependencies) and put them in a separate file, “functions.lua”, into the components folder

  4. Grab your level data files (exported from Tiled, written by yourself, whatever) and put them into a “level_data” folder

  5. Put images into an “images” directory (or “graphics”, whichever you like most)

Now your file tree should be a bit easier to understand. Now comes the trickier part, which may take some time.

  1. Create a class system

  6.1. Identify your major classes (= object types, if you don’t know any OO languages) like perhaps your player, basic enemy, level map, etc.

  6.2. For each major class, find the classes that could go “inside” them, like maybe “house object” could go in “level map” or “motorbike” could go in “player” or “sentient poisonous purple spot” could go inside “evil slug sorcerer”

  6.3. For each major class, create a new file named the class name and put it in a new folder named “components/objects”, for example, “objects/player.lua”, “objects/level_map.lua”, “objects/evil_slug_sorcerer.lua”

  6.4. As much as possible, detach each object’s creation code and put it into the class file for each object, under a method called “new”, “createObject”, or something like that

  6.5. Everything that that object  directly does should go inside the class definition and be added within the .new() function. I’ll give an example of this at the end.

  6.6. Put the minor class definitions (that you found in step 6.2) within their “parent” class file, if the result won’t be too big (otherwise, create a new class file for them). Put the code that creates them into a local function named something like “newMotorbike”, “newHouseObject”, or “newSentientPoisonousPurpleSpot”.

    6.6-b. If you had to create a new file for the minor class definition, require() it inside the “parent” class file, not your scene file

  1. Put the code that creates your level (not that which runs it, just the code that positions the ground and such) into a “load_level.lua” file inside the “components” directory. If you use something like Dusk or Lime that builds your level for you, you can skip this step

Example for this:

File tree:

[lua]

main.lua

config.lua

build.settings

maingame.lua – Your scene file

components/

  functions.lua

  load_level.lua

objects/

  evil_slug_sorcerer.lua

  player.lua

images/

  slug.png

  spot.png

  player.png

  motorbike.png

[/lua]

evil_slug_sorcerer.lua:

[lua]


–[[

Class: Evil Slug Sorcerer

Notes about the class go here in the header.

–]]


local class_evil_slug_sorcerer = {}


– New Sentient Poisonous Purple Spot


local function newSentientPoisonousPurpleSpot()

    local spot = display.newImage(“images/spot.png”)

    function spot:squish()

        print(“Sentient Poisonous Purple Spot is squishing!”)

    end

    return spot

end


– New Evil Slug Sorcerer


function class_evil_slug_sorcerer.new()

    local slug = display.newImage(“images/slug.png”)

    slug.life = 100

    slug.power = 10

    

    slug.spot = newSentientPoisonousPurpleSpot()

    ------------------------------------------------------------------------------

    – slug:sploosh()

    ------------------------------------------------------------------------------

    function slug:sploosh()

        print(“Evil Slug Sorcerer is splooshing!”)

    end

    ------------------------------------------------------------------------------

    – slug:castSpell()

    ------------------------------------------------------------------------------

    function slug:castSpell(player)

        print(“Evil Slug Sorcerer says, “Muhahaha! You are spellified!””)

        – Animate slug, etc.

        player:getDamaged(slug.power)

    end

    ------------------------------------------------------------------------------

    – slug:getDamaged()

    ------------------------------------------------------------------------------

    function slug:getDamaged(lifePoints)

        print(“Evil Slug Sorcer says, “Sniffle, sniffle.””)

        – Animate slug, etc.

        slug.life = slug.life - lifePoints

    end

    return slug

end

return class_evil_slug_sorcerer

[/lua]

class_player.lua:

[lua]


–[[

Class: Player

Notes about the class go here in the header.

–]]


local class_player = {}


– New Motorbike


local function newMotorbike()

    local motorbike = display.newImage(“images/motorbike.png”)

    function motorbike:vroom()

        print(“Motorbike is vroom-ing!”)

    end

    return motorbike

end


– New Player


function class_player.new()

    local player = display.newImage(“images/player.png”)

    player.life = 100

    player.power = 10

    player.motorbike = newMotorbike

    ------------------------------------------------------------------------------

    – player:runAround()

    ------------------------------------------------------------------------------

    function player:runAround()

        print(“Player is running around!”)

    end

    ------------------------------------------------------------------------------

    – player:attackSlug()

    ------------------------------------------------------------------------------

    function player:attackSlug(slug)

        print(“Player says, “Back, you evil thing!””)

        – Animate player, etc.

        slug:getDamaged(player.power)

    end

    ------------------------------------------------------------------------------

    – player:getDamaged()

    ------------------------------------------------------------------------------

    function player:getDamaged(lifePoints)

        print(“Player says, “Ow! Something pinched me!””)

        – Animate player, etc.

        player.life = player.life - lifePoints

    end

    return player

end

return class_player

[/lua]

Hope this helps! You might also look at this: http://en.wikipedia.org/wiki/Low-Coupling_/_High-Cohesion_pattern

  • Caleb