Json.encode not enough memory error

Solar2D
#1

I have a very large table that I want to json.encode but I get a “not enough memory” error. Has anyone else encountered this?
I thought about breaking up the operation and then piece back together the json like this:

    Local luaString
    For i,v in ipairs(data.layers) do
          luaString = luaString .. json.encode( data.layers[i] )
    end

It’s not working out for me… any ideas? Thanks in advance :slight_smile:

#2

Never.

Where?

  • On a PC or Mac in the simulator?
  • On a mobile device?

Again, never seen such a thing… however, looking at your code that string concatenation thing you’re doing is weird. I think that may be the problem. If data.layers is a table, just encode it in one shot.

   local encoded = json.encode(data.layers)

Question: You’re not trying to json encode display objects are you? If you are, that won’t work. They are no serializable in that way.

Question 2: Can you describe the data you are encoding? Maybe you can produce reduced table containing just the important parts and encode that?

#3

I’m using the PC simulator. The data in the table is the map of my game. It’s initially generated using the Tiled Map Editor. I encode it so I can then save it in a SQLite file. All my smaller maps work fine, its just my giant outside map that causes the problem.

At first I wasn’t seeing an error, it just wasn’t working. Then I split up my save game process using a Coroutine And fired each step off with a delay, thinking I needed to give each step of encoding and writing to SQL a little more time. Now I get the error message at the time its trying to encode the large map.

Looking at the raw data I now see there’s more in there that’s just a layers table. So my plan wouldn’t work anyway. Maybe I just cant have huge maps in my game…

#4

More questions…

  • Why sqlite? How does that help?
  • How many bytes is the source file you’re reading from disk?
  • Why not just leave it in a table in memory?
  • I still don’t understand the sequence of events. What tiled export type are you using? What formats are you converting to along the way? You should just be exporting to a lua file which returns a table when required. Unless you’re stitching a bunch of levels together I see no reason why you’d need anything more than this.
  • How many objects are represented in your map/level?
  • If you need more than one frame to read just the file representing your level… that sounds like a problem
#5

Thanks for helping me get to the bottom of this. I’m pretty sure I can solve my problem by making my map smaller. Ill try and answer your questions though, any advice is welcome.

  • I use sqlite to save my maps because the maps change during game play. Players can place objects, plant and harvest plants, place things in containers etc…
  • The file from tiled is exported in LUA and is about 6000kb.
  • The sequence goes like this:
    –Map files are exported from tiled as lua
    – Player initiates a new game
    – Map files go through some processing and turned into a new table in my game called mapData
    – when player saves game mapData is turned into json and written to a SQL file
    – player can resume game by turning the json back into mapData variable.
  • During gameplay I also add in a huge table containing pathifinding data for my spawns, after further testing this seems to be whats putting it over the top.

I’m going to analyze my data further and either shrink it or take a different approach to encoding into json. Ill get it sorted eventually…

#6

Cool. If you can’t trim down the total data, you’ll probably need to split it up into smaller chunks and save/load them separately.

One last note. In my experience, sqlite isn’t that fast and working with a table in memory is far quicker. That doesn’t solve the saving/loading time but I wonder if sqlite acesses aren’t part of the problem. That said since I mostly avoid using it, I could well be wrong here.

Best of luck! Please post back when you get t his solved so we can hear the good news and hopefully learn from your experience.