[RESOLVED] Problems with MY popular saveTable() function

Omnigeek_Media · October 12, 2012, 7:15pm

How often do you find the developer of something asking for help on code that he wrote for you to use…

HELP!!!

Here is my routine:

function M.saveTable(t, filename)  
 local path = system.pathForFile( filename, system.DocumentsDirectory)  
 local file = io.open(path, "w")  
 print("in saveTable. I think my table is....")  
 M.print\_r(t)  
 if file then  
 local contents = json.encode(t)  
 print("json data is")  
 print("\*" .. contents .. "\*")  
 file:write( contents )  
 io.close( file )  
 return true  
 else  
 return false  
 end  
end

This has been working flawlessly for me and many of you… until now.

I’m building a game where you can have multiple matches going on. Each match is ID’ed with a big hex number like:

t[50778c04ec27b6000f00066b] = {
[1] = {question=“somestring”, answer=true},
[2] = {question=“someother string”, answer=false}
}
t[50778c04ec27b6000f00077c] = {
[1] = {question=“somestring else”, answer=true},
[2] = {question=“yet someother string”, answer=false}
}

When I JSON encode “t” I get an empty string. Now it’s not quite that simple as all my values are of course coming from other variables.

I wrote a test case, figuring I found a bug… and found it’s working as I thought:

json = require("json")  
  
mytable = {}  
a = "50778c04ec27b6000f00066b"  
mytable[a] = {}  
mytable[a][1] = {}  
mytable[a][1]["question"] = "RYKF?"  
  
print("\*" .. json.encode(mytable) .. "\*")

outputs:

*{“50778c04ec27b6000f00066b”:[{“question”:“RYKF?”}]}*

So my first thought is that for some reason that big hex number might be treated as a number, not a string. I know Lua tables don’t like t[1] = nil; t[2] = “something” that many things see that first entry as nil and assumes there is no other data.

The bright light went on, so I went and changed all the the top level key’s to be:

t[tostring(reallybighexnumber)]

So it would be a table not an array and that would solve my problem. BZZZT wrong answer. Tonight I tried forcing it to be a string by concatenating that big hex number to the letter “Z” (i.e. “Z” … matchID and that’s in a variable so it should work as a key.

Sure enough when I run my print_r function, I see my table as I expect it, but when I json.encode it, I get an empty string.

I’m stumped. Why can’t json.encode deal with this table?

Oh here is the console dump from my function at the top:

2012-10-12 22:02:45.833 Corona Simulator[733:707] in saveTable. I think my table is.... 2012-10-12 22:02:45.833 Corona Simulator[733:707] table: 0x106232710 { 2012-10-12 22:02:45.834 Corona Simulator[733:707] [Z50778c04ec27b6000f00066b] =\> table: 0x106232710 { 2012-10-12 22:02:45.835 Corona Simulator[733:707] [1] =\> table: 0x106229800 { 2012-10-12 22:02:45.835 Corona Simulator[733:707] [question] =\> "TEST?" 2012-10-12 22:02:45.835 Corona Simulator[733:707] } 2012-10-12 22:02:45.836 Corona Simulator[733:707] } 2012-10-12 22:02:45.836 Corona Simulator[733:707] } 2012-10-12 22:02:45.836 Corona Simulator[733:707] json data is 2012-10-12 22:02:45.837 Corona Simulator[733:707] \*[]\*

The table looks like it should be saving properly, but json.encode is blowing up.
EDIT: Well this isn’t really resolved, but I switched to using SQLlite for this since I need a key lookup system rather than a big table to look through. But I think MY Developers found the problem with the cyclic table reference and that was likely what was breaking json.encode.

[import]uid: 19626 topic_id: 31912 reply_id: 331912[/import]

Omnigeek_Media · October 13, 2012, 9:51am

Any thoughts as to why JSON is having trouble with that table? [import]uid: 19626 topic_id: 31912 reply_id: 127320[/import]

Stephen_Lewis · October 13, 2012, 11:22am

A couple thoughts:

Since variable names can’t start with numbers maybe JSON is encoding the key 50778c04ec27b6000f00066b
as a variable name and since it starts with 5 JSON chokes.
Maybe the key in the lua table can’t be 50778c04ec27b6000f00066b since if it’s interpreted as a variable name it would be 50778c04ec27b6000f00066b=nil and a key in a lua table can’t be nil.

3.I’m pretty sure you can’t do t[50778c04ec27b6000f00066b] = “hello” but you CAN do t[“50778c04ec27b6000f00066b”] = “hello”.

You CAN do t[1] = “hello” because in that case the key is interpreted as an index. But I don’t know what happens if you try to JSON encode that.

[import]uid: 9422 topic_id: 31912 reply_id: 127323[/import]

Omnigeek_Media · October 13, 2012, 11:35am

I prepended the letter “Z” to the number which should force it to be a string making it:

[Z 50778c04ec27b6000f00066b]

And that didn’t work either.

#3, yes you can do that. I tried that early on. I may give up on using the matceID as an index and just make it a big array of tables and give each table a key that is the matchID and I’ll just have to loop over the the table looking for my match.

[import]uid: 19626 topic_id: 31912 reply_id: 127324[/import]

Stephen_Lewis · October 13, 2012, 11:59am

I recently started using Lua Glider and it is pretty good at pointing out errors in my code. If I enter

t={}  
t[50778c04ec27b6000f00066b] = "hello"  
t[Z50778c04ec27b6000f00066b] = "hello"  
t["50778c04ec27b6000f00066b"] = "hello"  
t["Z50778c04ec27b6000f00066b"] = "hello"

Glider says there is a"Parse Exception" error in Line 2, a warning that line 3 has something “possibly uninitialized”, and has no problem with lines 4 and 5.

For whatever it’s worth, there might be some clues for your problem there.
[import]uid: 9422 topic_id: 31912 reply_id: 127325[/import]

Omnigeek_Media · October 13, 2012, 12:11pm

Glider likes to throw too many of those possibly uninitialized errors for my liking. I stopped using it because they can’t get indents and back indents working correctly. I love their SVN integration.

Try:

a = “Z50778c04ec27b6000f00066b”
t[a]= “hello”
This is closer to what I’m actually doing. [import]uid: 19626 topic_id: 31912 reply_id: 127326[/import]

Stephen_Lewis · October 13, 2012, 12:37pm

Right, maybe I’m reading your original description wrong. It looked like to me your test code was working, as expected, using something like
a = “Z50778c04ec27b6000f00066b”
t[a]= “hello”
Which I agree should work.

But that your actual function wasn’t working as expected since you were passing in a table constructed more like this:
t[Z50778c04ec27b6000f00066b] = {“hello”}

i.e. missing quotes in the key, so it’s using a nil defined variable rather than an actual string as the key, which is a lua no-no. [import]uid: 9422 topic_id: 31912 reply_id: 127330[/import]

M.Y.developers · October 13, 2012, 2:09pm

@robmiracle,

There seems to be a fundamental problem which Stephen alluded to earlier. The code:

t={}  
t[Z50778c04ec27b6000f00066b] = {  
 [1] = {question="somestring", answer=true},  
 [2] = {question="someother string", answer=false}  
}

Z50778c04ec27b6000f00066b is an identifier and you have not defined it so it is equivalent to:

t={}  
t[nil] = {  
 [1] = {question="somestring", answer=true},  
 [2] = {question="someother string", answer=false}  
}

Which is going to be a problem.
Also note that"

t[tostring(Z50778c04ec27b6000f00066b )]

is also equivalent to:

t[tostring(nil )]

which also is a problem.

You probably meant to do this:

local json = require "json"  
t={}  
t[0x50778c04ec27b6000f00066b] = {  
 [1] = {question="somestring", answer=true},  
 [2] = {question="someother string", answer=false}  
}  
t[0x50778c04ec27b6000f00077c] = {  
 [1] = {question="somestring else", answer=true},  
 [2] = {question="yet someother string", answer=false}  
}  
print(json.encode(t))

which in this case is perfectly valid as those are now number literals instead of undefined identifiers. However, here is the output:
{“4294967295”:[{“answer”:true,“question”:“somestring else”},{“answer”:false,“question”:“yet someother string”}]}

Also, the giant hex number you have is a 96 bit number, if you use it as it is or even convert it to a string you will get an overflow. Integers are probably stored as 32 bit numbers.

Here is what you get when you print:

print(0x50778c04ec27b6000f00077c)  
print(0x00778c04ec27b6000f00066b)

4294967295
4294967295

and these numbers happen to equal FFFFFFFF in hex. As you can see, its an overflow. What will result is you will end up overwriting table values as lua thinks it is the same key.

Regards,
M.Y. Developers [import]uid: 55057 topic_id: 31912 reply_id: 127335[/import]

Omnigeek_Media · October 13, 2012, 7:11pm

It might help to see my actual code:

 answers[#answers + 1] = {}  
 answers[#answers].question = myQuestion  
 answers[#answers].answer = nil  
 storyboard.answers["Z" .. storyboard.matches[thisMatch].id] = answers  
 print("\*\*\*\*\*\*\*\*XXXXXXXX\*\*\*\*\*\*\*\* should be saving my table \*\*\*\*\*\*\*\*\*XXXXXXXXX\*\*\*\*\*\*\*\*\*\*\*\*\*")  
 utility.print\_r(storyboard.answers)  
 utility.saveTable(storyboard.answers,"answers.json")

In this case, lets say “thisMatch” = 1 and storyboard.matches[thisMatch].id is that hex number.

Here’s the actual output:

 \*\*\*\*\*\*\*\*XXXXXXXX\*\*\*\*\*\*\*\* should be saving my table \*\*\*\*\*\*\*\*\*XXXXXXXXX\*\*\*\*\*\*\*\*\*\*\*\*\*  
table: 0x10c128450 {  
 [Z50778c04ec27b6000f00066b] =\> table: 0x10c128450 {  
 [1] =\> table: 0x114847f70 {  
 [question] =\> "FUDGE?"  
 }  
 }  
 }  
in saveTable. I think my table is....  
table: 0x10c128450 {  
 [Z50778c04ec27b6000f00066b] =\> table: 0x10c128450 {  
 [1] =\> table: 0x114847f70 {  
 [question] =\> "FUDGE?"  
 }  
 }  
}  
json data is  
\*[]\*

So I’m pretty confused.
[import]uid: 19626 topic_id: 31912 reply_id: 127347[/import]

M.Y.developers · October 13, 2012, 9:29pm

Hello Rob,

Seems like you might have a cycle in your table?

The print function probably handles cycles, the Json encode probably can’t.

Regards,
M.Y. Developers [import]uid: 55057 topic_id: 31912 reply_id: 127354[/import]

Omnigeek_Media · October 14, 2012, 8:08am

That certainly would be problem.
I’m trying to do it with SQLite instead of a flat file and I"m running into a similar issue. It never actually seems to write to the database. I see the content is encoded correctly, I make a unique database key “Match” … biglonghexnumber and the result of the REPLACE INTO is a 1 indicating success.

But I probably still need to fix this cyclic reference.

Thanks!
Rob
[import]uid: 19626 topic_id: 31912 reply_id: 127369[/import]

Omnigeek_Media · October 13, 2012, 9:51am