Hey Dewey.
Have you considered grabbing the open source dkjson.lua file from the web and seeing if the bug exists there? And if so possibly fixing it, if the need is urgent to release your app?
Not saying you should have to do this, but afaik it’s a possible option.
Cheers
Yes, thank you.
That’s exactly what I intend to do just as soon as I:
Rob…can you find out which JSON library you guys are currently using?
Thx
Dewey
oh, perhaps you are telling me: is it dkjson??
We’re currently using DKJSON 2.2 and plan to update to 2.5 soon.
I don’t know if a more recent version of DKJSON fixes this issue (there’s a commit for 2.4 relating to “Fixed encoding and decoding of numbers in different numeric locales” which I guess might be apropos though it sounds unlikely).
Great!! I’ll go check the new version and see if its been fixed…I dont think my C is good enough to fix it if not
Thanks
Dewey
Oh, I forgot to mention …
I would just treat the returned ids as strings. They are opaque values which just happen to look like large numbers. You aren’t going to be doing math with them so just avoid having any part of the software try to convert them between representations.
DKJSON is a pure Lua implementation.
Super…guess I was looking at the wrong repo…here:
https://github.com/LuaDist/dkjson
I’ve got 8 tables that sync with the server…and the foreign key-names are not equivalent across tables…and I have greater-than (old vs new) comparisons (if stmts) that would have to change.
It would be very time consuming and laborious to go add this tostring casting into all my rest calls, queries and conditionals…
I’m hoping I don’t have to go that route…if it’s pure Lua, I’m hopeful…
Oh, if I load my own dkjson source, will it be able to see the CL internal LPEG library??
It will still work if not, albeit a bit slower…
Thanks again for looking at this!
D
This should do the trick:
local lpeg = require("lpeg")Just tested with 2.5…problem is still there.
Will report back later…
D
doohh…I clearly need some sleep 
yeah, he’s just looking for the std lpeg.so – so all good
Think I’ll shut it down tonight and look again with a clear head later in the weekend…
Thanks everyone for the tips and suggestions
Out of curiosity, I looked into the precise limits on integer size in Lua (our Lua, like most, uses 8 byte doubles to represent numbers).
Consider this code fragment:
local maxdouble = 2^53 -- one less than the maximum can be represented precisely print (string.format("%.0f",maxdouble-1)) --\> 9007199254740991 -- the maximum itself can be represented precisely print (string.format("%.0f",maxdouble)) --\> 9007199254740992 -- one more than the maximum gets rounded down print (string.format("%.0f",maxdouble+1)) --\> 9007199254740992 again
Output:
2014-05-27 11:24:20.996 Corona Simulator[60038:303] maxdouble-1 (9007199254740991): 9007199254740991 2014-05-27 11:24:20.996 Corona Simulator[60038:303] maxdouble (9007199254740992): 9007199254740992 2014-05-27 11:24:20.997 Corona Simulator[60038:303] maxdouble+1 (9007199254740992): 9007199254740992
So you can represent integers towards the upper end of 16 digits but the wheels come off when you go greater than 2^53.
Add a leading digit (to the example value supplied):
local mynumber = 23334445556667711 print (string.format("mynumber-1 (23334445556667710): %.0f",mynumber-1)) --\> should be 23334445556667710 print (string.format(" mynumber (23334445556667711): %.0f",mynumber)) --\>should be 23334445556667711 print (string.format("mynumber+1 (23334445556667712): %.0f",mynumber+1)) --\>should be 23334445556667712
And the output is incorrect (actually, and unfortunately, it’s close enough to the original number that you might not notice for a while that you’re in trouble):
2014-05-27 11:24:20.999 Corona Simulator[60038:303] mynumber-1 (23334445556667710): 23334445556667712 2014-05-27 11:24:21.000 Corona Simulator[60038:303] mynumber (23334445556667711): 23334445556667712 2014-05-27 11:24:21.001 Corona Simulator[60038:303] mynumber+1 (23334445556667712): 23334445556667712
Note that the same thing would happen if your 16 digit number happens to have greater than 90 for its first two digits.
So remember that Lua doesn’t actually support 64-bit integers, though it gets fairly close. Things that are opaque ids should be transmitted as strings. If you want to do math with them, then you’ll have make your own arrangements for arbitrary precision numbers.
agreed…and much of this is supposed to get fixed in 5.3 but for now I’ve had to patch dkjson as follows:
change line 183 (approx) in the encoder:
return replace(fsub(strformat("%.16g",num), numfilter, “”), decpoint, “.”) – my fixed version
– return replace(fsub(tostring(num), numfilter, “”), decpoint, “.”) – orig dkjson version of line 183 or thereabouts
– I think I made a comment or two at the top of the file so 183 may not be exact…
That seems to resolve my problems without my having to go cast back and forth (number to string) all over the place…