Problem with empty files when reading/writing data to files

Solar2D General Questions/Discussion
#1

Hello,

I’ve got a very tricky problem in my app. It happens only very occasionally, only to a very small amount of users. However, when it happens, they lose all their game progress. So I really would like to solve this issue.

I’m using GGData (Source code) for saving/reading data. Generally everything works fine. However, in some rare cases, the files in which the data is stored, are empty. They exist, but they are completely empty.

This can’t happen on a “normal” way, since the writing is done the following way, and json.encode will never return an empty string:

data = json.encode( data ) path = system.pathForFile( self.path .. "/" .. self.id .. ".box", system.DocumentsDirectory ) local file = io.open( path, "w" ) if not file then return end file:write( data )

I also built in a debug event to check whether data is nil before writing, which does not happen.

What I did find out so far is, that with opening the file in “w” mode, it becomes empty. So if the writing would fail afterwards, the file would stay empty (is there any alternative to that? Some kind of “safe” opening mode?)

I’m using 3 “GGData-box” objects parallel, so 3 files are involved as well. Could it be a problem when trying to write to 2 different files (that means using GGData:save()) at the same time?

The problem occurs both on iOS and Android.

Has anyone experienced something similar or has ideas or other input on this issue?

Best regards!

#2

why don’t you open the file with “r”?

“w” will overwrite your file.

#3

Of course I’m only opening the file with “w” when saving.

When loading the file I’m opening with “r”.

#4

sorry for the messy answer. I didn’t explain it correctly. (i had to go for a time…sent message without completing it)

Why don’t you open file first with “r”, ofc. save the result to a temporary variable or table…do whatever you already do with “w” to the old file…if it fails you can get back your old info because you have it in the temporary file. you can check if it fails with the size of the new file for example.

i also think it’s strange that this problem only affect a few devices and sometimes or all the time in those devices?

because the problem could be outside your app. there are apps that can clean all app data, even android let you clean your data. (don’t know if it will delete in system.DocumentsDirectory, never done those tests).

why don’t you use sqlite to save data? i use it in all my apps with 0 problems till now.

or, you can pass data to a remote database with the bonus that you can continue your progress in another device, even if you need to factory reset your device you can continue your game.

#5

I’m not sure how I feel about this:

data = json.encode( data )

First, we don’t know what data is. Is it a local? a global? You’re writing over the original data table with the encoded JSON string. You also don’t appear to be testing that the encoding was successful. 

Rob 

#6

The complete GGData save function looks like this (source code is linked above):

--- Saves this GGData object to disk. function GGData:save() -- Don't want this key getting saved out local integrityKey = self.integrityKey self.integrityKey = nil local data = {} -- Copy across all the properties that can be saved. for k, v in pairs( self ) do if type( v ) ~= "function" and type( v ) ~= "userdata" then data[k] = v end end -- Check for and create if necessary the boxes directory. local path = system.pathForFile( "", system.DocumentsDirectory ) local success = lfs.chdir( path ) if success then lfs.mkdir( self.path ) path = self.path else path = "" end data = json.encode( data ) path = system.pathForFile( self.path .. "/" .. self.id .. ".box", system.DocumentsDirectory ) local file = io.open( path, "w" ) if not file then return end file:write( data ) io.close( file ) file = nil -- Set the key back again self.integrityKey = integrityKey end

So data is local.

Also in my current productive version I have a debug event included like this:

if data then     file:write (data)     io.close (file)     file = nil else     -- fire debug event end

This event is never triggered.

#7

do you use GGData:verifyIntegrity?

you can create a backup file data when it success to write to a file. (copy file) (you need to check also the integrity of the backup file ofc)

you can use verifyIntegrity to check if the file your writing is valid at the end.

if it returns false you can copy back  your backup to original file.

still, I think going for internal database is 10x simpler, faster, and with 0 worries. There’s a code out there to simplify your life using databases if you don’t understand much about them. Just look in the forums. I passed it some days ago.

#8

why don’t you open the file with “r”?

“w” will overwrite your file.

#9

Of course I’m only opening the file with “w” when saving.

When loading the file I’m opening with “r”.

#10

sorry for the messy answer. I didn’t explain it correctly. (i had to go for a time…sent message without completing it)

Why don’t you open file first with “r”, ofc. save the result to a temporary variable or table…do whatever you already do with “w” to the old file…if it fails you can get back your old info because you have it in the temporary file. you can check if it fails with the size of the new file for example.

i also think it’s strange that this problem only affect a few devices and sometimes or all the time in those devices?

because the problem could be outside your app. there are apps that can clean all app data, even android let you clean your data. (don’t know if it will delete in system.DocumentsDirectory, never done those tests).

why don’t you use sqlite to save data? i use it in all my apps with 0 problems till now.

or, you can pass data to a remote database with the bonus that you can continue your progress in another device, even if you need to factory reset your device you can continue your game.

#11

I’m not sure how I feel about this:

data = json.encode( data )

First, we don’t know what data is. Is it a local? a global? You’re writing over the original data table with the encoded JSON string. You also don’t appear to be testing that the encoding was successful. 

Rob 

#12

The complete GGData save function looks like this (source code is linked above):

--- Saves this GGData object to disk. function GGData:save() -- Don't want this key getting saved out local integrityKey = self.integrityKey self.integrityKey = nil local data = {} -- Copy across all the properties that can be saved. for k, v in pairs( self ) do if type( v ) ~= "function" and type( v ) ~= "userdata" then data[k] = v end end -- Check for and create if necessary the boxes directory. local path = system.pathForFile( "", system.DocumentsDirectory ) local success = lfs.chdir( path ) if success then lfs.mkdir( self.path ) path = self.path else path = "" end data = json.encode( data ) path = system.pathForFile( self.path .. "/" .. self.id .. ".box", system.DocumentsDirectory ) local file = io.open( path, "w" ) if not file then return end file:write( data ) io.close( file ) file = nil -- Set the key back again self.integrityKey = integrityKey end

So data is local.

Also in my current productive version I have a debug event included like this:

if data then     file:write (data)     io.close (file)     file = nil else     -- fire debug event end

This event is never triggered.

#13

do you use GGData:verifyIntegrity?

you can create a backup file data when it success to write to a file. (copy file) (you need to check also the integrity of the backup file ofc)

you can use verifyIntegrity to check if the file your writing is valid at the end.

if it returns false you can copy back  your backup to original file.

still, I think going for internal database is 10x simpler, faster, and with 0 worries. There’s a code out there to simplify your life using databases if you don’t understand much about them. Just look in the forums. I passed it some days ago.