File error when trying to save data using JSON

Solar2D General Questions/Discussion
#1

I had encountered this issue with my own code, so this time I “borrowed” the loadsave code from Corona’s documentation in order to make sure it wasn’t just my code.

When I first start the app on simulator (or start it after having wiped the project’s sandbox), there are no issues. I create and save a table called “data” as JSON.

The problems arise when I start the app again and the data.json file exists in the project’s Documents directory. The table is loaded correctly, but if I try to save it again immediately, then I am met with the following error message: “File error: C:…\Documents\data.json: Permission denied”. According to the loadsave.lua file’s M.saveTable function, the file simply does not exist. However, if I try to save the table after a long delay by using timer.performWithDelay, like 15-20s, then the file does exist and everything works.

What is specifically the cause of this issue and how could I address it?

I have attached a sample project that demonstrates the issue. Just comment the rows 27 and 29 in main.lua to toggle between instant and delayed saving.

#2

I don’t have a PC but on my Mac and my android device, your code works as expected. No error. No delay needed.

That being said in general I wait until the system responds back that it is ready before I do any file operations. Not sure if this will fix your issue since I am not on a PC and couldn’t replicate your issue.

local function systemEvents( event ) delayedSave() end Runtime:addEventListener( "system", systemEvents )
#3

This sounds like an odd Windows caching issue. Perhaps Windows is holding on to a file handle too long, or a buffer hasn’t flushed yet. What do you need to write that immediately?

Rob

#4

Thanks for the replies.

On the first start up, I create and save a table for the general options and other variables as soon as the app starts, and this works without a hitch.

On future restarts, it isn’t that I am trying to write something immediately, but that some times it can take up to 14s until I can save the JSON file without encountering an error. In those 14s, I can go to the app’s options, change the app’s language, turn off music, etc. or maybe go to some level, perform some actions and still have time to spare. If I then close the app, all of those options should have been saved the moment that I made them. However, upon restart, I find that the changes weren’t saved and the console log keeps showing me the same error message every time I tried to save.

Once that varying time window since the app’s start has passed, I can save as frequently as I want without any errors.

#5

I save some 500KB locally and never had this issue.  Have you got some anti virus/malware scanner that is locking the file maybe?

#6

Seems like I need to be doing some testing with antivirus and related software then. Good to know that it is localised to my computer. 

#7

I’ve been trying to figure this one out for a while now and I’ve tried disabling all sorts of software that might be interfering, but I am at my wits end.

The duration for the file error also seems to be project specific. I used the following code to highlight the issue:

local function start() local count = 1 local function rapidSave() loadsave.saveTable( data, "data.json") print("Time:"..math.floor(system.getTimer()),"\tloop #:"..count) count = count+1 end timer.performWithDelay( 100, rapidSave, 50 ) end timer.performWithDelay( 2000, start )

Depending the project, it took anywhere from a few seconds to almost 20 seconds until the saving actually went through. And like before, once it goes through, then it just keeps on going without any interference. Here’s a few seconds long clip of the console log with the following code in a project of mine (https://gfycat.com/ImmenseWavyBrahmanbull). The console is barraged with errors, until everything just starts to work.

I would really appreciate it if anyone had a clue as to what is going on.

#8

can you try this?

local json = require "json" local data = {1,2,3,4,5,6,7,8,9,10} local function saveTable( dataTable, filename ) local myTable = json.encode( dataTable ) local path = system.pathForFile( filename, system.DocumentsDirectory ) local file = io.open( path, "w" ) if file then file:write( myTable ) io.close( file ) else print("error") end end local function start() local count = 1 local function rapidSave() local start = system.getTimer() saveTable( data, "data.json") print("Save in "..math.floor(system.getTimer() - start).."ms", "\tloop #:"..count) count = count + 1 end timer.performWithDelay( 100, rapidSave, 50 ) end timer.performWithDelay( 2000, start )

output for me is

15:51:07.423 Project sandbox folder: C:\Users\AppData\Local\Corona Labs\Corona Simulator\Sandbox\5A44ECC91411385BE32625F1BF14938E\Documents 15:51:09.556 Save in 2ms loop #:1 15:51:09.650 Save in 0ms loop #:2 15:51:09.775 Save in 2ms loop #:3 15:51:09.858 Save in 1ms loop #:4 15:51:09.952 Save in 1ms loop #:5 15:51:10.075 Save in 1ms loop #:6 15:51:10.148 Save in 2ms loop #:7 15:51:10.274 Save in 1ms loop #:8 15:51:10.378 Save in 2ms loop #:9 15:51:10.481 Save in 1ms loop #:10
#9

I tried it and I received no errors. I even tried again and included the errorString which also stated that there were no errors at any point.

EDIT: Well, I received no errors as I didn’t try to load the file first (if it exists) and then try to save it afterwards. The issue still exists, as is demonstrated by the code below in my new post.

#10

No idea, I’ve not looked at your project.

I just gave you the most simple saving code I could think of for you to test.

#11

Well, simple is usually the best, but it seems that I jumped the gun here.

The same problem as before still occur if I first check if the file exists and then load the file. Here’s the revised code:
 

local json = require "json" local data = {1,2,3,4,5,6,7,8,9,10} local function saveTable( dataTable, filename ) local myTable = json.encode( dataTable ) local path = system.pathForFile( filename, system.DocumentsDirectory ) local file = io.open( path, "w" ) if file then file:write( myTable ) io.close( file ) else print("error") end end local function loadTable( filename ) local path = system.pathForFile( filename, system.DocumentsDirectory ) local file = io.open( path, "w" ) if file then local contents = file:read( "\*a" ) local t = json.decode( contents ) io.close( file ) return t else print("error") end end local path = system.pathForFile( "data.json", system.DocumentsDirectory ) local file = io.open( path, "r" ) if file then saveData = loadTable( "data.json" ) end local function start() local count = 1 local function rapidSave() local start = system.getTimer() saveTable( data, "data.json") print("Save in "..math.floor(system.getTimer() - start).."ms", "\tloop #:"..count) count = count + 1 end timer.performWithDelay( 100, rapidSave, 50 ) end timer.performWithDelay( 2000, start )

My console output is:
 

18:30:23.238 Project sandbox folder: C:\Users\AppData\Local\Corona Labs\Corona Simulator\Sandbox\json file error-5C54F9E7F7FE5110E495ABFBB7F62136\Documents 18:30:23.249 error 18:30:25.366 error 18:30:25.366 Save in 0ms loop #:1 18:30:25.459 error 18:30:25.459 Save in 0ms loop #:2 18:30:25.563 error 18:30:25.563 Save in 0ms loop #:3 18:30:25.660 error 18:30:25.660 Save in 0ms loop #:4
#12

Well, I’ve done some extra digging and it seems that other users have had this issue as well in the past, for instance: https://forums.coronalabs.com/topic/59206-unable-to-save-score-between-sessions-error-could-not-read-nil/

I also made a new visual builds for win32 and Mac and asked friends to try them out. Those on a Mac had no issues, but everyone on Windows encountered the same issues as I’ve described. The first time that they run the project, everything works as expected, but every other start up it takes tens of attempts (official record is now 168 attempts at 100ms delay between attempts) until the save actually goes through.

The issue lies with io.open and seems to only affect Windows. It affects both running the code on simulator as well as building and running as a win32 build.

I would greatly appreciate it if anyone on Windows could confirm if they are also experiencing this issue.

Here is the updated code with visual indicators. I’ve also attached the project.

local json = require "json" local data = {1,2,3,4,5,6,7,8,9,10} local options = { text = "", x = display.contentCenterX, y = display.contentCenterY, width = display.actualContentWidth\*0.75, font = native.systemFont, fontSize = 18, align = "center" } local text = display.newText( options ) text:setFillColor(1,0,0) local tmr local function saveTable( dataTable, filename, count ) local myTable = json.encode( dataTable ) local path = system.pathForFile( filename, system.DocumentsDirectory ) local file, errorString = io.open( path, "w" ) if file then timer.cancel( tmr ) text:setFillColor(0,1,0) text.text = filename.." loaded\n\nAttempt #"..count file:write( myTable ) io.close( file ) else text.text = errorString.."\n\nAttempt #"..count end end local function loadTable( filename ) local path = system.pathForFile( filename, system.DocumentsDirectory ) local file = io.open( path, "w" ) if file then local contents = file:read( "\*a" ) local t = json.decode( contents ) io.close( file ) return t else print("Error when loading file") end end local path = system.pathForFile( "data.json", system.DocumentsDirectory ) local file = io.open( path, "r" ) -- if the file is loaded, then we run into issues if file then saveData = loadTable( "data.json" ) end local count = 1 local function rapidSave() saveTable( data, "data.json", count) count = count + 1 end tmr = timer.performWithDelay( 100, rapidSave, 0 )
#13

I don’t get why you need to open the file and then open it again in the save block? My code worked perfectly every time without this extra (and seemingly pointless) step.

You might have more luck if you close the file after the first open and before the second open.

-- if the file is loaded, then we run into issues if file then saveData = loadTable( "data.json" ) io.close(file) \<-- close before opening it later! end

I imagine, because you haven’t closed the file after the first read it remains locked for some OS specific time.

#14

I was already about to write a post on how I close the file after loading it on row 38, but after including io.close(file) where you suggested, the saves again went through straight away. There was still an issue with loading the file, but that was literally due to the data.json file having already been opened. Avoiding the double open, which seemingly happened, fixed it.

 

local path = system.pathForFile( "data.json", system.DocumentsDirectory ) local file = io.open( path, "r" ) local saveData if file then -- not using the loadTable function with additional io.open(file) local contents = file:read( "\*a" ) saveData = json.decode( contents ) io.close( file ) else print("Error when loading file") end

The extra and seemingly pointless step isn’t that at all, but it was the reason for my errors. I included it because I want to see if the data.json file exists. It may include save data or general options data which I want to load as the game starts, however, if the user makes any changes, I’d want to save those changes to the file as well. If the file doesn’t exist, then a file with default values was created and the double opening of the file didn’t occur, which meant that there were no issues.

 

Thank you for taking the time to help me out @SGS. Seems that my projects may have included this issue for years, but I just haven’t realised because the loading screens and logos have delayed the apps enough for the OS to close the files by itself.

#15

I don’t have a PC but on my Mac and my android device, your code works as expected. No error. No delay needed.

That being said in general I wait until the system responds back that it is ready before I do any file operations. Not sure if this will fix your issue since I am not on a PC and couldn’t replicate your issue.

local function systemEvents( event ) delayedSave() end Runtime:addEventListener( "system", systemEvents )
#16

This sounds like an odd Windows caching issue. Perhaps Windows is holding on to a file handle too long, or a buffer hasn’t flushed yet. What do you need to write that immediately?

Rob

#17

Thanks for the replies.

On the first start up, I create and save a table for the general options and other variables as soon as the app starts, and this works without a hitch.

On future restarts, it isn’t that I am trying to write something immediately, but that some times it can take up to 14s until I can save the JSON file without encountering an error. In those 14s, I can go to the app’s options, change the app’s language, turn off music, etc. or maybe go to some level, perform some actions and still have time to spare. If I then close the app, all of those options should have been saved the moment that I made them. However, upon restart, I find that the changes weren’t saved and the console log keeps showing me the same error message every time I tried to save.

Once that varying time window since the app’s start has passed, I can save as frequently as I want without any errors.

#18

I save some 500KB locally and never had this issue.  Have you got some anti virus/malware scanner that is locking the file maybe?

#19

Seems like I need to be doing some testing with antivirus and related software then. Good to know that it is localised to my computer. 

#20

I’ve been trying to figure this one out for a while now and I’ve tried disabling all sorts of software that might be interfering, but I am at my wits end.

The duration for the file error also seems to be project specific. I used the following code to highlight the issue:

local function start() local count = 1 local function rapidSave() loadsave.saveTable( data, "data.json") print("Time:"..math.floor(system.getTimer()),"\tloop #:"..count) count = count+1 end timer.performWithDelay( 100, rapidSave, 50 ) end timer.performWithDelay( 2000, start )

Depending the project, it took anywhere from a few seconds to almost 20 seconds until the saving actually went through. And like before, once it goes through, then it just keeps on going without any interference. Here’s a few seconds long clip of the console log with the following code in a project of mine (https://gfycat.com/ImmenseWavyBrahmanbull). The console is barraged with errors, until everything just starts to work.

I would really appreciate it if anyone had a clue as to what is going on.