Where to put my text (data) files?

Getting Started Newbie Questions
#1

Hi Guys.

Sorry for the really basic question… :huh:

So I’ve got the code of my first ever app to the point where I want to try it on my device.

My app references two text files - one is a (generally static) list (but editable from time to time) and the other is a dynamic list the app modifies as used.

I’ve developed my app on a Win10 machine in corona v 3.0.0 - build 2017.3184 with lots of referral to the API documentation from https://docs.coronalabs.com/api/ and in the simulator it seems to work fairly well.

So I’ve followed the guides and built a debug version and copied the .apk to my Samsung Galaxy Note 8 and installed it, and it does start.   :)     But my own precautionary code advises me that the app cannot find the two text files needed, for it to continue.   :frowning:

I was of the understanding that the ‘system.DocumentsDirectory’ referenced in the development/console environment equates to the existing ‘phone\documents’ folder, so I initially put the two text files in there - but that doesn’t work.

So I went back to the API and re-read the docs, and then copied the two text files needed into my app development folder where the .lua files are - before I re-ran the build, re-interpreting that the two text files might need to be inside the .apk (although I am not certain of this, nor, do i think that’s quite right and after the build process completes, I cannot be sure the files actually ARE in the .apk)

Well whatever, that doesn’t work either…   my app still can’t see the files.

WHERE should I put the text files on my phone, so that my app can see them?

Thanks for any assist…

Tez

#2

Hi Tez. This isn’t terribly complicated, but there are some concepts that don’t relate directly to what you might be used to in a desktop environment and we have to assume that you may want to build for platforms other than Android, so you have to also learn to think in a cross-platform mentality.

So let’s break it down, for security, all mobile apps run in a protected sandbox. The folders it can access (read and/or write ) are located in this locked down environment. Android has public folders that can be accessed by different apps, but iOS can’t. Desktops are less locked down. TV platforms are not known for having a lot of storage for you to use.

Let’s focus on the Sandbox since that’s what our API’s provide folders to:

system.ResourceDirectory – This is the folder where your main.lua lives and any sub-folders included in that folder. However, this is a READ-ONLY folder. Android and iOS do not allow you to write to your application for security reasons. There are additional restrictions on what files can be read from here on Android but you should not have issues reading text files. You cannot update files here without releasing a new update to your app and having people download the update.

system.DocumentsDirectory – This is a folder that lives in the sandbox that you can write to. Files here should be ones your app creates and are required for the app to function. This is potentially where your writable text files should live. But since they are in the sandbox, there isn’t a direct way to use the operating system to copy files there. You also don’t want your users to have to install files either. Most people will start with a copy of any files that you would need to update in system.ResourceDirectory and on first run, have the app copy the file into system.DocumentsDirectory and after that you can read/write the files there. These files should get backed up automatically.

system.CachesDirectory  – This is primarily an iOS feature, but to be cross-platform it works on Android and other. The idea is this is where you would store any file you would download from the Internet. Apple decided a while back that if you can download the file from the internet, that if it gets deleted you can always re-fetch it and more importantly these files, since they can be easily replaced, they won’t get backed up and take up space in your cloud storage. For Android, this maps to the same folder as system.TemporaryDirectory. If storage gets low, these files could get removed by the OS.

system.TemporaryDirectory – This is a folder where files can be read and written from with the understanding that the OS can remove them at any point. They are not backed up.

If you are only going to deploy to Android, there are plugins in the marketplace that can help you access folders in external storage.

Now that we have these locations somewhat defined, lets look at your two text files.  

  1. Static file that gets periodic updates.

This is a file that you have basically two (well three) options depending on how you want to do the updates. If you don’t mind issuing new app updates when you need to update the text file, simply include the file in with main.lua and read it from system.ResourceDirectory.  If you have a web server and each app user gets the same text file, you can host it on your web server and use network.request() and/or network.download() to get the file from the server and save it to system.CachesDirectory (or system.DocumentsDirectory).  Finally if the app will be updating the file, you can use either of the two previous methods, and then have the app update the ffile.

  1. dynamic list that updates frequently by the app

If you have an intial copy of the file you can put it in system.ResourceDirectory and on first run have the app copy it to system.DocumentsDirectory and then have your app update  the file in system.DocumentsDirectory. Depending on the data, you might want to consider an SQLite database, but you have to follow the same basic concept of having your app initialize the database in system.DocumentsDirectory on first run and then read/write/update to system.DocmentsDirectory.

You can check out this tutorial for an example/module for copying files from the resource directory to the documents directory:

https://coronalabs.com/blog/2015/05/19/tutorial-initializing-a-writable-sqlite-database-from-a-read-only-database/

Rob

#3

Hi Rob.

Thanks for the detailed reply.   That explanation makes things much clearer.    Seems I still have some work to do.   Having only ever developed fairly simple ‘programs’ - as they were called back then! :D  for the Windows environment and then, nothing for (say) 15 years, I now find myself on a steep learning curve, but I’m getting there!    Baby steps.

#4

Hi Tez. This isn’t terribly complicated, but there are some concepts that don’t relate directly to what you might be used to in a desktop environment and we have to assume that you may want to build for platforms other than Android, so you have to also learn to think in a cross-platform mentality.

So let’s break it down, for security, all mobile apps run in a protected sandbox. The folders it can access (read and/or write ) are located in this locked down environment. Android has public folders that can be accessed by different apps, but iOS can’t. Desktops are less locked down. TV platforms are not known for having a lot of storage for you to use.

Let’s focus on the Sandbox since that’s what our API’s provide folders to:

system.ResourceDirectory – This is the folder where your main.lua lives and any sub-folders included in that folder. However, this is a READ-ONLY folder. Android and iOS do not allow you to write to your application for security reasons. There are additional restrictions on what files can be read from here on Android but you should not have issues reading text files. You cannot update files here without releasing a new update to your app and having people download the update.

system.DocumentsDirectory – This is a folder that lives in the sandbox that you can write to. Files here should be ones your app creates and are required for the app to function. This is potentially where your writable text files should live. But since they are in the sandbox, there isn’t a direct way to use the operating system to copy files there. You also don’t want your users to have to install files either. Most people will start with a copy of any files that you would need to update in system.ResourceDirectory and on first run, have the app copy the file into system.DocumentsDirectory and after that you can read/write the files there. These files should get backed up automatically.

system.CachesDirectory  – This is primarily an iOS feature, but to be cross-platform it works on Android and other. The idea is this is where you would store any file you would download from the Internet. Apple decided a while back that if you can download the file from the internet, that if it gets deleted you can always re-fetch it and more importantly these files, since they can be easily replaced, they won’t get backed up and take up space in your cloud storage. For Android, this maps to the same folder as system.TemporaryDirectory. If storage gets low, these files could get removed by the OS.

system.TemporaryDirectory – This is a folder where files can be read and written from with the understanding that the OS can remove them at any point. They are not backed up.

If you are only going to deploy to Android, there are plugins in the marketplace that can help you access folders in external storage.

Now that we have these locations somewhat defined, lets look at your two text files.  

  1. Static file that gets periodic updates.

This is a file that you have basically two (well three) options depending on how you want to do the updates. If you don’t mind issuing new app updates when you need to update the text file, simply include the file in with main.lua and read it from system.ResourceDirectory.  If you have a web server and each app user gets the same text file, you can host it on your web server and use network.request() and/or network.download() to get the file from the server and save it to system.CachesDirectory (or system.DocumentsDirectory).  Finally if the app will be updating the file, you can use either of the two previous methods, and then have the app update the ffile.

  1. dynamic list that updates frequently by the app

If you have an intial copy of the file you can put it in system.ResourceDirectory and on first run have the app copy it to system.DocumentsDirectory and then have your app update  the file in system.DocumentsDirectory. Depending on the data, you might want to consider an SQLite database, but you have to follow the same basic concept of having your app initialize the database in system.DocumentsDirectory on first run and then read/write/update to system.DocmentsDirectory.

You can check out this tutorial for an example/module for copying files from the resource directory to the documents directory:

https://coronalabs.com/blog/2015/05/19/tutorial-initializing-a-writable-sqlite-database-from-a-read-only-database/

Rob

#5

Hi Rob.

Thanks for the detailed reply.   That explanation makes things much clearer.    Seems I still have some work to do.   Having only ever developed fairly simple ‘programs’ - as they were called back then! :D  for the Windows environment and then, nothing for (say) 15 years, I now find myself on a steep learning curve, but I’m getting there!    Baby steps.