Data Storage Question

Solar2D General Questions/Discussion
#1

My app was recently rejected by Apple due to their iOS Data Storage Guidelines. In particular, they found that on launch my app stores 2.37 MB.

The only file I read and write to (using system.DocumentsDirectory) is my “Player.json” file that holds the player’s high scores, option settings, etc.

The Android devices I have been testing the app on for the last month show my data storage to be at 11 - 13 MB.

What else gets stored that is causing this spike in my app’s data storage?  and if possible, what can I do to reduce the amount of data being stored to meet Apple’s guidelines.

Thank you for your help in advance.

#2

Have you checked into your sandbox and seen if there’s anything extra in there?

C

#3

Yes, Sir. The only file I see in my sandbox is the Player.json file.

#4

What’s the size of it? Text files (and therefore .json files) shouldn’t take up too much room… You’re not capturing any images or something, are you?

C

#5

I deleted my sandbox folder for the application and then reopened the application to get a fresh look. There is only one file in Documents named “Player.json” and it is 611 bytes.

Also I deleted my application from my Android device and then reinstalled the game. After opening the application, I exited out and checked the memory consumption. Data is at 2.94 MB.

I do save screenshots when the player uses the Share Feature in the game, but they are stored in the cache folder. The player has to tap a button to start that code.

This is the first time I have ran into a problem that I couldn’t solve by scanning through the forums or documentation. It is the only thing keeping me from having my first app published.
 

#6

The last thing I know to do is duplicate your project and see if it stil happens.

Other than that, I’m lost.

C

#7

Thank you, Caleb. I do very much appreciate you trying to help me solve this issue.

Hopefully someone from Corona can take the time to help me figure out why this is happening.

Does Corona create any additional data for me during a device build? or should the only data that is created for my app be the one file I created to hold the user’s data?

#8

I have discovered that the reason data is adding up so fast is due to my Audio files. When I commented out all my calls to audio.loadSound and audio.loadStream, my data stays at 8 KB.

I am using audio.dispose whenever I want to free up memory. I have 10 sounds that I load with audio.loudSound at the beginning of the application that puts my app’s data stored to 720 KB.

But when I call my main menu background song to play using audio.loadStream that’s when the app’s data storage jumps up to 3 MB. This eventually tops off at 11 - 13 MB after all my audio files have been called at least once.

These audio files are saved in my resources folder, so why is it causing my app’s data to rise?

How can I optimize this or have this audio data be stored in a different directory?

Thank you in advance.

#9

I converted all my MP3 files to AAC files to see if this reduced my app’s data. It seemed to drop my initial launch data consumption by 1 MB, but this is still not ideal and may not please Apple enough to accept the app.

How do other app’s use audio files and manage to keep their app’s data from bloating up?

Please help.

#10

Perhaps the loadSound() is converting the audio file when you load it (and storing it on the device)… If you use loadStream on your large sound file(s) (which supposedly only decodes a small chunk at a time), does it affect the usage?

Also as a test along these ilnes, if you replace the largest file with a small (short)  file, does that peak change?

#11

Thank you for your reply, mpappas.

I built 4 apks:

  1. No changes - launched application and Data is at 2.94 MB

  2. Used only loadStream - launched application and Data is at 2.94 MB

  3. Used only loadSound - launched application and Data is at 2.94 MB

  4. Replaced all music files with a sound effect file instead - launched application and Data is at 720 KB

It seems that the music audio files are causing my application’s data to rise.

The two music files that are loading upon launch are:

  1. Tutorial Background Music - 60 seconds long

  2. Main Menu Background Music - 90 seconds long

How can I fix this without getting rid of all my background audio?

Thank you again.

#12

The only thing I can think of is that the files (mp3) are crunched when packaged in the resource directory, and the sdk copies/decompresses them for the OS to play, and this is being picked up as new data storage (still don’t know where it is putting the files it sounds like, but I would have thought it would have to be the documents directory for Apple to complain).

Note that data you see in the sandbox on the ios simulator wouldn’t reflect any packaging compression / decompression that might be needed at runtime (since the app isn’t compressed into a package for simulator runs).

When I run my app (which doesn’t use mp3’s, just a few wavs) on the device and look at the file usage in xcode, I’m not seeing any obvious audio translation / temp files anywhere. Are you / can you look at the files on the device in xcode after a run? (Organizer - Devices: Applications: Your App, bring up a list of files in storage). There’s plenty of odd little system type cache files corona seems to make, but I’m not noticing anything weird, or out of the cache folder…

Some insight from corona on this would be helpful - you’ve got it pinned down to a pretty small, specific example.

#13

I changed both of the music .mp3 files I mentioned above to .wav files to see what kind of difference it would make. After starting the app and getting to the main menu, I checked the data usage on the app and its at 16.85 MB. The .wav format greatly increases the usage, which is what I expected since its a larger format.

Unfortunately I don’t own a Mac. I have been developing on my PC so I wouldn’t be able to check through XCode. I will have access to a Mac on Thursday though (Wish I could afford one).

Although I appreciate the suggestion you were right when you said I have this pinned to a pretty small, specific example. The audio files must be going through a decompression process and then saved into my documents directory. Otherwise Apple wouldn’t be giving me a hard time. I don’t understand why this information isn’t being saved in my temp folder or cache folder.

It would be very nice to hear from Corona, and possible get some options on how I can correct this. The only option I have at the moment is to get rid of these music files and give my users a boring experience. I would rather not do that.

There has to be another option. My app has already been accepted to the Google Play, Amazon, and Nook markets. Getting it on Apple means so much to me.

Thank you again, mpappas & Caleb. Hopefully Corona will reach out to me soon.

#14

Well… At least you’ve pinned down your problem.

When you get access to your Mac, try changing them all to .caf files (Apple’s audio format) and see if that helps. There was an app posted a while ago (2010, but it still works) for .wav to .caf: http://developer.coronalabs.com/code/ima4-dropper

If that doesn’t work, I’m out (again). I’m not much of an audio person :). (Hm… Maybe I should check my data size!)

C

#15

Thank you for the link, Caleb. I will give that a try as soon as I get the opportunity. Hopefully that will make enough of a difference to satisfy Apple’s guidelines, since I haven’t heard a thing from Corona.

They have options I would imagine to fix this situation so others don’t run into the same problem as I have. (1) Save the data in a different directory, other than Documents. Or (2) Mark this data as “Do Not Backup”.

If neither of those solutions are possible then it would be nice to hear from someone at Corona telling me so.

During the year I have been developing this app I have always found answers to my questions in previous post where Corona offered some assistance to the developer.

Corona, just help me out this one time. :smiley:

#16

Corona only extracts audio files to a hidden temporary/cache directory on Android if the files reside in the ResourceDirectory.  This is because the APK file which contains your resource files is really a zip file and the audio APIs that we use require direct file access, thus requiring us to extract the audio files first.  Please note that this is only the case on Android.  We never extract or copy audio files on iOS, Mac, or Windows.

Also, Corona never creates temporary files in the DocumentsDirectory… nor should it since that is what Apple’s iCloud syncs with.  Now, if you have have files in the DocumentsDirectory that you want iCloud to ignore, then you can use our native.setSync() function to explicitly tell which files iCloud should sync with.

   http://docs.coronalabs.com/api/library/native/setSync.html

That said, I’m a bit confused as to why Android is being brought up on this thread at all.  What does it have to do with Apple rejecting your iOS app?  Or am I missing something?

#17

Hi, Joshua. Thank you for the response.

Android was being brought up due to the fact that I don’t have a Mac to test on at the moment and I made the assumption that Corona handled the audio files the same way on both platforms. I apologize for my ignorance concerning that.

But here is what doesn’t make sense to me… Apple is telling me that my app was rejected due to my app’s data reaching 2.37 MB upon launch. When I launch my app on an Android device, the data is at 2.94 MB.

When I got rid of all my audio files on Android, the data was at 8 KB upon launch ( 8 KB = “Player.json” ). I assumed that if I built an iOS binary without Audio, then my data on an Apple device would drop significantly as well.

So if I am using the same code for both Apple and Android, and only creating one file in the Documents folder for a player profile (“Player.json”), then what else would be causing my iOS app’s data to spike upon launch if not the Audio?

#18

Thank you for that link as well. I was trying to find that feature all day on Monday.

When Apple rejected the app, I assumed the player profile was somehow causing the problem, since once again it is the only file I create and keep in Documents folder. But after removing the player profile, my app still jumped up to over 2 MB upon launch. This leaves me with nothing else to mark as “Do Not Backup”.

Thank you again though for that link.

#19

Hmm… the only other file that Corona creates on startup is for launchpad analytics, but that file is stored under a hidden directory under the Caches directory which iCloud does not sync with.  Plus, that file should not be 2 MB on launch either.

Would you mind running a quick test please?  If you add the following code to the bottom of your “main.lua”, it will print out your entire Documents directory contents to the log.  This will help us find where the 2 MBs is coming from.

local lfs = require("lfs") local documentsPath = system.pathForFile("", system.DocumentsDirectory) for file in lfs.dir(documentsPath) do print("File: " .. tostring(file)) end
#20

The log shows the following:

File: .

File: …

File: Player.json

I also modified the code to display the results using text so I could test this on an Android device build, and I got the same results as above.

This morning I converted all my audio files to a lower quality MP3 format using some free software I found online and this reduced my launch data usage from 2.94 MB to 1.36 MB on Android.

Tonight I will be testing the same thing on an iOS build to see if this reduces the data usage as well. I am also going to use the JonBeebe’s app that Caleb mentioned above to convert the audio to .CAF and compared the difference between the two iOS builds.

Let me know if there are any other tests I can run as well.

Thank you again for your support, Joshua.