How I load a 16-bit image?

Audio and Graphics
#1

Hello!

I am trying to load a alpha-less image on Corona, and it fails in some specific devices only.

Unfortunately for me, the devices are exactly the most popular ones in my target markets (developing nations), so I need to make it work on them.

I submitted a bug report, and got a reply:

We can only reproduce this issue on a Nexus One (Android 2.3) device.

The following low-end Android devices displayed the image without issue:

  • Droid (1st generation) running Android 2.2
  • HTC Evo 4g running Android 2.2
  • Galaxy SII running Android 2.3
  • Kindle Fire (1st generation) running Android 2.3
  • Nook Color running Android 2.2

What I’ve noticed is that our Try/Catch block is failing to catch that OutOfMemory exception just on that one device.  I’ve never seen this happen before, because we do successfully catch these errors and implement fallback mechanisms on other devices.  In fact, we have a lot of code in place to predict how much memory is available, and if we do not have enough, then we down-sample the loaded image (skipping every other pixel) to make it fit in memory.  We also down-sample the image if it is larger than OpenGL’s max texture size.  Now, if we still get an OutOfMemory exception (because some devices lie about how much memory is available), then we force garbage collection, reduce color quality from 32-bit to 16-bit (reducing the memory footprint in half), and then attempt to load the image one more time.  If we still fail to load the image from there, then we have no choice but to return “nil” in Lua.

But… since Java is failing to catch this OutOfMemory exception for your particular device, then unfortunately my only advise to you is to exclude support for the device this is happening to you in the app store, which you can do via the Google Play Developer Console.

Regards,
Joshua Quick

Mind you, the issue also happen on my device (Xperia Play with Android 2.3) and on many devices that are simple copies of the reference device (ie: Nexus One is a reference device, many manufacturers of cheap devices just copy it and use its implementation of Android).

So, if Joshua Quick is correct, Corona can load 16-bit images, so how I do that myself on my own? I don’t need those images to be 32-bit at all (even on devices that can load them at 32-bit).

Also, although I asked that by e-mail noone replied so far, CoronaLabs intend to fix their bug of not detecting the exception in those devices? (it outright crashes when the image is right on the start of the program, but LogCat does capture the exception, but when I load the image following my game flow, it throws a unhandled exception window instead of crashing).

And I would also like to know if I can load half-resolution image on my own too, avoiding duplicating assets and code (the images are imagesheet) that will only be used one the problematic devices.

#2

Regarding Corona loading images with 16-bit color quality, that’s simply how we load it on the Java side of an Android app due to low-end Android devices having a very small maximum heap size (worst case is 24 MB).  This is a fallback mechanism for low-end devices so that we can at least load the image successfully into memory.  In the end, that 16-bit color image is loading into OpenGL as a 32-bit texture on the native C/C++ side.  So, loading images as 16-bit color images will not save you any texture memory… if in fact that is the goal that you are after.

Now, out-of-memory exceptions are correctly caught by Corona if the image is too big.  When this happens, we print the exception to the Android log and then implement a fallback mechanism, such as down-sampling when loading the image (ie: skipping every other pixel) and reducing the color quality.  If none of our fallback mechanisms are able to load the image due to out-of-memory errors, then Corona has no choice but to give up and return nil to Lua.  Typically, the crash is caused on the Lua side due to the Lua script not expecting nil to be returned from display.newImage().  So what is crashing the app is a Lua runtime error, not a Java exception.

We find that this out of memory exception typically happens more often on modern Android devices due to larger higher res images being loaded.  Some Android devices have a lazy Java garbage collector which doesn’t free memory in time when loading images back-to-back.  We attempt force garbage collection when loading an image, but there is no guarantee that the garbage collector will immediately do this, which I find to be true with the Kindle Fire HDs.  You can work-around this by loading images via a timer.  You can also work-around this by setting the “build.settings” option “largeHeap” to true, which requests the Android OS (version 3.0 or higher) to give the app more memory.  This has solved the problem for many people.

   http://docs.coronalabs.com/guide/distribution/buildSettings/index.html#large-heap

   http://forums.coronalabs.com/topic/24584-very-weird-texture-memory-problem-on-android/?p=154069

Anyways, I hope this helps.

#3

@Joshua Quick

As your report mentions, the error is in Android 2.3 (both above and below work fine), so LargeHeap won’t help.

Also, as your report also mentions, it does try to load as 16-bit memory to workaround the out of memory error, if that can be done, I would appreciate it.

And finally, when the lua code is only a single line, that is load the problematic images, it outright crash, there is no Lua returning nil, it never finishes the routine on Corona side, it just crashes internally in Corona (in NativeToJavaBridge.java 1128 to be more exact), when the code that crashes when it is a single line is used normally on my game, it throws a runtime error popup instead.

Also I did tried heap workarounds, including loading zillions of smaller images (and it works, I can load 150mb of images on the vram), then unloading them and loading my image (it crashed on the already mentioned place).

Also tried to make a 100mb array of numbers, delete it, call garbage collector and then load the image (also crashed on the same .java file and line).

#4

Just found this post: http://forums.coronalabs.com/topic/33862-build-1085-breaks-android-app/

The same issue, it seems.

#5

By the way, if you make it really returns nil, it would make me happy enough, since I would be able to check if it returned nil, and then load my own workaround, but currently it crashes before returning anything, so I cannot check if I need or not to load a lower quality image.

#6

Alestane on the IRC suggested me test see if I can load that Image with regular Android…

I quickly coughed that up: https://docs.google.com/file/d/0BypJ08jsizKzRzhGNkdRZFhYYWM/edit?usp=sharing

It works fine on the Xperia Play with no issues, the pic even looks really awesome and crisp (probably because to use full memory I did not allowed it to scale).

#7

We already have try/catch blocks around all of our image loading code and our fallback mechanisms already attempt to load images as 16-bit color if we do not have enough memory.  Our past release versions of Corona already do this now.  Corona also does in fact return nil if we fail to load the image.  There is nothing more we can do on the Java end.  Everything you are asking us to… we are already doing it.  We have well over a 100 lines of fallback mechanisms to make image loading work on several low-end Android devices.

The only device where we’ve seen our try/catch blocks fail to catch an OutOfMemory exception is on the Nexus One.  The catch block simply fails to catch that exception on that device and immediately force quits.  That is a unique issue on that device.  Because of this, your only option is to either exclude Nexus One support on Google Play or change your app to only support lower res image files.

Bottom line, if that low-end Android device is giving you trouble, exclude support for it.  Probably not the answer you want to hear, but that’s why that option exists on Google Play.

#8

Hello Joshua!

I went around doing some research on why the Try Catch block might be failing to work.

Seemly the consensus is that something is breaking and killing the JVM outright, since the JVM is dead, it won’t catch any exceptions, because it is dead already.

So, does you call NDK-based code inside your try catch block?

#9

We have a try/catch block inside of our catch block, which is where our last fallback mechanism is invoked.  We attempt to force garbage collection reduce color quality to 16-bit color, and downsample the image.  If that fails, then the final catch block merely prints the exception to the Android log and we return null/nil.  It’s proven to work on all Android device except the Nexus One, where the JVM completely terminates the first time an OutOfMemory exception occurs.

Now, we do try to predict if the device has enough memory to load the image too, and if it doesn’t, then we attempt downsample the image before our first attempt to load the image.  But based on my experience, guessing how much memory is left for the app on Android has proven to be unreliable on some devices.  Some devices just flat out lie at how much heap memory that they have… or perhaps that is a more likely case on rooted Android devices where this value is wrongly hard coded.

In any case, the only way we can improve our image loading at this point is to load all images from C/C++ on Android, where the available memory is much much larger, and it would be faster too.  It would involve us using zlib to access the image files within the APK and libpng/libjpeg to load the image files.  It would pretty much be a full rewrite of our image loading code, which we’re not prepared to do at the moment, but we do recognize that we need to do this in the future.

#10

Joshua, I am asking if the point where the JVM terminates has native code (ie: not Java) running or not.

But by the way, improving loading speeds would make us very happy, all our games are very memory intensive, and are horribly slow on Android, sometimes taking more than a minute to load… And it is one of the major reasons we dumped Corona once (and are still considering doing so again, as soon we find something that has decent support beside community our thousand-level price for support).

#11

No, our code within the Java catch block does not call any native C/C++ code.  Although I imagine the Android BitmapFactory class, which we use to load bitmaps, calls upon native code.  So, it may just be a bad Android OS fork for the particular device that you are running… and with our Nexus One, because this issue does not happen on any other device we’ve tested on.  At least that’s my conclusion.

#12

check the image loading codes c#, maybe it will help you.

#13

Regarding that bug, we might be seeing the same thing. For us it’s a HTC Desire HD that throws the out of memory error popup when using expansion files. In our testing we found when not using an expansion file it worked.

#14

Regarding Corona loading images with 16-bit color quality, that’s simply how we load it on the Java side of an Android app due to low-end Android devices having a very small maximum heap size (worst case is 24 MB).  This is a fallback mechanism for low-end devices so that we can at least load the image successfully into memory.  In the end, that 16-bit color image is loading into OpenGL as a 32-bit texture on the native C/C++ side.  So, loading images as 16-bit color images will not save you any texture memory… if in fact that is the goal that you are after.

Now, out-of-memory exceptions are correctly caught by Corona if the image is too big.  When this happens, we print the exception to the Android log and then implement a fallback mechanism, such as down-sampling when loading the image (ie: skipping every other pixel) and reducing the color quality.  If none of our fallback mechanisms are able to load the image due to out-of-memory errors, then Corona has no choice but to give up and return nil to Lua.  Typically, the crash is caused on the Lua side due to the Lua script not expecting nil to be returned from display.newImage().  So what is crashing the app is a Lua runtime error, not a Java exception.

We find that this out of memory exception typically happens more often on modern Android devices due to larger higher res images being loaded.  Some Android devices have a lazy Java garbage collector which doesn’t free memory in time when loading images back-to-back.  We attempt force garbage collection when loading an image, but there is no guarantee that the garbage collector will immediately do this, which I find to be true with the Kindle Fire HDs.  You can work-around this by loading images via a timer.  You can also work-around this by setting the “build.settings” option “largeHeap” to true, which requests the Android OS (version 3.0 or higher) to give the app more memory.  This has solved the problem for many people.

   http://docs.coronalabs.com/guide/distribution/buildSettings/index.html#large-heap

   http://forums.coronalabs.com/topic/24584-very-weird-texture-memory-problem-on-android/?p=154069

Anyways, I hope this helps.

#15

@Joshua Quick

As your report mentions, the error is in Android 2.3 (both above and below work fine), so LargeHeap won’t help.

Also, as your report also mentions, it does try to load as 16-bit memory to workaround the out of memory error, if that can be done, I would appreciate it.

And finally, when the lua code is only a single line, that is load the problematic images, it outright crash, there is no Lua returning nil, it never finishes the routine on Corona side, it just crashes internally in Corona (in NativeToJavaBridge.java 1128 to be more exact), when the code that crashes when it is a single line is used normally on my game, it throws a runtime error popup instead.

Also I did tried heap workarounds, including loading zillions of smaller images (and it works, I can load 150mb of images on the vram), then unloading them and loading my image (it crashed on the already mentioned place).

Also tried to make a 100mb array of numbers, delete it, call garbage collector and then load the image (also crashed on the same .java file and line).

#16

Just found this post: http://forums.coronalabs.com/topic/33862-build-1085-breaks-android-app/

The same issue, it seems.

#17

By the way, if you make it really returns nil, it would make me happy enough, since I would be able to check if it returned nil, and then load my own workaround, but currently it crashes before returning anything, so I cannot check if I need or not to load a lower quality image.

#18

Alestane on the IRC suggested me test see if I can load that Image with regular Android…

I quickly coughed that up: https://docs.google.com/file/d/0BypJ08jsizKzRzhGNkdRZFhYYWM/edit?usp=sharing

It works fine on the Xperia Play with no issues, the pic even looks really awesome and crisp (probably because to use full memory I did not allowed it to scale).

#19

We already have try/catch blocks around all of our image loading code and our fallback mechanisms already attempt to load images as 16-bit color if we do not have enough memory.  Our past release versions of Corona already do this now.  Corona also does in fact return nil if we fail to load the image.  There is nothing more we can do on the Java end.  Everything you are asking us to… we are already doing it.  We have well over a 100 lines of fallback mechanisms to make image loading work on several low-end Android devices.

The only device where we’ve seen our try/catch blocks fail to catch an OutOfMemory exception is on the Nexus One.  The catch block simply fails to catch that exception on that device and immediately force quits.  That is a unique issue on that device.  Because of this, your only option is to either exclude Nexus One support on Google Play or change your app to only support lower res image files.

Bottom line, if that low-end Android device is giving you trouble, exclude support for it.  Probably not the answer you want to hear, but that’s why that option exists on Google Play.

#20

Hello Joshua!

I went around doing some research on why the Try Catch block might be failing to work.

Seemly the consensus is that something is breaking and killing the JVM outright, since the JVM is dead, it won’t catch any exceptions, because it is dead already.

So, does you call NDK-based code inside your try catch block?