Calling media.capturePhoto() cashes/restarts app (on a SGS5)

runewinse · March 24, 2015, 6:09pm

This is a follow-up to the thread:

http://forums.coronalabs.com/topic/55530-possible-to-resize-jpg-images/

In addition to the scaling problems described in the other thread it turns out that calling media.capturePhoto() is restarting my app about 50% of the time!

\_W = display.contentWidth \_H = display.contentHeight local function itemPhotoOnComplete( event ) print("itemPhotoOnComplete() - START") if (event.completed == true) then local photo = event.target photo.x = \_W/2 photo.y = \_H/2 print(" Original photo dimensions = (" .. photo.width .. " x " .. photo.height .. ")") local scaling = 0.25 local scaledW = math.floor(photo.width \* scaling) local scaledH = math.floor(photo.height \* scaling) -- Rescale, save and remove photo print(" Scaled photo dimensions = (" .. scaledW .. " x " .. scaledH .. ")") photo:scale(scaling, scaling) display.save(photo, "img.jpg", system.TemporaryDirectory) photo:removeSelf() photo = nil -- Show rescaled image loadedImage = display.newImage("img.jpg", system.TemporaryDirectory, \_W\*0.5, \_H\*0.5) print(" Loaded image size = (" .. loadedImage.width .. " x " .. loadedImage.height .. ")") end print("itemPhotoOnComplete() - END") end local function takePhotoTouchHandler(event) print("takePhotoTouchHandler - START") if (event.phase == "ended") then -- Get image from camera if media.hasSource( media.Camera ) then print("Camera detected - starting camera") media.capturePhoto( { listener = itemPhotoOnComplete } ) else native.showAlert("Error", "No camera found", { "OK" } ) end end print("takePhotoTouchHandler - END") end print("\n\nApp started") local takePhoto = display.newText("PRESS HERE TO TAKE PHOTO", \_W\*0.5, \_H\*0.9, native.systemFont, 40) takePhoto:addEventListener("touch", takePhotoTouchHandler)

A typical log looks like this:

I/Corona (16610): App started

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): takePhotoTouchHandler - END

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): takePhotoTouchHandler - END

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): Camera detected - starting camera

I/Corona (16610): takePhotoTouchHandler - END

V/Corona (16610): Downsampling image file ‘/storage/emulated/0/Pictures/Picture26.jpg’ to fit max pixel size of 4096.

I/Corona (16610): itemPhotoOnComplete() - START

I/Corona (16610): Original photo dimensions = (1494 x 2656)

I/Corona (16610): Scaled photo dimensions = (373 x 664)

V/Corona (16610): Downsampling image file ‘/storage/emulated/0/Pictures/Picture26.jpg’ to fit max pixel size of 4096.

I/Corona (16610): Loaded image size = (504 x 896)

I/Corona (16610): itemPhotoOnComplete() - END

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): takePhotoTouchHandler - END

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): takePhotoTouchHandler - END

I/Corona (16610): takePhotoTouchHandler - START

I/Corona (16610): Camera detected - starting camera

I/Corona (16610): takePhotoTouchHandler - END

V/Corona (20268): > Class.forName: network.LuaLoader

V/Corona (20268): < Class.forName: network.LuaLoader

V/Corona (20268): Loading via reflection: network.LuaLoader

I/Corona (20268): Platform: SM-G900F / ARM Neon / 5.0 / Adreno ™ 330 / OpenGL ES 3.0 V@84.0 AU@ (CL@) / 2015.2590

V/Corona (20268): > Class.forName: plugin.fuse.LuaLoader

V/Corona (20268): WARNING: Could not load ‘LuaLoader’

V/Corona (20268): > Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (20268): < Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (20268): Loading via reflection: CoronaProvider.licensing.google.LuaLoader

I/Corona (20268):

I/Corona (20268): App started

The first part of the log shows a “lucky” call to media.capturePhoto().

The second part of the log shows a “unlucky” call to media.capturePhoto(), where calling media.capturePhoto() results in the app restarting without any ado - no warnings, not error messages. The listener is never called. The app restarts somewhere between the call to media.capturePhoto() and the listener and I’m powerless to do anything about it.

Where do I go from here to find out what is happening?

bjsorrentino · March 24, 2015, 9:15pm

Hi @runewinse,

My first suggestion is to try commenting out certain aspects and then test if the crash occurs at the same frequency. For example, temporarily remove the lines where you load the captured image from the temporary directory, and test several times over. Also, in my experience, Android can be very fussy about “timing” of certain operations (what is performed in the same app time-step), and if you perform a certain action following a very small delay (10-50 milliseconds), sometimes that solves the problem entirely. So you might want to display.save() the photo after a small delay, instead of directly on the event completion. Even then, I don’t know if that will fix the issue, because every Android device has a different camera app (no standardization whatsoever like iOS) and that makes it difficult to account for behavior across the hundreds of manufacturers and thousands of devices with varying hardware and capabilities. However, do some testing and report back what happens.

Best regards,

Brent

runewinse · March 24, 2015, 10:39pm

Ok, I’ve now commented out so much that nothing really is happening, except for starting the camera. The only thing that happens in the listener is one print statement so that it is possible to see that it has been called at all:

local function itemPhotoOnComplete(event) if (event.completed == true) then print("--------------- itemPhotoOnComplete()") end end local function takePhotoTouchHandler(event) if (event.phase == "ended") then if media.hasSource( media.Camera ) then print("Camera detected - starting camera") media.capturePhoto( { listener = itemPhotoOnComplete } ) end end end print("\n\nApp started") local takePhoto = display.newText("PRESS HERE", display.contentWidth\*0.5, display.contentHeight\*0.9, native.systemFontBold, 50) takePhoto:addEventListener("touch", takePhotoTouchHandler)

But even with this bare minimum media.capturePhoto() the app restarts randomly

Starting app:

I/Corona (16322): ***** App started *****

I/Corona (16322):

First try to take a picture - app restarts:

I/Corona (16322): Camera detected - starting camera

V/Corona (16812): > Class.forName: network.LuaLoader

V/Corona (16812): < Class.forName: network.LuaLoader

V/Corona (16812): Loading via reflection: network.LuaLoader

I/Corona (16812): Platform: SM-G900F / ARM Neon / 5.0 / Adreno ™ 330 / OpenGL ES 3.0 V@84.0 AU@ (CL@) / 2015.2590

V/Corona (16812): > Class.forName: plugin.fuse.LuaLoader

V/Corona (16812): WARNING: Could not load ‘LuaLoader’

V/Corona (16812): > Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (16812): < Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (16812): Loading via reflection: CoronaProvider.licensing.google.LuaLoader

I/Corona (16812):

I/Corona (16812): ***** App started *****

I/Corona (16812):

Second try to take a picture - camera operation ok:

I/Corona (16812): Camera detected - starting camera

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812): --------------- itemPhotoOnComplete()

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812):

Third try to take a picture - camera operation ok:

I/Corona (16812): Camera detected - starting camera

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture48.jpg’ to fit max pixel size of 4096.

I/Corona (16812): --------------- itemPhotoOnComplete()

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture48.jpg’ to fit max pixel size of 4096.

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812):

Fourth try to take a picture - app restarts:

I/Corona (16812): Camera detected - starting camera

V/Corona (18340): > Class.forName: network.LuaLoader

V/Corona (18340): < Class.forName: network.LuaLoader

V/Corona (18340): Loading via reflection: network.LuaLoader

I/Corona (18340): Platform: SM-G900F / ARM Neon / 5.0 / Adreno ™ 330 / OpenGL ES 3.0 V@84.0 AU@ (CL@) / 2015.2590

V/Corona (18340): > Class.forName: plugin.fuse.LuaLoader

V/Corona (18340): WARNING: Could not load ‘LuaLoader’

V/Corona (18340): > Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (18340): < Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (18340): Loading via reflection: CoronaProvider.licensing.google.LuaLoader

I/Corona (18340):

I/Corona (18340): ***** App started *****

So in this example the behavior is just as I’ve been experiencing - about 50% of the time, the app restarts when using media.capturePhoto().

You advise me to add a delay. But what is there to be delayed in the bare minimum example above? There isn’t anything to delay!

One thing I notice is the apparent accumulation of the “Downsampling image file” lines. In one test run I managed to take 4 pictures in a row and then there were several of these lines in a row in the log.

Don’t know if it’s related to the app restart or not. Just mentioning it…

bjsorrentino · March 24, 2015, 11:37pm

Hello @runewinse,

Which build # of Corona are you using?

runewinse · March 24, 2015, 11:41pm

Hi,
I’m not at my pc at the moment, but according to my logs in my last post it’s 2015.2590.

bjsorrentino · March 25, 2015, 11:22pm

Hi @runewinse,

It’s possible that the device is low on memory and is forcefully terminating the Corona app, which is in the background while the camera app is in the foreground, because the camera app requires more memory from the OS to function. The OS typically logs when this situation happens, and a typical indication of this is that the app appears to “restart” when returning from the camera or photo gallery app.

Brent

runewinse · March 26, 2015, 9:28am

It’s seems a bit strange that if the camera app doesn’t have enough memory to function, the calling app is “killed” like this. If anything, it should be the camera app that should be failing. I cannot believe the OS does such a thing as restarting the app like this.

I’m more suspicious about what is happening behind these lines:

V/Corona (16812): Downsampling image file '/storage/emulated/0/Pictures/Picture48.jpg' to fit max pixel size of 4096.

This is a corona operation (it seems) that certainly craves memory to work. I wouldn’t be surprised if this operation fails and thus restarts the app.

But you’re probably right about the memory thing. By restarting the phone I’ve been able to do at least a dozen such photo captures in a row.

But I cannot have an app that behaves like this when the memory gets low. Blaming the OS is a bit of an easy way out. Even though it might be right, I would really like to get to the bottom of this.

How can you be sure that it is not the “Downsampling” thing that fails?
Is there some kind of debugging method for finding this out?

Thanks for your help so far!

bjsorrentino · March 26, 2015, 9:02pm

Hi @runewinse,

We (Corona) have no control over how the OS handles memory and what it decides to “close” as a result of low memory. This is not an easy excuse, it’s just a fact.

As for the other issue, truthfully I have never seen that “downsampling” console message before. I believe that you filed a bug report on this already. If so, we will attempt to reproduce it on our various Android devices, but it may also be an OS issue, a device memory issue, or something else that we cannot control. Again, this is not an excuse, it’s a fact of working within the parameters of the Android world with countless forked versions of the OS, thousands of devices with different capabilities and hardware, etc.

Best regards,

Brent

runewinse · March 26, 2015, 9:12pm

It’s also a fact that this happens to a Corona app and in the 1.5 years I’ve owned the phone I cannot recall something like that happening to any other app. Sure apps have crashed, but being restarted - no.

I didn’t file a bug report about the downsampling message specifically, but I may have mentioned that message in the bug report.

Do you think I should file a bug report about that message?

Because it IS a corona message since it starts with “V/Corona”, right?

bjsorrentino · March 26, 2015, 10:24pm

Hi @runewinse,

These console messages are Android log message headers. The “V” stands for “Verbose” and the “I” stands for “Information”:

http://developer.android.com/reference/android/util/Log.html

In regards to the “downsampling” message, I researched this a little further. Corona logs these messages on purpose when the image it’s trying to load is too large for the GPU to display. In this case, the image being loaded is larger than 4096 pixels wide or tall and it needs to be downscaled/downsampled to a smaller image so that the GPU can display it. So, this means that Corona is doing its job correctly by downsampling the image before pushing it to the GPU as a texture. Note that this is often the case for photos taken from the camera and photo gallery on both Android and iOS. The photos taken by these cameras are too large to be displayed by the device’s GPU and Corona has to automatically downscale/downsample them so that the GPU can display them.

Brent

bjsorrentino · March 24, 2015, 9:15pm

Hi @runewinse,

My first suggestion is to try commenting out certain aspects and then test if the crash occurs at the same frequency. For example, temporarily remove the lines where you load the captured image from the temporary directory, and test several times over. Also, in my experience, Android can be very fussy about “timing” of certain operations (what is performed in the same app time-step), and if you perform a certain action following a very small delay (10-50 milliseconds), sometimes that solves the problem entirely. So you might want to display.save() the photo after a small delay, instead of directly on the event completion. Even then, I don’t know if that will fix the issue, because every Android device has a different camera app (no standardization whatsoever like iOS) and that makes it difficult to account for behavior across the hundreds of manufacturers and thousands of devices with varying hardware and capabilities. However, do some testing and report back what happens.

Best regards,

Brent

runewinse · March 24, 2015, 10:39pm

Ok, I’ve now commented out so much that nothing really is happening, except for starting the camera. The only thing that happens in the listener is one print statement so that it is possible to see that it has been called at all:

local function itemPhotoOnComplete(event) if (event.completed == true) then print("--------------- itemPhotoOnComplete()") end end local function takePhotoTouchHandler(event) if (event.phase == "ended") then if media.hasSource( media.Camera ) then print("Camera detected - starting camera") media.capturePhoto( { listener = itemPhotoOnComplete } ) end end end print("\n\nApp started") local takePhoto = display.newText("PRESS HERE", display.contentWidth\*0.5, display.contentHeight\*0.9, native.systemFontBold, 50) takePhoto:addEventListener("touch", takePhotoTouchHandler)

But even with this bare minimum media.capturePhoto() the app restarts randomly

Starting app:

I/Corona (16322): ***** App started *****

I/Corona (16322):

First try to take a picture - app restarts:

I/Corona (16322): Camera detected - starting camera

V/Corona (16812): > Class.forName: network.LuaLoader

V/Corona (16812): < Class.forName: network.LuaLoader

V/Corona (16812): Loading via reflection: network.LuaLoader

I/Corona (16812): Platform: SM-G900F / ARM Neon / 5.0 / Adreno ™ 330 / OpenGL ES 3.0 V@84.0 AU@ (CL@) / 2015.2590

V/Corona (16812): > Class.forName: plugin.fuse.LuaLoader

V/Corona (16812): WARNING: Could not load ‘LuaLoader’

V/Corona (16812): > Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (16812): < Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (16812): Loading via reflection: CoronaProvider.licensing.google.LuaLoader

I/Corona (16812):

I/Corona (16812): ***** App started *****

I/Corona (16812):

Second try to take a picture - camera operation ok:

I/Corona (16812): Camera detected - starting camera

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812): --------------- itemPhotoOnComplete()

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812):

Third try to take a picture - camera operation ok:

I/Corona (16812): Camera detected - starting camera

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture48.jpg’ to fit max pixel size of 4096.

I/Corona (16812): --------------- itemPhotoOnComplete()

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture48.jpg’ to fit max pixel size of 4096.

V/Corona (16812): Downsampling image file ‘/storage/emulated/0/Pictures/Picture47.jpg’ to fit max pixel size of 4096.

I/Corona (16812):

Fourth try to take a picture - app restarts:

I/Corona (16812): Camera detected - starting camera

V/Corona (18340): > Class.forName: network.LuaLoader

V/Corona (18340): < Class.forName: network.LuaLoader

V/Corona (18340): Loading via reflection: network.LuaLoader

I/Corona (18340): Platform: SM-G900F / ARM Neon / 5.0 / Adreno ™ 330 / OpenGL ES 3.0 V@84.0 AU@ (CL@) / 2015.2590

V/Corona (18340): > Class.forName: plugin.fuse.LuaLoader

V/Corona (18340): WARNING: Could not load ‘LuaLoader’

V/Corona (18340): > Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (18340): < Class.forName: CoronaProvider.licensing.google.LuaLoader

V/Corona (18340): Loading via reflection: CoronaProvider.licensing.google.LuaLoader

I/Corona (18340):

I/Corona (18340): ***** App started *****

So in this example the behavior is just as I’ve been experiencing - about 50% of the time, the app restarts when using media.capturePhoto().

You advise me to add a delay. But what is there to be delayed in the bare minimum example above? There isn’t anything to delay!

One thing I notice is the apparent accumulation of the “Downsampling image file” lines. In one test run I managed to take 4 pictures in a row and then there were several of these lines in a row in the log.

Don’t know if it’s related to the app restart or not. Just mentioning it…

bjsorrentino · March 24, 2015, 11:37pm

Hello @runewinse,

Which build # of Corona are you using?

runewinse · March 24, 2015, 11:41pm

Hi,
I’m not at my pc at the moment, but according to my logs in my last post it’s 2015.2590.

bjsorrentino · March 25, 2015, 11:22pm

Hi @runewinse,

It’s possible that the device is low on memory and is forcefully terminating the Corona app, which is in the background while the camera app is in the foreground, because the camera app requires more memory from the OS to function. The OS typically logs when this situation happens, and a typical indication of this is that the app appears to “restart” when returning from the camera or photo gallery app.

Brent

runewinse · March 26, 2015, 9:28am

It’s seems a bit strange that if the camera app doesn’t have enough memory to function, the calling app is “killed” like this. If anything, it should be the camera app that should be failing. I cannot believe the OS does such a thing as restarting the app like this.

I’m more suspicious about what is happening behind these lines:

V/Corona (16812): Downsampling image file '/storage/emulated/0/Pictures/Picture48.jpg' to fit max pixel size of 4096.

This is a corona operation (it seems) that certainly craves memory to work. I wouldn’t be surprised if this operation fails and thus restarts the app.

But you’re probably right about the memory thing. By restarting the phone I’ve been able to do at least a dozen such photo captures in a row.

But I cannot have an app that behaves like this when the memory gets low. Blaming the OS is a bit of an easy way out. Even though it might be right, I would really like to get to the bottom of this.

How can you be sure that it is not the “Downsampling” thing that fails?
Is there some kind of debugging method for finding this out?

Thanks for your help so far!

bjsorrentino · March 26, 2015, 9:02pm

Hi @runewinse,

We (Corona) have no control over how the OS handles memory and what it decides to “close” as a result of low memory. This is not an easy excuse, it’s just a fact.

As for the other issue, truthfully I have never seen that “downsampling” console message before. I believe that you filed a bug report on this already. If so, we will attempt to reproduce it on our various Android devices, but it may also be an OS issue, a device memory issue, or something else that we cannot control. Again, this is not an excuse, it’s a fact of working within the parameters of the Android world with countless forked versions of the OS, thousands of devices with different capabilities and hardware, etc.

Best regards,

Brent

runewinse · March 26, 2015, 9:12pm

It’s also a fact that this happens to a Corona app and in the 1.5 years I’ve owned the phone I cannot recall something like that happening to any other app. Sure apps have crashed, but being restarted - no.

I didn’t file a bug report about the downsampling message specifically, but I may have mentioned that message in the bug report.

Do you think I should file a bug report about that message?

Because it IS a corona message since it starts with “V/Corona”, right?

bjsorrentino · March 26, 2015, 10:24pm

Hi @runewinse,

These console messages are Android log message headers. The “V” stands for “Verbose” and the “I” stands for “Information”:

http://developer.android.com/reference/android/util/Log.html

In regards to the “downsampling” message, I researched this a little further. Corona logs these messages on purpose when the image it’s trying to load is too large for the GPU to display. In this case, the image being loaded is larger than 4096 pixels wide or tall and it needs to be downscaled/downsampled to a smaller image so that the GPU can display it. So, this means that Corona is doing its job correctly by downsampling the image before pushing it to the GPU as a texture. Note that this is often the case for photos taken from the camera and photo gallery on both Android and iOS. The photos taken by these cameras are too large to be displayed by the device’s GPU and Corona has to automatically downscale/downsample them so that the GPU can display them.

Brent