Image capture/cropping using display.save( )

bob.dickinson · February 5, 2012, 11:43pm

In my application I have the need to take a snapshot of a section of the display and save it to a file. Specifically, I have a level builder (a group with physics objects) and I want the level designer to be able to select the region of the screen that represents the level to use as a “preview” image in the game.

I’m fairly close to having a game ready to ship, but I don’t think I can release it without the ability to save a rectangular region of the screen (or a rectangular region of a display object).

I explain my problem and my original workaround in the discussion on the documentation page for display.save( ):

https://developer.anscamobile.com/reference/index/displaysave

Basically, I make an onscreen camera object that can be dragged around. The users scales/pans the game level, positions the camera, and then pushes the “take picture” button. At that point I call display.save( ) with an empty/transparent rectangle object that contains the section of the screen that I want (basically the viewfinder part of the camera object). The idea is to save the selected area as an image.

This works great on the Windows simulator and on Android. But on iOS devices you get a black rectangle instead of the contents of the screen under the invisible rectangle (not exactly inconsistent with the documentation, but certainly different than Windows/Android).

FWIW, on the Mac simulator display.save( ) appears to be completely broken (I have the latest daily build - even the ScreenCaptureToFile app doesn’t work). But I think that is known and being worked on.

I have done a significant amount of research on this issue. There are a lot of people who seem to like to suggest “hey, just use a mask!” every time someone mentions a problem like this. I don’t think a mask really solves the problem (you might be able to save lots of metadata and try to be really clever when later displaying the image - loading full screen images and sucking up all the texture memory for them, just to mask off that part you want, which is also borderline impossible since you can’t create a mask at a specific size/shape in code).

In looking for a workaround, I found a lot of other people who seem to want to do the same thing (crop an image in their UI). Here is a partial list:

http://developer.anscamobile.com/forum/2012/01/31/cropping-images

http://developer.anscamobile.com/forum/2011/01/16/crop-image

http://developer.anscamobile.com/forum/2011/07/29/simple-photo-cropping-possible

http://developer.anscamobile.com/forum/2011/08/09/cropping-photo-bit-mask

http://developer.anscamobile.com/forum/2011/10/19/capture-specified-area-screen

http://developer.anscamobile.com/forum/2010/10/19/clip-or-crop-images

There are also several people who commented on the display.save( ) documentation page that wanted this kind of cropping / region saving.

The closest thing I could find to a working solution is this one:

http://www.youtube.com/watch?v=23qu-b8mmRU

And that involved sending the image to be cropped to a remote server that did the cropping and sent the cropped image back (super clever, but that’s an awfully long way to go).

For my purposes, I could envision either making the iOS version of display.save( ) work like Android and document that (that it is truly a screenshot of the region defined by the object, and not “sometimes a screenshot and sometimes a rendering of the object, depending on what platform you are on”). Alternatively, adding an optional “region” parameter to display.save that captured the screen contents of the specified object inside of that region would work (region could be a rectangle object, a set of top, left, height, width coords, whatever). Either approach would allow someone to pass in the stage, or a group, or even an individual object, and save a rectangular region to a file.

I think if I added up all of the people that have asked for a version of this in the last year I’m at at least “+20” for this feature. I’m reasonably familiar with the underlying OpenGL mechanics and APIs on both iOS and Android, and I have a really hard time imagining that this change would be all that complex internally.

I’d be happy to help any way I can. I can provide some non-trival sample code. I can help test the solution (I develop for both iOS and Android on both Windows and Mac and have every generation of every iOS device as well as a pretty healthy stable of Android devices).

But for right now, I’m stuck, and I don’t see a good path to shipping my app using Corona SDK that doesn’t involve this issue getting addressed by ANSCA.

[import]uid: 111481 topic_id: 21444 reply_id: 321444[/import]

Satheesh · March 3, 2012, 5:54am

Check out this crop library
http://developer.anscamobile.com/code/useful-crop-library [import]uid: 64174 topic_id: 21444 reply_id: 90947[/import]

bob.dickinson · March 3, 2012, 12:33pm

@Satheesh - That’s a very interesting library (especially being able to make a custom mask from a polygon).

My problem above is not really a “masking” type of cropping problem (as your library solves very nicely). I wanted to save a region of the screen to a file (where that region could be composed of a complex collection of objects). I did eventually figure out a way to do this, but it was not pretty. [import]uid: 111481 topic_id: 21444 reply_id: 90987[/import]

paul10 · March 25, 2012, 7:44pm

Bob - how did you do this? [import]uid: 90248 topic_id: 21444 reply_id: 96192[/import]

bob.dickinson · April 1, 2012, 5:50pm

Actually, the really clever way that I got cropping to work has been broken by a couple of recent Corona changes (namely, that display.save() no longer works the way it used to on Android so you can’t use if for capture anymore, and they changed what the height/width parameters do on image objects, so you can’t use those for cropping anymore either).

So as far as I know, given the current daily build, it is again not possible to crop an image in Corona.

I’ve filed a bug report on the image height/width change (though it might have been intentional for all I know) and another one asking (yet again) for a supported display object cropping API. I’m am currently VERY stuck (my app requires cropping, and also some other bug fixes that aren’t available in the last stable release). [import]uid: 111481 topic_id: 21444 reply_id: 97825[/import]

bob.dickinson · April 6, 2012, 11:03am

OK, I did finally figure out another way to get image cropping to work with current daily builds. This takes advantage of the new image sheet API. I basically take a screen capture, save it to a file, load that file into an image sheet, then reference the area that I want to capture in that image sheet creating an display object, and then saving that to a file. Whew.

It’s ugly and very slow, especially on Android (~5 seconds on a Kindle Fire) because of all of the file IO. But it does work.

I’m *REALLY* hoping our Corona overlords will just add a parameter to display.save() / display.capture() to support cropping soon!

local function cropImage(filename, x, y, width, height)  
  
 -- Make a screen capture and save it to a file.  
 local imgFullScreen = display.captureScreen(false)  
 display.save(imgFullScreen, "screen.jpg", system.DocumentsDirectory)  
 imgFullScreen:removeSelf()  
 imgFullScreen = nil  
  
 -- x, y, width, height are in scaled units - convert to image pixel values.  
 --  
 img\_x = (x - display.screenOriginX) / display.contentScaleX  
 img\_y = (y - display.screenOriginY) / display.contentScaleY  
 img\_width = width / display.contentScaleX  
 img\_height = height / display.contentScaleY  
  
 -- Make an image sheet from the screen capture file, with a single frame consisting  
 -- of the crop target.  
 local options = {  
 frames = {  
 {  
 x = img\_x - img\_width/2,  
 y = img\_y - img\_height/2,  
 width = img\_width,  
 height = img\_height,  
 },  
 },  
 }  
 local screenImageSheet = graphics.newImageSheet("screen.jpg", system.DocumentsDirectory, options)  
  
 -- Create an image from the image sheet, then save it to a file.  
 local captureImg = display.newImage(screenImageSheet, 1, true)  
 captureImg.x = x  
 captureImg.y = y  
 captureImg.width = width  
 captureImg.height = height  
 display.save(captureImg, filename, system.DocumentsDirectory)  
  
 -- Clean up  
 captureImg:removeSelf()  
 captureImg = nil  
 screenImageSheet = nil  
 os.remove(system.pathForFile("screen.jpg", system.DocumentsDirectory))  
end

[import]uid: 111481 topic_id: 21444 reply_id: 99034[/import]

pau7 · May 28, 2014, 2:19pm

I have recently found this post and I’am using the code bob.dickinson posted above. It was working fine and suddenly it started to give me this horrible error:

malloc: *** mach_vm_map(size=18446743113140129792) failed (error code=3)
*** error: can’t allocate region
*** set a breakpoint in malloc_error_break to debug

Does anybody tested it? It seems that corona display.save() is not working properly.

Does Corona plan to add a feature to trim transparent pixels when saving a display group into a file?

Thank you!

pau7 · May 28, 2014, 4:23pm

I think I have found why it crashes. I seems display.save() crashes when the image to save is outside the content area. Make sure your display object is inside the display area and it will work. Hope helps someone.

pau7 · May 28, 2014, 2:19pm