CORONA BUG - display.capture does not work on images larger than screen size

Yes, this is true. Display.capture does not work on images larger than screen size. The following code was submitted as a BUG REPORT to Corona and is now Case 42777.

However, if you want to see this bug for yourself, run this code in a new main.lua and run it. The code itself uses text to explain what is going on so you can easily understand the BUG.

PLEASE FIX THIS ASAP as my project uses thousands of roundedRects and fixing this bug will increase my app performance substantially.

display.setStatusBar(display.HiddenStatusBar)
display.setDefault( “anchorX”, 0 )
display.setDefault( “anchorY”, 0 )

local VW = display.viewableContentWidth
local VH = display.viewableContentHeight
local cz=VH*.05
local fs=VH*.03

temp=display.newRoundedRect(0,0,VW,VH*.9,cz)  --yellow rounded rect
temp:setFillColor(1,1,0)
temp2=display.newRoundedRect(VW*.1,VH*.8,VW*.8,VH,cz)  --blue rounded rect
temp2:setFillColor(0,0,1)
local grp=display.newGroup()
grp:insert(temp)
grp:insert(temp2)

local grpT=display.newGroup()
local ts=“1) scroll up and down a little to see how this example group height (of two roundedRects) is greater than the screen height.\n2) Click capture to see how the height attribute of the captured image doesn’t change, although the image is reduced to the proportional height of the screen (DEFINATELY A BUG)”
local temp3 = display.newText({text=ts, x = VW*.15,y = temp2.y+10,width = VW*.7, height = 0, font = native.systemFont, fontSize = fs, align = “left”})
ts=“Screen Height:”…VH…"\nCombined Rects Height:"…grp.height
local temp4 = display.newText({text=ts, x = VW*.1,y = temp.y+(temp.height*.5),width = VW*.8, height = 0, font = native.systemFont, fontSize = fs, align = “left”})
temp4:setFillColor(0,0,0)
grpT:insert(temp3)
grpT:insert(temp4)

grp.y=-grp.height*.2  --show the group.y scrolled up a little
grpT.y=-grp.height*.2

local grpY,touchY,y  --allow us to scroll and see the height of the example
local function scroll(event)
    if event.phase==“began” then
        grpY=grp.y
        touchY=event.y
    elseif event.phase==“moved” then
        if grpY~=nil and touchY~=nil then
            y=grpY-(touchY-event.y)
            grp.y=y
            grpT.y=y
        end
    elseif event.phase==“ended” then
        grpY=nil
    end
end
grp:addEventListener(“touch”,scroll)

local buttonGrp=display.newGroup()
local buttonRect=display.newRoundedRect(VW*.5,VH*.3,VW*.4,VH*.1,cz)
buttonRect:setFillColor(0,0,1)
buttonRect.strokeWidth=VW*.01
local buttonText=display.newText(“CAPTURE”,(7777),(7777),native.systemFont,VH*.04)
buttonText.x=buttonRect.x+(buttonRect.width*.5)-(buttonText.width*.5)
buttonText.y=buttonRect.y+(buttonRect.height*.5)-(buttonText.height*.5)
buttonGrp:insert(buttonRect)
buttonGrp:insert(buttonText)

local function buttonPress(event)
    if event.phase==“began” then
        newImage=display.capture( grp, {isFullResolution=true } )    
        display.remove(grp)
        display.remove(buttonGrp)
        ts=“Screen Height:”…VH…"\nNewImage Height:"…newImage.height
        temp4.text=ts
        ts=“You are now looking at the captured image. And you can see that the newImage.height attribute appears correct. But when you scroll this newImage, you will see that the objects in the group VISIBLE ON THE SCREEN ONLY were captured and NOT the entire group per the API.\n\nIn fact, you can see how the capture process actually resigned the heights of the roundedRects by looking at the corners.”
        temp3.text=ts
        grpT:toFront()
        touchY=nil
        
        local newImageY,touchY,y  --allow us to scroll and see the height of the example
        local function scroll2(event)
            if event.phase==“began” then
                newImageY=newImage.y
                touchY=event.y
            elseif event.phase==“moved” then
                if newImageY~=nil and touchY~=nil then
                    newImage.y=newImageY-(touchY-event.y)
                end
            elseif event.phase==“ended” then
                newImageY=nil
            end
        end
        newImage:addEventListener(“touch”,scroll2)
        
    end
end
buttonGrp:addEventListener(“touch”,buttonPress)

Can you please take this, package it with a config.lua and bulld.settings and put it in a zip file and use the Report a Bug link at the top of the page.

You will get an email acknowledging the submission, there will be a Case ID in the subject. Please post ti back here when you’re done.

Rob

This bug has been reported as requested by Rob Miracle -

Case 42777

Rob, any idea when this will be fixed? My app is 85% complete - so I hope it will be fixed within the next 30 days.

Hi @troylyndon,

May I inquire why you believe this is a bug? The documentation for “display.capture()” states that it’s essentially a hybrid between “display.captureScreen()” and “display.save()”, and for the latter, “The object to be saved must be on the screen and fully within the screen boundaries.”.

Best regards,

Brent

At the bottom of the API description is written “display.capture() can be thought of as a hybrid between display.save() and display.captureScreen()”

There is simply no reference to the fact that the capture is limited to the size of the screen resolution for each device, and for this reason, I had thought it could be used to combine a group of display objects into one. In my App, I have more than 1,000 rects that need to be combined to save memory - anyway, I had thought this feature was not limited to screen size. Any chance you could perhaps create a new display.combine to do this? I have the feeling it wouldn’t take more than an hour or two since display.capture does this already, albeit with screen resolution restriction.

By the way, I have already written the code to reload textures on Android, and it works - so ideally, display.save could keep the resolution of a combine instead of saving the image restricted to screen resolution.

Hi @troylyndon,

I apologize for the miscommunication. This actually should be possible if you specify the “isFullResolution” boolean as true during the API call. I will update the documentation on this point soon, so that it’s clearer.

Of course, it appears that you’re already doing so… I’ll need to check out your bug report code soon to see what might be occurring.

Thanks,

Brent

Thank you, Brent. Please see what you can do to have this bug fixed soon - it’s a big deal to my app and shouldn’t take very long to fix, since it appears to have been an oversight - perhaps even a single line flag that was set for debugging.

You can work-around this issue by calling display.save() with the isFullResolution flag.

   https://docs.coronalabs.com/api/library/display/save.html

That’ll save content outside the bounds of the screen to file.  You can then display that file via display.newImageRect(), feeding the same content width and height of the original object that was captured.

Also note that if you plan on targeting Android *and* you plan on displaying the captured image for a long period of time, then you’ll need to use display.save() anyways.  This is because Android dumps all loaded textures/images from memory when your app gets suspended, such as when you press the Home button.  When you resume your app, Corona is forced to reload all images into memory from file, but Corona cannot recover the images they were generated via the display.capture*() function because they were deleted from memory and are not persisted to file anywhere.  So, the only way to recover captured images after a suspend/resume to make sure that they’re saved to file.

Hi Joshua,

I’m sorry to report that your work around does not work. Essentially, the display.save command is also broken when isFullResolution=true, as it only saves a file that matches the screen resolution and ignores the actual resolution of the defined group or object. This ‘related issue’ is asked here, but has not received a worth reply yet: https://forums.coronalabs.com/topic/59273-displaysave-and-isfullresolution-param-issue/

Can you answer another important question?

The ORIGINAL structure of the code simply 1) performed a screen capture and then 2) saved the image (to be loaded upon an Android resume). Now, based upon your recommendation, the NEW code 1) saves the image first and then 2) loads it from disk.

Which takes more CPU time to perform? The ORIGINAL or NEW code? And most importantly, I’m sorry to put you on the spot, but when will you and the Corona Team be able to provide us (the Corona community) with this feature that operates and functions as written in the API?

Very kindly as always, -Troy

Hi Troy,

I responded to the post that you mention, which should clarify some of this:

https://forums.coronalabs.com/topic/59273-displaysave-and-isfullresolution-param-issue/

Brent

>> I’m sorry to report that your work around does not work.

Yes, it does.  We’ve confirmed it on our end.  The thing you have to remember about display.save() is that it saves the screen capture in *pixels*… and not in Corona’s scaled content coordinates.  This means that the capture image that gets saved to file will be larger for high resolution screens and smaller for low resolution screens.  It makes sense for it to work this way.

What you need to do is use display.newImageRect() and load the capture image flle to the width and height you want it to be in your app’s Corona content scaled coordinate system.  This way Corona will automatically scale the loaded image from pixels to your app’s scaled coordinate system that you’ve defined in your “config.lua”.  A simple way to do this so as follows…

display.save(myGroup, "myCapture.png", system.DocumentsDirectory) local myCaptureImage = display.newImageRect ( "myCapture.png", system.DocumentsDirectory, myGroup.contentWidth, myGroup.contentHeight ) myCaptureImage.x = myGroup.x myCaptureImage.y = myGroup.y myGroup:removeSelf()

>> Which takes more CPU time to perform?

Yes, the display.capture*() APIs will perform faster than display.save(), but the work-around I’m providing above also handles the suspend/resume case on Android so that you can restore your captured images onscreen upon resume.

One more thing to note…

It is highly unusual to capture content larger than the screen.  The biggest issue with this is that GPUs (ie: the graphics processor) have what’s called a “max texture size” where there is limit to how tall/wide an image/texture can be when loaded into the GPU.  Some devices have a max texture size that match the height of the screen, such as 2048 or 4096 pixels.  So, the issue with this is that if you attempt to load image (such as your capture) that is larger than the GPU’s supported max texture size, Corona is forced to downscale/downsample the image before loading it as a texture into the GPU.  The result of this will be a blurry picture because the captured image will have been downscaled/downsampled and then the texture will be scaled up back to what the original size of the image was in the file.  This effect typically goes unnoticed for camera photos, but it tends to be highly noticeable for screenshots having a lot of solid colored regions.  So, even if we were to *fix* the display.capture*() APIs to display the captured image at the original size, this max texture size downscaling/downsampling issue would still happen and that’s not something we can fix.  That’s a technical issue on the GPU side that is best to be avoided.  You may need to reconsider your app’s design.  Just my 2 cents.

I’ve done some tests and display.save() is saving off screen bits.  Can you try this code:

local background = display.newRect( display.contentCenterX, display.contentCenterY, display.contentWidth, display.contentHeight) background:setFillColor( 0, 0.2, 0) local myObject0 = display.newRect( 0, 0, 300, 300) myObject0:setFillColor(1, 0, 0) local myObject1 = display.newRect( 150, 150, 100, 150 )  -- Create a rectangle object local myObject2 = display.newCircle( 100, 300, 50 )    -- Create a circle object local myObject3 = display.newRect( display.contentWidth, display.contentHeight, 300, 300) myObject3:setFillColor(0, 0, 1) local group = display.newGroup() group:insert( background ) group:insert( myObject1 ) group:insert( myObject2 ) group:insert( myObject0 ) group:insert( myObject3 ) timer.performWithDelay(1000, function()     display.save( group, { filename="entireGroup.png", baseDir=system.DocumentsDirectory, isFullResolution=true, backgroundColor={0,0,0,0} } ) end) timer.performWithDelay(2000, function()     local img = display.newImage("entireGroup.png", system.DocumentsDirectory)     img.x = display.contentCenterX     img.y = display.contentCenterY     img:scale(0.5, 0.5)     background:setFillColor(0.5, 0, 0.25) end)

Joshua, very good comment but…if a group contains 10 screens high of objects and the player can scroll that group on every device, and the visibility of such group works flawlessly, does Corona manage the coming in and out of smaller texture pieces by itself so that a group of objects can be infinitely large? Or are groups limited to the size of texture memory?

Rob, I appreciate you dedicating time to some code on this, really - may I ask a dumb question? Which objects in your code represent pixels outside the resolution of the content width or height?

Joshua one more thing - using display.newImageRect is clearly documented for use with content scaling. I have no content scaling and maximize screen space on devices with larger areas. I will look more at Rob’s code and report my findings.

>> if a group contains 10 screens high of objects and the player can scroll that group on every device, and the visibility of such group works flawlessly, does Corona manage the coming in and out of smaller texture pieces by itself so that a group of objects can be infinitely large?

For objects such as rectangles and images, yes.  And images you use more than once are loaded as a single texture and reference counted by Corona.

More complex objects such as circles would be less efficient because implementing a smooth rounded edge via OpenGL and the GPU would require a lot of vertices (ie: a lot of polygons with straight edges so small that you can’t see them) to achieve that effect.  This is very expensive.  If you have a lot of circular/rounded polygons, then it might be better not to use them.  For example, if you’re using rounded rectangles, then it might be better to replace those with a “9-patch image” where you display images for the corners and sides, which is a much cheaper operation to perform on the GPU side.  Note that the 9-patch approach is what Google uses for its UI on Android.  It provides good performance and it makes it easier to edit the theme.

And you can still use display.newImageRect() even though you’re not using Corona’s content scaling.  Although you’ll only benefit from it if the group you were capturing was scaled.  All that display.newImageRect() does is tell Corona to scale the loaded image to the width and height you’ve provided… and that function can scale both up and down.

Object0 and Object3 are partially off screen. For instance Object0 is centered on 0,0 so 3/4 of it is off screen. Only the bottom right quadrant on screen.  Object3 is at display.contentWidth, display.contentHeight. 3/4 of it is off screen, only it’s top, left quadrant is on screen.

Kudos to Brent, Joshua and Rob for help with this issue, although not yet resolved.

Rob’s example works, no doubt, in creating a PNG file that is larger than the screen resolution. But this doesn’t explain why this doesn’t occur in my App, despite a lot of time spent on it.

I will pursue this further and report my findings to you.

Thanks again, guys.

Happy to help!

One more thing.  One of our QA guys here verified that display.capture*() with the “saveToPhotoLibrary” argument set to true will actually save the captured are outside of the screen at the correct resolution to file just like how display.save() does it.  The issue is that the image displayed is downscaled.

Also, if you’re displaying your captured image file via display.newImage() still, then make sure to set the “isFullResolution” argument to true in that function as well.  Otherwise OS X and iOS will automatically downscale the loaded image to fit within the bounds of the screen.  You might be getting burned by this.  Note that you don’t have to worry about this with display.newImageRect().

   https://docs.coronalabs.com/api/library/display/newImage.html#syntax

okay guys. run this code and it explains the entire problem with display.save and steps you through how display.save() is broken and how in 30 seconds you can witness the problem I’m having in my App.

display.setStatusBar(display.HiddenStatusBar)
display.setDefault( “anchorX”, 0 )
display.setDefault( “anchorY”, 0 )
W=display.contentWidth
H=display.contentHeight
local ts=“What we have here is a colored grid that is 2 screens wide and 2 screens high. Move grid around to see that it’s larger than one screen.”
TextObject=display.newText({text=ts,x=0,y=0,width=W,height,0,font=native.systemFont,fontSize=H*.05,align=“left”})
–====================================================================
function saveAsImage()
    TextObject:removeEventListener(“touch”,saveAsImage)
    – save the display group
    local gw=G_grid.width
    local gh=G_grid.height
    
    display.save( G_grid, { filename=“entireGroup.png”, baseDir=system.DocumentsDirectory, isFullResolution=true, backgroundColor={0,0,0,0} } )
    
    – delete original display objects and group now that we have saved the file
    local a
    for a=1,#G_grid do
        display.remove(D_grid)
    end
    display.remove(G_grid)
    G_grid=nil
    
    – now load the image and scale it down as defined by variable ‘sc’
    local img = display.newImage(“entireGroup.png”, system.DocumentsDirectory)
    local sc=50  – % percent of scale after load
    local ts=“This displays the saved image at”…sc…"%. When set to 50%, this should fill screen, because 50% of 2x2 screens is a 1x1 filled screen. "
    ts=ts…"However, if you analyze the ‘entireGroup.png’ file, you will see that it was saved with the proper resolution. "
    ts=ts…“However, only what is visible on the screen at the time of the save is actually visible in the file itself.”
    ts=ts…“This is not evident in a ‘preview’ program, but loading the ‘entireGroup.png’ file up in photoshop makes this clear.\n\n”
    ts=ts…“Also, the loaded Image Resolution of “…img.width…“x”…img.height…” is actually smaller than the saved image file resolution of “…gw…“x”…gh…” because the non-displayed area which wasn’t visible on screen at the time of the display.save was transparent and is cropped when loaded.”
    TextObject.text=ts
    TextObject:toFront()
    img.x = (W*.5)-(gw*sc*.01*.5)
    img.y = (H*.5)-(gh*sc*.01*.5)
    img.width=gw*sc*.01
    img.height=gh*sc*.01
    
    – this code allows you to move the newly loaded image around to see it’s size
    local imgX,imgY,touchX,touchY,x,y  --allow us to scroll and see the height of the example
    local function scroll(event)
        if event.phase==“began” then
            imgX=img.x
            imgY=img.y
            touchX=event.x
            touchY=event.y
        elseif event.phase==“moved” then
            if (imgY~=nil and touchY~=nil) or (imgX~=nil and touchX~=nil) then
                x=imgX-(touchX-event.x)
                y=imgY-(touchY-event.y)
                img.x=x
                img.y=y
            end
        elseif event.phase==“ended” then
            imgX=nil
            imgY=nil
        end
    end
    img:addEventListener(“touch”,scroll)
end
TextObject:addEventListener(“touch”, saveAsImage)
–====================================================================
function createGrid()
    – create grid to see the position of field
    G_grid=display.newGroup()
    D_grid={}
    local z1,z2,x,y
    for z1=1,20 do
        for z2=1,20 do
            if anchor==0 then
                x=(z1-.5)*(W*.1)
                y=(z2-.5)*(H*.1)
            else
                x=(z1-.5)*(W*.1)
                y=(z2-.5)*(H*.1)
            end
            – create obj in grid
            D_grid[#D_grid+1]=display.newRect(x,y,W*.1,H*.1)
            D_grid[#D_grid]:setFillColor(z1*.05,z2*.05,(z1+z2)*.025)
            
            – add object to group
            G_grid:insert(D_grid[#D_grid])
        end
    end
    --because the grid is 2 screens wide and high, let’s center what we see on group
    G_grid.x=-W*.5
    G_grid.y=-H*.5
    
    – this code allows us to scroll the grid around, since it is larger than one screen
    local img=G_grid
    local imgX,imgY,touchX,touchY,x,y  --allow us to scroll and see the height of the example
    function scrollGrid(event)
        if event.phase==“began” then
            imgX=img.x
            imgY=img.y
            touchX=event.x
            touchY=event.y
        elseif event.phase==“moved” then
            if (imgY~=nil and touchY~=nil) or (imgX~=nil and touchX~=nil) then
                x=imgX-(touchX-event.x)
                y=imgY-(touchY-event.y)
                img.x=x
                img.y=y
            end
        elseif event.phase==“ended” then
            imgX=nil
            imgY=nil
            TextObject.text=“You can keep moving the image around if you like, but when you are ready, click on this text object to perform the display.save and display.newImage - after which the code will reload the saved image at 50% resolution. This can be adjusted in the code by modifying the variable ‘sc’.”
            TextObject:toFront()            
        end
    end
    G_grid:addEventListener(“touch”,scrollGrid)
end
–====================================================================
createGrid()
TextObject:toFront()
 

I have the project in zip file if you’d like me to post it as a bug, again.