Increased gaussian blur causes increased CPU load

Audio and Graphics
#1

I’ve wanted to create blurred circles without using an image, so I figured that applying gaussian blur to a snapshot would allow blurs to be applied to circles produced with newCircle. Unfortunately, this not only does not work, but the load on the GPU in even the simulator is so much, beyond a certain value, that the operation of the whole machine becomes choppy. I’ve tried this in the mac sim using the latest daily build 2093:

pWidth, pHeight = display.actualContentWidth, display.actualContentHeight local circle = display.newCircle( 0, 0, pWidth\*.25 ) circle.fill = RGB.red local snapshot = display.newSnapshot( pWidth, pHeight ) snapshot:translate( pWidth\*.5, pHeight\*.5 ) snapshot.canvasMode = "discard" snapshot.canvas:insert( circle ) local blur = 2 snapshot.fill.effect = "filter.blurGaussian" snapshot.fill.effect.horizontal.blurSize = 1 snapshot.fill.effect.vertical.blurSize = 1 snapshot:invalidate( "canvas" ) -- accumulate changes w/o clearing timer.performWithDelay( 1, function()     blur = blur + 1     snapshot.fill.effect.horizontal.blurSize = blur     snapshot.fill.effect.vertical.blurSize = blur     snapshot:invalidate( "canvas" )     print(blur) end, 300 )  
#2

In your example your maximum horizontal/vertical blur size is 300! That’s enough to kill any GPU.
Keep in mind that this means that the GPU will have to calculate 300x300 = 90.000 new pixel values for *every* pixel in your original image, and you’re changing the value every millisecond.

#3

What value do you suppose iOS uses for the control panel?

Even if you reduce the number to 60 and add 10 circles to the snapshot and it’s just as bad.

#4

OK I think I know what’s going on here.

The problem is that the effect isn’t rendered to the snapshot texture itself. It’s applied to the snapshot on every single frame causing an overload on the GPU. I guess you want to apply the effect to the snapshot texture once, right? In that case you’ll need to put the snapshot within another snapshot so that the first (inner) snapshot is only rendered once.

Something like this (I’ve added an extra square object to the inner snapshot):

pWidth, pHeight = display.contentWidth, display.contentHeight local circle = display.newCircle( 0, 0, pWidth\*.25 ) circle:setFillColor(1,0,0); local square = display.newRect(100, 100, 100, 100 ) square:setFillColor(0,1,0); local innerSnapshot = display.newSnapshot( pWidth, pHeight ) innerSnapshot.group:insert( circle ) innerSnapshot.group:insert( square ) local blur = 300 innerSnapshot.fill.effect = "filter.blurGaussian" innerSnapshot.fill.effect.horizontal.blurSize = blur innerSnapshot.fill.effect.vertical.blurSize = blur local outerSnapshot = display.newSnapshot(innerSnapshot.width, innerSnapshot.height ) outerSnapshot:translate( pWidth\*0.5, pHeight\*0.5 ) outerSnapshot.group:insert(innerSnapshot);
#5

In my case, yes, I want to apply the blur effect to a single object, then move that around. Specifically, I want translucent coloured blobs floating around the screen. I was hoping that there would be an easy way to create a circle from the display.newCircle function, apply a varying level of blur and scaling and then just use a transition. As it is, it seems that I need to use an image with a blurry edge, which does not afford me as much control as I would like.

#6

I’ve just written this, which is about what I’d like. What I’d really like is to make the spots draggable by the perimeter of the circles, rather than the snapshot, which is proving tricky. I’ve tried a number of things, including putting the snapshots inside a parent group then putting a touch layer object on top of the snapshot, in the same group. Doesn’t work.

-- snapshot blur test display.setStatusBar( display.HiddenStatusBar ) display.setDefault( "background", 1, 1, 1 ) math.randomseed( os.time() ) stage = display.getCurrentStage() pWidth, pHeight = display.contentWidth, display.contentHeight local group = display.newGroup() local xSpan, ySpan = pWidth\*.5, pHeight\*.5 local function newEndpoint( isStart ) local x = math.random( 0, pWidth ) - pWidth\*.5 local y = pHeight if (isStart) then y = -y end return x, y end function newDot( x, y, r, c, b ) local circleSnapshot = display.newSnapshot( 600, 600 ) circleSnapshot.circle = display.newCircle( 0, 0, r ) circleSnapshot.circle:setFillColor(c[1],c[2],c[3],c[4] or 1) circleSnapshot.group:insert( circleSnapshot.circle ) circleSnapshot.fill.effect = "filter.blurGaussian" circleSnapshot.fill.effect.horizontal.blurSize = b circleSnapshot.fill.effect.vertical.blurSize = b circleSnapshot.x, circleSnapshot.y = x, y return circleSnapshot end local radius = 150 local blur = 50 local alpha = .5 a = newDot( 200, 200, radius, {1,0,0,alpha}, blur ) b = newDot( 350, 200, radius, {0,1,0,alpha}, blur ) c = newDot( 275, 350, radius, {0,0,1,alpha}, blur ) function touch(e) if (e.phase == "began") then e.target.hasFocus = true stage:setFocus(e.target) e.target.x, e.target.y = e.x, e.y return true elseif (e.target.hasFocus) then e.target.x, e.target.y = e.x, e.y if (e.phase == "moved") then -- nothing else e.target.hasFocus = false stage:setFocus(nil) end return true end return false end a:addEventListener("touch",touch) b:addEventListener("touch",touch) c:addEventListener("touch",touch)
#7

I think my previous code was a failure because I was expecting the blur effect to be applied each time the objects which were being blurred was changed. It can, of course, be applied to each object separately.

#8

Well, here is some code which, as far as I can tell, does not require any extra CPU/GPU load to perform the blur because it’s already been calculated. I think. This code, however, starts to run extremely slowly if the spot count goes above 10 or the max blur value is increased. As it is, it runs almost just fine on an iPhone5s - though the device noticeably heats up.

Download: https://dl.dropboxusercontent.com/u/10254959/SnapshotBlurTest/SnapshotBlurTest.zip

main.lua

-- snapshot blur test require("rgb") display.setStatusBar( display.HiddenStatusBar ) display.setDefault( "background", 1, 1, 1 ) math.randomseed( os.time() ) stage = display.getCurrentStage() pWidth, pHeight = display.contentWidth, display.contentHeight local colours = { "red", "green", "blue", "purple", "yellow", "dodgerblue", "lightseagreen", "gold", "maroon", } local spotcount = 10 local group = display.newGroup() group.x, group.y = pWidth\*.5, pHeight\*.5 local xSpan, ySpan = pWidth\*.5, pHeight\*.5 local function newEndpoint( isStart ) local x = math.random( 0, pWidth ) - pWidth\*.5 local y = pHeight if (not isStart) then y = -y end return x, y end function newDot( p, x, y, r, c, a, b ) print( p, x, y, r, c, a, b ) local circleSnapshot = display.newSnapshot( p, 600, 600 ) circleSnapshot.circle = display.newCircle( 0, 0, r ) circleSnapshot.circle:setFillColor(c[1],c[2],c[3],a) circleSnapshot.group:insert( circleSnapshot.circle ) circleSnapshot.fill.effect = "filter.blurGaussian" circleSnapshot.fill.effect.horizontal.blurSize = b circleSnapshot.fill.effect.vertical.blurSize = b circleSnapshot.x, circleSnapshot.y = x, y return circleSnapshot end local function launch(dot) if (dot.name == "timer") then local colour = RGB[colours[ math.random(1,#colours)] ] dot = newDot( group, 0, 0, math.random(10, 40), colour, 1/(100/math.random(20,70)), math.random(10, 30) ) end local sx, sy = newEndpoint( true ) local ex, ey = newEndpoint( false ) dot.x, dot.y = sx, sy transition.to( dot, { time=math.random(4000,15000), x=ex, y=ey, onComplete=launch } ) end for i=1, spotcount do timer.performWithDelay( math.random(1000,7000), launch, 1 ) end

rgb.lua

-- rgb --Colors referenced from http://www.tayloredmktg.com/rgb/ RGB = { neoncyan = {16, 174, 239}, -- Neon Cyan neonyellow = {231, 228, 37}, -- Neon Yellow neonpink = {231, 83, 177}, -- Neon Pink neongreen = {4, 228, 37}, -- Neon Green iosblue = {0, 122, 255}, -- iOS Blue iosgrey = {146, 146, 146}, -- iOS Grey iosgreen = {50, 155, 43}, -- iOS Green iosdarkgreen = {35, 105, 28}, -- iOS Dark Green white = {255, 255, 255}, --White orange = {255, 132, 66}, --Orange pink = {255, 192, 203}, --Pink red = {255, 0, 0}, --Red green = {0, 255, 0}, --Green blue = {0, 0, 255}, --Blue purple = {160, 32, 240}, --Purple black = {0, 0, 0}, --Black yellow = {255, 255, 0}, --Yellow grey = {150, 150, 150}, --Grey whitesmoke = {245, 245, 245}, --White Smoke skyblue = {135, 206, 250}, --Sky Blue deepskyblue = {0, 191, 255}, --Deep sky blue dodgerblue = {30, 144, 255}, --Dodger Blue navy = {0, 0, 128}, --Navy lightskyblue = {135, 206, 250}, --Light Sky Blue lightcyan = {224, 255, 255}, --Light Cyan powderblue = {176, 224, 230}, --Powder Blue darkgreen = {0, 100, 0}, --Dark Green darkolivegreen = {85, 107, 47}, -- Dark Olive Green yellowgreen = {154, 205, 50}, --Yellow Green khaki = {240, 230, 140}, --Khaki kellygreen = {76, 187, 23}, --Kelly Green northtexasgreen = {5, 144, 51}, --North Texas Green mediumspringgreen = {0, 250, 154}, -- Medium Spring Green honeydew = {240, 255, 240}, --Honeydew lightseagreen = {32, 178, 170}, --Light Sea Green palegreen = {152, 251, 152}, --Pale Green limegreen = {50, 205, 50}, --Lime Green forestgreen = {34, 139, 34}, --Forest Green gold = {255, 215, 0}, --Gold mustard = {255, 192, 3}, --Mustard lemonchiffon = {255, 250, 205}, --Lemon Chiffon lightgoldenrodyellow = {250, 250, 210}, --Light Goldenrod Yellow darkred = {160, 0, 0}, --Dark Red peach = {255, 185, 143}, --Peach hotpink = {255, 105, 180}, --Hot Pink deeppink = {255, 20, 147}, --Deep Pink turquoise = {64, 224, 208}, --Turquoise kcred = {207, 0, 0}, --KC Red petal = {173, 90, 255}, --Petal indianred = {205, 92, 92}, --Indian Red violetred = {208, 32, 144}, --Violet Red darksalmon = {233, 150, 122}, --Dark Salmon lavenderblush = {255, 240, 245}, --Lavender Blush wheat = {245, 222, 179}, --Wheat thistle = {216, 191, 216}, --Thistle maroon = {176, 48, 96}, --Maroon bamboo = {216, 199, 169}, --Bamboo greyflannel = {195, 196, 192}, -- Grey Flannel oldlace = {249, 240, 226}, --Old Lace brownbag = {192, 137, 103}, -- Brown Bag mediumgrey = {175, 175, 175}, -- Med Grey darkwood = {133, 94, 66}, -- Dark Wood woolgrey = {143, 148, 152}, -- Wool Grey tan = {139, 90, 43}, --Tan darkgrey = {100, 100, 100}, -- Dark Grey darkbrown = {95, 59, 24}, --Dark Brown } -- added since v.1225 local function convertForGfx2() for k,v in pairs(RGB) do for i=1, #v do v[i] = v[i] / 255 end end end convertForGfx2() local colours = { primary = { "red", "blue", "kellygreen", }, moderate = { "red", "blue", "kellygreen", "deeppink", "deepskyblue", "gold", }, advanced = { "red", "blue", "kellygreen", "deeppink", "deepskyblue", "gold", "darkbrown", "lightseagreen", "mediumspringgreen", }, } function getColourBars() local group = display.newGroup() local scrollview = widget.newScrollView { top = 0, left = 0, width = pWidth, height = pHeight, scrollWidth = pWidth, scrollHeight = pHeight, } group:insert(scrollview) local text = display.newText( group, "", 0,50, nil, 48 ) function touch(e) text.text = e.target.text text.x = pWidth/2 return true end local y = 0 if (true) then for k,v in pairs(RGB) do local rect = display.newRect( 0, 0, pWidth, 50 ) scrollview:insert(rect) rect.y=25+y rect.text = k rect:setFillColor( v[1], v[2], v[3] ) rect:addEventListener("tap",touch) y=y+50 end else for k,v in pairs(colours) do for i=1, #v do local rect = display.newRect( 0, 0, pWidth, 50 ) scrollview:insert(rect) rect.y=25+y rect.text = v[i] local col = RGB[v[i]] rect:setFillColor( col[1], col[2], col[3] ) rect:addEventListener("tap",touch) y=y+50 end y=y+150 end end return scrollView end
#9

The reason things start to go slow is because the blur effect is never rendered to the texture. The blur is being calculated for every object on every frame.

If you want the blur to only be rendered once, then you must have a second snapshot which contains the first snapshot (with the blur effect) as I described above.

#10

I modified your newDot function as follows.

After doing this I was able to increase the spotCount to 50 and the blur up to 100 and still see good performance. The more spots and the wider blur you set, there may be an initial stuttering as the rendering is being done for the first time, but after that the animation is smooth.

I used the snapshot canvas here. I’m not sure if it’s necessary, but I think that less resources will be used by doing so.

function newDot( p, x, y, r, c, a, b ) print( p, x, y, r, c, a, b ) local circle = display.newCircle( 0, 0, r ) circle:setFillColor(c[1],c[2],c[3],a) local circleSnapshot = display.newSnapshot(r\*2+b\*2, r\*2+b\*2 ) circleSnapshot.canvas:insert( circle ) circleSnapshot.canvasMode = "discard"; circleSnapshot.fill.effect = "filter.blurGaussian" circleSnapshot.fill.effect.horizontal.blurSize = b circleSnapshot.fill.effect.vertical.blurSize = b circleSnapshot:invalidate("canvas"); local outerSnapshot = display.newSnapshot(p, circleSnapshot.width, circleSnapshot.height) outerSnapshot.canvas:insert(circleSnapshot); outerSnapshot.canvasMode = "discard"; outerSnapshot:invalidate("canvas"); outerSnapshot.x, outerSnapshot.y = x, y return outerSnapshot; end
#11

Awesome, thank you. I’m just not clear on why two snapshots are required. Can the snapshot be told simply to render once? If so, I would have thought that only one is required. If snapshots are redrawing surely that means that both are redrawing? (Yes, there is obviously something I have not grasped here.) However, your code works really well, so I’ve updated my snippet to include advice from Danny, too…

https://dl.dropboxusercontent.com/u/10254959/SnapshotBlurTest/SnapshotBlurTest.2.zip

#12

The whole idea with snapshots is that the children of the snapshot are rendered to texture only once (or until invalidate() is called).

However, it’s only the children that are rendered to texture. Any effects on the snapshot are *not* included when rendering to the texture. 

That’s why a second snapshot is needed. The second snapshot will have the first snapshot as its child, and since children get rendered to texture (effects and all), the blur effect on the first snapshot will ultimately be rendered to the texture of the second snapshot only once.

#13

So it performs better with two because the outer snapshot is not applying any filters to it’s own content?

#14

Precisely.

I’m not certain if using the snapshot canvas in this case has any advantage over the group though. I’m just assuming that rendering from the canvas in “discard” mode will have a smaller memory footprint than if you’d use the group. I may be wrong about that though…

#15

In your example your maximum horizontal/vertical blur size is 300! That’s enough to kill any GPU.
Keep in mind that this means that the GPU will have to calculate 300x300 = 90.000 new pixel values for *every* pixel in your original image, and you’re changing the value every millisecond.

#16

What value do you suppose iOS uses for the control panel?

Even if you reduce the number to 60 and add 10 circles to the snapshot and it’s just as bad.

#17

OK I think I know what’s going on here.

The problem is that the effect isn’t rendered to the snapshot texture itself. It’s applied to the snapshot on every single frame causing an overload on the GPU. I guess you want to apply the effect to the snapshot texture once, right? In that case you’ll need to put the snapshot within another snapshot so that the first (inner) snapshot is only rendered once.

Something like this (I’ve added an extra square object to the inner snapshot):

pWidth, pHeight = display.contentWidth, display.contentHeight local circle = display.newCircle( 0, 0, pWidth\*.25 ) circle:setFillColor(1,0,0); local square = display.newRect(100, 100, 100, 100 ) square:setFillColor(0,1,0); local innerSnapshot = display.newSnapshot( pWidth, pHeight ) innerSnapshot.group:insert( circle ) innerSnapshot.group:insert( square ) local blur = 300 innerSnapshot.fill.effect = "filter.blurGaussian" innerSnapshot.fill.effect.horizontal.blurSize = blur innerSnapshot.fill.effect.vertical.blurSize = blur local outerSnapshot = display.newSnapshot(innerSnapshot.width, innerSnapshot.height ) outerSnapshot:translate( pWidth\*0.5, pHeight\*0.5 ) outerSnapshot.group:insert(innerSnapshot);
#18

In my case, yes, I want to apply the blur effect to a single object, then move that around. Specifically, I want translucent coloured blobs floating around the screen. I was hoping that there would be an easy way to create a circle from the display.newCircle function, apply a varying level of blur and scaling and then just use a transition. As it is, it seems that I need to use an image with a blurry edge, which does not afford me as much control as I would like.

#19

I’ve just written this, which is about what I’d like. What I’d really like is to make the spots draggable by the perimeter of the circles, rather than the snapshot, which is proving tricky. I’ve tried a number of things, including putting the snapshots inside a parent group then putting a touch layer object on top of the snapshot, in the same group. Doesn’t work.

-- snapshot blur test display.setStatusBar( display.HiddenStatusBar ) display.setDefault( "background", 1, 1, 1 ) math.randomseed( os.time() ) stage = display.getCurrentStage() pWidth, pHeight = display.contentWidth, display.contentHeight local group = display.newGroup() local xSpan, ySpan = pWidth\*.5, pHeight\*.5 local function newEndpoint( isStart ) local x = math.random( 0, pWidth ) - pWidth\*.5 local y = pHeight if (isStart) then y = -y end return x, y end function newDot( x, y, r, c, b ) local circleSnapshot = display.newSnapshot( 600, 600 ) circleSnapshot.circle = display.newCircle( 0, 0, r ) circleSnapshot.circle:setFillColor(c[1],c[2],c[3],c[4] or 1) circleSnapshot.group:insert( circleSnapshot.circle ) circleSnapshot.fill.effect = "filter.blurGaussian" circleSnapshot.fill.effect.horizontal.blurSize = b circleSnapshot.fill.effect.vertical.blurSize = b circleSnapshot.x, circleSnapshot.y = x, y return circleSnapshot end local radius = 150 local blur = 50 local alpha = .5 a = newDot( 200, 200, radius, {1,0,0,alpha}, blur ) b = newDot( 350, 200, radius, {0,1,0,alpha}, blur ) c = newDot( 275, 350, radius, {0,0,1,alpha}, blur ) function touch(e) if (e.phase == "began") then e.target.hasFocus = true stage:setFocus(e.target) e.target.x, e.target.y = e.x, e.y return true elseif (e.target.hasFocus) then e.target.x, e.target.y = e.x, e.y if (e.phase == "moved") then -- nothing else e.target.hasFocus = false stage:setFocus(nil) end return true end return false end a:addEventListener("touch",touch) b:addEventListener("touch",touch) c:addEventListener("touch",touch)
#20

I think my previous code was a failure because I was expecting the blur effect to be applied each time the objects which were being blurred was changed. It can, of course, be applied to each object separately.