Mode 7 / 3D at 60fps

Solar2D Advanced Graphics
#1

For the Graphics 2 competition a while ago I built a concept 3D engine which plotted 2D tiles, read from a tile map in 3D space. This was made possible by the new feature that allowed us to warp textures.

https://www.youtube.com/watch?v=X52EKFIeKhY

Unfortunately it was very CPU intensive and had the usual 3D issues of polygons / textures warping when points go behind the camera and also artefacts where the textures doesn’t line up exactly. On an iPhone 6 it would hold about 30fps, 60fps if the tiles in view were pre-calculated. On my iPod Touch 5 ( cpu one before A7 ) it would sit around 20fps with a lookup table.

Just a few days ago, after some very quick support from the Corona devs, we were given canvases. This would allow me to render a texture, say a segment of a racing track, as in the first demo, and then pass that data to an OpenGL shader to convert into 3D.

https://www.youtube.com/watch?v=qD6OLH8t0Aw

What you are seeing above are 2 seperate textures being rendered into 2 separate shaders. The ground shader is set to repeat and it is fed an animated texture, moving for effect. The background layer can also accept a tiled layer of 128 x 128 tiles.

The track layer is another tiled layer of 128 x 128 tiles, the background layer is shifted down for a 3D effect. I also render a shadow of the track onto the background layer.

The yellow dots however are not rendered by the shader. These are superimposed afterwards, with Corona using my original 3D engine to plot the corners of the tiles in the 3D space. Just as you would with sprites of the other cars, work out their position on the tile map, pass that to the 3D engine and it returns x , y  and scale for each sprite in screen space.

At the moment though the 3D code in the shader and the 3D code in Corona is quite different. I tried to write the fastest code for each with means its more voodoo code that just kinda works because.

Aligning the output from the shader to Corona space took a lot more voodoo and some weird values in the 3D engine, but it all now works and matches up. Voodoo!

Issues -

  1. I really could use some fast, 3D code, that I can use in both Corona and the Shader and give it the same values to achieve the same viewing angle. At the moment it took a lot of tweaking to get both to match up which means I can only do rotation around the Y axis. It’s enough and matches what I wanted to achieve, but better, easier to understand code would be much great. It would allow me to setup a screen for each device, rather than the 1024 screen shared across all.

The pink grid is iPhone 4 / 5 / 6 / 6+ and the simulator is running at 1024 x 1024 at the moment. Each device crops to the right borders, meaning everything is the same size on each device.

  1. To achieve the same view across different devices I have to render to a 1024 x 1024 texture. If I make the texture the size of the screen then the 3D goes wonky at the aspect ration changes and I can’t seem to account for that.

This needs work as a lower end device still ends up rendering the 1024 texture ( scaled x 1/4 for pixellated effect ), but its still a waste of GPU time. Even if I am not actually drawing those pixels on the devices ( OpenGL shader discard for out of bounds ) it still takes time, though not as much at the texture lookups.

  1. After that I’m going to make each tile animate, such as the lights flashing and the 2D track collision data will be generated from the tile map.

This is how it looks at the correct resolution on device.

mode7.png

#2

Wow, very nice.  :slight_smile:

Assuming this is an ongoing project and not simply a demo, will you have jumps, too? (With the camera zooms that implies, and any complications that would arise.)

#3

I’ve been pushing for this since 2010! https://forums.coronalabs.com/topic/1273-simple-3d-transforms/

At different times I’ve written a 2D physics based racing game, a 3D engine and now 3D shaders and given it gets great performance on older devices ( and not optimised at all ) I’m thinking its time to give it a go.

#4

If anyone knows of an OpenGL shader routine where I can warp a texture like the above, and it uses normal maths I can use in Corona then let me know. I will try it in the engine.

#5

Awesome stuff Matt! Very impressed :slight_smile:

#6

Hmm, it’s hard to give a shader for “voodoo”.  :smiley: Anything more specific? How are you with vectors / matrices and such?

Since you bring up rotation, this might be worth keeping in mind. Not sure where that falls on the “normal maths” spectrum. I made a 3D scene in its sample. It’s all wireframe, but maybe I could use it to explain some stuff. (I’m considering exploring depth buffers now that we have this new feature, but I don’t know how far you could push that with continuous full-screen updates.)

EDIT : Skimming the old thread. I have actually been enjoying (and failing at!) Axelay these last few days.  :slight_smile:

#7

Axelay is one of the effects I was going for! Again from 2010 https://forums.coronalabs.com/topic/1349-simple-3d-texture-background/

I have yet to find any workable samples of shader code for transforming a square texture into the above effect. If I can get that and transpose the code into Corona then I can probably build a module that would share common values such as pitch, yaw, zoom, fov.

At the moment the shader is hard coded with me just throwing random values at it until it gave me what I wanted, mixing a bunch of different examples online plus a few other hacks to get everything to line up.

It really is a mess at the moment, code wise bits everywhere and lots of inefficiencies.

Looking at shader lab they seem to be able to render 3d worlds, planes etc… independent of the screen size / aspect. The trouble there is that a lot of the code is too complex and would kill device performance.

This is my first try with shaders. Im doing it all in the fragment shader. At a guess I think I should construct and transform a plane in the vertex shader and then map to that.

#8

I’ve been too busy dodging enemies and bullets to pick Axelay’s background apart much.  :D If “the above effect” refers to that, rather than the F-Zero-ish star of the thread, I might have an idea. Assuming the checkerboard image approximates the behavior well, it can probably be modeled as a sector of a fat cylinder (for some code, I did a page curl using this same idea). I could point you to one of my in-the-works (ugly) docs on this, if it would be of interest.

I’m not sure if by “shader lab” you meant Shadertoy (where a lot of effects divide by iResolution.xy up front) or Unity’s system (where it’s in some uniforms) or something else. What you might be after is CoronaTexelSize (see here). This will let you step from one pixel to the next, respecting the screen / content dimensions. But I haven’t seen your code, so just a guess.

#9

Awesome! Although can i be the first to say – Why would someone want to make a 3d game in a 2d engine? Wouldn’t it be much easier to just go into unity and put something like this in a matter of hours if your a beginner? Not hating haha it’s awesome! But why?

–SonicX278

#10

@SratCrunch

That effect was the best I could do at the time, there wasn’t any image manipulation tools in corona so I saw just slicing an image into sprites and scaling. I could do that much better now.

The mode 7 effect is what I’m after. It could be used in an Axelay style game or a Mario Kart stye game.  

#11

@SonicX278 cheers. Yeah Unity would seem the idea choice and I have explored doing it in that. Unity likes to complicate things though and once my camera is sorted the rest would be much easier in Corona ( to achieve the 2.5D effect I’m looking for ).

#12

Awesome, Mario Kart/FZero here we come :slight_smile:

#13

Axelay = Yes

#14

I puttered around for a few minutes with the idea mentioned in my Axelay comments above and came up with this:

local old = display.getDefault("textureWrapY") display.setDefault("textureWrapY", "repeat") local image = display.newImageRect("Image1.jpg", display.contentWidth, .4 \* display.contentHeight) -- your image here (ideally one that tiles vertically) -- can shrink these dimensions to see what's going on image.x, image.y = display.contentCenterX, display.contentCenterY + .3 \* display.contentHeight display.setDefault("textureWrapY", old) -- -- do local kernel = { category = "filter", group = "custom", name = "roll" } kernel.vertexData = { { index = 0, name = "to\_angle", min = -90, max = 80, default = 40 -- cylinder angle (starting from 90 at top) of bottom of rect }, { index = 1, name = "stretch", min = 0, max = display.contentWidth, default = 50 -- scale factor used to widen bottom of rect }, { index = 2, name = "t", min = 0, default = 0 -- rotation (0-1, periodic) } } kernel.vertex = [[P\_POSITION vec2 VertexKernel (P\_POSITION vec2 pos) { // texture coords for a basic rect go: // (0, 0) - (1, 0) // (0, 1) - (1, 1) pos.x += 2. \* (CoronaTexCoord.x - .5) \* (CoronaTexCoord.y \* CoronaVertexUserData.y); // widen the bottom of the rect return pos; }]] kernel.fragment = [[P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { P\_UV float yf = cos(radians(90. - CoronaVertexUserData.x)); P\_UV float y = mix(1., yf, uv.y); P\_UV float angle = 1. - acos(y) / 3.14159 + CoronaVertexUserData.z; return CoronaColorScale(texture2D(CoronaSampler0, vec2(uv.x, angle))); }]] graphics.defineEffect(kernel) end image.fill.effect = "filter.custom.roll" local Period = 5000 Runtime:addEventListener("enterFrame", function(event) image.fill.effect.t = (event.time % Period) / Period end)

It needs some work (e.g. the underlying triangles are somewhat obvious in the way things warp around one of the diagonals, and maybe a radius for the cylinder would add some flexibility), but might be useful to get going.

#15

Gave it a little more love. The issues with the triangles simply go away when using rect paths:

local old = display.getDefault("textureWrapY") display.setDefault("textureWrapY", "repeat") local image = display.newImageRect("Image1.jpg", display.contentWidth, .4 \* display.contentHeight) -- your image here (ideally one that tiles vertically) -- can shrink these dimensions to see what's going on image.x, image.y = display.contentCenterX, display.contentCenterY + .3 \* display.contentHeight display.setDefault("textureWrapY", old) -- -- do local kernel = { category = "filter", group = "custom", name = "roll" } kernel.vertexData = { { index = 0, name = "to\_angle", min = -90, max = 80, default = 40 -- cylinder angle (starting from 90 at top) of bottom of rect }, { index = 1, name = "t", min = 0, default = 0 -- rotation (0-1, periodic) } } kernel.fragment = [[P\_COLOR vec4 FragmentKernel (P\_UV vec2 uv) { P\_UV float yf = cos(radians(90. - CoronaVertexUserData.x)); P\_UV float y = mix(1., yf, uv.y); P\_UV float angle = 1. - acos(y) / 3.14159 + CoronaVertexUserData.y; return CoronaColorScale(texture2D(CoronaSampler0, vec2(uv.x, angle))); }]] graphics.defineEffect(kernel) end image.path.x2 = -110 image.path.x3 = 110 image.fill.effect = "filter.custom.roll" local Period = 5000 Runtime:addEventListener("enterFrame", function(event) image.fill.effect.t = (event.time % Period) / Period end)

Not a lot occurs to me about how to improve the rolling effect, but any number of improvements could be made to the basic uv.x lookup. (I tried with repeating x texture mode and doing simple scrolls and subtle sine waves based on time and both looked pretty good.)

#16

Thats a lot like it. The idea I suppose would be to feed a tile map into a canvas and then warp the canvas using the shader. As you move through the level the tile map is moved in the container so it shows movement.

Thats how I’m doing the mode 7 stuff.

#17

Matthew - just out of curiosity, why are you using a shader for the 3D effect, as opposed to simply distorting the canvas?
Either way, you need a general set of functions to convert from 3D to 2D and back, which from what I read above is causing you some problems (I *think* you have hard-coded some values, hence the problems doing the above). If you care to share your methodology a bit more or some code, I’ll try to help you with this if I can, so you can have a general routine that works for any view values.

#18

If you remember the old way, I was doing real 3D calculations and then warping tiles to form the 3D world. As in the top video.

That was quite slow on older devices, the textures warped when a point went behind the camera ( requiring subdivision ) and we had artefacts where the different polygons nearly, but didn’t always line up 100% ( requiring each point to be offset slightly ).

In the end, all those extra calculations slowed things down on old hardware and that was without a good way of clipping. I was using pre-calculated lookup tables, which needed to be calculated for each screen size, twice, once for floor and once for the track.

The shader way removes all that. I just pass it a canvas of the track and it warps it to the same perspective, without all the issues from above. Movement means updating the canvas and moving the track, the shader only calculates rotation.

I can get my old 3D code to plot points that now match what the shader does meaning I can mix 2D sprites with corona x & y locations with the track shader output.

Being able to repeat the ground texture to infinity also means less processing of the ground tiles as I used to need to draw a lot more to fill the screen.

If you send me your email I will send you the project ( its messy at the moment )

#19

Ah, when I meant distorting the canvas, I wasn’t referring to the individual tiles. What I did in my mode 7 test was to draw a tile-map translated and rotated into a snapshot to match my view values, then distort just *that* snapshot. So it wasn’t in fact any 3D calculations (other than the original distorted shape of the snapshot) just to draw the ground. You essentially treated the game like a top down view, and the snapshot took care of the rest.

There were some problems with this approach - for anything approaching a reasonably flat projection (as in, you aren’t that high above the ground surface) you ended up having to distort the bottom two snapshot corners so much that it produced (on apple devices at least) some serious artifacts (I think apple used low precision calculations so things went wrong more often there).
Another problem is that because the snapshot would end up with a lot off screen left and right it meant you were drawing and updating a significant amount of tiles that were never seen by the player. There isn’t really a way around this sadly :frowning:
I did come up with a way of dealing with the first issue, but snapshots weren’t up to the task (basically split the single snapshot into several horizontal snapshots all using the same source image - the problem being, you couldn’t have several snapshots referencing the same source snapshot. Although, now with canvases… hrrrrm :smiley: ).

The drawing to infinity does sound cool though, and is not something you can really do with the snapshot approach, so I’d love to see this, and we can chalk this up as a win for the shader :slight_smile:

I’ll PM you here my email.

#20

Also, to explain my old way of doing things, there’s this: https://github.com/Rakoonic/Mode-7

I really really doubt it’ll work out of the box though, it is old :slight_smile: