Text-rendering plugin, including meshes, triangulation, etc. (WIP)

Corona Marketplace Corona Community Plugins
#1

Hi.

I’ve mentioned a text-rendering plugin at least a couple of times, in here and here. This is something I put together a while ago, even submitted, yet have since sat on (it’s available on the Marketplace but still hidden), mostly owing to docs and wanting better examples. Actually, docs will still probably be a big bottleneck with everything that follows.  :stuck_out_tongue:

It’s a pretty low-level library, rendering text as blobs of bytes, basically a thin wrapper over the TrueType component of stb, a collection of useful libraries.

I’ve finally started getting back to it. The library also exposes the intermediate vertices used to render glyphs, which I’ve been getting in order. The geometry is exposed as contours, so I’ve also now bound libtess2 (the same author wrote the SVG library Corona uses). That module might have been a good answer to sporkfin’s question in this topic, but I wasn’t aware of it. :) Since it uses contours as well, I’ve also been binding the Clipper library I mentioned in that same topic, but this still needs testing.

I’ve included a project, with binaries and an example. There are some rendering bits (a scaled string and unscaled “a”, plus a printout in the console) and then the mesh bits: one by one, I etch out a series of characters, both by tracing dots around their outlines, and as geometry. The code is quite messy as I’m still just putzing around in it. The strings used are in there and can obviously be edited. The geometry mode can be toggled between “BOUNDARY_CONTOURS” and “POLYGONS” to tessellate in the appropriate way (in theory the connected polygons mode should work too, but I haven’t tried porting it). This is still very much a WIP with other options left to explore.

(Unfortunately, since the plugin needs to read the font, it’s subject to pathForFile’s gotchas on Android, thus the text files in the example.)

If you’re on Windows and any of this sounds remotely interesting, you can take the DLL files from the binaries

directory and drop them into a plugin folder that you create at this location:

print(system.pathForFile("", system.PluginsDirectory))

I think Bytemap is still the same as the Marketplace version, but GlyphRenderer currently isn’t, so comment them out from build.settings accordingly. I don’t have Mac builds yet.

Text formatting additions
#2

It occurred to me later that I probably should have included a video, so here goes:
 
[media]https://www.youtube.com/watch?v=tr9nl0VGL84[/media]
 
Some commentary:
 
The “Heljo World” strings comes from stb_truetype’s example. It was meant to demonstrate some quirks between the letters “l” and “j” that come from rendering a whole string, I believe with Arial fonts. In this case, with the Mayan font, things seem to manifest around the “r” and “d” instead. The authors use this to point out that the library is really meant for rendering characters one by one.
 
I scaled the string to 5x just to see what the alpha looked like. The “a” in the middle is normal size. In addition to just some basic rendering, I’m thinking of an example or two using the output as masks.
 
The example is also going through the characters in the string “Mil7kst 88 or 3?!” (nonsense, but just contains various shapes I wanted to test, e.g. the loops in the "8"s and the separated dots in the “i”, “?”, and “!”). In the upper-left you see circles being traced gradually around in the contour in the order I pull them out of the shape. Marching slowly across the screen, on the other hand, you see a polygonal tessellation of those same contours. This is “POLYGONS” mode, but as I mentioned there’s also a “BOUNDARY_CONTOURS” that will just give you the outline shapes. (I haven’t played with “CONNECTED_POLYGONS” but I suppose it would let you gather, say, the polygons from the main shape in the “i”, “?”, or “!”, then later all those from the dot.)

#3

Nice, I’m linking to DropBox now, thanks!

#4

Dude.  Nice post, but what happened to that video?  The low-res makes it hard to truly appreciate. :frowning:

#5

@roaminggamer  With any luck the new result should be better, in spite of a number of setbacks along the way!

@sporkfin and others: I did do builds for Mac and other platforms, though so far this has been just to get projects up and running, without much if any testing (for several plugins, not just those shown). I’ve added the binaries (except Bytemap, but as mentioned above the Marketplace version should do), but probably won’t get much additional work done over the weekend.

#6

The Mac version does seem to work (although it flushed out a minor problem in Bytemap which I thought was Windows-only, when textures aren’t some multiple-of-4 bytes wide), so I’ll try to eventually test the mobile versions as well and update the binaries.

I’m now trying to come up with a sample that also tests for connected polygons and winding modes. My thinking is that I’ll copy out the contours of a few glyphs that were shown in the video (e.g. “8”, “!”, “o”, “7”) that were representative of the important geometric cases, then maybe do some other stuff with non-glyph shapes, probably drawing from Songho and the OpenGL book. Assuming all goes well, I think  libtess2  is ready to submit. Whenever that goes live I’ll try to work the example in the video back into the  truetype  sample (I’m thinking I can enlarge on this over time with interesting text examples).

Here are some reasonably complete docs for libtess2.

I’ve also been scraping up comments from stb_truetype lately and updating the truetype docs as well, although most of this hasn’t landed yet.

On that note, I’ve been trying to ride the momentum of building and documenting to move a few other plugins forward. With any luck I’ll get them out the door soon too:

clipper, mentioned above. Not specifically relevant to text, but it uses contours, so  libtess2 gives it some oomph. The documentation for clipper  proper is quite good, but for Delphi / C++ / C#, so mostly this is just an editing job.

msquares, for turning byte data into triangles. Again not immediately relevant to text, though you should be able to use the renders from some of the truetype  functions as input. Once again, the author has quite good docs in the form of a blog post, so I’m hoping this won’t be too painful.

object3d, a 3D-scene-to-texture utility that I mentioned here and in a few follow-up posts as well as rather vaguely in some older posts. msquares actually can emit OBJ files (as well as SVG), so it ties to that.  :) Docs are about halfway along now.

Lastly, luaffifb, a desktop-only means of working with structured binary data and interface with dynamic libraries. This doesn’t really tie in to anything above, but I’ve been sitting on it forever and rode this wave of sudden activity to tidy it up and get everything almost in order (there’s one case in the tests that still fails on Mac, but it might be obscure enough to overlook and fix later). I need to add a few notes but otherwise the docs are mostly complete.

EDIT : The erstwhile “GlyphRenderer” is now truetype , since the name is less unwieldy / more memorable and my earlier guess that this would encompass more than stb_truetype has been quite wrong. Docs are redirected here, with marketplace and sample to follow; I’m leaving the details in earlier posts as is.

#7

I can’t wait to spend some time in this new playground you are constructing!  Who know what new inventions it will lead to?  Count me in for buying the plugins!

#8

As an update on this, I’ve uploaded newer binaries for truetype (see edit in previous post about name change) and basically fleshed out the docs. There seems to be a hiccup on iOS, but it should go live once that’s settled.

I’ve also got a sample I’m happy with for libtess2 :

[media]https://www.youtube.com/watch?v=ZR5oBmIFZ_4[/media]

That will probably make more sense if you read some of the docs and comments above.  :slight_smile:

If all goes well building a new round of binaries tomorrow I’ll go ahead and submit.

Next up is probably Clipper , though we’ll see.

#9

The truetype and libtess2 plugins are now live.

The sample for the former is pretty bare, as is its marketplace page, obviously. I’ll see if I can’t at least tidy up and incorporate the stuff shown in the first video above.

Some future ideas for the truetype sample:

  • Text as a mask

  • Text with strokes, via the contours

  • Somewhat along those lines, I’ve considered pulling the signed distance field functions here out into their own small plugin

  • Text meshes that travel around on curved paths and subtly deform to match

  • A friend of mine was always impressed by the text in the N64 game Paper Mario, which would have differently formatted, animated words in the middle of the some text, e.g. for emphasis when a character was shouting (I’m not sure this does it justice, but video searches have failed me), so something along those lines, as well as emoticons et al.

  • Building a font bitmap

  • Probably other things I’ve considered that are slipping my mind

I’m trying to learn what the “text rendering plugin” ETA’d for Q3 2018 is. I suspect something different and truetype won’t be stepping on toes, but we’ll see. Ideally, of course, it will allow for enhancements.  :slight_smile:

As for the others, looks like either msquares or luaffifb is next out of the gate.

#10

Wow, you have been busy.  I got both plugins.  Do you have demos for them?  Thanks!

#11

Sorry, sample links were in the docs. I’ve just added them to the descriptions on the product pages as well.

The truetype sample is a little lame at the moment, as it’s basically just a port of some of stb_truetype’s unit tests, but I should be able to bring it at least to parity with the first video by cleaning up the material in the zipped project; the sample there ought to work with these plugins aside from changing require(“plugin.GlyphRenderer”) to require(“plugin.truetype”). A couple items in my last post’s list look like low-hanging fruit, too.