Color-Emoji support without native.* really not doable?

Solar2D General Questions/Discussion
#1

Dear Corona community,

what I’d like to do:

Allow the user to type in not just text, but also colored emojis. Native.newTextBoxes can do that. Then store the input in a mysql database and show it on other devices in a scrollView or container, so basically an absolute requirement in todays social media apps. The strings can be accross multiple lines.

what the problems are:

  • Display.newText objects dont support colored emojis (only black and white).

  • webviews reside on top of other stuff, so I cannot use them in a scrollview for example

  • same with native.newtextboxes, they also sit on top of other stuff

  • Splitting strings at some sort of markers within and inject images in between doesnt really work either because I cannot extract the .x - position of those markers

so my question is:

Is there any way to support colored emojis in Corona within a user-input string? Like, ANY way? I’d even consider hiring someone to write a plugin if there is anyone out there who says, yeah, I can do that :slight_smile:

/edit: This is how it looks like when you have a native.newTextField on top and copy its input into a display.newText() below:

Here is zipped project with a very simple demonstration:

Would appreciate any ideas or hints.

Thanks,

Chris

#2

No & Yes

No, not through Corona’s built-in display.* features directly.

Yes, write your own layer over display.newText() (or follow my suggestion below)

I’m not saying it will be easy, but you can do it.

You’ll need emoji art to support this.

Suggestion

  • Your best bet is to download a BitMapped font library you can use and understand. 

  • Generate a set of fonts to use with it.

  • Modify the module/library to support emojis.

  • Supply a standard set of emojis.

Note: I believe, you’ll you’ll need to support unicode.  This could be a bit of work.

PS - Thanks for the downloadable example.  You did however forget to include the font in you example, and I don’t see how this demonstrates the issue.

#3

Question, how do you expect the text to be provided? 

i.e. Is this entirely internal with an arbitrary encoding for the emojis or are you trying to make something that can take text input from the users’s device that is then displayed as a text object with the same emoji’s used on their device?

I mean do you expect this to work:

  1. User types in some text and emojis into native.newTextField().

  2. You extract the ‘string’ from the text field.

  3. You display it using Corona’s display.new*() features.  

#4

Download this to see an hacky solution and tool I made to demonstrate the singular fact that ‘it can (almost) always be done with Corona’:
 
https://github.com/roaminggamer/RG_FreeStuff/raw/master/AskEd/2018/10/emojis.zip
 
(emoji_mod subfolder)
 
Run using a simulated android device for mouse hover to work.
 
 
>>> If you like my effort click here:  Thanks RoamingGamer!
 
Image below shows demo running.
 
Image description top-to-bottom:

  • line 1 - What you get when printing strings containing emoji using display.newText()
  • line 2 - The output of my hacky extension display.newEmojiText()
  • remainder - A tool for getting sheet index for emojis from a sheet of emojis I found online.

emojis.png
 
 
 
PS - I know  those are probably NOT the right emojis.  I randomly selected two for the mapping.

#5

Ed, it’s just always a pleasure reading your replies to either my or someone else’s questions. It’s not that often that people put that much effort into a reply and even upload a demo of a - hacky or not - solution. I’ve said it many times: Thank you :slight_smile:

I’ve downloaded your mod and I’m going to try to understand every single line and function tomorrow - just too tired right now, too many hours of coding :slight_smile:

After having a quick look it seems that your newEmojiText function sort of splits an input text into multiple parts: “Normal” characters are treated with display.newText - objects, emojis instead are added via newImageRect right next to the last string by setting curX + tmp.contentWidth.

I had something similar in mind, but stumbled when the userinput is longer (wider) than display.contentWidth (or any width value specified) since then tmp.contentWidth needs to be fixed in advance. So, I guess the demo project is a very good start, the next challenge though would be to modify it for multiline input. Hope it gets clear what I am trying to explain.

And, to answer your questions:

How does my demo project demonstrate the issue?

When you run it on a device and type in an emoji via the normal keyboard, you get the result my screenshot shows. The missing font is the result of my copy and paste process to make it a mini-version, but that shouldnt be a problem since devices just use a standard font instead.

how do you expect the text to be provided?

  • User types in some text and emojis into native.newText Box ().

  • This input is stored in a module and is retreived in another scene where it is - or should be - put into a scrollview

  • If user is happy with his input, the input is send to my Google App Engine where it’s stored in a mysql database (utf8)

  • other devices can request this input and show it again in a scrollview, so yes, display it using Corona’s display.new*() features

  • The input can be multiline

  • it would be perfect if the user could use all the emojis his keyboard offers

I’ll have a closer look tomorrow, but if this sounds like a ‘hire a hitman’ project for you and you feel like you could solve this even for newTextBox - inputs, please let me know :slight_smile:

#6

Argh, this whole thing wont let me sleep. Just made your hacky solution even dirtier :slight_smile:

I modified your newEmojiText a little bit and

  • added currentWidth = 0

  • added currentY = 0

Now, with every Character or emoji, I add their contentWidth to currentWidth. As soon as currentWidth is equal or more than my width-limit, I set back currentWidth to 0 and currentY to group.contentHeight. This way, the next character is added into the next line.

Guess you’re right, this whole thing might be doable with pure Corona, even though there is - of course - a lot of improvements and work to do.

For the moment I am happy though that for a longer uniString in your examle I did get a multiline result with smiling emojis. Good night :slight_smile:

#7

After the alignment issues, the sheer number of emojis to map is daunting.  I do suspect however that someone out there must have made a collection of emojis or a sheet that is ordered to match the standard emojis list (assuming there is such a thing.)

Whatever the case, hacky solution is far from perfect, but I do think it will make a first version that can be refined over and over to suit your needs.

Good luck with this.

#8

Played around with it this morning and I am facing tthree main issues here:

  1. A user types in an emoji that his keyboard offers. Later, this emoji is replaced by the corresponding emoji from an image sheet. They do look similar, but I do see young people claiming: “Wait a minute, that’s not exactly what I typed”

  2. y-Alignment within one line: display.newText(…) and display.newImageRect are perfect on the same y-level in simulator, but accross testdevices they’re not. I know this has something to do with rendering of the newText, and if I remember correctly you can never tell the exact y-position of a text-element.

  3. Each input string right now is split into a display-object for each and every character (char or emoji). So, for an input string like “Hello :slight_smile: how are we doing today ;)” we get like 30 display-objects. Now, I have a scrollview that shows many of those input strings, so I’d get 300 objects (lets keep it simple) for ten strings. At least to me this seems like this is not good at all memory-wise.

Any ideas how this could be approached?

#9
  1. once you have built your row or the whole paragraph just take a snapshot of it and use the snapshot in the scrollview
#10

…and then remove all the other objects that have been created before I guess!

Good point Arteficio, that would indeed be an option to solve at least problem nr. 3 :slight_smile:

Thank you for your input :slight_smile:

#11

Hi again.  That was about all I had to offer.  

re: #2 - Yes.  y-alignment will be an issue. This is why I suggest using and then modifying an existing Bitmapped text library (PonyFont or other) instead of plain text.  Bitmapped fonts should all be pre-aligned.

#3 I did letter by letter, but you can actually accumulate all letters and spaces into one text object till you hit an emoji.

My plain text hack is just a starting-point/concept for a real solution.

#12

I skyped with Hassaan today and we discussed the whole thing in detail. He agress that your suggestion of using a bitmap font might be my best option here. I already downloaded PonyFont, found a .ttf file with all emojis in there (EmojiOne 4.0) and am currently trying to make bitmap font out of their ttf using Glyph Designer. Right now I’ve now idea how to get this work, but I’ll get there :slight_smile:

#13

I am not suggesting you use PonyFont unmodified.

I do not think you will be able to get colored Emojis and bitmapped fonts.

You will need to modify PonyFont to identify emojies and instead of using a letter texture, use a emoji image from the sheet like I did.

i.e. You probably cannot have a single sheet of letters and emojis.  No tool for creating Font Sheets that I know of works that way.

#14

Final note regarding y-align.

Did you try to use baselineOffset?

https://docs.coronalabs.com/api/type/TextObject/baselineOffset.html

With that info you may be able to get by with my hacky solution and the addition of taking a snapshot to reduce the number of total objects.

#15

Ok then, seems like I misunderstood something. Wow, this whole thing is like the biggest challenge for me so far in Corona. Guess I’ll take a deep look into PonyFont and try to modify it the way you suggested.

And - concerning your second question: Yes, I came accross baselineOffset today, but then switched to the bitmap font solution. I now have those two options, we’ll see if I can make one work :slight_smile:

#16

I’d experiment with baselineOffset first.
 
That is a few minutes of work.
 
Modifying PonyFont will be much more time.
 
 
Adding baselineOffset should literally be changing this:
 

tmp = display.newText( text, string.char(codepoint), curX, 0, font, fontSize ) tmp:translate( 0, tmp.baselineOffset )

to this:
 

tmp = display.newText( text, string.char(codepoint), curX, 0, font, fontSize ) tmp:translate( 0, tmp.baselineOffset )

PASTED WRONG CODE ORIGINALLY

#17

Ugh, now I am curious and have to find out… I’ll check back in a bit.

#18

OK, I’m not 100% sure, but this might be it:

tmp = display.newText( text, string.char(codepoint), curX, 0, font, fontSize ) tmp:translate( 0, tmp.baselineOffset \* tmp.anchorY )

I have not tested this on device.

Here is an update to my demo (I used a TTF to test):

https://github.com/roaminggamer/RG_FreeStuff/raw/master/AskEd/2018/10/emojis.zip (same link; download again)

I also tweaked the function to allow you to provide an ‘alternate’ emoji’ size to make the fit better.

#19

I tried it with baselineOffset * 0.5 this morning, not really understanding why the *0.5 needed to be included, but your  * tmp.anchorY now makes sense. Yes, it does work on my Huawei P20. The emojis are a little bit bigger than a single char, but their y-alignment looks good and I guess the size for emojis should be easily customizable (fontSize * 0.9 or something).

One step closer :slight_smile:

So, right now I am thinking of providing two imageSheets: One with iOS emojis, one for Android (ignoring copyright for the moment, I’ll check that out later). Same mapping table though. This way, iPhone user would see their apple emojis, android users would receive the same emoji in their familiar Android way.

So, step by step :

  1. User types in using the standard iOS or Android keyboard into a native.newTextBox()

  2. User sees his “normal” emojis since its all native. String is stored in a module

  3. In other scenes I fetch the string and shoot it into your newEmojiText function.

  4. Within this function, the string is split into single elements (chars and emojis)

  5. for chars, use display.newText, for emojis use a spriteSheet (iOS or Android) and newImageRects.

  6. add those chars and symbols next to each other, or if end of line (specified limit) is reached, change curY and reset curX to 0.

  7. This way, all emojis are presented the way the user is familiar with and all should look fine.

Sounds not too bad to me. I’ll send another beer via paypal if that works out  :slight_smile:

#20

Good idea on the per-platform emoji sheets.  Remember to make one of them the default in case of a failure to identify the OS.

I keep thinking there should be some clever way to get the emojis by downloading them or something, but making a sheet or finding one is probably the safest bet.

Of course, the remapping is going to be a hassle if nobody has a ordered sheet already.

Tip: Don’t forget to provide a nice way for handling ‘unknown’/‘unmapped’ emojis IDs.  Right now I just make a unfilled white rectangle, but you might want to fill with a transparent texture or default texture instead.