Compare UTF8 with non UTF8 strings

Solar2D General Questions/Discussion
#1

Following problem:

I think it relates to UTF8.

I read some data from a public API with network.request().

The data gets returned as JSON data.

Now I want to compare a text string from within the data with a hardcoded string.

I.e. the german word “Außen” which has a special character in it.

So in the JSON data (which I saved to DocumentsDirectory and read it again), I can see the word “Außen”.

Let’s say the JSON decoded table path to the loaded word is like this:

data.devices.module.name

So when I do this

print( data.devices.module.name )

it properly prints the word “Außen” to console.

But then when I try to compare it like this

if data.devices.module.name == "Außen"

it gives me a false result.

Any hints how to solve this?

#2

lua’s string equality check is just a byte-by-byte check - so even if two chars look the same on screen, if they have different internal coding an equality test will fail.  maybe one or both aren’t normalized (or normalized to different forms)  perhaps loop over your strings with string.byte() to see if you could spot the problem.  then depending on which one seems “weird”, either look into the json parsing or the constant.  fwiw, hth

#3

thank you davebollinger I will try a byte by byte comparison. Maybe it helps…

#4

I would suggest to use : https://coronalabs.com/blog/2016/03/21/introducing-the-utf-8-string-plugin/

#5

thanks nmichaud, I did already try a utf8.match() but it did not work either.

#6

lua’s string equality check is just a byte-by-byte check - so even if two chars look the same on screen, if they have different internal coding an equality test will fail.  maybe one or both aren’t normalized (or normalized to different forms)  perhaps loop over your strings with string.byte() to see if you could spot the problem.  then depending on which one seems “weird”, either look into the json parsing or the constant.  fwiw, hth

#7

thank you davebollinger I will try a byte by byte comparison. Maybe it helps…

#8

I would suggest to use : https://coronalabs.com/blog/2016/03/21/introducing-the-utf-8-string-plugin/

#9

thanks nmichaud, I did already try a utf8.match() but it did not work either.