https://supertokens.com/ logo
c

CaptainPhoton

06/27/2022, 10:13 AM
I am migrating users, and I have a problem with some characters in the metadata. Characters like "ä" are "ä" in the database. I am using the supertokens Metadata recipe to insert the metadata. What charsets and collations does Supertokens expect? Is there anything I can do?
the database is using the utf8mb4_unicode_ci collation and utf8mb4 charset
r

rp

06/27/2022, 10:15 AM
hey @porcellus can help with this
c

CaptainPhoton

06/27/2022, 10:15 AM
hopefully!
p

porcellus

06/27/2022, 10:19 AM
Hi. I'd expect it uses utf8, but I'm not a 100% sure, I'll check. ä becoming 2 chars feels like a normalization going wrong.
c

CaptainPhoton

06/27/2022, 10:20 AM
okay! Is it fixable?
p

porcellus

06/27/2022, 10:28 AM
It should be 🙂 but I need to check what/where it goes wrong
Btw, are you getting the wrong info back or is it just an issue of what you see in the db?
I mean: is the metadata recipe returning the malformed string?
c

CaptainPhoton

06/27/2022, 10:31 AM
this is what I see in the DB, and also what it returns
p

porcellus

06/27/2022, 10:31 AM
I see, thanks.
what kind of stack (backend/db) are you using?
c

CaptainPhoton

06/27/2022, 10:33 AM
mariadb and node/express (with Sails)
but the metadata is going through the Supertokens SDK
p

porcellus

06/27/2022, 10:34 AM
Yeah, what I want to check is that the node SDK does this encoding correctly 🙂
c

CaptainPhoton

06/27/2022, 10:34 AM
ah yes!
p

porcellus

06/27/2022, 10:39 AM
I'm looking at the same thing. we are using axios to post/get data
c

CaptainPhoton

06/27/2022, 10:40 AM
yeah I saw that in the code 😄
encoding issues are nasty :=
p

porcellus

06/27/2022, 10:43 AM
They are. equality checks through encoding/normalizations are even more nasty. I'm happy we are not doing that 😄
yep, so axios is not sending a content-encoding header
so the java implementation doesn't try to parse it as utf8
I'll come up w/ and provide a fix tonight I think
(most likely to the node SDK, but I'll discuss it with @rp )
c

CaptainPhoton

06/27/2022, 12:27 PM
great, thanks! 🙏
r

rp

06/28/2022, 8:04 AM
hey @CaptainPhoton we have fixed this issue (hopefully) - try out version 10.0.1 of the node SDK
c

CaptainPhoton

06/28/2022, 8:04 AM
Great, will try right now! thanks
@rp @porcellus IT WORKS, amazing! 😄 thanks for the fast fix
r

rp

06/28/2022, 8:12 AM
all thanks to @porcellus
c

CaptainPhoton

06/28/2022, 8:12 AM
👏