`TLS certificate verification failed: hostname mismatch` on IDN

[mrusme]

After digging through how Ejabberd and, to that extent, XMPP is handling IDNs, I have switched from using a punycode domain (xn--gckvb8fzb.com) to the unicode representation (マリウス.com). Upon connecting to the new account with Profanity, however, I'm now getting the following error:

15:46:34 - Connecting with account marius as marius@マリウス.com/profanity.Aojj
15:46:36 -
15:46:36 - TLS certificate verification failed: hostname mismatch
15:46:36 - Certificate:
15:46:36 -   Subject:
15:46:36 -     Common name        : xn--gckvb8fzb.com
15:46:36 -   Issuer:
15:46:36 -     Common name        : E5
15:46:36 -     Organisation       : Let's Encrypt
15:46:36 -     Country            : US
15:46:36 -   Version             : 3
15:46:36 -   Serial number       : 06EEFFFBA22B4909933291398261B3EE8017
15:46:36 -   Key algorithm       : id-ecPublicKey
15:46:36 -   Signature algorithm : ecdsa-with-SHA384
15:46:36 -   Start               : Jul 10 19:04:37 2025 GMT
15:46:36 -   End                 : Oct  8 19:04:36 2025 GMT
15:46:36 -   Fingerprint         : 31266909a7d1748f09e6f0b31c2cb0705122cb8f

While I could simply /tls always, I'm still wondering why this messages shows up.

[sjaeckel]

hmm, seems like the cert only contains the punycode name and is missing the unicode one.

Having the unicode name in the CN would be possible, but not in the SAN.

Can you please try adding the unicode name to the CN and only having the punycode version (with all the other subdomains) in the SAN, to check if that helps?

[mrusme]

@sjaeckel I tried %s/xn--gckvb8fzb.com/マリウス.com/gI in lego's domains.txt and regenerated the certs, but it doesn't seem like Let's Encrypt is issuing unicode entries anywhere. I have the new cert running (Start field is up to date) but I experience the same issue.

Shouldn't profanity treat the certificate as a valid cert in first place? I don't see this issue occurring in e.g. Conversations.

[sjaeckel]

Yeah, I see that the cert is renewed, and after thinking once more over it (and skimming various RFCs) I'm not sure whether my proposal made even sense 🙈

IIUC it's not profanity that is complaining, but OpenSSL resp. GnuTLS, depending on which TLS library you're using, which doesn't like the mismatch and fires the verify callback, [0] resp. [1].

FMU the TLS library should do the de-/encode from/to punycode and compare that!?

Which TLS library do you use?

[0] https://docs.openssl.org/3.2/man3/SSL_CTX_set_verify/
[1] https://www.gnutls.org/manual/gnutls.html#gnutls_005fcertificate_005fset_005fverify_005ffunction

[mrusme]

$ ldd /usr/bin/profanity
...
        libssl.so.3 => /usr/lib64/libssl.so.3 (0x00007f89cb62c000)
...
$ equery belongs /usr/lib64/libssl.so.3
 * Searching for /usr/lib64/libssl.so.3 ...
dev-libs/openssl-3.4.1 (/usr/lib64/libssl.so.3)

$ equery m dev-libs/openssl
 * dev-libs/openssl [gentoo]
Maintainer:  base-system@gentoo.org (Gentoo Base System)
Upstream:    Remote-ID:   cpe:/a:openssl:openssl (cpe)
             Remote-ID:   https://github.com/openssl/openssl (github)
Homepage:    https://openssl-library.org/
Location:    /var/db/repos/gentoo/dev-libs/openssl
...

FMU the TLS library should do the de-/encode from/to punycode and compare that!?

Thought the same, however, it seems like this is not how it works:

openssl s_client -connect c3.xn--gckvb8fzb.com:5443 -servername "マリウス.com" -verify_hostname "マリウス.com"
Connecting to 70.34.209.135
CONNECTED(00000003)
depth=0 CN=xn--gckvb8fzb.com
verify error:num=62:hostname mismatch
verify return:1
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=E5
verify return:1
depth=0 CN=xn--gckvb8fzb.com
verify return:1
...

The moment one changes this to the punycode:

openssl s_client -connect c3.xn--gckvb8fzb.com:5443 -verify_hostname "xn--gckvb8fzb.com"
Connecting to 70.34.209.135
CONNECTED(00000003)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=E5
verify return:1
depth=0 CN=xn--gckvb8fzb.com
verify return:1
---
...

I suspect that the client has to do the translation?

[sjaeckel]

FMU the TLS library should do the de-/encode from/to punycode and compare that!?

Thought the same, however, it seems like this is not how it works:

openssl s_client -connect c3.xn--gckvb8fzb.com:5443 -servername "マリウス.com" -verify_hostname "マリウス.com"
Connecting to 70.34.209.135
CONNECTED(00000003)
depth=0 CN=xn--gckvb8fzb.com
verify error:num=62:hostname mismatch
[...]

You're seeing the same error!

FMU this is an OpenSSL bug (and I couldn't find a corresponding open issue on their GH).

GnuTLS handles this properly.

$ gnutls-cli  --verify-hostname="マリウス.com" 70.34.209.135:5223     
[...]
- Status: The certificate is trusted. 
[...]
- Handshake was completed
[...]

Here a negative test

$ gnutls-cli  --verify-hostname="foobar.com" 70.34.209.135:5223    
[...]
- Status: The certificate is NOT trusted. The name in the certificate does not match the expected. 
*** PKI verification of server certificate failed...
*** Fatal error: Error in the certificate.

CC @levitte (Sorry for highlighting you here, but you're the only OpenSSL guy I had contact with.)