Technical note · Tokenkeep PQC · 13 Nov 2026

ML-DSA certificate chains: the 184 bytes that break TLS handshakes

Swap a three-certificate chain from ECDSA to ML-DSA-65 and its TLS 1.3 Certificate message grows from 1,157 bytes to 16,568. The default mbedTLS input buffer holds 16,384. That's 1.1% over, which is enough to fail every handshake that presents the chain. Nothing in your crypto inventory will flag it.

Where the bytes come from

ML-DSA-65 (FIPS 204) has a 1,952-byte public key and a 3,309-byte signature. In an X.509 certificate, each of those sits inside a DER BIT STRING: a tag byte, a length, and one "unused bits" byte before the payload. Once the content passes 255 bytes, the length takes three bytes (0x82 plus two):

The 244 bytes is a deliberately minimal profile, measured against certificates generated with OpenSSL 3.6.1. Real certificates carry SANs, policies, SCTs and AIA extensions, so treat every number below as a lower bound.

TLS 1.3 (RFC 8446 §4.4.2) wraps the chain in a Certificate handshake message. It has an 8-byte header: handshake type (1), length (3), request-context length (1) and certificate-list length (3). Each certificate entry adds 5 bytes: a 3-byte length and 2 bytes of extensions length.

Why 16,384 is a wall

mbedTLS reassembles each incoming handshake message in its input buffer, and MBEDTLS_SSL_IN_CONTENT_LEN sets that buffer's size. The default is 2¹⁴ = 16,384 bytes, which is also the largest plaintext record TLS allows (RFC 8446 §5.1). A Certificate message larger than the buffer can't be processed, so the handshake fails.

Tokenkeep classifies this sink as a CRITICAL hard ceiling, because the setting is already at the protocol's record maximum: no config change gets you more room. Embedded builds often shrink the buffer to 4 or 8 KiB to save RAM, and then even an ML-DSA-44 chain breaks.

How the algorithms compare

Bytes. Every certificate in the chain uses the same algorithm for its subject key and its issuer's signature, with minimal 244 B metadata.

Why your inventory misses it

A Cryptographic Bill of Materials finds the call that signs a certificate and reports ECDSA. It's right about that. What it doesn't see is the rest of the picture, which is spread across files no crypto scanner reads:

• the buffer size, which is a #define in a C header;
• how many certificates get sent, which is set in deployment config;
• how the DER and TLS framing turn a 3,309-byte signature into a 3,314-byte field inside a 5,520-byte entry.

The failure only shows up when those three facts are put together, which is what Tokenkeep's constraint graph does. The scanner reads MBEDTLS_SSL_IN_CONTENT_LEN from the header and the chain from a [tls_chain] record, then solves the projected message size for any target algorithm.

$ tokenkeep-pqc-scan --path . --target-algo ML-DSA-65
 [CRITICAL] tls/mbedtls_config.h:2   16,568 B > 16,384 B   +184 B · 1.01× limit
            RUNTIME FAILURE IMMINENT: Certificate message exceeds the TLS
            stack's record buffer; handshake aborts

Four ways out

1. Send the leaf alone. A one-certificate message is 5,528 bytes. Intermediates are fetched through the Authority Information Access extension (RFC 5280 §4.2.2.1). This is the template Tokenkeep PQC Enterprise emits, and it only works if your clients can fetch AIA.
2. Shorten the chain. Two ML-DSA-65 certificates come to 11,048 bytes and fit with room to spare. If an intermediate exists only for organisational reasons, dropping it buys back 5,520 bytes.
3. Pick the parameter set per tier. ML-DSA-44 at all three tiers is 11,981 bytes. Many PKIs could use ML-DSA-65 for the root and intermediates and a smaller set for short-lived leaves. Run the scan once per candidate before you commit.
4. Don't count on compression. TLS certificate compression (RFC 8879) does well on names and extensions, but PQC keys and signatures are high-entropy, so most of the 16,568 bytes won't shrink.

Related

• ML-DSA signature and key sizes
• SLH-DSA signature sizes
• Tokenkeep PQC: post-quantum migration analysis