Support binary PSK identities with an optional Mbed TLS backend

[KRZ303]

Why this is needed

My Samsung oven's owner UUID contains a NUL byte. I recovered the correct PSK, but OpenSSL couldn't send the full identity.

This follows #85. Its guard is correct: removing it would truncate the identity. This patch uses Mbed TLS for that case and preserves all 16 bytes.

What changed

• Use Mbed TLS only for PSK identities containing NUL.
• Keep certificates and other PSK identities on OpenSSL.
• Pass the identity with an explicit length through a small native wrapper built against Mbed TLS 3.6.
• Reuse the existing CoAP session, retry, observation, and cancellation code.
• Reject missing or incompatible native builds before networking. Never fall back to OpenSSL for a NUL-containing identity.
• Limit this backend to DTLS 1.2 and TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA256.

What worked on my oven

My oven reports board prefix LCD_R18_SCO_QMD_EU_22K and SmartThings profile DA-KS-OVEN-0105X. I haven't checked the retail model number yet.

I recovered its existing OwnerPSK from SmartThings on rooted Android 17. The recovery scripts and instructions are here.

On October 3, Mbed TLS 3.6.7 authenticated with the complete identity, and GET /oic/d returned my oven's expected device ID. I got the same result with the standalone native probe and the patched Python session, including inside my HA Linux runtime.

After installing both patches, I imported the PSK and added the oven in HA. My certificate-based washer still loads too.

The oven currently exposes only a connection-mode sensor. Authentication works; full oven entity coverage doesn't. I haven't tested heating commands or used these tools to write OCF security resources.

Tests

The pre-deployment checks for e41c902 passed:

• 943 tests on each of Python 3.11, 3.12, 3.13, and 3.14.
• Python 3.11 with cbor2==5.6.0 and pyOpenSSL==23.1.0.
• Nine native tests against the Linux build used in HA.
• Wheel/sdist, public API, share-safety, and whitespace checks.

A fresh Python 3.14 run also passed all 943 tests before I opened this PR.

Installation limits

The wheel includes C source, not a compiled native module. You need to build _mbedtls_native.so for your architecture and C library. Nothing compiles during authentication.

For HA, I built a static Mbed TLS 3.6.7 module in a disposable container matching my HA image. I didn't replace HA's installed Mbed TLS package.

The existing CI doesn't build this optional module, so its native tests skip there. Native binary distribution still needs sorting out. I've tested macOS and my Linux HA target, not Windows or other appliances.

The companion LocalThings PR lets the integration accept identities supported by this transport. Credential extraction and oven capability support stay separate.

[QuiteYellow]

@KRZ303 A change of direction on this one, since it affects your patch.

I went to package the Mbed TLS backend as prebuilt wheels, because shipping .c to compile on install does not work under Home Assistant. That turned out to be bigger than I expected: this project builds with hatchling, which has no C-extension support, so it means swapping the build backend and then keeping a wheel matrix across musllinux and manylinux on two architectures, rebuilt on every Mbed TLS security release.

Before giving up on OpenSSL I checked whether it had any length-carrying PSK path at all. psk_use_session_cb fired zero times for DTLS 1.2 across three configurations, and when it is the only callback registered OpenSSL reports no ciphers available. It is TLS 1.3 only, as you said.

So I tried the other direction: a DTLS 1.2 client speaking only TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA256, written in Python on cryptography. 567 lines, no build step, certificates left on OpenSSL.

Against a server built from the appliance's own Mbed TLS 2.7.8, configured the way ca_adapter_net_ssl.c configures a PSK session, it completes the handshake with a 16-byte identity carrying a zero byte at offset 8 intact, decrypts application data both ways, and closes cleanly.

That server runs on a laptop, though. Your oven is the only real PSK device anyone working on this can reach.

Would you run this against it? One script, no build, read-only: handshake, GET /oic/d, close.

git clone --depth 1 --branch scratch/test-scripts \
  https://github.com/QuiteYellow/SmartThings-Local.git psk-test
cd psk-test/pr-115-purepy-psk

That branch is orphaned off main and holds nothing but scripts, so the clone is two files. Both are readable in the browser first if you would rather look before running: pr-115-purepy-psk.

It takes the host, identity and key from environment variables, which the script lists if you run it without them. The device id is reported as a match against one you supply, so the output is safe to paste here.

I am leaving this PR open until that result is in. What happens to it afterwards depends on how the engine does on real hardware, and there is no sense settling that beforehand.

[KRZ303]

@KRZ303 A change of direction on this one, since it affects your patch.

I went to package the Mbed TLS backend as prebuilt wheels, because shipping .c to compile on install does not work under Home Assistant. That turned out to be bigger than I expected: this project builds with hatchling, which has no C-extension support, so it means swapping the build backend and then keeping a wheel matrix across musllinux and manylinux on two architectures, rebuilt on every Mbed TLS security release.

Before giving up on OpenSSL I checked whether it had any length-carrying PSK path at all. psk_use_session_cb fired zero times for DTLS 1.2 across three configurations, and when it is the only callback registered OpenSSL reports no ciphers available. It is TLS 1.3 only, as you said.

So I tried the other direction: a DTLS 1.2 client speaking only TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA256, written in Python on cryptography. 567 lines, no build step, certificates left on OpenSSL.

Against a server built from the appliance's own Mbed TLS 2.7.8, configured the way ca_adapter_net_ssl.c configures a PSK session, it completes the handshake with a 16-byte identity carrying a zero byte at offset 8 intact, decrypts application data both ways, and closes cleanly.

That server runs on a laptop, though. Your oven is the only real PSK device anyone working on this can reach.

Would you run this against it? One script, no build, read-only: handshake, GET /oic/d, close.

git clone --depth 1 --branch scratch/test-scripts \
  https://github.com/QuiteYellow/SmartThings-Local.git psk-test
cd psk-test/pr-115-purepy-psk

That branch is orphaned off main and holds nothing but scripts, so the clone is two files. Both are readable in the browser first if you would rather look before running: pr-115-purepy-psk.

It takes the host, identity and key from environment variables, which the script lists if you run it without them. The device id is reported as a match against one you supply, so the output is safe to paste here.

I am leaving this PR open until that result is in. What happens to it afterwards depends on how the engine does on real hardware, and there is no sense settling that beforehand.

Hah I was just experimenting with Mbed prebuilt wheels!
Thank you I will try to test and comeback with reply tomorrow - sorry for accidental close/open PR, missclicked!

[KRZ303]

@QuiteYellow Following up on your test request: I tested the Python client against my oven inside HA Core on October 3. It worked after fixing the engine and probe.

The handshake used the full 16-byte PSK identity containing a zero byte. The corrected probe produced:

identity      : 16 bytes, contains a zero byte: yes
handshake     : OK in 0.1s (DTLS 1.2, ECDHE-PSK-AES128-CBC-SHA256)
GET /oic/d    : OK (2.05, CBOR map, expected device ID matched)
close_notify  : sent

Exit code was zero, with no stderr. This was one read-only GET; no appliance controls or security-resource writes. close_notify: sent means local UDP submission; I did not measure receipt on the oven.

The changes are in #116, targeting your scratch/test-scripts branch:

• Handshake transcript: retain the initial ClientHello when the server skips the cookie exchange. The cookie path still resets the transcript before the replacement ClientHello.
• HelloVerifyRequest framing: accept DTLS 1.0 record headers for the initial plaintext HelloVerifyRequest, keeping DTLS 1.2 checks elsewhere.
• UDP transport: use an unconnected socket so replies from a different source port can arrive. Filter by the target IP and allow reuse of the integration's local port.
• Probe results and cleanup: correlate CoAP responses, handle empty ACKs and separate CON replies, and require CoAP 2.05, valid CBOR, and the expected device ID. Attempt shutdown and socket closure on success or failure, and omit identifiers and credentials from output.

I temporarily unloaded only the oven entry, then restored it. The oven resumed Observe mode, and the washer stayed loaded. The installed integration and protocol library were unchanged.

The PR also documents runtime dependencies and includes 26 passing offline regression tests, including real OpenSSL exchanges with and without cookies. Since the hardware run, only documentation and tests changed; the engine and executable probe code are unchanged.

This establishes a successful authenticated read on my oven. #116 contains the standalone script fixes and tests; library integration remains separate.

[QuiteYellow]

@KRZ303 Merged as 961bf60, and thank you for running it. Your 26 tests pass here, and all four of my existing suites pass against your engine with no regressions.

I reproduced the transcript failure this fix repairs. Building the reference server with mbedtls_ssl_conf_dtls_cookies(&conf, NULL, NULL, NULL) so it skips the cookie exchange, the old engine fails with fatal alert 50 and the server reports -0x7E80, which is MBEDTLS_ERR_SSL_BAD_HS_FINISHED. Your version completes the handshake against the same server.

The cause was a comment of mine that was right about the specification and wrong about when it applies. RFC 6347 4.2.1 excludes the initial ClientHello from the transcript, but only when a cookie round trip actually happens. I applied it unconditionally. The server now takes a nocookie argument and both configurations run every time.

The socket change was a good spot, because this repository already fixed that pattern in a879ef4.

Now it is confirmed working, I ran a pass over this repository's earlier protocol fixes to check each one had been carried into the engine. One had not: it answered only the first HelloVerifyRequest and ignored any later one, which is the shape localthings#504 describes. Each cookie is answered now, with a cap so a server that only ever sends them cannot hold the handshake open. That is 4ec1c64 on the branch, so the engine has moved since your run.

One question I could not settle by experiment. Did the oven actually frame its HelloVerifyRequest as DTLS 1.0, or was that change defensive? I rebuilt the reference server without its version pinning and still got 1.2 framing, so I cannot reproduce the condition your fix handles. The fix is correctly scoped either way and I have kept it.

If you still have the probe output or a capture from that run, the record header on the HelloVerifyRequest is the only part I need.

[QuiteYellow]

@Jason-Morcos this lands on your PskAuth design, so: the engine is now #117, a draft against main. The module, 331 tests, and cryptography declared as the direct dependency it has quietly been since August. CI is green across 3.11 to 3.14 and both dependency sets.

Nothing imports it, so it cannot merge as it stands. The ask is in the body, along with one question for you: routing PSK through this needs a verb AuthenticationProvider does not have.

[KRZ303]

@QuiteYellow I ran another read-only probe against the oven and recorded the incoming HelloVerifyRequest headers.

The oven used DTLS 1.2 framing (FE FD). Its complete 13-byte record header was:

16 fe fd 00 00 00 00 00 00 00 00 00 2f

The handshake type was 03 (HelloVerifyRequest), and the server_version inside its body was also FE FD.

So the FE FF exception was not needed for this oven run. That case is reproduced by the real OpenSSL cookie-exchange test: it enables the cookie exchange, then observes OpenSSL emitting an FE FF HelloVerifyRequest. The test does not force the record version.

I should have distinguished that fixture-backed compatibility fix from the hardware result in my earlier reply. The original October 3 run did not record headers, so I cannot establish which framing the oven used then.

I also tested your exact commit 4ec1c642ad0e75c55a481fbc98ae2d412d9b0263 against the oven today. It authenticated with the full 16-byte identity containing a zero byte and produced:

identity      : 16 bytes, contains a zero byte: yes
handshake     : OK in 0.1s (DTLS 1.2, ECDHE-PSK-AES128-CBC-SHA256)
GET /oic/d    : OK (2.05, CBOR map, expected device ID matched)
close_notify  : sent

Exit code was zero, with no stderr. close_notify: sent records local UDP submission; I did not measure its receipt on the oven.

That run also received one HelloVerifyRequest with FE FD framing. The header recorder forwarded incoming bytes unchanged and omitted credentials and cookie contents.

All 26 regression tests pass on 4ec1c64. Separate local tests using synthetic DTLS 1.2 challenges also passed for changed-cookie responses, duplicate-cookie timer preservation, and the retry cap. The oven did not issue a second challenge, so the hardware run confirms the updated engine works on it, while repeated-challenge handling was exercised locally.

[QuiteYellow]

@KRZ303 Closing this in favour of #117, which carries a pure-Python DTLS 1.2 ECDHE-PSK engine. Your oven runs are what showed the approach works, the engine includes your fixes from #116, and the one you tested at 4ec1c64 is the same engine #117 carries.

#117 wires the engine to nothing: PskAuth still raises, and validate_identity still rejects an identity carrying a zero byte, because routing the transport through it needs a connection seam AuthenticationProvider lacks. The #85 guard therefore stays, and stays correct. Until the seam lands, your branch here is the only code that has authenticated one of these identities end to end on an installed library. This was essential for me to build 117!

What this patch and your work on it established stands regardless of which engine ships:

• A recovered OwnerPSK identity containing a zero byte is valid appliance data, and the appliance resolves it by raw bytes and length.
• OpenSSL's DTLS 1.2 PSK client callback cannot carry such an identity. You found that, and it is why a replacement transport exists at all.
• An authenticated /oic/d matching the expected device id distinguishes a working credential from a plausible one.
• Three of the engine's defects were yours: a transcript that excluded the initial ClientHello whenever a server skipped the cookie exchange, a HelloVerifyRequest record-version check too strict for OpenSSL, and a connected socket in the probe that this repository had already fixed once, in a879ef4.
• Recording the oven's own HelloVerifyRequest header settled something I could not reach by experiment. 16 fe fd 00 00 00 00 00 00 00 00 00 2f says the appliance frames DTLS 1.2, so that fix is an OpenSSL compatibility one, and my reference server was faithful on an axis I had started to doubt.
• Your recovery tooling stands on its own and is unaffected by any of this.

If you are willing to be tagged when a PSK change needs a hardware check before a release, say so and I will tag you.

[KRZ303]

@QuiteYellow count me in, will be happy to test and generally contribute what I can :)

[Jason-Morcos]

@KRZ303 Thanks for doing the hardware runs and separating the oven's actual FE FD header from the OpenSSL fixture's FE FF behavior. That clears up an important detail.

One correction to my earlier note on #16: the separately built Mbed TLS backend is no longer the only demonstrated way to carry that identity. Your standalone Python-engine runs establish another working path on the oven; #117 still needs the library connection wiring before normal PskAuth users gain it.

I've now run #117's 331 new tests and replayed a retained refrigerator PSK server flight through that engine with synthetic credentials. It gets through ClientKeyExchange with the full identity. I've put the details, a reproduced repeated-cookie framing bug, and the connection-factory suggestion on #117 so they stay with the implementation. Our live washer probe today was inconclusive, so I'm not counting that as an additional hardware success.