Add a server role: synchronized multi-client audio push, mDNS discovery/advertising, and supervised connections

[davidgraeff]

Closes #86

What this adds

sendspin-rs currently implements only the client/receiver role. This PR adds a server role (src/server/) so the crate can accept Sendspin clients, sync their clocks, and push synchronized audio out to them — built on the existing protocol::messages schema and binary frame layout, reusing those types.

The concrete driver is a Home Assistant add-on that streams audio to Home Assistant Voice PE, whose ESPHome sendspin: component (sendspin-cpp) only ever runs its own embedded WebSocket server and never dials out. Supporting that device forces both connection directions — inbound accept and discover-then-dial — so the server role handles both.

Everything here has been validated against the real, unmodified aiosendspin[server] reference and tested against real Voice PE hardware on the network.

New public API

pub use server::{ServerConnection, ServerListener};
// and, within the server module:
//   Advertisement, ClientBrowser        // mDNS advertise / discover
//   dial_client                          // one-shot dial-out
//   ClientManager, ClientEvent           // supervised discovery + reconnect
//   Group, DEFAULT_SEND_AHEAD_US         // synchronized multi-client playback
//   ServerConnectionGuard, ServerSender
//   encode_audio_frame                   // encode-side mirror of AudioChunk::from_bytes

How it's built (commit by commit)

The branch is structured so each commit is a reviewable, self-contained step:

1. feat(server): prototype the server role: ServerListener (accept + server-side handshake, mirroring ProtocolListener for the opposite role), ServerConnection/ServerSender (per-client actor with the same writer-task/message-router split as protocol::client, time-sync echo stamped at actual send time, stream/start → binary chunks → stream/end), and binary::encode_audio_frame (the encode-side mirror of the client's parse-only AudioChunk::from_bytes). Scoped to a single client, single format, player role.

2. test(conformance): prove the server role matches real aiosendspin: Drives both the real aiosendspin[server] and the new Rust server through an identical canned pcm track, points a real aiosendspin protocol client at each, and diffs what it observed. Audio bytes are sha256-identical and every event (ordering + field values) matches.

3. feat(server): multi-client synchronized playback groups: Group: the actual multi-room correctness property. Every member is sent the same audio bytes tagged with the same server-clock timestamp for a given push, computed once per push; each client's own clock offset then handles sample-accurate simultaneous playback, so the server never compares members' clocks to each other. Broadcasts concurrently (one slow member can't block the others); a member whose send fails is pruned automatically.

4. feat(server): mDNS advertising via _sendspin-server._tcp.local.: Advertisement, so clients can discover a running server without a configured address (distinct service type from _sendspin._tcp.local., which is what a client advertises). Promotes mdns-sd from a dev-dependency to a real dependency.

5. feat(server): discover and dial clients that only run their own server: ClientBrowser (browses _sendspin._tcp.local., resolves each client to a ws:// URL) and dial_client (connects out and drives the same server-role handshake as accepted connections — the protocol roles don't depend on dial direction). This is the path Voice PE requires.

6. feat(server): ClientManager: continuous discovery + reconnect-with-backoff: Closes the robustness gap: dial_client is one-shot. ClientManager supervises every discovered client — dials on discovery, retries with capped exponential backoff (matching aiosendspin's MAX_RECONNECT_BACKOFF_S/STABLE_SERVER_INITIATED_SESSION_S), re-dials promptly when the same device (tracked by stable mDNS fullname, not ephemeral address) reappears at a new address. Surfaces Connected/Message/Disconnected as one event stream. This is what real deployments should use.

7. docs(server): close out late-join scope: records the decision that late-join historical replay is not implemented, with a test proving the actual behavior a mid-stream joiner gets.

8. examples: add play_wav: a manual hardware test tool: streams a WAV file to whatever client(s) connect, running all three connection sources (accept, discover+dial, direct --dial) into one Group.

Scope

This is a v1/prototype scope, not full parity with aiosendspin. Everything deferred is tracked in src/server/mod.rs's module docs.

Deferred (follow-up): per-client codec transcoding/resampling (v1 is PCM-only, one format per group); non-player roles (color, visualizer, artwork, controller, metadata); external player registration.

Not planned, by decision: late-join catch-up / historical buffer replay. aiosendspin's version of this exists almost entirely to re-encode cached history through a newly-joined role's codec/format when it differs from what's cached. A mid-stream joiner gets stream/start plus the next push, in sync with everyone else, but nothing from before it joined; Can result in a short join gap, but everything else would have exploded the code lines of this PR. Current behaviour proven by a_late_joiner_gets_current_stream_start_and_only_subsequent_audio in tests/group_sync.rs.

Testing

• Conformance (tests/conformance/): observable equivalence to the real aiosendspin[server] — sha256-identical audio, identical event sequence/fields. See tests/conformance/README.md (including a real bug in the oracle driver this caught: registering the event listener after start_server(), which opens the TCP listener before finishing its own mDNS setup).
• Unit/integration: tests/server_listener.rs, tests/group_sync.rs, tests/discovery.rs (real mDNS browser), tests/dial_discovery.rs (fake self-advertising client), tests/client_manager.rs.
• Real hardware: tested against Home Assistant Voice PE devices on the network, and end-to-end via examples/play_wav.rs.

Dependency change

mdns-sd moves from a dev-dependency (client examples only) to a real dependency — the server role needs it to advertise and discover.

Notes for reviewers

Happy to adjust module layout, naming, or the public re-export surface. The scope boundaries above are deliberate for a first milestone; I'm open to pulling any of the deferred items forward if you'd prefer them in the initial merge.

[davidgraeff]

Because there was no review yet, I have for pushed two small fix (use SO_REUSEADDR for server, ClientManager: Propagate fullname) into the existing commits.

[teancom]

@davidgraeff Hello! Thank you so much for taking a crack at this, it's been on my todo list for a long time and I hadn't gotten around to it. Please understand that the following feedback is with the intent of landing the PR, not blowing you off. It took me a few days to respond to the PR because I wanted to give you real, actionable feedback.

Let's start with: please take out all of the conformance adapter stuff. We have a separate conformance repo - github.com/Sendspin/conformance/ - that has a fleet of scenarios that, while not comprehensive (yet!) ensure interoperability between implementations, and is the proper place for that. You can run it locally, adding sendspin-rs as a server, and test this against all of the clients, not just ourselves + aiosendspin. I've found it invaluable for finding bugs. NB: head of aiosendspin as of this writing is in a bit of a broken state when acting as a server as they're adding encryption support, which was recently added to the spec. That should be ignorable for your purposes, just letting you know.

Next, Group should anchor stream_position_us = now + lead on first push (or start_stream), advance it per push by samples * 1_000_000 / rate with a residue accumulator (it already knows the format from StreamPlayerConfig), rebase forward only when the timeline falls behind now + min_lead, and re-anchor on stream/clear. That makes push_audio pacing-insensitive, deletes the need for play_wav's interval-tick workaround, and matches the oracle it's claiming equivalence with. This is the biggest issue by far, as it's the cause of all of the bad audio you were fighting against.

I was looking at play_wav, and was going to mention that you had a problem where you're holding onto the group mutex the whole time preventing add_member during playback and you should switch to a lock per push, but that made me look deeper. And having Group conflate the control plane and the data plane behind one &mut self is the underlying issue. Even with lock-per-push, push_audio doesn't return until every member's actual socket write completes — ServerSender::send_audio_chunk awaits an ack that the writer task fires only after sink.send().await finishes, and broadcast_and_prune joins all members, so each push blocks for the slowest member with no timeout anywhere. A member that errors gets pruned; a member that stalls (TCP zero-window, half-dead Wi-Fi device) blocks push_audio indefinitely. Combine with the mutex and one sick client freezes the entire group: audio to everyone else, membership, everything. So even the "correct" usage pattern has an unbounded starvation hazard baked into the API.

But! Each connection already has a per-client unbounded channel + dedicated writer task — the exact machinery for non-blocking fan-out. push_audio blocks only because it on awaits per-write acks (which exist to detect dead members for pruning). If broadcast just enqueued into each member's channel and pruning happened on writer-task death instead of per-send error, push_audio would be near-instant, a slow member couldn't block the group (its frames queue; its problem), and the whole mutex question would become academic.

So, either give Group interior mutability (&self methods, members behind an internal lock) with non-blocking enqueue broadcast, or restructure it as an actor task that owns membership and takes commands over a channel (same thing, different syntax). Either approach will need some sort of eviction policy so we don't have unbounded memory growth: queue depth, age watermark, something like that. Then play_wav can demonstrate the canonical API usage, including mid-playback joins (yay).

ClientManager address-change swallows Disconnected. When the browser sees the same fullname at a new URL, it abort()s the live supervisor task. A currently-connected client dies without a Disconnected event, so caller bookkeeping desyncs until a send fails. A possible fix would be to stop using abort() as the signaling mechanism: give each supervisor a tokio::sync::watch::Receiver<String> (or a cancellation token + new URL), select! it against the drain, and on change close the connection, emit Disconnected, and redial at the new URL.

There are some references in code comments to your local agent plan docs (spikes/03-sendspin-pushstream). In fact, there are various comments that seem to narrate what happened during a coding session - stuff like:

// ABOUTME: Not a test binary itself — `tests/common/` (mod.rs, not
// ABOUTME: common.rs) is the standard way to share code between
// ABOUTME: integration test files without cargo compiling it as its own test.

or

//   - Clients that only run their own embedded WebSocket server and never
//     dial out — this is the primary topology for real embedded hardware
//     (e.g. ESPHome's `sendspin:` component / Home Assistant Voice PE, via
//     sendspin-cpp), confirmed by reading esphome/esphome's sendspin_hub.cpp:
//     its setup() only ever calls start_server(), never an outbound connect.

that don't belong (amongst other examples). That should all be cleaned up. Oh, and src/server/listener.rs:30 references examples/minimal_server.rs, which doesn't exist.

connection_reason is hardcoded to Discovery even on the dial path.

Discovery is IPv4-only, we should accept IPv6 as well (aiosendspin is biased towards IPv4 but does work with IPv6)

resolve_client_url filters link-local/unspecified but not loopback — sorted-first selection would prefer 127.x if ever advertised.

push_audio clones the payload once per member; an Arc<[u8]>/Bytes would drop that to one allocation.

No mDNS ServiceRemoved handling — a permanently-gone device is redialed forever at the 300s cap. You implemented most of the aiosendspin approach, but only for persistent clients that have retry_indefinitely=True set.

Speaking of, your mDNS tests do real multicast which is fine for now, but should be gated behind something like an env var (SENDSPIN_NET_TESTS=1 or whatever).

Feel free to ask questions here or in the discord and push back if I got something wrong, it's just my brain + Fable over here, so mistakes are more than possible, they're likely. 😄

[davidgraeff]

Hi @teancom,

Thanks for the detailed review! It’s usually not my style to surprise an open-source project with a 3k-line PR. I did consider splitting it up beforehand, but couldn't quite find a clean or practical way to do so.

While play_wav is my primary test application, I admittedly didn't account for edge cases like stalls caused by poor connections. In my small, stable local network with only three sendspin clients, that issue simply never came up.

Thank you also for pointing out the Sendspin/conformance suite, I clearly didn't search thoroughly enough for that.

I will push all upcoming changes as separate commits (no force-pushes) in case you want to track the updates commit by commit.

[davidgraeff]

Thank you again for the feedback, none of it read as blowing me off.
I think I have addressed all of your very valid comments;
details below, point by point in your order.

Conformance adapter — removed. Dropped tests/conformance/ and the
conformance_server.rs example entirely. Instead of a bespoke oracle I wrote a
sendspin-rs server adapter for your conformance repo and opened it as a draft:
Sendspin/conformance#98. It drives
this server role through client-initiated-pcm, server-initiated-pcm, and
server-initiated-pcm-24bit, all passing against the sendspin-rs client
adapter (canonical PCM hashes match). It's draft because it depends on this PR's
server role and because aiosendspin HEAD as a client trips the client/init
handshake (see the last note below).

Group timeline anchoring — done. This was the big one and you were right
about the cause. Group now anchors next_ts_us = now + send_ahead_us on the
first push after a start/clear, advances it by each chunk's exact duration with
a fractional-microsecond residue accumulator (it takes the format from
StreamPlayerConfig), re-anchors forward only when the timeline falls behind
now + send_ahead_us/2, and resets on start_stream/end_stream. push_audio
is pacing-insensitive now, and play_wav's absolute-schedule ticker workaround
is gone.

Group control/data-plane split + non-blocking broadcast — done. You
correctly identified the starvation hazard baked into the API. Group now takes
&self with its members behind an internal lock, and broadcast is non-blocking:
ServerConnection gained enqueue_audio, which hands the frame to the existing
per-connection writer task and returns immediately — no per-write ack. Pruning
now happens on writer-task death (the send fails) rather than per-send error, so
a stalled member can't block the group. I went with interior mutability rather
than a full actor, as the smaller change that reuses the existing writer/channel
machinery.

Eviction: each connection has a bounded audio backlog
(MAX_QUEUED_AUDIO_FRAMES, currently 32 frames); once it's full, further frames
for that member are dropped rather than growing memory without bound — its audio
suffers, nobody else's does. Happy to make that configurable or tune the number;
Maybe you have a smarter idea.

push_audio per-member clone — fixed. Each chunk is encoded once into a
Bytes and fanned out to members as refcount clones — one allocation per push
instead of one per member.

play_wav — reworked. It uses the Group directly now (no external
Mutex, no ticker), holds no lock while streaming, and so demonstrates
mid-playback joins: clients discovered mid-stream are added concurrently by the
accept/discovery loops and join the group in progress.

ClientManager swallowing Disconnected on address change — fixed.
Replaced the abort() signaling with a watch directive channel per
supervisor. An address change now sends Dial(new_url), which makes the
supervisor drop the current connection, emit Disconnected, and redial the
new URL; select! runs the directive against the drain, exactly as you
suggested.

Agent-plan narration / stray comments — cleaned up. Removed the
coding-session narration (the ESPHome/aiosendspin travelogues, the tutorial
asides, the spikes/… references) and tightened the ABOUTME headers to match
the terse two-line style already used across the crate. Also fixed
src/server/listener.rs's reference to the nonexistent
examples/minimal_server.rs.

connection_reason hardcoded to Discovery on the dial path — fixed. The
dial path now sends Playback. I left the inbound accept path as Discovery
(the client initiated, so the server is just present/available) — happy to make
both Playback if you'd prefer.

Discovery IPv4-only — fixed. resolve_client_url now uses
get_addresses() and handles IPv6 (bracketed in the URL) alongside IPv4.

resolve_client_url loopback — fixed. It now filters loopback in addition
to link-local/unspecified, so 127.x/::1 can't be picked as the sorted-first
address.

No ServiceRemoved handling — fixed. ClientBrowser now surfaces
Removed alongside Found (a Discovered enum via next_event()), and
ClientManager stops supervising a device whose mDNS advertisement is removed
(gracefully, emitting Disconnected for a live connection) instead of redialing
a gone device forever.

mDNS tests doing real multicast — gated. The multicast integration tests
(discovery, dial_discovery, client_manager) are now behind
SENDSPIN_NET_TESTS; a plain cargo test no longer depends on multicast.

---

When using the conformance suite, I noticed that aiosendspin now
opens with client/init (the encryption negotiation you mentioned is in flight?).
The v1 server expects client/hello first.

Should the server skip unknown pre-hello messages for forward-compatibility, or
is that better handled once the encryption handshake is specced? I didn't change
it here.

cargo test / cargo fmt --check are green (17/17 test binaries;
tests/group_sync.rs covers the reworked Group; identical timestamped audio to
all members, a dead member pruned without blocking the survivor, and correct
late-join). Pre-existing chunks_exact clippy lints in src/audio/decode/pcm.rs
are left untouched as out of scope.

[teancom]

Thanks for the follow-up! Two findings this go 'round:

Control and close messages can still be starved by a stalled audio writer

enqueue_audio() is non-blocking from the group’s perspective, but audio, control, and close commands still share the same per-connection writer queue.

The writer handles an audio command with:
sink.send(WsMessage::Binary(frame)).await

If that socket stalls, the writer cannot process the following commands until the send completes. Consequently, a later stream/end, stream/clear, player command, or Close command can remain queued indefinitely. ServerConnectionGuard::disconnect() also waits for the close acknowledgement, so it can hang in this situation.

This is slightly different from the original group-level blocking problem: the group can enqueue audio without waiting, but lifecycle and cleanup operations for a stalled member can still be blocked behind that audio.

It would probably be useful to add a regression test with a deliberately stalled socket and a subsequent control or close command.

Stream lifecycle transitions are not serialized with audio enqueueing

start_stream() updates stream_config and releases the group lock before it broadcasts stream/start. A concurrent push_audio() can therefore enqueue an audio frame before stream/start.

The inverse happens in end_stream(): it clears the stream state and releases the lock before broadcasting stream/end. A concurrent push_audio() can enqueue another audio frame while the end transition is still in progress.

Because these operations ultimately use the same per-connection writer queue, the client can observe invalid ordering such as:
audio
stream/start

or:
stream/end
audio

The existing tests cover the sequential behavior, but not these concurrent interleavings. The lifecycle transition and audio enqueue path need some sort of shared ordering mechanism so that stream/start, audio, and stream/end are serialized consistently. A concurrent start/push/end test would help demonstrate the guarantee.