Always-async client with coexisting SQLite + Postgres backends (handoff prototype)

[tylerhawkes]

Always-async libxmtp client with coexisting SQLite + Postgres backends (handoff)

Status: prototype / handoff. Opened as one PR for continuity as XMTP winds down —
not intended to merge as-is. It captures a working spike so someone can pick it up.
CI will not be green (see Dependencies and Known state below).

What this is

Reworks the client to be unconditionally async with two storage backends selected by
cargo feature, coexisting in one codebase:

• sqlite (default, dominant) — diesel / encrypted SQLite. What mobile / node / wasm ship.
  Every sqlx impl is gated not(feature = "sqlite"), so --all-features still compiles the
  SQLite backend and keeps its on-disk format.
• sqlx — sqlx / Postgres, for server-side deployments (herald). Async all the way down.

The old blocking/maybe_await! split is gone — the client is async everywhere; SQLite runs its
synchronous work behind an immediately-ready future.

Structure (15 commits, oldest → newest)

1. prototype(xmtp_db,xmtp_mls,openmls_pg_storage): async storage track over the maybe_async openmls fork (= the old tyler/asyncdb base)
2. async storage: Postgres Client path, CBOR openmls serialization, nested SAVEPOINT, integration coverage
3. xmtp_db: typed storage accessors replacing the generic byte-KV
4. thread .await through the storage-layer tests after Store/Fetch became async
5. build: exclude async-only openmls_pg_storage from the sync workspace build
6. finish AFIT .await threading in apps + bindings (fake-executor at the sync FFI getters)
7. xmtp_db: stand up is_sync-off + diesel (ready-future SQLite) config
8. wasm: untangle hakari two-openmls + wasm ready-future SQLite build
9. storage-shape A/B microbench (xmtp_db_shape_bench)
10. xmtp_mls: drop nightly async_fn_traits/unboxed_closures from the ready-future shape
11. async test-utils on both shapes (mock threaded, d14n, test-suite maybe_await)
12. retire the blocking feature — the client is unconditionally async
13. xmtp_mls: inline maybe_await! to .await and delete the macro
14. make the sync (SQLite) track dominant so both features coexist in one build
15. rename storage-backend features: sync → sqlite, async → sqlx

Dependencies (why CI won't be green here)

This builds on the maybe_async openmls fork, which is itself a stack of open PRs on
xmtp/openmls:

• Streaming support #58 — catch the fork's main up to upstream openmls v0.9.0 (validated, MERGEABLE).
• refactor: add vodozemac dep, exclude unused crates #57 (draft, DO NOT MERGE) — the maybe_async storage-trait rework (#1923) + sqlx Postgres CI,
  plus the Send/MaybeSend-for-wasm, CBOR-derive, and test-framework-gating follow-ons.
• Still open on the openmls side: reconciling the maybe_async work onto the v0.9.0 main
  (a ~14-file conflict merge across the core MLS group logic — deferred, not done here).

Point the openmls dep at the fork's maybe_async branch to build the sqlx shape.

Build / feature model

• sqlite and sqlx select the backend; sqlite is the default and dominant one.
• The sqlx path uses a few nightly features (async_fn_traits, unboxed_closures), scoped to
  xmtp_db/xmtp_mls via .cargo/config.toml RUSTC_BOOTSTRAP so the rest builds on stable.
• Storage enums serialize as bincode under SQLite and CBOR under Postgres; a custom serde derive
  accepts either the storage tag or the variant name so both share the enums with no migration.
• hakari excludes openmls so the workspace-hack can't unify backend-specific openmls features
  (which would change the SQLite on-disk format).

Known state / next steps for whoever picks this up

• xmtp_db (sqlx) compiles and has live-Postgres + 2-client round-trip coverage
  (xmtp_mls_pg_tests, an excluded crate needing a live node + Postgres).
• xmtp_mls async integration is in progress; the sqlx shape depends on the openmls fork work
  landing / being reconciled with v0.9.0.
• Excluded crates (xmtp_db_pg_tests, xmtp_mls_pg_tests, xmtp_db_shape_bench) carry their own
  lockfiles that stale when the workspace-hack changes.

Note

Make all DB query traits async and add coexisting Postgres/sqlx backend

• Converts every Query* trait, Store/Fetch/StoreOrIgnore trait, and StorageProvider method from synchronous returns to async futures (impl Future + MaybeSend), so all callers across xmtp_mls, bindings, and tools must .await database operations
• Gates all Diesel-specific derives, imports, and impl blocks behind #[cfg(feature = "sqlite")] and adds a new #[derive(PgModel)] macro (pg_model.rs) that generates sqlx::FromRow impls and table/column metadata for the Postgres track
• Adds a new PgDb handle (pg.rs) with pooled/transaction-aware connection management, plus new pg_impl modules in every encrypted-store file providing async sqlx-based implementations of all query traits
• Introduces Postgres schema migrations (up.sql) and an openmls_pg_storage crate with a full StorageProvider impl for OpenMLS entities backed by sqlx
• On the SQLite backend, transactions/savepoints now accept async closures that are driven to completion via drive_to_completion (transactions.rs), which panics if the closure's future is not immediately ready
• Behavioral Change: on wasm and mobile bindings, several methods that remain synchronous now use now_or_never().expect(...) and will panic if the underlying future is not immediately ready; all other methods became async and return Promises/futures
• Risk: drive_to_completion in transactions.rs will panic on any suspending await inside a SQLite transaction body; now_or_never().expect(...) calls in wasm/mobile bindings (conversation.rs, mls.rs) will panic if a storage future is not ready on first poll; the pg_impl module's SELECT query in d14n_migration_cutover.rs contains a literal \ + sequence in the SQL string that reviewers should verify

^{Macroscope summarized 4c5144f.}