diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 5e29915..2580402 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -78,6 +78,14 @@ jobs: - name: Test run: cargo test --target ${{ matrix.target }} + - name: Run examples + shell: bash + run: | + set -euo pipefail + for example in singleflight rate_limiter lru_session_store; do + cargo run --example "$example" --target ${{ matrix.target }} + done + coverage: name: Code Coverage runs-on: ubuntu-latest diff --git a/CHANGELOG.md b/CHANGELOG.md index 699d3b5..ac905b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Declare `rust-version` (MSRV 1.75) and crates.io `keywords` / `categories` metadata - Pin dev-dependency versions (`criterion = "0.8"`, `rand = "0.10"`) +### Documentation + +- Add runnable examples: `singleflight` (request coalescing), `rate_limiter` + (per-key token bucket) and `lru_session_store` (bounded session cache) +- README: add installation/MSRV section, capability comparison matrix, + non-reentrancy caveat and explicit dual-license statement + ### CI - Run Miri on the whole library test suite (`cargo miri test --lib`) diff --git a/README.md b/README.md index 94bf3ef..496713d 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,23 @@ A high-performance, thread-safe HashMap and LRU cache for Rust with **fine-grained per-key locking**. +Correctness is taken seriously: every commit is verified in CI with **Miri** and **ThreadSanitizer** over the full concurrent test suite, plus an MSRV check, `clippy -D warnings`, and tests across five OS/architecture targets. + +## Installation + +```bash +cargo add lockmap +``` + +Or add it to your `Cargo.toml` manually: + +```toml +[dependencies] +lockmap = "0.2" +``` + +The minimum supported Rust version (MSRV) is **1.75**, verified in CI. + ## Data Structures | Type | Description | @@ -29,7 +46,7 @@ Unlike standard concurrent maps that might lock the entire map or large buckets, * **Deadlock Prevention**: Provides `batch_lock` to safely acquire locks on multiple keys simultaneously using a deterministic order. * **Non-Blocking Locking**: `try_entry` / `try_entry_by_ref` return `None` instead of blocking when a key is already held. * **Iteration**: `for_each` / `retain` visit all entries shard by shard, without a global lock. -* **Pluggable Hasher**: both maps accept a custom `BuildHasher` via `with_hasher` constructors (default: `foldhash`). +* **Pluggable Hasher**: Both maps accept a custom `BuildHasher` via `with_hasher` constructors (default: `foldhash`). * **Single Hash Computation**: Each key is hashed once; the pre-computed hash is stored alongside the key and reused for shard selection, table probing, and rehashing. * **No Key Duplication**: Uses `hashbrown::HashTable` so each key is stored only once, inside the entry state. * **Entry API**: Ergonomic unified RAII guard (`Entry`) for managing locks. @@ -132,6 +149,35 @@ assert_eq!(cache.pop_lru(), Some(("key".to_string(), "new_value".to_string()))); // entries are automatically evicted. ``` +## Examples + +Runnable, self-checking examples live in [`examples/`](examples/): + +| Example | What it shows | +|---------|---------------| +| [`singleflight.rs`](examples/singleflight.rs) | Request coalescing: 8 threads miss the same key, the backend is hit exactly once | +| [`rate_limiter.rs`](examples/rate_limiter.rs) | Per-user token-bucket rate limiter with atomic refill-and-consume per key | +| [`lru_session_store.rs`](examples/lru_session_store.rs) | Bounded session store using LRU eviction, `peek` and `pop_lru` | + +```bash +cargo run --example singleflight +``` + +## Comparison with Alternatives + +Capability matrix (see the [Benchmarks](#benchmarks) section for performance trade-offs): + +| Capability | `lockmap` | `dashmap` | `moka` | +|------------|-----------|-----------|--------| +| Exclusive lock granularity | per key | per shard | no lock API | +| Holding a guard blocks | that key only | the whole shard | — | +| Locking a not-yet-existing key | ✅ | ✅ (holds the shard) | ❌ | +| Deadlock-safe multi-key locking | ✅ `batch_lock` | ❌ | ❌ | +| Bounded capacity with eviction | ✅ per-shard LRU | ❌ | ✅ TinyLFU | +| Non-promoting read / explicit pop | ✅ `peek` / `pop_lru` | — | ❌ | + +Rule of thumb: pick `lockmap` when you need **exclusive per-key critical sections** (read-modify-write, request coalescing, per-key state machines) or a **throughput-oriented bounded cache**; pick `dashmap` for a general concurrent map with short operations; pick `moka` when **cache hit rate** is the primary concern. + ## Benchmarks The repository ships two Criterion benchmark suites: @@ -152,12 +198,17 @@ run the suite on your own hardware and workload before drawing conclusions. ## Important Caveats -### 1. No Lock Poisoning +### 1. Locks Are Not Reentrant + +Calling any map operation for a key **while already holding that key's `Entry`** deadlocks — this includes `get`, `insert`, `remove`, `entry`, and whole-map operations such as `clear`, `for_each` and `retain`. +> **Note**: Drop the guard first, or use `try_entry` / `try_entry_by_ref` when you cannot statically rule out re-entry. + +### 2. No Lock Poisoning Unlike `std::sync::Mutex`, **this library does not implement lock poisoning**. If a thread panics while holding an `Entry`, the lock is released immediately (via Drop) to avoid deadlocks, but the data is **not** marked as poisoned. > **Warning**: Users must ensure exception safety. If a panic occurs during a partial update, the data associated with that key may be left in an inconsistent state for subsequent readers. -### 2. `get()` Performance +### 3. `get()` Performance The `map.get(key)` method clones the value while holding an internal shard lock. > **Note**: If your value type `V` is expensive to clone (e.g., deep copy of large structures), or if `clone()` acquires other locks, use `map.entry(key).get()` instead. This moves the clone operation outside the internal map lock, preventing blocking of other threads accessing the same shard. @@ -168,4 +219,13 @@ See [CHANGELOG.md](CHANGELOG.md) for release notes and migration guides. ## License +Licensed under either of + +* Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or ) +* MIT license ([LICENSE-MIT](LICENSE-MIT) or ) + +at your option. + +Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions. + [![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2FSF-Zhou%2Flockmap.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2FSF-Zhou%2Flockmap?ref=badge_large) diff --git a/examples/lru_session_store.rs b/examples/lru_session_store.rs new file mode 100644 index 0000000..ed5ac30 --- /dev/null +++ b/examples/lru_session_store.rs @@ -0,0 +1,66 @@ +//! Bounded session store built on `LruLockMap`. +//! +//! - Capacity-bounded: the least recently used session is evicted +//! automatically when the store is full. +//! - `peek` inspects a session without marking it as recently used. +//! - `pop_lru` explicitly reclaims the coldest session. +//! - Sessions held by an entry guard are never evicted, even at capacity. +//! +//! Run with: `cargo run --example lru_session_store` + +use lockmap::LruLockMap; + +#[derive(Clone, Debug)] +struct Session { + user: String, + hits: u32, +} + +fn main() { + // A tiny single-shard store so the eviction order is easy to follow. + let sessions = LruLockMap::::with_options(3, 3, 1); + + for id in 1..=3u64 { + sessions.insert( + id, + Session { + user: format!("user-{id}"), + hits: 0, + }, + ); + } + + // Touch session 1 through the entry API: this promotes it to + // most-recently-used and updates it atomically. + { + let mut entry = sessions.entry(1); + if let Some(session) = entry.get_mut() { + session.hits += 1; + } + } + + // Inserting a 4th session evicts the LRU one. Session 2 is now the + // coldest (session 1 was just promoted), so it is the one that goes. + sessions.insert( + 4, + Session { + user: "user-4".into(), + hits: 0, + }, + ); + assert!(sessions.peek(&2).is_none(), "session 2 was evicted"); + assert!(sessions.peek(&1).is_some(), "session 1 survived (promoted)"); + + // `peek` does not disturb the LRU order, so it is safe for monitoring. + for id in [1u64, 3, 4] { + if let Some(session) = sessions.peek(&id) { + println!("session {id}: {session:?}"); + } + } + + // Explicitly drain the coldest sessions, e.g. on memory pressure. + while let Some((id, session)) = sessions.pop_lru() { + println!("reclaimed session {id} of {}", session.user); + } + assert!(sessions.is_empty()); +} diff --git a/examples/rate_limiter.rs b/examples/rate_limiter.rs new file mode 100644 index 0000000..85dd055 --- /dev/null +++ b/examples/rate_limiter.rs @@ -0,0 +1,82 @@ +//! Per-key token-bucket rate limiter. +//! +//! Each user owns an independent bucket protected by its own lock, so the +//! refill-and-consume critical section of one user never contends with +//! another user's requests. +//! +//! Run with: `cargo run --example rate_limiter` + +use lockmap::LockMap; +use std::time::{Duration, Instant}; + +struct Bucket { + tokens: f64, + last_refill: Instant, +} + +struct RateLimiter { + buckets: LockMap, + capacity: f64, + refill_per_sec: f64, +} + +impl RateLimiter { + fn new(capacity: f64, refill_per_sec: f64) -> Self { + Self { + buckets: LockMap::new(), + capacity, + refill_per_sec, + } + } + + /// Tries to take one token from `user`'s bucket. + /// + /// The whole read-modify-write below is atomic for this user because the + /// entry guard holds the per-key lock, yet requests from other users + /// proceed in parallel. + fn try_acquire(&self, user: &str) -> bool { + let mut entry = self.buckets.entry_by_ref(user); + let now = Instant::now(); + let bucket = entry.or_insert_with(|| Bucket { + tokens: self.capacity, + last_refill: now, + }); + + let elapsed = now.duration_since(bucket.last_refill).as_secs_f64(); + bucket.tokens = (bucket.tokens + elapsed * self.refill_per_sec).min(self.capacity); + bucket.last_refill = now; + + if bucket.tokens >= 1.0 { + bucket.tokens -= 1.0; + true + } else { + false + } + } +} + +fn main() { + // 5 tokens burst capacity, refills at 10 tokens per second. + let limiter = RateLimiter::new(5.0, 10.0); + + // A burst of 8 requests from alice: 5 pass, 3 are rejected. + let (mut allowed, mut rejected) = (0, 0); + for _ in 0..8 { + if limiter.try_acquire("alice") { + allowed += 1; + } else { + rejected += 1; + } + } + println!("alice burst: {allowed} allowed, {rejected} rejected"); + assert_eq!((allowed, rejected), (5, 3)); + + // bob has his own bucket and is unaffected by alice's burst. + assert!(limiter.try_acquire("bob")); + println!("bob: allowed"); + + // After 300ms alice's bucket has refilled ~3 tokens. + std::thread::sleep(Duration::from_millis(300)); + assert!(limiter.try_acquire("alice")); + println!("alice after refill: allowed"); +} diff --git a/examples/singleflight.rs b/examples/singleflight.rs new file mode 100644 index 0000000..700cc71 --- /dev/null +++ b/examples/singleflight.rs @@ -0,0 +1,56 @@ +//! Request coalescing ("singleflight"): when many threads miss the same cache +//! key simultaneously, only one of them performs the expensive load; the +//! others block on that key's lock — and only that key — then reuse the +//! freshly inserted value. +//! +//! This is the signature use case for per-key locking: a shard-level lock +//! (e.g. `dashmap`) would also stall unrelated keys that happen to live in the +//! same shard, while a global lock would stall the entire cache. +//! +//! Run with: `cargo run --example singleflight` + +use lockmap::LockMap; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::thread; +use std::time::Duration; + +static BACKEND_LOADS: AtomicUsize = AtomicUsize::new(0); + +/// Simulates a slow backend (database, RPC, disk...). +fn load_from_backend(key: &str) -> String { + BACKEND_LOADS.fetch_add(1, Ordering::Relaxed); + thread::sleep(Duration::from_millis(100)); + format!("value-of-{key}") +} + +/// Cache-aside read with request coalescing. +fn get_or_load(cache: &LockMap, key: &str) -> String { + // Fast path: no per-key lock is taken if the value is already cached. + if let Some(value) = cache.get(key) { + return value; + } + + // Slow path: lock this key exclusively. Exactly one thread runs the + // closure; the others wait on the same key and then find the value + // already present, so the backend is hit only once. + let mut entry = cache.entry_by_ref(key); + entry.or_insert_with(|| load_from_backend(key)).clone() +} + +fn main() { + let cache = LockMap::::new(); + + thread::scope(|s| { + for i in 0..8 { + let cache = &cache; + s.spawn(move || { + let value = get_or_load(cache, "hot-key"); + println!("thread {i}: got {value:?}"); + }); + } + }); + + let loads = BACKEND_LOADS.load(Ordering::Relaxed); + println!("backend loads for 8 concurrent requests: {loads}"); + assert_eq!(loads, 1, "the backend must be hit exactly once"); +}