Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,14 @@ jobs:
- name: Test
run: cargo test --target ${{ matrix.target }}

- name: Run examples
shell: bash
run: |
set -euo pipefail
for example in singleflight rate_limiter lru_session_store; do
cargo run --example "$example" --target ${{ matrix.target }}
done

coverage:
name: Code Coverage
runs-on: ubuntu-latest
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Declare `rust-version` (MSRV 1.75) and crates.io `keywords` / `categories` metadata
- Pin dev-dependency versions (`criterion = "0.8"`, `rand = "0.10"`)

### Documentation

- Add runnable examples: `singleflight` (request coalescing), `rate_limiter`
(per-key token bucket) and `lru_session_store` (bounded session cache)
- README: add installation/MSRV section, capability comparison matrix,
non-reentrancy caveat and explicit dual-license statement

### CI

- Run Miri on the whole library test suite (`cargo miri test --lib`)
Expand Down
66 changes: 63 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,23 @@

A high-performance, thread-safe HashMap and LRU cache for Rust with **fine-grained per-key locking**.

Correctness is taken seriously: every commit is verified in CI with **Miri** and **ThreadSanitizer** over the full concurrent test suite, plus an MSRV check, `clippy -D warnings`, and tests across five OS/architecture targets.

## Installation

```bash
cargo add lockmap
```

Or add it to your `Cargo.toml` manually:

```toml
[dependencies]
lockmap = "0.2"
```

The minimum supported Rust version (MSRV) is **1.75**, verified in CI.

## Data Structures

| Type | Description |
Expand All @@ -29,7 +46,7 @@ Unlike standard concurrent maps that might lock the entire map or large buckets,
* **Deadlock Prevention**: Provides `batch_lock` to safely acquire locks on multiple keys simultaneously using a deterministic order.
* **Non-Blocking Locking**: `try_entry` / `try_entry_by_ref` return `None` instead of blocking when a key is already held.
* **Iteration**: `for_each` / `retain` visit all entries shard by shard, without a global lock.
* **Pluggable Hasher**: both maps accept a custom `BuildHasher` via `with_hasher` constructors (default: `foldhash`).
* **Pluggable Hasher**: Both maps accept a custom `BuildHasher` via `with_hasher` constructors (default: `foldhash`).
* **Single Hash Computation**: Each key is hashed once; the pre-computed hash is stored alongside the key and reused for shard selection, table probing, and rehashing.
* **No Key Duplication**: Uses `hashbrown::HashTable` so each key is stored only once, inside the entry state.
* **Entry API**: Ergonomic unified RAII guard (`Entry`) for managing locks.
Expand Down Expand Up @@ -132,6 +149,35 @@ assert_eq!(cache.pop_lru(), Some(("key".to_string(), "new_value".to_string())));
// entries are automatically evicted.
```

## Examples

Runnable, self-checking examples live in [`examples/`](examples/):

| Example | What it shows |
|---------|---------------|
| [`singleflight.rs`](examples/singleflight.rs) | Request coalescing: 8 threads miss the same key, the backend is hit exactly once |
| [`rate_limiter.rs`](examples/rate_limiter.rs) | Per-user token-bucket rate limiter with atomic refill-and-consume per key |
| [`lru_session_store.rs`](examples/lru_session_store.rs) | Bounded session store using LRU eviction, `peek` and `pop_lru` |

```bash
cargo run --example singleflight
```

## Comparison with Alternatives

Capability matrix (see the [Benchmarks](#benchmarks) section for performance trade-offs):

| Capability | `lockmap` | `dashmap` | `moka` |
|------------|-----------|-----------|--------|
| Exclusive lock granularity | per key | per shard | no lock API |
| Holding a guard blocks | that key only | the whole shard | — |
| Locking a not-yet-existing key | ✅ | ✅ (holds the shard) | ❌ |
| Deadlock-safe multi-key locking | ✅ `batch_lock` | ❌ | ❌ |
| Bounded capacity with eviction | ✅ per-shard LRU | ❌ | ✅ TinyLFU |
| Non-promoting read / explicit pop | ✅ `peek` / `pop_lru` | — | ❌ |

Rule of thumb: pick `lockmap` when you need **exclusive per-key critical sections** (read-modify-write, request coalescing, per-key state machines) or a **throughput-oriented bounded cache**; pick `dashmap` for a general concurrent map with short operations; pick `moka` when **cache hit rate** is the primary concern.

## Benchmarks

The repository ships two Criterion benchmark suites:
Expand All @@ -152,12 +198,17 @@ run the suite on your own hardware and workload before drawing conclusions.

## Important Caveats

### 1. No Lock Poisoning
### 1. Locks Are Not Reentrant

Calling any map operation for a key **while already holding that key's `Entry`** deadlocks — this includes `get`, `insert`, `remove`, `entry`, and whole-map operations such as `clear`, `for_each` and `retain`.
> **Note**: Drop the guard first, or use `try_entry` / `try_entry_by_ref` when you cannot statically rule out re-entry.
Comment on lines +203 to +204

### 2. No Lock Poisoning

Unlike `std::sync::Mutex`, **this library does not implement lock poisoning**. If a thread panics while holding an `Entry`, the lock is released immediately (via Drop) to avoid deadlocks, but the data is **not** marked as poisoned.
> **Warning**: Users must ensure exception safety. If a panic occurs during a partial update, the data associated with that key may be left in an inconsistent state for subsequent readers.

### 2. `get()` Performance
### 3. `get()` Performance

The `map.get(key)` method clones the value while holding an internal shard lock.
> **Note**: If your value type `V` is expensive to clone (e.g., deep copy of large structures), or if `clone()` acquires other locks, use `map.entry(key).get()` instead. This moves the clone operation outside the internal map lock, preventing blocking of other threads accessing the same shard.
Expand All @@ -168,4 +219,13 @@ See [CHANGELOG.md](CHANGELOG.md) for release notes and migration guides.

## License

Licensed under either of

* Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or <http://www.apache.org/licenses/LICENSE-2.0>)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or <http://opensource.org/licenses/MIT>)

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2FSF-Zhou%2Flockmap.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2FSF-Zhou%2Flockmap?ref=badge_large)
66 changes: 66 additions & 0 deletions examples/lru_session_store.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
//! Bounded session store built on `LruLockMap`.
//!
//! - Capacity-bounded: the least recently used session is evicted
//! automatically when the store is full.
//! - `peek` inspects a session without marking it as recently used.
//! - `pop_lru` explicitly reclaims the coldest session.
//! - Sessions held by an entry guard are never evicted, even at capacity.
//!
//! Run with: `cargo run --example lru_session_store`

use lockmap::LruLockMap;

#[derive(Clone, Debug)]
struct Session {
user: String,
hits: u32,
}

fn main() {
// A tiny single-shard store so the eviction order is easy to follow.
let sessions = LruLockMap::<u64, Session>::with_options(3, 3, 1);

for id in 1..=3u64 {
sessions.insert(
id,
Session {
user: format!("user-{id}"),
hits: 0,
},
);
}

// Touch session 1 through the entry API: this promotes it to
// most-recently-used and updates it atomically.
{
let mut entry = sessions.entry(1);
if let Some(session) = entry.get_mut() {
session.hits += 1;
}
}

// Inserting a 4th session evicts the LRU one. Session 2 is now the
// coldest (session 1 was just promoted), so it is the one that goes.
sessions.insert(
4,
Session {
user: "user-4".into(),
hits: 0,
},
);
assert!(sessions.peek(&2).is_none(), "session 2 was evicted");
assert!(sessions.peek(&1).is_some(), "session 1 survived (promoted)");

// `peek` does not disturb the LRU order, so it is safe for monitoring.
for id in [1u64, 3, 4] {
if let Some(session) = sessions.peek(&id) {
println!("session {id}: {session:?}");
}
}

// Explicitly drain the coldest sessions, e.g. on memory pressure.
while let Some((id, session)) = sessions.pop_lru() {
println!("reclaimed session {id} of {}", session.user);
}
assert!(sessions.is_empty());
}
82 changes: 82 additions & 0 deletions examples/rate_limiter.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
//! Per-key token-bucket rate limiter.
//!
//! Each user owns an independent bucket protected by its own lock, so the
//! refill-and-consume critical section of one user never contends with
//! another user's requests.
//!
//! Run with: `cargo run --example rate_limiter`

use lockmap::LockMap;
use std::time::{Duration, Instant};

struct Bucket {
tokens: f64,
last_refill: Instant,
}

struct RateLimiter {
buckets: LockMap<String, Bucket>,
capacity: f64,
refill_per_sec: f64,
}

impl RateLimiter {
fn new(capacity: f64, refill_per_sec: f64) -> Self {
Self {
buckets: LockMap::new(),
capacity,
refill_per_sec,
}
}

/// Tries to take one token from `user`'s bucket.
///
/// The whole read-modify-write below is atomic for this user because the
/// entry guard holds the per-key lock, yet requests from other users
/// proceed in parallel.
fn try_acquire(&self, user: &str) -> bool {
let mut entry = self.buckets.entry_by_ref(user);
let now = Instant::now();
let bucket = entry.or_insert_with(|| Bucket {
tokens: self.capacity,
last_refill: now,
});

let elapsed = now.duration_since(bucket.last_refill).as_secs_f64();
bucket.tokens = (bucket.tokens + elapsed * self.refill_per_sec).min(self.capacity);
bucket.last_refill = now;

if bucket.tokens >= 1.0 {
bucket.tokens -= 1.0;
true
} else {
false
}
}
}

fn main() {
// 5 tokens burst capacity, refills at 10 tokens per second.
let limiter = RateLimiter::new(5.0, 10.0);

// A burst of 8 requests from alice: 5 pass, 3 are rejected.
let (mut allowed, mut rejected) = (0, 0);
for _ in 0..8 {
if limiter.try_acquire("alice") {
allowed += 1;
} else {
rejected += 1;
}
}
println!("alice burst: {allowed} allowed, {rejected} rejected");
assert_eq!((allowed, rejected), (5, 3));

// bob has his own bucket and is unaffected by alice's burst.
assert!(limiter.try_acquire("bob"));
println!("bob: allowed");

// After 300ms alice's bucket has refilled ~3 tokens.
std::thread::sleep(Duration::from_millis(300));
assert!(limiter.try_acquire("alice"));
println!("alice after refill: allowed");
}
56 changes: 56 additions & 0 deletions examples/singleflight.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
//! Request coalescing ("singleflight"): when many threads miss the same cache
//! key simultaneously, only one of them performs the expensive load; the
//! others block on that key's lock — and only that key — then reuse the
//! freshly inserted value.
//!
//! This is the signature use case for per-key locking: a shard-level lock
//! (e.g. `dashmap`) would also stall unrelated keys that happen to live in the
//! same shard, while a global lock would stall the entire cache.
//!
//! Run with: `cargo run --example singleflight`

use lockmap::LockMap;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::thread;
use std::time::Duration;

static BACKEND_LOADS: AtomicUsize = AtomicUsize::new(0);

/// Simulates a slow backend (database, RPC, disk...).
fn load_from_backend(key: &str) -> String {
BACKEND_LOADS.fetch_add(1, Ordering::Relaxed);
thread::sleep(Duration::from_millis(100));
format!("value-of-{key}")
}

/// Cache-aside read with request coalescing.
fn get_or_load(cache: &LockMap<String, String>, key: &str) -> String {
// Fast path: no per-key lock is taken if the value is already cached.
if let Some(value) = cache.get(key) {
return value;
}

// Slow path: lock this key exclusively. Exactly one thread runs the
// closure; the others wait on the same key and then find the value
// already present, so the backend is hit only once.
let mut entry = cache.entry_by_ref(key);
entry.or_insert_with(|| load_from_backend(key)).clone()
}

fn main() {
let cache = LockMap::<String, String>::new();

thread::scope(|s| {
for i in 0..8 {
let cache = &cache;
s.spawn(move || {
let value = get_or_load(cache, "hot-key");
println!("thread {i}: got {value:?}");
});
}
});

let loads = BACKEND_LOADS.load(Ordering::Relaxed);
println!("backend loads for 8 concurrent requests: {loads}");
assert_eq!(loads, 1, "the backend must be hit exactly once");
}
Loading