Skip to content

AGENTS.md

Latest commit

 

History

History
156 lines (102 loc) · 11.5 KB

AGENTS.md

File metadata and controls

156 lines (102 loc) · 11.5 KB
Raw

AGENTS.md

This file provides guidance to AI coding agents working in this repository.

Build

mkdir build && cd build
cmake -G Ninja <SOURCE_DIR>
ninja
# or: ninja fdbserver fdbcli fdbclient  (specific targets)

Key cmake options:

  • -DCMAKE_BUILD_TYPE=Debug (default Release)
  • -DUSE_WERROR=ON for development
  • -DOPEN_FOR_IDE=ON to generate IDE project without building

macOS: add -DBUILD_SWIFT_BINDING=OFF if Swift issues arise.

Linux with Clang: CC=clang CXX=clang++ cmake -DUSE_LD=LLD -DUSE_LIBCXX=1 -G Ninja ..

Testing

Unit tests use TEST_CASE("/path/to/test") macros (defined in flow/include/flow/UnitTest.h). Prefer the narrowest dedicated unit-test target instead of fdbserver -r unittests; these targets compile and run faster. Use fdbclient_test for client tests or the relevant fdbserver_<library>_test target defined via add_fdbserver_unit_test (for example, fdbserver_core_test, fdbserver_kvstore_test, or fdbserver_storageserver_test). Find the applicable target in the affected component's CMakeLists.txt:

ninja fdbclient_test
bin/fdbclient_test                                  # all tests in the target
bin/fdbclient_test -f "<test-prefix>"                # tests matching a prefix

ninja fdbserver_core_test
bin/fdbserver_core_test

Use fdbserver for simulation or broader end-to-end validation, not as the default unit-test runner.

Simulation tests use TOML workload definitions in tests/fast/, tests/slow/, etc.:

bin/fdbserver -r simulation -f tests/fast/CycleTest.toml

Via ctest:

ctest -L fast                    # all fast tests
ctest -R "StatusDuringOutage"    # by name pattern

Enable simulation tests in cmake: -DENABLE_SIMULATION_TESTS=ON

Adding new tests:

  • For a new otherwise-unreferenced translation unit containing TEST_CASE, add a forceLinkXxxTests() stub and call it from fdbserver/workloads/UnitTests.cpp, or the linker can drop the tests.
  • Register new .toml or .txt tests under tests/ in tests/CMakeLists.txt via add_fdb_test. configure_testing(... ERROR_ON_ADDITIONAL_FILES) and verify_testing() should catch unregistered files during CMake configuration.

Joshua: bulk correctness/simulation testing runs on the internal Joshua cluster against optimized Release builds. Local fdbserver -r simulation runs are for iteration; large-scale shake-out happens on Joshua.

Trace logs: simulation and test runs emit trace.*.xml (or .json) files in the run directory. TraceEvent(...) is the canonical logging mechanism — search trace files by event name when debugging a simulation failure.

  • A SevError (Severity 40) TraceEvent fails a simulation/Joshua run. Use SevWarnAlways for "shouldn't happen but non-fatal." Errors carrying an injected fault are auto-downgraded; genuine SevErrors are not. Expected errors must be suppressed/allowlisted, never emitted at SevError.
  • Keep a TraceEvent small: any single detail() value over 495 bytes is truncated (...), and an event whose fields total over 4000 bytes is dropped entirely and re-logged as TraceEventOverflow — at SevError in simulation, so an oversized event fails the test. Chunk long payloads (joined lists, command lines, serialized blobs) across multiple events.
  • The Time field is now(), which does not advance between co_awaits, so spans within one run-loop slice read as zero. g_traceBatch events also carry MonotonicTime for this.

Architecture

FoundationDB is a distributed ordered key-value store with strict serializability. The codebase is organized into ~12 subsystems. For background on subsystems before diving into code, the design/ directory holds human-authored design docs and design/AI-generated/ holds subsystem maps and per-subsystem diagrams (start with design/AI-generated/foundationdb_subsystem_map.md).

Concurrency Model: C++ Coroutines

FDB uses cooperative single-threaded concurrency. Asynchronous functions use standard C++ coroutines with co_await and co_return in ordinary .cpp and .h files. See design/coroutines.md for coroutine and cancellation semantics.

Key types: Future<T>, Promise<T>, PromiseStream<T>, Reference<T> (ref-counted pointer), Optional<T>, ErrorOr<T>, Arena (region-based allocation).

Common pitfalls

  • C++ coroutines: co_await is not allowed inside a catch handler. Capture the error, exit the catch, then co_await outside.
  • Errors are integer codes (flow/include/flow/error_definitions.h), not exceptions with messages. When you catch (Error& e), re-throw actor_cancelled (and never silently swallow broken_promise) — eating cancellation causes hangs and leaks. Transaction retry goes through tr.onError(e), not a bare loop.
  • StringRef/KeyRef/ValueRef are non-owning views into an Arena. Returning or storing one past its arena's lifetime is a dangling-reference bug; use Standalone<> (or Key/Value) when you need to own the bytes.
  • a && b on Future<Void> is quorum(x, 2, 2), not a join: the first error completes it and drops the other input, cancelling it if nothing else holds it, and an already-failed lhs discards rhs unreferenced. Never accumulate with f = f && g; use waitForAllReady() to wait for everything regardless of errors.
  • Subspace::range() spans prefix + '\x00' to prefix + '\xff', so it excludes the subspace's own key: a value written at s.pack(x) is not returned by scanning s.get(x).range().

Core Subsystems

  • flow/ — Async runtime, event loop, tracing, deterministic random, arenas
  • fdbrpc/ — Endpoint-addressed RPC, peer management, failure monitor, Sim2 (deterministic simulation network)
  • fdbclient/ — Transaction API (NativeAPI.cpp), read-your-writes (ReadYourWrites.cpp), location cache, multi-version client
  • fdbserver/ — All server roles, organized by subdirectory:
    • clustercontroller/ — Leader election, role recruitment, ServerDBInfo broadcasting
    • coordinator/ — Paxos-based coordination state (generation registers)
    • commitproxy/, grvproxy/, resolver/ — Transaction commit pipeline
    • tlog/, logsystem/ — Durable mutation log (tag-partitioned)
    • storageserver/ — Serves reads, applies mutations from TLogs
    • kvstore/ — Pluggable storage engines (RocksDB, SQLite, memory/DiskQueue)
    • datadistributor/ — Shard management, team building, data movement
    • ratekeeper/ — Back-pressure and throttling
    • workloads/ — Simulation test workloads

Simulation Testing

fdbserver -r simulation runs the entire cluster in a single process using Sim2, a deterministic simulated network. buggify() calls (flow/include/flow/Buggify.h) inject faults (delays, failures, corruption). Tests are TOML files that compose workloads. This is FDB's primary testing strategy.

  • Determinism is per seed and binary. Any code change shifts the deterministicRandom() sequence, so the same seed on a patched binary is a different execution, not an A/B comparison. Reproduce a failure on the unpatched binary, and judge a fix by failure rates over many seeds.
  • A continuation can resume with a different simulated process current than the one that started waiting, e.g. when a future is shared between processes on one simulated machine. Code that binds to g_simulator->getCurrentProcess() should capture it on entry and co_await g_simulator->onProcess(...) back before binding.

Knobs

Runtime-tunable parameters are declared in fdbserver/core/include/fdbserver/core/Knobs.h, fdbclient/include/fdbclient/Knobs.h, and flow/include/flow/Knobs.h (UPPER_CASE names). Access them via SERVER_KNOBS, CLIENT_KNOBS, or FLOW_KNOBS. Defaults and buggification ranges live in fdbserver/core/ServerKnobs.cpp, fdbclient/ClientKnobs.cpp, and flow/Knobs.cpp.

Compatibility

Before changing a serialized type that persists on disk, inspect its serializer() and adjacent versioning or downgrade comments. Do not reorder, remove, or change encoded fields without the appropriate version gate or migration and compatibility coverage.

Naming Conventions

  • Variables, functions, methods: camelCase (smoothTotal, getWorkerForRole)
  • Classes, structs, enums: PascalCase (StorageServer, RecoveryState)
  • Knobs/constants: UPPER_CASE (MAX_BATCH_SIZE, VERSIONS_PER_SECOND)
  • Globals: g_ prefix for important ones (g_network, g_simulator)

Code Formatting

clang-format is used. Python code uses black and flake8 (pre-commit hooks: pip install pre-commit && pre-commit install).

Edit tracked .cpp and .h sources, not generated output under the build directory.

Source File Headers

Every new .cpp / .h file starts with the standard Apache 2.0 license block, with the filename on line 2 and the current year on the copyright line. Copy from any existing file in the tree (e.g. flow/Knobs.cpp). Add file-purpose comments after the license block, not in place of it.

Code Comments

General guidance: comments should maximize clarity. They should be clear without being verbose. They should be concise without being cryptic.

Scope: the guidance here mainly governs inline and implementation comments. Comments on public-facing APIs are contract statements and must be clear, precise, and complete.

Audience: assume the readers of FDB are senior engineers fluent in asynchronous and concurrent programming, distributed systems, and stateful complex systems generally. Calibrate "non-obvious" to that reader — do not explain the language, the standard library, Flow/actor idioms, or established patterns they already know. Reserve comments for what such a reader could not quickly infer from the code itself.

Implementation comments should focus on important, non-obvious fundamental rationale, motivation, and explanation of extremely subtle phenomena. Writing down important invariants — and the preconditions a piece of code requires of its caller (e.g. a lock that must be held, or a version that must already be durable) — is also a good idea.

Do not write comments about mundane day-to-day development refactorings — junk like // This used to be part of xyzFunc in Foo.cpp but we moved it here so that aaaFunc could also call it.

Do not write caller-location / "happening elsewhere" comments — narration of who invokes this code or where it is used ("called from X", "used by Y"). They duplicate what a search answers instantly and rot when callers change; describe what the code requires and guarantees on its own terms. Stating a real invariant is fine — "recovery is the only path that repairs" — naming the callers that happen to exist today is not.

After writing or changing a bunch of code, make a pass over it one last time to review the comments. Ask yourself: am I rambling? Am I repeating what the code is showing? Am I just telling stories here? Is the information I'm writing going to be relevant and important a week from now, a month from now, a year from now? Is it genuinely essential and non-obvious from the code itself?

Code Review

When reviewing, state explicitly: what the change is for, whether it is correct, bugs, omissions, better approaches, and a clear LGTM / no / not-yet. Read a construct's history (git log -L, git blame) before proposing to change something that looks odd.

Branching

PRs target main. Release branches receive cherry-picks rather than direct PRs — don't open backport PRs without confirming first.

Design Docs

Before a design doc is considered finished (not at the start), reconcile it against design/design-doc-template.md.