A resource-optimized, full-stack Retrieval-Augmented Generation (RAG) system that serves as a lore archivist for Brandon Sanderson's Cosmere universe.
The system ingests and indexes 4,740 documents and 19,819 passages (over 2.68M words) crawled from the Coppermind wiki, performing low-latency semantic search and answering user questions in an in-universe scholarly archivist persona powered by Google Gemini. The entire backend is designed to run within 512MB RAM free-tier cloud container limits.
All metrics below are measured directly from the indexed Cosmere corpus on an Apple Silicon (M-series) environment and are reproducible via benchmark_suite.py and benchmark_retrieval.py:
| Metric | Measured Value | Measurement Details & Scope |
|---|---|---|
| Indexed Lore Knowledge Base | 4,740 docs • 19,819 chunks • 2.68M words | Exact count from meta.jsonl (median chunk: 578 chars, p95: 2,228 chars) |
| Metadata Table Heap Footprint | 0.70 MB RAM (vs 25.32 MB for parsed JSON) | Measured via tracemalloc for 64-bit offsets in LineIndexedJSONL (47.4 bytes/record) |
| Metadata File Seek Latency | 0.007 ms p50 • 0.011 ms p95 | Direct |
| Query Embedding Latency (CPU) | 9.79 ms p50 • 12.60 ms p95 | Dense vectorization using all-MiniLM-L6-v2
|
| Vector Search Latency (FAISS) | 0.81 ms p50 • 0.99 ms p95 (top-5) | Exact inner-product scan on 19,819 normalized 384-d vectors (IndexFlatIP) |
| Vector Search vs NumPy BLAS | 0.81 ms vs 1.03 ms (~21% lower latency) | FAISS C++ SIMD scan compared against vectorized NumPy matmul (vectors @ q) |
| Cross-Encoder Rerank Latency | 118.69 ms p50 • 164.25 ms p95 (20 pairs) | Optional ms-marco-MiniLM-L-6-v2 re-scoring (disabled by default) |
| Retrieval Accuracy (Eval Set) | 80.0% Recall@5 • 0.585 MRR@10 | Measured on 20 hand-labeled domain queries (Recall@1: 45%, Recall@3: 65%) |
| Container RAM Budget |
512 MB cap (-m 512m) |
CPU-only PyTorch, OMP_NUM_THREADS=1, single Uvicorn worker |
Note
- Process Memory vs Heap: The
0.70 MBmetric is the Python heap footprint of the byte-offset lookup table. The overall running backend process (including the Python runtime, PyTorch CPU libraries, and FAISS C++ binaries) consumes ~150 MB – 220 MB of Resident Set Size (RSS), fitting safely within the 512MB container limit. - End-to-End Latency: Vector search (0.81 ms) and metadata lookup (0.007 ms) take under 1 ms combined. Total user-perceived query latency is dominated by local CPU query embedding (~10 ms) and the external Google Gemini API round-trip over HTTPS (~800 ms – 2,500 ms depending on network conditions and output token length).
- Header-Aware Chunking Pipeline: Crawled raw articles split by Markdown
##headings, then token-bounded into 500-token chunks with 100-token sliding overlap viatiktoken. - MD5 Deduplication: Deduplication pass eliminates identical text passages across articles.
- Custom Byte-Offset Metadata Engine (
LineIndexedJSONL): Replaces heavy in-memory JSON dictionaries with a 64-bit file byte-offset table, slashing metadata RAM usage by over 96% with thread-safe sub-millisecond on-demand disk seeks. - Memory-Mapped Vector Search: 384-dimensional dense embeddings (
all-MiniLM-L6-v2) indexed with FAISSIndexFlatIPusingIO_FLAG_MMAPto prevent duplicating index memory in user space. - Two-Stage Retrieval Pipeline: Fast vector similarity search with optional Cross-Encoder reranking (
cross-encoder/ms-marco-MiniLM-L-6-v2) toggled via environment configuration (ENABLE_RERANKER=true). - Persona-Driven Generative Q&A: Formats context into prompts for Google Gemini (
gemini-2.5-flash-lite) adopting the scholarly "CosmereArchivist" voice, with dynamic context fallbacks when query relevance is low. - 512MB RAM Container Engineering: Containerized using Docker with CPU-only PyTorch, pre-baked model weights, single-threaded BLAS/OMP caps, and single-worker Uvicorn to fit strictly within 512MB cloud free tiers.
- Modern Full-Stack UI: Responsive Next.js 15 interface with React 19, Tailwind CSS v4, and cold-boot status tracking / auto-retries for container spin-up.
- Backend: Python 3.13 • FastAPI • Uvicorn • Pydantic v2
- Vector Search & ML: FAISS (
faiss-cpu) • SentenceTransformers (all-MiniLM-L6-v2) • CrossEncoder (ms-marco-MiniLM-L-6-v2) • PyTorch (CPU-only) - Generative AI: Google Gemini API (
gemini-2.5-flash-liteviagoogle-generativeai) - Frontend: Next.js 15 • React 19 • Tailwind CSS v4
- Infrastructure & Testing: Docker • Vercel (Frontend) • Render (Backend) • Pytest
[ Crawled Wiki Docs (4,740) ]
│
▼
[ Ingestion & Header-Aware Chunking ] ──► [ MD5 Deduplication ] (19,819 chunks)
│
├──► Embeddings (all-MiniLM-L6-v2) ──► FAISS Index (IndexFlatIP, MMAP)
│
└──► Metadata (LineIndexedJSONL) ──► 64-bit Byte Offset Table (<1 MB heap)
-
User Query: The Next.js chat frontend sends
POST /askwith the question. - Embedding: Backend embeds the query text into a normalized 384-dimensional vector on CPU (~10 ms).
- Candidate Search: FAISS performs fast inner-product search across 19,819 vectors (~0.8 ms).
-
Metadata Lookup:
LineIndexedJSONLexecutes$O(1)$ file seeks to fetch chunk text for the candidate IDs (~0.01 ms). -
Optional Reranking: If
ENABLE_RERANKER=true, candidates are scored via Cross-Encoder (~120 ms). -
Prompt Assembly & Generation: Chunks are formatted into a prompt and dispatched to
gemini-2.5-flash-liteover HTTPS. - Response: Generated in-universe archivist response is returned to the user.
FastAPI server serves the following endpoints (with Swagger UI at http://localhost:8000/docs):
Runs full RAG pipeline (search → optional rerank → generate answer via Gemini).
- Request Body:
{ "question": "Who is Kaladin Stormblessed?" } - Response:
{ "answer": "Kaladin Stormblessed is a Windrunner of the Knights Radiant..." }
Retrieves top-k candidate chunks without invoking the LLM generation step.
- Request Body:
{ "question": "What are the Ten Surges?" } - Response:
{ "chunks": [ { "doc_id": "Surgebinding", "chunk_index": 2, "text": "The Ten Surges are fundamental forces..." } ] }
- Python 3.11+ (Python 3.13 recommended)
- Node.js 18+ and npm
- A Google Gemini API Key
git clone https://github.com/AJ576/loreGPT.git
cd loreGPTcd backend
# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Set your Gemini API key
export GEMINI_API_KEY="your-gemini-api-key"
# Or create backend/.env with: GEMINI_API_KEY=your-gemini-api-key
# Start the FastAPI server
uvicorn src.api.app:app --reload --port 8000The API is now running at http://localhost:8000.
In a new terminal window:
cd frontend
# Install npm packages
npm install
# Start development server
npm run devOpen http://localhost:3000 to interact with the Cosmere Archivist interface.
All test suites and benchmarks run without external server dependencies.
# From backend directory
pytest tests
# Or skip the slow embedding model download test
pytest tests -m "not slow"Measures metadata memory footprint, disk seek times, query embedding latency, and FAISS vector retrieval:
python3 backend/benchmarks/benchmark_suite.py
# Optional flags:
# --rerank include Cross-Encoder latency
# --llm include real Gemini API calls (uses API quota)
# --quality include retrieval hit-rate evaluationResults are saved to backend/benchmarks/measured_results.json.
Evaluates Hit Rate / Recall @ 1, 3, 5, 10 and MRR @ 10 on 20 curated Cosmere queries:
python3 backend/benchmarks/benchmark_retrieval.py
# With cross-encoder reranking:
python3 backend/benchmarks/benchmark_retrieval.py --rerankMeasures throughput and latency percentiles under concurrent client loads:
# Make sure backend is running on port 8000 first:
# uvicorn src.api.app:app --port 8000
python3 backend/benchmarks/benchmark_api.pyTo build and run the backend container locally matching the 512MB RAM production profile:
cd backend
# Build Docker image
docker build -t cosmere-archivist-backend .
# Run container with 512MB memory limit
docker run -p 8000:8000 -m 512m --memory-swap 512m -e GEMINI_API_KEY="your-api-key" cosmere-archivist-backendThis project is licensed under the MIT License. Fictional Cosmere lore content is based on the works of Brandon Sanderson, documented by the community at Coppermind.net.