Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 33 additions & 31 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ license = "Apache-2.0"
publish = false
repository = "https://github.com/bunkerlab-net/graphify"
rust-version = "1.95"
version = "0.9.0"
version = "0.9.2"

[workspace.dependencies]
anyhow = "1"
Expand Down
7 changes: 7 additions & 0 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,10 @@ Save a Q&A result back into `graphify-out/memory/` so it gets re-extracted into
Pass `--outcome useful|dead_end|corrected` (and `--correction "<what worked>"` for the `corrected` case) to record
a work-memory signal that `graphify reflect` later aggregates into `LESSONS.md`. An out-of-set `--outcome` is rejected.

The answer can be passed inline with `--answer "<text>"` or read from a file with `--answer-file <path>`; the
latter avoids fragile shell quoting for long or multiline answers (#1502). At least one is required, and
`--answer-file` takes precedence when both are given.

```bash
graphify save-result \
--question "how is auth scoped" \
Expand Down Expand Up @@ -703,6 +707,9 @@ completes the feature.)
| `GRAPHIFY_CLUSTER_BACKEND` | `leiden` (default) or `louvain` to force the fallback. |
| `GRAPHIFY_ALLOW_LOCAL_PROVIDERS` | Opt in to loading a project-local `.graphify/providers.json` (ignored by default; see Custom providers). |
| `OLLAMA_BASE_URL` | Ollama endpoint (default `http://localhost:11434/v1`); a link-local/cloud-metadata host is refused, a general non-loopback host warns. |
| `GRAPHIFY_API_TIMEOUT` | LLM HTTP request timeout in seconds (default 600); bounds a runaway connection during semantic extraction. |
| `GRAPHIFY_MAX_RETRIES` | Times a rate-limited (HTTP 429) LLM request is retried before its chunk is dropped (default 6); `0` disables retries (#1523). |
| `GRAPHIFY_RETRY_BASE_MS` | Base backoff delay in milliseconds for the 429 retry path; the wait grows exponentially per attempt (default 500). `0` disables the sleep (#1523). |

### LLM backends

Expand Down
4 changes: 3 additions & 1 deletion crates/graphify-cache/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ mod store;
pub use error::CacheError;
pub use hash::{body_content, file_hash};
pub use paths::{EXTRACTOR_VERSION, cache_dir, cache_dir_versioned};
pub use semantic::{SemanticCacheSplit, check_semantic_cache, save_semantic_cache};
pub use semantic::{
SemanticCacheSplit, check_semantic_cache, prune_semantic_cache, save_semantic_cache,
};
pub use stat_index::{
_reset_stat_index_for_tests, ensure_atexit_flush_registered, flush_stat_index,
};
Expand Down
48 changes: 48 additions & 0 deletions crates/graphify-cache/src/semantic.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
//! Semantic-extraction cache: per-source-file nodes/edges/hyperedges.

use std::collections::HashSet;
use std::path::{Path, PathBuf};

use indexmap::IndexMap;
Expand Down Expand Up @@ -121,3 +122,50 @@ pub fn save_semantic_cache(
}
Ok(saved)
}

/// Remove orphaned semantic cache entries, returning the count pruned.
///
/// The semantic cache is content-hash-keyed (`{file_hash}.json` under
/// `cache/semantic/`) and deliberately UNVERSIONED — entries are produced by the
/// LLM from file contents, so invalidating them on every release would re-bill
/// extraction. Because it is unversioned it is never swept by the AST
/// version-cleanup, so every content change or file deletion leaves a permanent
/// orphan that accumulates unbounded (#1527).
///
/// This sweeps `cache/semantic/*.json` and deletes any entry whose stem (the
/// content hash) is not in `live_hashes` — the hashes of the current live
/// document set. `*.tmp` atomic-write temporaries are skipped, and only this
/// directory is touched (never `cache/ast/**`). Best-effort: each unlink failure
/// is ignored (worst case is a surviving orphan, never wrong output). Mirrors
/// Python `prune_semantic_cache`.
#[must_use]
pub fn prune_semantic_cache<S: std::hash::BuildHasher>(
root: &Path,
live_hashes: &HashSet<String, S>,
) -> usize {
let semantic_dir = crate::paths::out_base(root).join("cache").join("semantic");
if !semantic_dir.is_dir() {
return 0;
}
let Ok(entries) = std::fs::read_dir(&semantic_dir) else {
return 0;
};
let mut pruned = 0;
for entry in entries.flatten() {
let path = entry.path();
// Only `*.json`; `*.tmp` atomic-write temporaries are left untouched.
if path.extension().and_then(|e| e.to_str()) != Some("json") {
continue;
}
let Some(stem) = path.file_stem().and_then(|s| s.to_str()) else {
continue;
};
if live_hashes.contains(stem) {
continue;
}
if std::fs::remove_file(&path).is_ok() {
pruned += 1;
}
}
pruned
}
Loading
Loading