Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ The latest release is

**New features**

- Assistants searching your history through MCP can now ask for sessions that
covered several ideas at once, such as an auth change and its retry policy,
even when they came up in different parts of the conversation. Pass two to
five `concepts` to `search_content`. Each result shows the passage that
matched each idea. This needs the semantic search index.

- Hosted operators can retry failed raw parse jobs after fixing their cause with
`pg raw-reparse --repair`. Each bounded batch gives selected failed jobs at
this executable's processing version a fresh attempt budget.
Expand Down
12 changes: 12 additions & 0 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,18 @@ without it they return a "not available" error. Exact `session_id`,
before the final limit. Limits default to 10 and go up to 50; a value outside
that range falls back to the default.

To find one past session that covered several ideas, pass `concepts` instead of
`pattern`: two to five distinct natural-language strings. The mode defaults to
`semantic`, and any other explicit mode is rejected. AgentsView searches each
concept separately and keeps only sessions with evidence for all of them, even
when different exchanges carry that evidence. Results rank by the mean of each
concept's strongest score. Each match lists `concept_evidence` in request order:
the concept, its snippet, score, ordinal, `ordinal_range`, and
`transcript_revision`. Concept searches do not page, so `cursor` is rejected.
`candidate_truncated` is true when a session may be missing: one concept's
candidate search filled up or stopped at its fetch cap, or more sessions matched
than the limit allowed.

A `terms` snippet shows about 60 characters of context around the first
occurrence of each term. Terms that sit far apart in a long exchange produce
separate windows joined by `...`, so snippet size follows the number of terms,
Expand Down
33 changes: 29 additions & 4 deletions docs/semantic-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -829,6 +829,30 @@ falls back to the default. Responses state the effective mode and scope and
which default exclusions applied; `next_cursor` is present when another page
exists.

### MCP multi-concept semantic search

Pass `concepts` with two to five distinct natural-language ideas when recall
must find one session that covers all of them. Do not also pass `pattern`.
`mode` defaults to `semantic`; an explicit mode must also be `semantic`. The
HTTP API takes the same list as repeated `concepts` query parameters and
requires the `X-AgentsView-Search-Intent: semantic` header.

AgentsView runs one semantic search per concept and keeps only the sessions
every search found. The concepts may match different exchanges. A session's
score is the mean of its best score for each concept; equal means keep the order
of the per-concept rankings. Every result includes `concept_evidence` with the
supporting snippet, score, and conversation-unit range for each concept.

Project, branch, agent, session, date, scope, one-shot, automated-session, and
current-session filters apply to every concept search before the intersection.
Each concept search considers five times the limit, at most 250 candidates.
When filters leave fewer candidates than that, the concept search fetches again
with twice as many vector candidates, up to the same 4096-chunk cap described
under [Limitations](#limitations). A session that one concept ranks outside its
window is dropped from the intersection. The response sets
`candidate_truncated` whenever a window fills, the cap stops a concept search
before the vector index runs out, or more sessions match than the limit.

### Inline context: `--context N`

```bash
Expand Down Expand Up @@ -1068,10 +1092,11 @@ keyword leg is the mirror's recency-ordered term match, as on PostgreSQL.
When the recent-activity cutoff rejects otherwise eligible hits, semantic
and hybrid searches fetch again with twice as many candidates. They start at
`max(4 * limit, 200)` and stop at 4096 chunks on every store, which allows
at most six fetches. Other queries use a single candidate batch. Narrow
filters or scope can still return fewer than `--limit` results when eligible
matches lie beyond that budget. The hybrid keyword leg keeps its own bounded
search budget.
at most six fetches. Multi-concept searches expand the same way whenever
filters leave a concept short. Other queries use a single candidate batch.
Narrow filters or scope can still return fewer than `--limit` results when
eligible matches lie beyond that budget. The hybrid keyword leg keeps its own
bounded search budget.
- **Legacy no-`source_uuid` rows re-embed on ordinal shifts.** Each embedded
document is keyed by its first message's stable per-message UUID when the
parser recorded one, or by `(session_id, ordinal)` when it didn't.
Expand Down
16 changes: 16 additions & 0 deletions frontend/src/lib/api/generated/models/dbConceptEvidence.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions frontend/src/lib/api/generated/models/dbContentMatch.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions frontend/src/lib/api/generated/models/index.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions frontend/src/lib/api/generated/search/search.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

60 changes: 50 additions & 10 deletions internal/apiclient/client.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 43 additions & 3 deletions internal/db/search_content.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,22 @@ type ContentSearchFilter struct {

Limit int
Cursor int

// Candidates, when set, makes mode "semantic" and "hybrid" keep
// fetching vector candidates while fewer than Limit units pass the
// filters, until the vector index runs out or the candidate cap stops
// the search. It records which of the two happened. Multi-concept
// search sets it on each concept's search to tell a complete short
// result from a truncated one.
Candidates *VectorCandidates
}

// VectorCandidates reports how a semantic search's vector candidate fetch
// ended.
type VectorCandidates struct {
// Exhausted is true when the vector index had no candidates beyond the
// ones fetched, so every eligible unit was considered.
Exhausted bool
}

// ContentMatch is one matching message or tool call. Snippet is built from the
Expand Down Expand Up @@ -112,6 +128,23 @@ type ContentMatch struct {
// the storage query that produced this evidence. An empty value means the
// backend cannot provide revision-bound evidence for this result.
TranscriptRevision string `json:"transcript_revision,omitempty"`
// ConceptEvidence holds each requested concept's strongest match in this
// session, in request order. It is set only by multi-concept searches.
ConceptEvidence []ConceptEvidence `json:"concept_evidence,omitempty"`
}

// ConceptEvidence is one concept's strongest semantic match in a session
// returned by a multi-concept search.
type ConceptEvidence struct {
Concept string `json:"concept"`
Ordinal int `json:"ordinal"`
OrdinalRange [2]int `json:"ordinal_range"`
Snippet string `json:"snippet"`
Score float64 `json:"score"`
// TranscriptRevision is the revision this concept's search observed. Each
// concept is a separate query, so it can differ from the match's revision
// when the session changed between them.
TranscriptRevision string `json:"transcript_revision,omitempty"`
}

// ContentSearchPage is a page of matches with an optional next cursor.
Expand Down Expand Up @@ -1138,7 +1171,8 @@ func classifyFTSError(err error) error {
const SemanticOverfetchMin = 200

// maxActivityExpansionCandidates caps the candidate count when an activity
// cutoff makes SelectSurvivingVectorHits double its fetch. Starting from
// cutoff or ContentSearchFilter.Candidates makes SelectSurvivingVectorHits
// double its fetch. Starting from
// SemanticOverfetchMin, the cap allows at most six fetches. It equals
// sqlite-vec's KNN ceiling, and it also bounds the KNN and hydration work
// that PostgreSQL and ClickHouse would otherwise do for large limits.
Expand All @@ -1147,7 +1181,9 @@ const maxActivityExpansionCandidates = 4096
// SelectSurvivingVectorHits returns the ranked vector hits that pass the
// filter's session scope. When an activity cutoff rejects hits and leaves
// fewer than f.Limit units, it fetches again with twice the candidates, until
// the engine is exhausted or the candidate cap is reached. allowedSessions
// the engine is exhausted or the candidate cap is reached. When
// f.Candidates is set it expands the same way whenever fewer than f.Limit
// units survive, and records whether the engine was exhausted. allowedSessions
// returns an entry for each otherwise eligible session; false marks a
// session that the activity cutoff rejects.
func SelectSurvivingVectorHits(
Expand Down Expand Up @@ -1175,8 +1211,12 @@ func SelectSurvivingVectorHits(
if err := ctx.Err(); err != nil {
return nil, err
}
if f.Candidates != nil {
f.Candidates.Exhausted = exhausted
}
next := clampRecallVectorCandidates(k*2, ceiling)
if !activityRejected || units >= f.Limit || exhausted || next <= k {
expand := activityRejected || f.Candidates != nil
if !expand || units >= f.Limit || exhausted || next <= k {
return surviving, nil
}
k = next
Expand Down
1 change: 1 addition & 0 deletions internal/mcp/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ func newServer(opts ServeOptions) *mcp.Server {
"prior work and answering contextual questions, especially when the exact wording is unknown. " +
"If these modes report not available, use terms when every literal term must occur within one " +
"user/assistant exchange, or substring/regex for exact error messages, identifiers, and code fragments. " +
"Use concepts with two to five semantic ideas to keep only sessions with evidence for every idea. " +
"The default mode remains substring. Set context to include N messages of " +
"surrounding conversation with each match. Matches from the last 10 minutes (including the " +
"current conversation) are excluded unless include_active is set; current_session_id replaces " +
Expand Down
Loading
Loading