diff --git a/docs/changelog.md b/docs/changelog.md index b8cf62239d..5a0f5b9a17 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -11,6 +11,12 @@ The latest release is **New features** +- Assistants searching your history through MCP can now ask for sessions that + covered several ideas at once, such as an auth change and its retry policy, + even when they came up in different parts of the conversation. Pass two to + five `concepts` to `search_content`. Each result shows the passage that + matched each idea. This needs the semantic search index. + - Hosted operators can retry failed raw parse jobs after fixing their cause with `pg raw-reparse --repair`. Each bounded batch gives selected failed jobs at this executable's processing version a fresh attempt budget. diff --git a/docs/mcp.md b/docs/mcp.md index bf81a9c536..ad80da9ed0 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -219,6 +219,18 @@ without it they return a "not available" error. Exact `session_id`, before the final limit. Limits default to 10 and go up to 50; a value outside that range falls back to the default. +To find one past session that covered several ideas, pass `concepts` instead of +`pattern`: two to five distinct natural-language strings. The mode defaults to +`semantic`, and any other explicit mode is rejected. AgentsView searches each +concept separately and keeps only sessions with evidence for all of them, even +when different exchanges carry that evidence. Results rank by the mean of each +concept's strongest score. Each match lists `concept_evidence` in request order: +the concept, its snippet, score, ordinal, `ordinal_range`, and +`transcript_revision`. Concept searches do not page, so `cursor` is rejected. +`candidate_truncated` is true when a session may be missing: one concept's +candidate search filled up or stopped at its fetch cap, or more sessions matched +than the limit allowed. + A `terms` snippet shows about 60 characters of context around the first occurrence of each term. Terms that sit far apart in a long exchange produce separate windows joined by `...`, so snippet size follows the number of terms, diff --git a/docs/semantic-search.md b/docs/semantic-search.md index bdaef0e84e..fa5fc926b8 100644 --- a/docs/semantic-search.md +++ b/docs/semantic-search.md @@ -829,6 +829,30 @@ falls back to the default. Responses state the effective mode and scope and which default exclusions applied; `next_cursor` is present when another page exists. +### MCP multi-concept semantic search + +Pass `concepts` with two to five distinct natural-language ideas when recall +must find one session that covers all of them. Do not also pass `pattern`. +`mode` defaults to `semantic`; an explicit mode must also be `semantic`. The +HTTP API takes the same list as repeated `concepts` query parameters and +requires the `X-AgentsView-Search-Intent: semantic` header. + +AgentsView runs one semantic search per concept and keeps only the sessions +every search found. The concepts may match different exchanges. A session's +score is the mean of its best score for each concept; equal means keep the order +of the per-concept rankings. Every result includes `concept_evidence` with the +supporting snippet, score, and conversation-unit range for each concept. + +Project, branch, agent, session, date, scope, one-shot, automated-session, and +current-session filters apply to every concept search before the intersection. +Each concept search considers five times the limit, at most 250 candidates. +When filters leave fewer candidates than that, the concept search fetches again +with twice as many vector candidates, up to the same 4096-chunk cap described +under [Limitations](#limitations). A session that one concept ranks outside its +window is dropped from the intersection. The response sets +`candidate_truncated` whenever a window fills, the cap stops a concept search +before the vector index runs out, or more sessions match than the limit. + ### Inline context: `--context N` ```bash @@ -1068,10 +1092,11 @@ keyword leg is the mirror's recency-ordered term match, as on PostgreSQL. When the recent-activity cutoff rejects otherwise eligible hits, semantic and hybrid searches fetch again with twice as many candidates. They start at `max(4 * limit, 200)` and stop at 4096 chunks on every store, which allows - at most six fetches. Other queries use a single candidate batch. Narrow - filters or scope can still return fewer than `--limit` results when eligible - matches lie beyond that budget. The hybrid keyword leg keeps its own bounded - search budget. + at most six fetches. Multi-concept searches expand the same way whenever + filters leave a concept short. Other queries use a single candidate batch. + Narrow filters or scope can still return fewer than `--limit` results when + eligible matches lie beyond that budget. The hybrid keyword leg keeps its own + bounded search budget. - **Legacy no-`source_uuid` rows re-embed on ordinal shifts.** Each embedded document is keyed by its first message's stable per-message UUID when the parser recorded one, or by `(session_id, ordinal)` when it didn't. diff --git a/frontend/src/lib/api/generated/models/dbConceptEvidence.ts b/frontend/src/lib/api/generated/models/dbConceptEvidence.ts new file mode 100644 index 0000000000..a31cb4bae3 --- /dev/null +++ b/frontend/src/lib/api/generated/models/dbConceptEvidence.ts @@ -0,0 +1,16 @@ +/** + * Generated by Orval. Do not edit manually. + */ + +export interface DbConceptEvidence { + concept: string; + ordinal: number; + /** + * @minItems 2 + * @maxItems 2 + */ + ordinal_range: number[]; + score: number; + snippet: string; + transcript_revision?: string; +} diff --git a/frontend/src/lib/api/generated/models/dbContentMatch.ts b/frontend/src/lib/api/generated/models/dbContentMatch.ts index bba0ad5791..79923c313f 100644 --- a/frontend/src/lib/api/generated/models/dbContentMatch.ts +++ b/frontend/src/lib/api/generated/models/dbContentMatch.ts @@ -1,10 +1,12 @@ /** * Generated by Orval. Do not edit manually. */ +import type { DbConceptEvidence } from "./dbConceptEvidence.ts"; import type { DbMessage } from "./dbMessage.ts"; export interface DbContentMatch { agent: string; + concept_evidence?: DbConceptEvidence[]; context_after?: DbMessage[]; context_before?: DbMessage[]; /** @nullable */ diff --git a/frontend/src/lib/api/generated/models/getApiV1SearchContentParams.ts b/frontend/src/lib/api/generated/models/getApiV1SearchContentParams.ts index 0bfd74bac4..2d0e8a5278 100644 --- a/frontend/src/lib/api/generated/models/getApiV1SearchContentParams.ts +++ b/frontend/src/lib/api/generated/models/getApiV1SearchContentParams.ts @@ -6,9 +6,13 @@ import type { GetApiV1SearchContentScope } from "./getApiV1SearchContentScope.ts export type GetApiV1SearchContentParams = { /** - * Pattern to search for + * Pattern to search for; mutually exclusive with concepts */ - pattern: string; + pattern?: string; + /** + * Two to five semantic concepts that must all have evidence in one session; repeatable and mutually exclusive with pattern + */ + concepts?: string[]; /** * Search mode */ diff --git a/frontend/src/lib/api/generated/models/index.ts b/frontend/src/lib/api/generated/models/index.ts index 82e555b174..d1809d3c07 100644 --- a/frontend/src/lib/api/generated/models/index.ts +++ b/frontend/src/lib/api/generated/models/index.ts @@ -56,6 +56,7 @@ export * from "./dbCodeAttribution.ts"; export * from "./dbCodeAttributionSource.ts"; export * from "./dbCompactEstimate.ts"; export * from "./dbCompactResult.ts"; +export * from "./dbConceptEvidence.ts"; export * from "./dbContentMatch.ts"; export * from "./dbCursorAttributionMetrics.ts"; export * from "./dbCursorConversationCount.ts"; diff --git a/frontend/src/lib/api/generated/models/serviceContentSearchResult.ts b/frontend/src/lib/api/generated/models/serviceContentSearchResult.ts index db838e8515..e53780d57e 100644 --- a/frontend/src/lib/api/generated/models/serviceContentSearchResult.ts +++ b/frontend/src/lib/api/generated/models/serviceContentSearchResult.ts @@ -6,6 +6,7 @@ import type { ServiceMemoryCoverage } from "./serviceMemoryCoverage.ts"; export interface ServiceContentSearchResult { active_filter_applied: boolean; + candidate_truncated?: boolean; coverage: ServiceMemoryCoverage; matches: DbContentMatch[]; next_cursor?: number; diff --git a/frontend/src/lib/api/generated/search/search.ts b/frontend/src/lib/api/generated/search/search.ts index ade84e163e..7237ebb7aa 100644 --- a/frontend/src/lib/api/generated/search/search.ts +++ b/frontend/src/lib/api/generated/search/search.ts @@ -37,11 +37,11 @@ export const getApiV1Search = async ( }); }; -export const getGetApiV1SearchContentUrl = (params: GetApiV1SearchContentParams) => { +export const getGetApiV1SearchContentUrl = (params?: GetApiV1SearchContentParams) => { const normalizedParams = new URLSearchParams(); Object.entries(params || {}).forEach(([key, value]) => { - const explodeParameters = ["exclude_session"]; + const explodeParameters = ["concepts", "exclude_session"]; if (Array.isArray(value) && explodeParameters.includes(key)) { value.forEach((v) => { @@ -66,7 +66,7 @@ export const getGetApiV1SearchContentUrl = (params: GetApiV1SearchContentParams) * @summary Search session content */ export const getApiV1SearchContent = async ( - params: GetApiV1SearchContentParams, + params?: GetApiV1SearchContentParams, options?: Parameters[1], ): Promise => { return orvalFetch(getGetApiV1SearchContentUrl(params), { diff --git a/internal/apiclient/client.gen.go b/internal/apiclient/client.gen.go index 73792d2251..abca3510b4 100644 --- a/internal/apiclient/client.gen.go +++ b/internal/apiclient/client.gen.go @@ -21223,8 +21223,11 @@ func (g GetAPIV1SearchQuery) Validate() error { } type GetAPIV1SearchContentQuery struct { - // Pattern Pattern to search for - Pattern string `json:"pattern" validate:"required"` + // Pattern Pattern to search for; mutually exclusive with concepts + Pattern *string `json:"pattern,omitempty"` + + // Concepts Two to five semantic concepts that must all have evidence in one session; repeatable and mutually exclusive with pattern + Concepts []string `json:"concepts,omitempty"` // Mode Search mode Mode *GetAPIV1SearchContentQueryMode `json:"mode,omitempty"` @@ -21304,9 +21307,6 @@ type GetAPIV1SearchContentQuery struct { func (g GetAPIV1SearchContentQuery) Validate() error { var errors runtime.ValidationErrors - if err := typesValidator.Var(g.Pattern, "required"); err != nil { - errors = errors.Append("Pattern", err) - } if g.Mode != nil { if v, ok := any(g.Mode).(runtime.Validator); ok { if err := v.Validate(); err != nil { @@ -25410,12 +25410,45 @@ type DBCompactEstimate struct { type DBCompactResult = db.CompactResult +type DBConceptEvidence struct { + Concept string `json:"concept" validate:"required"` + Ordinal int64 `json:"ordinal"` + OrdinalRange []int64 `json:"ordinal_range" validate:"required"` + Score float64 `json:"score"` + Snippet string `json:"snippet" validate:"required"` + TranscriptRevision *string `json:"transcript_revision,omitempty"` +} + +func (d DBConceptEvidence) Validate() error { + var errors runtime.ValidationErrors + if err := typesValidator.Var(d.Concept, "required"); err != nil { + errors = errors.Append("Concept", err) + } + if err := typesValidator.Var(d.OrdinalRange, "required"); err != nil { + errors = errors.Append("OrdinalRange", err) + } + if d.OrdinalRange != nil && len(d.OrdinalRange) < 2 { + errors = errors.Add("OrdinalRange", fmt.Sprintf("must have at least 2 items, got %d", len(d.OrdinalRange))) + } + if len(d.OrdinalRange) > 2 { + errors = errors.Add("OrdinalRange", fmt.Sprintf("must have at most 2 items, got %d", len(d.OrdinalRange))) + } + if err := typesValidator.Var(d.Snippet, "required"); err != nil { + errors = errors.Append("Snippet", err) + } + if len(errors) == 0 { + return nil + } + return errors +} + type DBContentMatch struct { - Agent string `json:"agent" validate:"required"` - ContextAfter []DBMessage `json:"context_after,omitempty"` - ContextBefore []DBMessage `json:"context_before,omitempty"` - DisplayName *string `json:"display_name" validate:"required"` - IsSidechain *bool `json:"is_sidechain,omitempty"` + Agent string `json:"agent" validate:"required"` + ConceptEvidence []DBConceptEvidence `json:"concept_evidence,omitempty"` + ContextAfter []DBMessage `json:"context_after,omitempty"` + ContextBefore []DBMessage `json:"context_before,omitempty"` + DisplayName *string `json:"display_name" validate:"required"` + IsSidechain *bool `json:"is_sidechain,omitempty"` // Location Where the match occurred: message, thinking, tool_input, or tool_result. Location string `json:"location" validate:"required"` @@ -25444,6 +25477,13 @@ func (d DBContentMatch) Validate() error { if err := typesValidator.Var(d.Agent, "required"); err != nil { errors = errors.Append("Agent", err) } + for i, item := range d.ConceptEvidence { + if v, ok := any(item).(runtime.Validator); ok { + if err := v.Validate(); err != nil { + errors = errors.Append(fmt.Sprintf("ConceptEvidence[%d]", i), err) + } + } + } for i, item := range d.ContextAfter { if v, ok := any(item).(runtime.Validator); ok { if err := v.Validate(); err != nil { diff --git a/internal/db/search_content.go b/internal/db/search_content.go index 1f0b9ae6c8..984904762b 100644 --- a/internal/db/search_content.go +++ b/internal/db/search_content.go @@ -62,6 +62,22 @@ type ContentSearchFilter struct { Limit int Cursor int + + // Candidates, when set, makes mode "semantic" and "hybrid" keep + // fetching vector candidates while fewer than Limit units pass the + // filters, until the vector index runs out or the candidate cap stops + // the search. It records which of the two happened. Multi-concept + // search sets it on each concept's search to tell a complete short + // result from a truncated one. + Candidates *VectorCandidates +} + +// VectorCandidates reports how a semantic search's vector candidate fetch +// ended. +type VectorCandidates struct { + // Exhausted is true when the vector index had no candidates beyond the + // ones fetched, so every eligible unit was considered. + Exhausted bool } // ContentMatch is one matching message or tool call. Snippet is built from the @@ -112,6 +128,23 @@ type ContentMatch struct { // the storage query that produced this evidence. An empty value means the // backend cannot provide revision-bound evidence for this result. TranscriptRevision string `json:"transcript_revision,omitempty"` + // ConceptEvidence holds each requested concept's strongest match in this + // session, in request order. It is set only by multi-concept searches. + ConceptEvidence []ConceptEvidence `json:"concept_evidence,omitempty"` +} + +// ConceptEvidence is one concept's strongest semantic match in a session +// returned by a multi-concept search. +type ConceptEvidence struct { + Concept string `json:"concept"` + Ordinal int `json:"ordinal"` + OrdinalRange [2]int `json:"ordinal_range"` + Snippet string `json:"snippet"` + Score float64 `json:"score"` + // TranscriptRevision is the revision this concept's search observed. Each + // concept is a separate query, so it can differ from the match's revision + // when the session changed between them. + TranscriptRevision string `json:"transcript_revision,omitempty"` } // ContentSearchPage is a page of matches with an optional next cursor. @@ -1138,7 +1171,8 @@ func classifyFTSError(err error) error { const SemanticOverfetchMin = 200 // maxActivityExpansionCandidates caps the candidate count when an activity -// cutoff makes SelectSurvivingVectorHits double its fetch. Starting from +// cutoff or ContentSearchFilter.Candidates makes SelectSurvivingVectorHits +// double its fetch. Starting from // SemanticOverfetchMin, the cap allows at most six fetches. It equals // sqlite-vec's KNN ceiling, and it also bounds the KNN and hydration work // that PostgreSQL and ClickHouse would otherwise do for large limits. @@ -1147,7 +1181,9 @@ const maxActivityExpansionCandidates = 4096 // SelectSurvivingVectorHits returns the ranked vector hits that pass the // filter's session scope. When an activity cutoff rejects hits and leaves // fewer than f.Limit units, it fetches again with twice the candidates, until -// the engine is exhausted or the candidate cap is reached. allowedSessions +// the engine is exhausted or the candidate cap is reached. When +// f.Candidates is set it expands the same way whenever fewer than f.Limit +// units survive, and records whether the engine was exhausted. allowedSessions // returns an entry for each otherwise eligible session; false marks a // session that the activity cutoff rejects. func SelectSurvivingVectorHits( @@ -1175,8 +1211,12 @@ func SelectSurvivingVectorHits( if err := ctx.Err(); err != nil { return nil, err } + if f.Candidates != nil { + f.Candidates.Exhausted = exhausted + } next := clampRecallVectorCandidates(k*2, ceiling) - if !activityRejected || units >= f.Limit || exhausted || next <= k { + expand := activityRejected || f.Candidates != nil + if !expand || units >= f.Limit || exhausted || next <= k { return surviving, nil } k = next diff --git a/internal/mcp/server.go b/internal/mcp/server.go index 62a03436c1..bb65df4f73 100644 --- a/internal/mcp/server.go +++ b/internal/mcp/server.go @@ -197,6 +197,7 @@ func newServer(opts ServeOptions) *mcp.Server { "prior work and answering contextual questions, especially when the exact wording is unknown. " + "If these modes report not available, use terms when every literal term must occur within one " + "user/assistant exchange, or substring/regex for exact error messages, identifiers, and code fragments. " + + "Use concepts with two to five semantic ideas to keep only sessions with evidence for every idea. " + "The default mode remains substring. Set context to include N messages of " + "surrounding conversation with each match. Matches from the last 10 minutes (including the " + "current conversation) are excluded unless include_active is set; current_session_id replaces " + diff --git a/internal/mcp/tools.go b/internal/mcp/tools.go index d64a02d5c9..0bd806d4f4 100644 --- a/internal/mcp/tools.go +++ b/internal/mcp/tools.go @@ -732,23 +732,24 @@ func (t *toolset) getMessageBodyContinuation( // --- search_content --- type searchContentIn struct { - Pattern string `json:"pattern" jsonschema:"Natural-language query for semantic/hybrid, whitespace-separated literals for terms, or exact substring/regex for lexical search across message text, thinking, and tool inputs/results."` - Mode string `json:"mode,omitempty" jsonschema:"substring (default), regex, terms, semantic, or hybrid. Prefer hybrid or semantic for contextual questions when a vector search index is configured."` - Scope string `json:"scope,omitempty" jsonschema:"Semantic/hybrid/terms result scope: top, all, or subordinate (default all)."` - Project string `json:"project,omitempty" jsonschema:"Restrict to one project."` - Agent string `json:"agent,omitempty" jsonschema:"Restrict to one agent."` - SessionID string `json:"session_id,omitempty" jsonschema:"Restrict to one exact full stored session ID."` - GitBranch string `json:"git_branch,omitempty" jsonschema:"Restrict to one exact git branch."` - CurrentSessionID string `json:"current_session_id,omitempty" jsonschema:"Exclude this exact full stored session ID instead of applying the recent-activity heuristic."` - DateFrom string `json:"date_from,omitempty" jsonschema:"Only sessions on or after this date (YYYY-MM-DD)."` - DateTo string `json:"date_to,omitempty" jsonschema:"Only sessions on or before this date (YYYY-MM-DD)."` - Limit int `json:"limit,omitempty" jsonschema:"Max matches, default 10, max 50."` - Cursor int `json:"cursor,omitempty" jsonschema:"Pagination cursor from a previous next_cursor; also pass activity_cutoff when the recent-activity heuristic applies."` - ActivityCutoff string `json:"activity_cutoff,omitempty" jsonschema:"RFC3339 cutoff returned by the previous page. Required with a positive cursor when the recent-activity heuristic applies."` - IncludeActive bool `json:"include_active,omitempty" jsonschema:"Include matches from sessions active in the last 10 minutes. Default false: the conversation you are in right now is also recorded, so without this exclusion you would find yourself."` - IncludeOneShot bool `json:"include_one_shot,omitempty" jsonschema:"Include one-shot sessions. Default false."` - IncludeAutomated bool `json:"include_automated,omitempty" jsonschema:"Include automated sessions. Default false."` - Context int `json:"context,omitempty" jsonschema:"Messages of context before/after each match (max 10)."` + Pattern string `json:"pattern,omitempty" jsonschema:"Natural-language query for semantic/hybrid, whitespace-separated literals for terms, or exact substring/regex for lexical search across message text, thinking, and tool inputs/results. Mutually exclusive with concepts."` + Concepts []string `json:"concepts,omitempty" jsonschema:"Two to five semantic concepts that must all have evidence in the same session, even in different messages. Implies mode semantic, does not page, and is mutually exclusive with pattern."` + Mode string `json:"mode,omitempty" jsonschema:"substring (default), regex, terms, semantic, or hybrid. Prefer hybrid or semantic for contextual questions when a vector search index is configured."` + Scope string `json:"scope,omitempty" jsonschema:"Semantic/hybrid/terms result scope: top, all, or subordinate (default all)."` + Project string `json:"project,omitempty" jsonschema:"Restrict to one project."` + Agent string `json:"agent,omitempty" jsonschema:"Restrict to one agent."` + SessionID string `json:"session_id,omitempty" jsonschema:"Restrict to one exact full stored session ID."` + GitBranch string `json:"git_branch,omitempty" jsonschema:"Restrict to one exact git branch."` + CurrentSessionID string `json:"current_session_id,omitempty" jsonschema:"Exclude this exact full stored session ID instead of applying the recent-activity heuristic."` + DateFrom string `json:"date_from,omitempty" jsonschema:"Only sessions on or after this date (YYYY-MM-DD)."` + DateTo string `json:"date_to,omitempty" jsonschema:"Only sessions on or before this date (YYYY-MM-DD)."` + Limit int `json:"limit,omitempty" jsonschema:"Max matches, default 10, max 50."` + Cursor int `json:"cursor,omitempty" jsonschema:"Pagination cursor from a previous next_cursor; also pass activity_cutoff when the recent-activity heuristic applies."` + ActivityCutoff string `json:"activity_cutoff,omitempty" jsonschema:"RFC3339 cutoff returned by the previous page. Required with a positive cursor when the recent-activity heuristic applies."` + IncludeActive bool `json:"include_active,omitempty" jsonschema:"Include matches from sessions active in the last 10 minutes. Default false: the conversation you are in right now is also recorded, so without this exclusion you would find yourself."` + IncludeOneShot bool `json:"include_one_shot,omitempty" jsonschema:"Include one-shot sessions. Default false."` + IncludeAutomated bool `json:"include_automated,omitempty" jsonschema:"Include automated sessions. Default false."` + Context int `json:"context,omitempty" jsonschema:"Messages of context before/after each match (max 10)."` } // contextMessage is a truncated view of a service-level db.Message, used @@ -792,9 +793,10 @@ type contentMatch struct { // ContextBefore/ContextAfter are populated when Context > 0: the N // messages immediately before/after this match, content truncated to // 500 characters. - ContextBefore []contextMessage `json:"context_before,omitempty"` - ContextAfter []contextMessage `json:"context_after,omitempty"` - TranscriptRevision string `json:"transcript_revision,omitempty" jsonschema:"Transcript revision observed with this evidence; pass it as expected_revision to get_messages."` + ContextBefore []contextMessage `json:"context_before,omitempty"` + ContextAfter []contextMessage `json:"context_after,omitempty"` + TranscriptRevision string `json:"transcript_revision,omitempty" jsonschema:"Transcript revision observed with this evidence; pass it as expected_revision to get_messages."` + ConceptEvidence []db.ConceptEvidence `json:"concept_evidence,omitempty" jsonschema:"With concepts: each concept's strongest match in this session, in request order. score is then the mean of these scores."` } type searchContentOut struct { @@ -804,11 +806,12 @@ type searchContentOut struct { ExcludedActive int `json:"excluded_active,omitempty"` // EffectiveMode and EffectiveScope report the defaults the search // resolved, which the caller cannot see in its own request. - EffectiveMode string `json:"effective_mode"` - EffectiveScope string `json:"effective_scope,omitempty"` - Exclusions searchContentExclusions `json:"exclusions"` - RevisionBound bool `json:"revision_bound" jsonschema:"True when every returned match was captured with a transcript revision and can be opened with a revision-bound get_messages call."` - Coverage service.MemoryCoverage `json:"coverage" jsonschema:"Current lexical and semantic memory coverage from the same readiness provider as get_memory_status."` + EffectiveMode string `json:"effective_mode"` + EffectiveScope string `json:"effective_scope,omitempty"` + Exclusions searchContentExclusions `json:"exclusions"` + RevisionBound bool `json:"revision_bound" jsonschema:"True when every returned match was captured with a transcript revision and can be opened with a revision-bound get_messages call."` + Coverage service.MemoryCoverage `json:"coverage" jsonschema:"Current lexical and semantic memory coverage from the same readiness provider as get_memory_status."` + CandidateTruncated bool `json:"candidate_truncated,omitempty" jsonschema:"With concepts: true when a concept's candidate window filled or more sessions matched than limit, so some sessions may be missing."` } type memoryStatusIn struct{} @@ -853,7 +856,19 @@ func validateSearchDates(from, to string) error { func (t *toolset) searchContent( ctx context.Context, _ *mcp.CallToolRequest, in searchContentIn, ) (*mcp.CallToolResult, searchContentOut, error) { - if in.Scope != "" && !db.ContentSearchModeSupportsScope(in.Mode) { + if strings.TrimSpace(in.Pattern) == "" && len(in.Concepts) == 0 { + return nil, searchContentOut{}, errors.New("pattern or concepts required") + } + limit := clampLimit(in.Limit, defaultSearchLimit, maxContentSearchLimit) + normalized, err := service.NormalizeContentSearchRequest(service.ContentSearchRequest{ + Pattern: in.Pattern, Concepts: in.Concepts, Mode: in.Mode, + Limit: limit, Cursor: in.Cursor, + }) + if err != nil { + return nil, searchContentOut{}, err + } + mode := normalized.Mode + if in.Scope != "" && !db.ContentSearchModeSupportsScope(mode) { return nil, searchContentOut{}, errors.New(db.ContentSearchScopeUnsupportedMsg) } if err := validateSearchDates(in.DateFrom, in.DateTo); err != nil { @@ -861,7 +876,6 @@ func (t *toolset) searchContent( } var cutoffTime time.Time if in.ActivityCutoff != "" { - var err error cutoffTime, err = time.Parse(time.RFC3339, in.ActivityCutoff) if err != nil { return nil, searchContentOut{}, errors.New("invalid activity_cutoff: use RFC3339") @@ -884,6 +898,7 @@ func (t *toolset) searchContent( } req := service.ContentSearchRequest{ Pattern: in.Pattern, + Concepts: in.Concepts, Mode: in.Mode, Scope: in.Scope, Project: in.Project, @@ -892,7 +907,7 @@ func (t *toolset) searchContent( GitBranchExact: in.GitBranch, DateFrom: in.DateFrom, DateTo: in.DateTo, - Limit: clampLimit(in.Limit, defaultSearchLimit, maxContentSearchLimit), + Limit: limit, Cursor: in.Cursor, Context: in.Context, IncludeOneShot: in.IncludeOneShot, @@ -908,16 +923,17 @@ func (t *toolset) searchContent( out := searchContentOut{ Matches: make([]contentMatch, 0, len(res.Matches)), ActivityCutoff: cutoff, - EffectiveMode: cmp.Or(in.Mode, "substring"), + EffectiveMode: cmp.Or(mode, "substring"), Exclusions: searchContentExclusions{ CurrentSessionID: currentSession, RecentActive: currentSession == "" && !in.IncludeActive, OneShot: !in.IncludeOneShot, Automated: !in.IncludeAutomated, }, - RevisionBound: res.RevisionBound, - Coverage: service.NormalizeMemoryCoverage(res.Coverage), + RevisionBound: res.RevisionBound, + Coverage: service.NormalizeMemoryCoverage(res.Coverage), + CandidateTruncated: res.CandidateTruncated, } - if db.ContentSearchModeSupportsScope(in.Mode) { + if db.ContentSearchModeSupportsScope(mode) { out.EffectiveScope = cmp.Or(in.Scope, "all") } activity := make(map[string]string) @@ -942,6 +958,7 @@ func (t *toolset) searchContent( ContextBefore: toContextMessages(m.ContextBefore), ContextAfter: toContextMessages(m.ContextAfter), TranscriptRevision: m.TranscriptRevision, + ConceptEvidence: m.ConceptEvidence, }) } if res.NextCursor > 0 && len(res.Matches) > 0 { diff --git a/internal/mcp/tools_test.go b/internal/mcp/tools_test.go index 22f782f8de..c8b343df33 100644 --- a/internal/mcp/tools_test.go +++ b/internal/mcp/tools_test.go @@ -1381,13 +1381,16 @@ func TestServer_SearchContentIncludeOneShot(t *testing.T) { require.True(t, ok, name) assert.Equal(t, "boolean", property["type"]) } - assert.NotContains(t, schema["required"], "include_one_shot") - assert.NotContains(t, schema["required"], "include_automated") + // pattern and concepts are each optional, so the schema may list no + // required fields at all. + required, _ := schema["required"].([]any) + assert.NotContains(t, required, "include_one_shot") + assert.NotContains(t, required, "include_automated") for name, wantType := range map[string]string{"cursor": "integer", "activity_cutoff": "string"} { property, ok := properties[name].(map[string]any) require.True(t, ok, name) assert.Equal(t, wantType, property["type"]) - assert.NotContains(t, schema["required"], name) + assert.NotContains(t, required, name) } raw, err = json.Marshal(searchTool.OutputSchema) require.NoError(t, err) @@ -1545,6 +1548,58 @@ func TestSearchContent_RecallContractMapping(t *testing.T) { require.Len(t, out.Matches, 1) } +func TestSearchContent_ConceptsMapping(t *testing.T) { + evidence := []db.ConceptEvidence{ + {Concept: "auth model", Ordinal: 4, OrdinalRange: [2]int{3, 5}, Score: 0.7}, + {Concept: "retry policy", Ordinal: 12, OrdinalRange: [2]int{11, 13}, Score: 0.9}, + } + fake := &fakeContentSearchService{result: &service.ContentSearchResult{ + Matches: []db.ContentMatch{{ + SessionID: "both", Location: "message", Ordinal: 4, + OrdinalRange: [2]int{3, 5}, Score: new(0.8), ConceptEvidence: evidence, + }}, + CandidateTruncated: true, + }} + ts := &toolset{svc: fake, now: func() time.Time { return fixedNow }} + + _, out, err := ts.searchContent(t.Context(), nil, searchContentIn{ + Concepts: []string{"auth model", "retry policy"}, Scope: "top", + IncludeActive: true, + }) + require.NoError(t, err) + assert.Equal(t, []string{"auth model", "retry policy"}, fake.lastReq.Concepts) + assert.Equal(t, "top", fake.lastReq.Scope) + assert.Equal(t, "semantic", out.EffectiveMode) + assert.Equal(t, "top", out.EffectiveScope) + assert.True(t, out.CandidateTruncated) + require.Len(t, out.Matches, 1) + assert.Equal(t, evidence, out.Matches[0].ConceptEvidence) +} + +func TestSearchContent_ConceptsRejectedBeforeService(t *testing.T) { + for _, tc := range []struct { + name string + in searchContentIn + want string + }{ + {"neither", searchContentIn{}, "pattern or concepts required"}, + {"both", searchContentIn{Pattern: "x", Concepts: []string{"a", "b"}}, "mutually exclusive"}, + {"lexical mode", searchContentIn{Concepts: []string{"a", "b"}, Mode: "regex"}, "mode must be semantic"}, + {"cursor", searchContentIn{Concepts: []string{"a", "b"}, Cursor: 10}, "cursor is not supported"}, + } { + t.Run(tc.name, func(t *testing.T) { + fake := &fakeContentSearchService{result: &service.ContentSearchResult{}} + ts := &toolset{svc: fake, now: func() time.Time { return fixedNow }} + tc.in.IncludeActive = true + _, _, err := ts.searchContent(t.Context(), nil, tc.in) + require.Error(t, err) + assert.Contains(t, err.Error(), tc.want) + assert.Empty(t, fake.lastReq.Concepts, "a rejected request must not reach the service") + assert.Empty(t, fake.lastReq.Pattern, "a rejected request must not reach the service") + }) + } +} + func TestSearchContent_OutOfRangeLimitUsesDefault(t *testing.T) { for _, limit := range []int{-1, 51} { t.Run(fmt.Sprintf("limit=%d", limit), func(t *testing.T) { diff --git a/internal/postgres/vector_search.go b/internal/postgres/vector_search.go index d31332ded6..3a0841d05b 100644 --- a/internal/postgres/vector_search.go +++ b/internal/postgres/vector_search.go @@ -94,11 +94,11 @@ func (v *vectorSearcher) PrepareSemanticSearch( return nil, 0, err } return func(ctx context.Context, limit int) ([]db.VectorHit, bool, error) { - chunks, err := v.knnChunks(ctx, vec, extSchema, limit) + chunks, rowCap, err := v.knnChunks(ctx, vec, extSchema, limit) if err != nil { return nil, false, err } - exhausted := len(chunks) < limit + exhausted := knnExhausted(len(chunks), limit, rowCap) docs, err := kitvec.RollupByDocument(chunks) if err != nil { return nil, false, fmt.Errorf("rolling up vector hits: %w", err) @@ -114,13 +114,23 @@ func (v *vectorSearcher) PrepareSemanticSearch( }, 0, nil } +// knnExhausted reports whether a KNN that asked for limit rows and returned +// rows read every chunk. Only an HNSW index scan stops at rowCap below limit; +// a sequential scan has no cap. A short result therefore proves exhaustion +// unless it is exactly the cap, which a capped index scan also returns when +// more rows remain. +func knnExhausted(rows, limit, rowCap int) bool { + return rows < limit && (rowCap >= limit || rows != rowCap) +} + // chunkHit is one chunk-level KNN neighbor: the document it belongs to, which // chunk matched, and its cosine similarity (1 - cosine distance, higher is // better). type chunkHit = kitvec.Hit[string] // knnChunks runs the chunk-level KNN, returning up to exactly limit neighbors -// ordered best (nearest) first. Fetching exactly `limit` chunks matches the +// ordered best (nearest) first, and the most rows the scan could return (see +// tuneHNSWRecall). Fetching exactly `limit` chunks matches the // local sqlite-vec searcher's candidate pool (kit's queryGenerationSQL also // runs its KNN with LIMIT k before rollup), per the backend-parity rule: after // rollup the doc pool can be smaller than `limit` when a high-ranking run doc @@ -140,11 +150,11 @@ type chunkHit = kitvec.Hit[string] // ceiling, and past that ceiling iterative scan keeps filling. func (v *vectorSearcher) knnChunks( ctx context.Context, vec []float32, extSchema string, limit int, -) ([]chunkHit, error) { +) ([]chunkHit, int, error) { k := max(limit, 0) literal, err := halfvecLiteral(vec) if err != nil { - return nil, fmt.Errorf("query embedding: %w", err) + return nil, 0, fmt.Errorf("query embedding: %w", err) } dist := fmt.Sprintf("embedding OPERATOR(%s.<=>) $1::%s.halfvec", extSchema, extSchema) q := fmt.Sprintf(` @@ -155,16 +165,17 @@ SELECT doc_key, chunk_index, 1 - (%s) AS score tx, err := v.pg.BeginTx(ctx, &sql.TxOptions{ReadOnly: true}) if err != nil { - return nil, fmt.Errorf("begin chunk knn tx: %w", err) + return nil, 0, fmt.Errorf("begin chunk knn tx: %w", err) } defer func() { _ = tx.Rollback() }() - if err := tuneHNSWRecall(ctx, tx, k); err != nil { - return nil, err + rowCap, err := tuneHNSWRecall(ctx, tx, k) + if err != nil { + return nil, 0, err } rows, err := tx.QueryContext(ctx, q, literal, k) if err != nil { - return nil, fmt.Errorf("chunk knn query: %w", err) + return nil, 0, fmt.Errorf("chunk knn query: %w", err) } defer rows.Close() defer func() { _ = rows.Close() }() @@ -174,15 +185,15 @@ SELECT doc_key, chunk_index, 1 - (%s) AS score var h chunkHit var score float64 if err := rows.Scan(&h.Doc, &h.ChunkIndex, &score); err != nil { - return nil, fmt.Errorf("scanning chunk knn row: %w", err) + return nil, 0, fmt.Errorf("scanning chunk knn row: %w", err) } h.Score = float32(score) hits = append(hits, h) } if err := rows.Err(); err != nil { - return nil, fmt.Errorf("iterating chunk knn rows: %w", err) + return nil, 0, fmt.Errorf("iterating chunk knn rows: %w", err) } - return hits, nil + return hits, rowCap, nil } // tuneHNSWRecall raises pgvector's per-scan HNSW candidate pool for the current @@ -200,7 +211,8 @@ SELECT doc_key, chunk_index, 1 - (%s) AS score // going past ef_search. Older pgvector lacks that GUC and would abort the tx on // an unknown-GUC error, so it is probed first with current_setting(..., true) // (missing_ok), which yields NULL instead of erroring; a NULL result skips the -// setting and leaves the 1000-row cap in place. +// setting and leaves the 1000-row cap in place. tuneHNSWRecall returns the +// most rows the scan can return: k, or 1000 when that cap stays in place. // hnswEfSearch clamps the KNN candidate-pool size k to pgvector's usable // hnsw.ef_search range [40, 1000]: never below the stock default 40 (so small-k // searches keep default recall) and never above pgvector's 1000 ceiling (past @@ -210,29 +222,29 @@ func hnswEfSearch(k int) int { return min(max(k, 40), 1000) } -func tuneHNSWRecall(ctx context.Context, tx *sql.Tx, k int) error { +func tuneHNSWRecall(ctx context.Context, tx *sql.Tx, k int) (int, error) { efSearch := hnswEfSearch(k) if _, err := tx.ExecContext(ctx, fmt.Sprintf("SET LOCAL hnsw.ef_search = %d", efSearch)); err != nil { - return fmt.Errorf("setting hnsw.ef_search: %w", err) + return 0, fmt.Errorf("setting hnsw.ef_search: %w", err) } - if k <= 1000 { - return nil + if k <= efSearch { + return k, nil } var current sql.NullString if err := tx.QueryRowContext(ctx, `SELECT current_setting('hnsw.iterative_scan', true)`, ).Scan(¤t); err != nil { - return fmt.Errorf("probing hnsw.iterative_scan: %w", err) + return 0, fmt.Errorf("probing hnsw.iterative_scan: %w", err) } if !current.Valid { - return nil + return efSearch, nil } if _, err := tx.ExecContext(ctx, "SET LOCAL hnsw.iterative_scan = 'relaxed_order'"); err != nil { - return fmt.Errorf("setting hnsw.iterative_scan: %w", err) + return 0, fmt.Errorf("setting hnsw.iterative_scan: %w", err) } - return nil + return k, nil } // vectorDoc is the subset of a vector_documents row needed to hydrate a chunk diff --git a/internal/postgres/vector_search_pg_test.go b/internal/postgres/vector_search_pg_test.go index a6d22e2195..e10b03a89b 100644 --- a/internal/postgres/vector_search_pg_test.go +++ b/internal/postgres/vector_search_pg_test.go @@ -338,3 +338,46 @@ func TestPGVectorSearcherEfSearchRecall(t *testing.T) { assert.Len(t, hits, docCount, "ef_search raised to k returns all %d docs, not the ~40 default cap", docCount) } + +// TestPGVectorSearcherCappedScanIsNotExhausted pins the exhaustion signal past +// hnsw.ef_search's 1000-row ceiling. Without iterative scan (pgvector < 0.8) +// an HNSW index scan for k=1600 returns 1000 rows; with it, 1600. Either way +// rows remain, so the page must not report the index exhausted. The table is +// large enough that the planner scans the index instead of sorting rows. +func TestPGVectorSearcherCappedScanIsNotExhausted(t *testing.T) { + pgURL := testPGURL(t) + pg := newVectorSearchTestPG(t, pgURL, "agentsview_vector_search_capped_test") + ctx := t.Context() + + genID, err := ensureVectorGeneration(ctx, pg, "fp-capped", "m", 4) + require.NoError(t, err, "ensureVectorGeneration") + require.NoError(t, ensureVectorChunkTable(ctx, pg, genID, 4), "ensureVectorChunkTable") + extSchema, err := vectorExtensionSchema(ctx, pg) + require.NoError(t, err, "vectorExtensionSchema") + table := vectorChunkTable(genID) + + const docCount, k = 20000, 1600 + _, err = pg.ExecContext(ctx, ` +INSERT INTO vector_documents ( + doc_key, session_id, source_uuid, ordinal, ordinal_end, + subordinate, offsets, content, content_hash) +SELECT 'd' || i, 'S', '', i, i, FALSE, '[]', 'doc ' || i, 'h' + FROM generate_series(0, $1 - 1) AS i`, docCount) + require.NoError(t, err, "insert docs") + _, err = pg.ExecContext(ctx, ` +INSERT INTO `+table+` (doc_key, chunk_index, embedding) +SELECT 'd' || i, 0, ('[1,' || (i::float8 / $1) || ',' || ((i % 97)::float8 / 97) || ',0]')::`+extSchema+`.halfvec + FROM generate_series(0, $1 - 1) AS i`, docCount) + require.NoError(t, err, "insert chunks") + _, err = pg.ExecContext(ctx, `ANALYZE `+table) + require.NoError(t, err, "analyze chunks") + + searcher := NewVectorSearcher( + pg, genID, 4, searchMaxInputChars, fixedEncoder([]float32{1, 0, 0, 0})) + page, _, err := searcher.PrepareSemanticSearch(ctx, "query") + require.NoError(t, err, "PrepareSemanticSearch") + hits, exhausted, err := page(ctx, k) + require.NoError(t, err, "page") + assert.GreaterOrEqual(t, len(hits), 1000) + assert.False(t, exhausted, "%d of %d docs returned", len(hits), docCount) +} diff --git a/internal/postgres/vector_search_unit_test.go b/internal/postgres/vector_search_unit_test.go index 57c51c8001..8c3be9dccc 100644 --- a/internal/postgres/vector_search_unit_test.go +++ b/internal/postgres/vector_search_unit_test.go @@ -60,6 +60,25 @@ func TestHNSWEfSearch(t *testing.T) { } } +func TestKNNExhausted(t *testing.T) { + for _, tc := range []struct { + name string + rows, limit, cap int + want bool + }{ + {"short uncapped scan", 10, 200, 200, true}, + {"full uncapped scan", 200, 200, 200, false}, + {"capped index scan", 1000, 1600, 1000, false}, + {"capped index scan below its cap", 640, 1600, 1000, true}, + {"sequential scan past the cap", 1200, 1600, 1000, true}, + {"full sequential scan past the cap", 1600, 1600, 1000, false}, + } { + t.Run(tc.name, func(t *testing.T) { + assert.Equal(t, tc.want, knnExhausted(tc.rows, tc.limit, tc.cap)) + }) + } +} + func TestSemanticUnavailableError(t *testing.T) { s := &Store{} diff --git a/internal/server/huma_routes_search.go b/internal/server/huma_routes_search.go index ce3466d3a8..336cab9bb2 100644 --- a/internal/server/huma_routes_search.go +++ b/internal/server/huma_routes_search.go @@ -37,7 +37,8 @@ type searchInput struct { } type contentSearchInput struct { - Pattern string `query:"pattern" required:"true" doc:"Pattern to search for"` + Pattern string `query:"pattern" doc:"Pattern to search for; mutually exclusive with concepts"` + Concepts []string `query:"concepts,explode" doc:"Two to five semantic concepts that must all have evidence in one session; repeatable and mutually exclusive with pattern"` Mode contentSearchMode `query:"mode" enum:"substring,regex,fts,terms,semantic,hybrid" doc:"Search mode"` Scope contentSearchScope `query:"scope" enum:"top,all,subordinate" doc:"Semantic/hybrid/terms result scope: top, all, or subordinate (default all)"` SearchIntent string `header:"X-AgentsView-Search-Intent" doc:"Required for semantic/hybrid GET searches"` @@ -106,15 +107,26 @@ func (s *Server) humaSearchContent( ctx context.Context, in *contentSearchInput, ) (*jsonOutput[*service.ContentSearchResult], error) { + if strings.TrimSpace(in.Pattern) == "" && len(in.Concepts) == 0 { + return nil, apiError(http.StatusBadRequest, "pattern or concepts required") + } + normalized, err := service.NormalizeContentSearchRequest(service.ContentSearchRequest{ + Pattern: in.Pattern, Concepts: in.Concepts, Mode: string(in.Mode), + Limit: in.Limit, Cursor: in.Cursor, + }) + if err != nil { + return nil, apiError(http.StatusBadRequest, err.Error()) + } + mode := contentSearchMode(normalized.Mode) if in.Reveal && !isLocalhostContext(ctx) { return nil, apiError(http.StatusForbidden, "reveal is only permitted from localhost") } - if requiresSemanticSearchIntent(in.Mode) && + if requiresSemanticSearchIntent(mode) && in.SearchIntent != service.SemanticSearchIntentValue { return nil, apiError(http.StatusForbidden, "semantic and hybrid search require "+service.SemanticSearchIntentHeader) } - if in.Scope != "" && !db.ContentSearchModeSupportsScope(string(in.Mode)) { + if in.Scope != "" && !db.ContentSearchModeSupportsScope(string(mode)) { return nil, apiError(http.StatusBadRequest, db.ContentSearchScopeUnsupportedMsg) } @@ -131,6 +143,7 @@ func (s *Server) humaSearchContent( } res, err := s.sessions.SearchContent(ctx, service.ContentSearchRequest{ Pattern: in.Pattern, + Concepts: in.Concepts, Mode: string(in.Mode), Sources: sources, ExcludeSystem: in.ExcludeSystem, diff --git a/internal/server/search_scope_test.go b/internal/server/search_scope_test.go index cf45362034..797dd7437d 100644 --- a/internal/server/search_scope_test.go +++ b/internal/server/search_scope_test.go @@ -209,3 +209,82 @@ func TestSearchContentSemanticResponseCarriesUnitRangeAndLineage(t *testing.T) { "zero-valued lineage keys must be omitted from the wire") } } + +// queryHitsVectorSearcher returns canned semantic hits per query text, so a +// multi-concept search sees different evidence for each concept. +type queryHitsVectorSearcher map[string][]db.VectorHit + +func (f queryHitsVectorSearcher) PrepareSemanticSearch( + _ context.Context, query string, +) (db.VectorSearchPage, int, error) { + return func(context.Context, int) ([]db.VectorHit, bool, error) { + return f[query], true, nil + }, 0, nil +} + +func (f queryHitsVectorSearcher) ResolveMessageUnits( + _ context.Context, refs []db.MessageRef, +) ([]db.UnitRef, error) { + return make([]db.UnitRef, len(refs)), nil +} + +func TestSearchContentConceptsIntersectDifferentExchangesThroughHTTP(t *testing.T) { + te := setup(t) + for _, id := range []string{"both", "auth-only"} { + te.seedSession(t, id, "proj", 4) + te.seedMessages(t, id, 4, func(i int, m *db.Message) { + m.Content = "message" + if i%2 == 1 { + m.Role = "assistant" + } + }) + } + te.db.SetVectorSearcher(queryHitsVectorSearcher{ + "auth model": { + {SessionID: "auth-only", Ordinal: 0, OrdinalStart: 0, OrdinalEnd: 1, Score: 0.95}, + {SessionID: "both", Ordinal: 0, OrdinalStart: 0, OrdinalEnd: 1, Score: 0.7}, + }, + "retry policy": { + {SessionID: "both", Ordinal: 2, OrdinalStart: 2, OrdinalEnd: 3, Score: 0.9}, + }, + }) + + w := te.wrappedRequest(http.MethodGet, + "/api/v1/search/content?concepts=auth+model&concepts=retry+policy&scope=top&limit=10", + withHeader("X-AgentsView-Search-Intent", "semantic")) + assertStatus(t, w, http.StatusOK) + res := decode[service.ContentSearchResult](t, w) + require.Len(t, res.Matches, 1) + assert.Equal(t, "both", res.Matches[0].SessionID) + require.NotNil(t, res.Matches[0].Score) + assert.InDelta(t, 0.8, *res.Matches[0].Score, 0.0001) + require.Len(t, res.Matches[0].ConceptEvidence, 2) + assert.Equal(t, "auth model", res.Matches[0].ConceptEvidence[0].Concept) + assert.Equal(t, [2]int{0, 1}, res.Matches[0].ConceptEvidence[0].OrdinalRange) + assert.Equal(t, "retry policy", res.Matches[0].ConceptEvidence[1].Concept) + assert.Equal(t, [2]int{2, 3}, res.Matches[0].ConceptEvidence[1].OrdinalRange) +} + +func TestSearchContentConceptsRejectedRequestsThroughHTTP(t *testing.T) { + te := setup(t) + intent := withHeader("X-AgentsView-Search-Intent", "semantic") + for _, tc := range []struct { + name string + query string + opts []requestOpt + status int + want string + }{ + {"neither", "", []requestOpt{intent}, http.StatusBadRequest, "pattern or concepts required"}, + {"both", "pattern=x&concepts=a&concepts=b", []requestOpt{intent}, http.StatusBadRequest, "mutually exclusive"}, + {"one concept", "concepts=a", []requestOpt{intent}, http.StatusBadRequest, "between 2 and 5"}, + {"lexical mode", "concepts=a&concepts=b&mode=substring", []requestOpt{intent}, http.StatusBadRequest, "mode must be semantic"}, + {"no intent header", "concepts=a&concepts=b", nil, http.StatusForbidden, "X-AgentsView-Search-Intent"}, + } { + t.Run(tc.name, func(t *testing.T) { + w := te.wrappedRequest(http.MethodGet, "/api/v1/search/content?"+tc.query, tc.opts...) + assertStatus(t, w, tc.status) + assert.Contains(t, w.Body.String(), tc.want) + }) + } +} diff --git a/internal/service/direct.go b/internal/service/direct.go index 85898eabbb..558639b464 100644 --- a/internal/service/direct.go +++ b/internal/service/direct.go @@ -1137,9 +1137,12 @@ const maxContentSearchContext = 10 func (b *directBackend) SearchContent( ctx context.Context, req ContentSearchRequest, ) (*ContentSearchResult, error) { + req, err := NormalizeContentSearchRequest(req) + if err != nil { + return nil, err + } var excludeActiveSince time.Time if req.ExcludeActiveSince != "" { - var err error excludeActiveSince, err = time.Parse(time.RFC3339, req.ExcludeActiveSince) if err != nil || excludeActiveSince.IsZero() { return nil, &db.SearchInputError{Msg: "search: invalid exclude_active_since: use RFC3339"} @@ -1172,7 +1175,7 @@ func (b *directBackend) SearchContent( if err != nil { return nil, err } - page, err := b.db.SearchContent(ctx, db.ContentSearchFilter{ + filter := db.ContentSearchFilter{ Pattern: req.Pattern, Mode: req.Mode, Sources: req.Sources, @@ -1201,13 +1204,20 @@ func (b *directBackend) SearchContent( RevealSecrets: req.Reveal, Limit: req.Limit, Cursor: req.Cursor, - }) + } + var page db.ContentSearchPage + candidateTruncated := false + if len(req.Concepts) > 0 { + page, candidateTruncated, err = b.searchContentConcepts(ctx, filter, req.Concepts) + } else { + page, err = b.db.SearchContent(ctx, filter) + } if err != nil { return nil, err } revisionBound := true for i := range page.Matches { - if page.Matches[i].TranscriptRevision == "" { + if !matchRevisionBound(page.Matches[i]) { revisionBound = false break } @@ -1236,6 +1246,7 @@ func (b *directBackend) SearchContent( NextCursor: page.NextCursor, RevisionBound: revisionBound, Coverage: coverage, + CandidateTruncated: candidateTruncated, }, nil } diff --git a/internal/service/search_concepts.go b/internal/service/search_concepts.go new file mode 100644 index 0000000000..d92a6de0ff --- /dev/null +++ b/internal/service/search_concepts.go @@ -0,0 +1,200 @@ +package service + +import ( + "cmp" + "context" + "fmt" + "slices" + "strings" + + "go.kenn.io/agentsview/internal/db" + "go.kenn.io/kit/search/rrf" +) + +const ( + minContentConcepts = 2 + maxContentConcepts = 5 + maxConceptSearchLimit = 50 + // maxConceptCandidates bounds each concept's semantic search, so a + // five-concept request reads at most 5 * 250 units. + maxConceptCandidates = 250 + conceptRankConstant = 60 +) + +// NormalizeContentSearchRequest validates the multi-concept contract and +// returns a copy with trimmed concepts and the semantic mode they require. A +// request without concepts is returned unchanged. +func NormalizeContentSearchRequest(req ContentSearchRequest) (ContentSearchRequest, error) { + if len(req.Concepts) == 0 { + return req, nil + } + if strings.TrimSpace(req.Pattern) != "" { + return req, &db.SearchInputError{Msg: "search: pattern and concepts are mutually exclusive"} + } + if len(req.Concepts) < minContentConcepts || len(req.Concepts) > maxContentConcepts { + return req, &db.SearchInputError{Msg: fmt.Sprintf( + "search: concepts must contain between %d and %d values", + minContentConcepts, maxContentConcepts)} + } + concepts := make([]string, len(req.Concepts)) + for i, concept := range req.Concepts { + concepts[i] = strings.TrimSpace(concept) + if concepts[i] == "" { + return req, &db.SearchInputError{Msg: "search: concepts must not be blank"} + } + if slices.Contains(concepts[:i], concepts[i]) { + return req, &db.SearchInputError{Msg: "search: concepts must be distinct"} + } + } + if req.Mode != "" && req.Mode != "semantic" { + return req, &db.SearchInputError{Msg: "search: mode must be semantic when concepts are supplied"} + } + if req.Cursor != 0 { + return req, &db.SearchInputError{Msg: "search: cursor is not supported with concepts"} + } + if req.Limit < 0 || req.Limit > maxConceptSearchLimit { + return req, &db.SearchInputError{Msg: fmt.Sprintf( + "search: limit must be between 1 and %d with concepts", maxConceptSearchLimit)} + } + req.Concepts = concepts + req.Mode = "semantic" + return req, nil +} + +// conceptMember points at one session's best match in one concept's leg. +type conceptMember struct { + leg, index int +} + +// searchContentConcepts runs one semantic search per concept and keeps the +// sessions every concept found, even when different messages carry the +// evidence. Results rank by the mean of each concept's best score; the legs +// share one vector generation, so their scores are comparable. truncated +// reports that a full candidate window, the candidate fetch cap, or the +// limit may have hidden sessions. +func (b *directBackend) searchContentConcepts( + ctx context.Context, f db.ContentSearchFilter, concepts []string, +) (page db.ContentSearchPage, truncated bool, err error) { + limit := f.Limit + if limit <= 0 { + limit = maxConceptSearchLimit + } + candidateLimit := min(limit*5, maxConceptCandidates) + best := make([][]db.ContentMatch, len(concepts)) + legs := make([]rrf.GroupLeg[string, conceptMember], len(concepts)) + for i, concept := range concepts { + legFilter := f + legFilter.Pattern = concept + legFilter.Limit = candidateLimit + legFilter.Cursor = 0 + legFilter.Candidates = &db.VectorCandidates{} + legPage, err := b.db.SearchContent(ctx, legFilter) + if err != nil { + return db.ContentSearchPage{}, false, err + } + // A short leg is complete only when its search ran out of vector + // candidates; one stopped by the fetch cap may have missed sessions. + if len(legPage.Matches) >= candidateLimit || legPage.NextCursor > 0 || + !legFilter.Candidates.Exhausted { + truncated = true + } + best[i] = bestMatchPerSession(legPage.Matches) + groups := make([]rrf.Group[string, conceptMember], len(best[i])) + for j, match := range best[i] { + groups[j] = rrf.Group[string, conceptMember]{ + Key: match.SessionID, Members: []conceptMember{{leg: i, index: j}}, + } + } + legs[i] = rrf.GroupLeg[string, conceptMember]{ + Name: fmt.Sprintf("concept-%d", i), Weight: 1, Groups: groups, + } + } + hits, err := rrf.FuseGroupsEvery(conceptRankConstant, legs) + if err != nil { + return db.ContentSearchPage{}, false, err + } + + type ranked struct { + match db.ContentMatch + score float64 + } + results := make([]ranked, 0, len(hits)) + for _, hit := range hits { + evidence := make([]db.ConceptEvidence, len(concepts)) + var sum float64 + // Each leg has one member per session, so there is exactly one + // alternate per concept. + for _, alt := range hit.Alternates { + match := best[alt.Member.leg][alt.Member.index] + score := matchScore(match) + sum += score + evidence[alt.Member.leg] = db.ConceptEvidence{ + Concept: concepts[alt.Member.leg], Ordinal: match.Ordinal, + OrdinalRange: match.OrdinalRange, Snippet: match.Snippet, + Score: score, TranscriptRevision: match.TranscriptRevision, + } + } + first := hit.Alternates[0].Member + match := best[first.leg][first.index] + mean := sum / float64(len(concepts)) + match.Score = &mean + match.ConceptEvidence = evidence + results = append(results, ranked{match: match, score: mean}) + } + // FuseGroupsEvery returns reciprocal-rank order, which breaks score ties. + slices.SortStableFunc(results, func(a, b ranked) int { + return cmp.Compare(b.score, a.score) + }) + if len(results) > limit { + results = results[:limit] + truncated = true + } + page.Matches = make([]db.ContentMatch, len(results)) + for i := range results { + page.Matches[i] = results[i].match + } + return page, truncated, nil +} + +// bestMatchPerSession keeps each session's highest-scoring match, preferring +// the earlier ordinal on a tie. Sessions keep the order in which the leg +// first ranked them. +func bestMatchPerSession(matches []db.ContentMatch) []db.ContentMatch { + index := make(map[string]int, len(matches)) + out := make([]db.ContentMatch, 0, len(matches)) + for _, match := range matches { + i, ok := index[match.SessionID] + if !ok { + index[match.SessionID] = len(out) + out = append(out, match) + continue + } + current := out[i] + if matchScore(match) > matchScore(current) || + (matchScore(match) == matchScore(current) && match.Ordinal < current.Ordinal) { + out[i] = match + } + } + return out +} + +func matchScore(match db.ContentMatch) float64 { + if match.Score == nil { + return 0 + } + return *match.Score +} + +// matchRevisionBound reports whether a match and all of its concept evidence +// carry the transcript revision their queries observed. +func matchRevisionBound(match db.ContentMatch) bool { + if match.TranscriptRevision == "" { + return false + } + for _, evidence := range match.ConceptEvidence { + if evidence.TranscriptRevision == "" { + return false + } + } + return true +} diff --git a/internal/service/search_content_test.go b/internal/service/search_content_test.go index 95c684c3de..12d872e981 100644 --- a/internal/service/search_content_test.go +++ b/internal/service/search_content_test.go @@ -75,9 +75,11 @@ func TestDirectSearchContentFTSSourceGuard(t *testing.T) { // test path reached it (none of these tests exercise anything else). type fakeContentStore struct { db.Store - page db.ContentSearchPage - lastFilter db.ContentSearchFilter - windows map[string][]db.Message // keyed by contextWindowKey + page db.ContentSearchPage + pagesByPattern map[string]db.ContentSearchPage + lastFilter db.ContentSearchFilter + filters []db.ContentSearchFilter + windows map[string][]db.Message // keyed by contextWindowKey } func (f *fakeContentStore) HasFTS(context.Context) bool { return true } @@ -92,9 +94,240 @@ func (f *fakeContentStore) SearchContent( _ context.Context, filter db.ContentSearchFilter, ) (db.ContentSearchPage, error) { f.lastFilter = filter + f.filters = append(f.filters, filter) + // The fake's pages are complete result sets. + if filter.Candidates != nil { + filter.Candidates.Exhausted = true + } + if page, ok := f.pagesByPattern[filter.Pattern]; ok { + return page, nil + } return f.page, nil } +func TestDirectSearchContentConceptsIntersectAndRankSessions(t *testing.T) { + store := &fakeContentStore{pagesByPattern: map[string]db.ContentSearchPage{ + "auth model": {Matches: []db.ContentMatch{ + {SessionID: "both", Ordinal: 4, OrdinalRange: [2]int{3, 5}, Snippet: "auth evidence", Score: new(0.7), TranscriptRevision: "rev-1"}, + {SessionID: "both", Ordinal: 8, OrdinalRange: [2]int{8, 9}, Snippet: "weaker auth", Score: new(0.4), TranscriptRevision: "rev-1"}, + {SessionID: "auth-only", Ordinal: 2, OrdinalRange: [2]int{2, 2}, Snippet: "auth only", Score: new(0.95), TranscriptRevision: "rev-1"}, + {SessionID: "tie", Ordinal: 1, OrdinalRange: [2]int{1, 2}, Snippet: "tie auth", Score: new(0.6), TranscriptRevision: "rev-1"}, + }}, + "retry policy": {Matches: []db.ContentMatch{ + {SessionID: "both", Ordinal: 12, OrdinalRange: [2]int{11, 13}, Snippet: "retry evidence", Score: new(0.9), TranscriptRevision: "rev-2"}, + {SessionID: "tie", Ordinal: 6, OrdinalRange: [2]int{6, 7}, Snippet: "tie retry", Score: new(0.8), TranscriptRevision: "rev-1"}, + }}, + }} + be := service.NewReadOnlyBackend(store) + + res, err := be.SearchContent(t.Context(), service.ContentSearchRequest{ + Concepts: []string{" auth model ", "retry policy"}, + Project: "agentsview", Agent: "codex", SessionID: "target", + GitBranchExact: "feature/memory", DateFrom: "2026-08-01", + DateTo: "2026-09-20", ExcludeSessionIDs: []string{"current"}, + Scope: "top", Limit: 10, + }) + require.NoError(t, err) + require.Len(t, res.Matches, 2) + assert.False(t, res.CandidateTruncated) + assert.True(t, res.RevisionBound) + + both := res.Matches[0] + assert.Equal(t, "both", both.SessionID) + require.NotNil(t, both.Score) + assert.InDelta(t, 0.8, *both.Score, 0.0001) + require.Len(t, both.ConceptEvidence, 2) + assert.Equal(t, db.ConceptEvidence{ + Concept: "auth model", Ordinal: 4, OrdinalRange: [2]int{3, 5}, + Snippet: "auth evidence", Score: 0.7, TranscriptRevision: "rev-1", + }, both.ConceptEvidence[0]) + assert.Equal(t, db.ConceptEvidence{ + Concept: "retry policy", Ordinal: 12, OrdinalRange: [2]int{11, 13}, + Snippet: "retry evidence", Score: 0.9, TranscriptRevision: "rev-2", + }, both.ConceptEvidence[1]) + + assert.Equal(t, "tie", res.Matches[1].SessionID) + require.NotNil(t, res.Matches[1].Score) + assert.InDelta(t, 0.7, *res.Matches[1].Score, 0.0001) + + require.Len(t, store.filters, 2) + assert.Equal(t, "auth model", store.filters[0].Pattern) + assert.Equal(t, "retry policy", store.filters[1].Pattern) + for _, filter := range store.filters { + assert.Equal(t, "semantic", filter.Mode) + assert.Equal(t, "agentsview", filter.Project) + assert.Equal(t, "codex", filter.Agent) + assert.Equal(t, "target", filter.SessionID) + assert.Equal(t, "feature/memory", filter.GitBranchExact) + assert.Equal(t, "2026-08-01", filter.DateFrom) + assert.Equal(t, "2026-09-20", filter.DateTo) + assert.Equal(t, []string{"current"}, filter.ExcludeSessionIDs) + assert.Equal(t, "top", filter.Scope) + assert.Equal(t, 50, filter.Limit) + assert.Zero(t, filter.Cursor) + } +} + +func TestDirectSearchContentConceptsRejectInvalidRequests(t *testing.T) { + for _, tc := range []struct { + name string + req service.ContentSearchRequest + want string + }{ + {"pattern", service.ContentSearchRequest{Pattern: "x", Concepts: []string{"a", "b"}}, "pattern and concepts"}, + {"one", service.ContentSearchRequest{Concepts: []string{"a"}}, "between 2 and 5"}, + {"six", service.ContentSearchRequest{Concepts: []string{"a", "b", "c", "d", "e", "f"}}, "between 2 and 5"}, + {"blank", service.ContentSearchRequest{Concepts: []string{"a", " "}}, "must not be blank"}, + {"duplicate", service.ContentSearchRequest{Concepts: []string{"a", " a"}}, "must be distinct"}, + {"mode", service.ContentSearchRequest{Concepts: []string{"a", "b"}, Mode: "hybrid"}, "mode must be semantic"}, + {"cursor", service.ContentSearchRequest{Concepts: []string{"a", "b"}, Cursor: 1}, "cursor is not supported"}, + {"limit", service.ContentSearchRequest{Concepts: []string{"a", "b"}, Limit: 51}, "limit must be between 1 and 50"}, + } { + t.Run(tc.name, func(t *testing.T) { + store := &fakeContentStore{} + _, err := service.NewReadOnlyBackend(store).SearchContent(t.Context(), tc.req) + var inputErr *db.SearchInputError + require.ErrorAs(t, err, &inputErr) + assert.Contains(t, err.Error(), tc.want) + assert.Empty(t, store.filters) + }) + } +} + +func TestDirectSearchContentConceptsBoundCandidatesAndReportTruncation(t *testing.T) { + sessions := func(ids ...string) db.ContentSearchPage { + page := db.ContentSearchPage{} + for _, id := range ids { + page.Matches = append(page.Matches, db.ContentMatch{SessionID: id, Score: new(0.5)}) + } + return page + } + tests := []struct { + name string + limit int + a, b db.ContentSearchPage + wantLegLimit int + wantSessions []string + wantTruncated bool + }{ + { + name: "default limit", limit: 0, + a: sessions("s1"), b: sessions("s1"), + wantLegLimit: 250, wantSessions: []string{"s1"}, + }, + { + // Equal mean scores keep the legs' reciprocal-rank order. + name: "limit cuts the intersection", limit: 1, + a: sessions("z", "a"), b: sessions("z", "a"), + wantLegLimit: 5, wantSessions: []string{"z"}, wantTruncated: true, + }, + { + name: "full candidate window", limit: 1, + a: sessions("s1", "s2", "s3", "s4", "s5"), b: sessions("s1"), + wantLegLimit: 5, wantSessions: []string{"s1"}, wantTruncated: true, + }, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + store := &fakeContentStore{pagesByPattern: map[string]db.ContentSearchPage{ + "a": tc.a, "b": tc.b, + }} + res, err := service.NewReadOnlyBackend(store).SearchContent(t.Context(), service.ContentSearchRequest{ + Concepts: []string{"a", "b"}, Limit: tc.limit, + }) + require.NoError(t, err) + ids := make([]string, 0, len(res.Matches)) + for _, m := range res.Matches { + ids = append(ids, m.SessionID) + } + assert.Equal(t, tc.wantSessions, ids) + assert.Equal(t, tc.wantTruncated, res.CandidateTruncated) + require.Len(t, store.filters, 2) + assert.Equal(t, tc.wantLegLimit, store.filters[0].Limit) + }) + } +} + +// conceptVectorSearcher returns a fixed ranking for each concept and records +// every candidate count it was asked for. +type conceptVectorSearcher struct { + hits map[string][]db.VectorHit + ks map[string][]int +} + +func (s *conceptVectorSearcher) PrepareSemanticSearch( + _ context.Context, query string, +) (db.VectorSearchPage, int, error) { + return func(_ context.Context, k int) ([]db.VectorHit, bool, error) { + s.ks[query] = append(s.ks[query], k) + hits := s.hits[query] + return hits[:min(k, len(hits))], len(hits) < k, nil + }, 0, nil +} + +func (s *conceptVectorSearcher) ResolveMessageUnits( + _ context.Context, refs []db.MessageRef, +) ([]db.UnitRef, error) { + return make([]db.UnitRef, len(refs)), nil +} + +func TestDirectSearchContentConceptsSearchPastFilteredCandidates(t *testing.T) { + d := dbtest.OpenTestDB(t) + seedServiceSearchSession(t, d, "target", "agentsview", "auth model and retry policy") + seedServiceSearchSession(t, d, "noise", "elsewhere", "auth model") + unit := func(session string, ordinal int) db.VectorHit { + return db.VectorHit{ + SessionID: session, Ordinal: ordinal, OrdinalStart: ordinal, + OrdinalEnd: ordinal, Score: 0.5, + } + } + for _, tc := range []struct { + name string + noise int + wantSessions []string + wantTruncated bool + wantKs []int + }{ + { + // The first window holds only another project's units. + name: "match past the first window", noise: 200, + wantSessions: []string{"target"}, wantKs: []int{200, 400}, + }, + { + name: "match past the candidate cap", noise: 5000, + wantSessions: []string{}, wantTruncated: true, + wantKs: []int{200, 400, 800, 1600, 3200, 4096}, + }, + } { + t.Run(tc.name, func(t *testing.T) { + auth := make([]db.VectorHit, 0, tc.noise+1) + for i := range tc.noise { + auth = append(auth, unit("noise", i)) + } + searcher := &conceptVectorSearcher{ + hits: map[string][]db.VectorHit{ + "auth model": append(auth, unit("target", 0)), + "retry policy": {unit("target", 1)}, + }, + ks: map[string][]int{}, + } + d.SetVectorSearcher(searcher) + res, err := service.NewReadOnlyBackend(d).SearchContent(t.Context(), service.ContentSearchRequest{ + Concepts: []string{"auth model", "retry policy"}, + Project: "agentsview", Limit: 1, + }) + require.NoError(t, err) + ids := make([]string, 0, len(res.Matches)) + for _, m := range res.Matches { + ids = append(ids, m.SessionID) + } + assert.Equal(t, tc.wantSessions, ids) + assert.Equal(t, tc.wantTruncated, res.CandidateTruncated) + assert.Equal(t, tc.wantKs, searcher.ks["auth model"]) + }) + } +} + func TestDirectSearchContentTermsAndExactFilters(t *testing.T) { store := &fakeContentStore{} be := service.NewReadOnlyBackend(store) diff --git a/internal/service/service.go b/internal/service/service.go index aa173f3915..f9ca19fd88 100644 --- a/internal/service/service.go +++ b/internal/service/service.go @@ -178,7 +178,10 @@ type SessionSearchResult struct { // ContentSearchRequest is the transport-neutral content-search input. type ContentSearchRequest struct { - Pattern string `json:"pattern"` + Pattern string `json:"pattern"` + // Concepts runs one semantic search per concept and keeps only sessions + // with evidence for every one. See NormalizeContentSearchRequest. + Concepts []string `json:"concepts,omitempty"` Mode string `json:"mode,omitempty"` // substring|regex|fts|terms|semantic|hybrid Sources []string `json:"sources,omitempty"` ExcludeSystem bool `json:"exclude_system,omitempty"` @@ -212,6 +215,10 @@ type ContentSearchResult struct { NextCursor int `json:"next_cursor,omitempty"` RevisionBound bool `json:"revision_bound"` Coverage MemoryCoverage `json:"coverage"` + // CandidateTruncated reports that a multi-concept search may have missed + // sessions: a concept's candidate window filled, or more sessions matched + // every concept than the limit allowed. + CandidateTruncated bool `json:"candidate_truncated,omitempty"` } // RecallFilter mirrors GET /api/v1/recall/entries query parameters. diff --git a/internal/servicehttp/http.go b/internal/servicehttp/http.go index ed13d0f93e..2bef0c052c 100644 --- a/internal/servicehttp/http.go +++ b/internal/servicehttp/http.go @@ -816,7 +816,10 @@ func (b *httpBackend) SearchContent( ctx context.Context, req service.ContentSearchRequest, ) (*service.ContentSearchResult, error) { q := &apiclient.GetAPIV1SearchContentQuery{} - q.Pattern = req.Pattern + if req.Pattern != "" { + q.Pattern = new(req.Pattern) + } + q.Concepts = req.Concepts if req.Mode != "" { q.Mode = new(apiclient.GetAPIV1SearchContentQueryMode(req.Mode)) } @@ -915,7 +918,7 @@ func (b *httpBackend) SearchContent( q.Context = new(int64(req.Context)) } var editors []runtime.RequestEditorFn - if req.Mode == "semantic" || req.Mode == "hybrid" { + if req.Mode == "semantic" || req.Mode == "hybrid" || len(req.Concepts) > 0 { editors = append(editors, func(_ context.Context, r *http.Request) error { r.Header.Set(service.SemanticSearchIntentHeader, service.SemanticSearchIntentValue) return nil diff --git a/internal/servicehttp/http_internal_test.go b/internal/servicehttp/http_internal_test.go index c4bcb1aabf..da3990dcaf 100644 --- a/internal/servicehttp/http_internal_test.go +++ b/internal/servicehttp/http_internal_test.go @@ -339,3 +339,24 @@ func TestUsageModelGroupsQuery(t *testing.T) { }) } } + +func TestSearchContentForwardsConceptsAndSemanticIntent(t *testing.T) { + t.Parallel() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + query := r.URL.Query() + assert.Equal(t, []string{"auth model", "retry policy"}, query["concepts"]) + assert.False(t, query.Has("pattern")) + assert.Equal(t, service.SemanticSearchIntentValue, r.Header.Get(service.SemanticSearchIntentHeader)) + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"matches":[],"candidate_truncated":true}`)) + })) + t.Cleanup(srv.Close) + + res, err := NewHTTPBackend(srv.URL, "", false, "").SearchContent( + t.Context(), service.ContentSearchRequest{ + Concepts: []string{"auth model", "retry policy"}, + }, + ) + require.NoError(t, err) + assert.True(t, res.CandidateTruncated) +} diff --git a/openapi.yaml b/openapi.yaml index 7b6dd1f5eb..4523b206ea 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -1223,11 +1223,44 @@ components: - reclaimed_bytes - duration_millis type: object + DbConceptEvidence: + additionalProperties: false + properties: + concept: + type: string + ordinal: + format: int64 + type: integer + ordinal_range: + items: + format: int64 + type: integer + maxItems: 2 + minItems: 2 + type: array + score: + format: double + type: number + snippet: + type: string + transcript_revision: + type: string + required: + - concept + - ordinal + - ordinal_range + - snippet + - score + type: object DbContentMatch: additionalProperties: false properties: agent: type: string + concept_evidence: + items: + $ref: "#/components/schemas/DbConceptEvidence" + type: array context_after: items: $ref: "#/components/schemas/DbMessage" @@ -6695,6 +6728,8 @@ components: properties: active_filter_applied: type: boolean + candidate_truncated: + type: boolean coverage: $ref: "#/components/schemas/ServiceMemoryCoverage" matches: @@ -18413,14 +18448,22 @@ paths: get: operationId: get-api-v1-search-content parameters: - - description: Pattern to search for + - description: Pattern to search for; mutually exclusive with concepts explode: false in: query name: pattern - required: true schema: - description: Pattern to search for + description: Pattern to search for; mutually exclusive with concepts type: string + - description: Two to five semantic concepts that must all have evidence in one session; repeatable and mutually exclusive with pattern + explode: true + in: query + name: concepts + schema: + description: Two to five semantic concepts that must all have evidence in one session; repeatable and mutually exclusive with pattern + items: + type: string + type: array - description: Search mode explode: false in: query