Summary
HNSW vector search latency appears to scale linearly with the number of indexed vectors rather than logarithmically. Measured on Harper 5.1.19 across three tables in the same instance, cost is roughly 25–30 µs per indexed vector, and is independent of the requested neighbour count.
At current sizes this is tolerable (~65 ms on 2.7k vectors) but it sets the ceiling for how far a vector index can grow: extrapolating, a 27k-vector table would put a single search near 650 ms.
I may well be holding it wrong — if there's a query shape or index option that changes this, I'd be glad to hear it.
Measurements
Three tables in one instance, each embedding: [Float] @indexed(type: "HNSW", distance: "cosine"), 768-dim vectors (nomic-embed-text). Timing is measured server-side inside the resource around the search call only, so it excludes network, request parsing, and embedding of the query.
| table |
indexed vectors |
search time (steady state) |
µs / vector |
| skills |
23 |
3 ms |
130 |
| docs |
2,729 |
63–66 ms |
24 |
| code |
4,395 |
131–142 ms |
31 |
The query is:
const results = table.search({
sort: { attribute: 'embedding', target: queryVector }, // 768 floats
limit,
select: ['id', '$distance'],
});
for await (const row of results) { /* collect */ }What this is not
- Not a cold cache. The same query repeated 5× stays at 63–66 ms and never drops below it. First call on a table is higher (~85–98 ms), then it settles — so there is a warm-up, but the steady state is still corpus-proportional.
- Not a missing index.
describe_table confirms the index is live:
{"attribute":"embedding","type":"array","elements":"Float",
"indexed":{"type":"HNSW","distance":"cosine"}}
- Not per-returned-row cost. Sweeping
limit across 5 / 40 / 80 / 200 / 400 on the 2,729-vector table produced no measurable trend (65–213 ms, all noise). Cost tracks corpus size, not result count.
A possibly related observation
On the same 2,729-vector table, a single search with limit greater than the table size returns only ~221 rows rather than everything reachable:
limit: 2502 → 221 rows returned
I was trying to enumerate the graph to detect unreachable vectors, and this made that approach unusable. It suggests traversal breadth is capped independently of limit, which may or may not be connected to the timing above.
Reproduction
- Create tables with an HNSW-indexed
[Float] attribute at different row counts (a few dozen, a few thousand).
- Populate with 768-dim vectors.
- From a resource, time only the
table.search({sort:{attribute, target}, limit, select:['id','$distance']}) iteration.
- Compare µs-per-vector across the tables, and sweep
limit within one table.
Environment
- Harper 5.1.19 (
registration_info reports deprecated: true, so this may already be addressed on a newer build — happy to re-measure on current)
- Harper Fabric, 2-node cluster, 16 HTTP workers
- Vectors written via the operations API (
insert/update), embeddings precomputed client-side
- Distance: cosine, 768 dimensions
Why it matters to us
This is a documentation-search application where vector search is ~half of query latency, and the corpus grows with the docs. The linear component is the part that worries us over time rather than today's absolute number.
Summary
HNSW vector search latency appears to scale linearly with the number of indexed vectors rather than logarithmically. Measured on Harper 5.1.19 across three tables in the same instance, cost is roughly 25–30 µs per indexed vector, and is independent of the requested neighbour count.
At current sizes this is tolerable (~65 ms on 2.7k vectors) but it sets the ceiling for how far a vector index can grow: extrapolating, a 27k-vector table would put a single search near 650 ms.
I may well be holding it wrong — if there's a query shape or index option that changes this, I'd be glad to hear it.
Measurements
Three tables in one instance, each
embedding: [Float] @indexed(type: "HNSW", distance: "cosine"), 768-dim vectors (nomic-embed-text). Timing is measured server-side inside the resource around the search call only, so it excludes network, request parsing, and embedding of the query.The query is:
What this is not
describe_tableconfirms the index is live:{"attribute":"embedding","type":"array","elements":"Float", "indexed":{"type":"HNSW","distance":"cosine"}}limitacross 5 / 40 / 80 / 200 / 400 on the 2,729-vector table produced no measurable trend (65–213 ms, all noise). Cost tracks corpus size, not result count.A possibly related observation
On the same 2,729-vector table, a single search with
limitgreater than the table size returns only ~221 rows rather than everything reachable:I was trying to enumerate the graph to detect unreachable vectors, and this made that approach unusable. It suggests traversal breadth is capped independently of
limit, which may or may not be connected to the timing above.Reproduction
[Float]attribute at different row counts (a few dozen, a few thousand).table.search({sort:{attribute, target}, limit, select:['id','$distance']})iteration.limitwithin one table.Environment
registration_inforeportsdeprecated: true, so this may already be addressed on a newer build — happy to re-measure on current)insert/update), embeddings precomputed client-sideWhy it matters to us
This is a documentation-search application where vector search is ~half of query latency, and the corpus grows with the docs. The linear component is the part that worries us over time rather than today's absolute number.