Skip to content

fix(import): classify label entities when restoring a bank (#3236) - #3237

Merged
nicoloboschi merged 1 commit into
mainfrom
fix/import-label-entity-kind
Aug 7, 2026
Merged

fix(import): classify label entities when restoring a bank (#3236)#3237
nicoloboschi merged 1 commit into
mainfrom
fix/import-label-entity-kind

Conversation

@nicoloboschi

Copy link
Copy Markdown
Collaborator

Fixes #3236.

Found while benchmarking #3208/#3214 against a real production export.

The bug

import_bank_async resolved the target bank's config before import_bank
restored the archive's bank row:

resolved_config = await self._config_resolver.resolve_full_config(bank_id, request_context)
return await import_bank(..., config=resolved_config, ...)

The bank cannot exist at that point — import explicitly refuses to write into an
existing bank — so that resolve only ever sees global + tenant config. The bank's
own entity_labels, which the archive carries in banks.json, arrives later. Retain
Phase 1 then classified every label entity in the imported bank as regular.

Consequences:

The fix

import_bank now takes resolve_config (a callback) instead of a resolved config,
and calls it right after the bank row is restored. This is deliberately not an
optional extra parameter alongside config: the only correct moment to resolve is
inside import_bank, so making it the only way to supply one keeps the next caller
from reintroducing the bug. import_bank_async is the sole caller.

Measured on a real export

862 documents / 38,162 facts, bank has a free-text brief_bio label group. Imported
into an emptied instance (all other banks dropped — the trigram index has no
bank_id, so a leftover bank pollutes any measurement here):

pre-#3187 baseline main as shipped main + this fix
entity resolution during import 471.75 s 382.61 s 26.01 s
share of import spent resolving 45.8% 40.9% 3.1%
import wall-clock 1030.76 s 934.93 s 850.00 s
entities stamped label n/a (no column) 0 5,719

18.1× less entity-resolution work than the baseline and 14.7× less than main as
shipped
— on this bank the defect was hiding nearly the whole benefit of #3214.

(The 500 label rows visible on unfixed main in the issue came from a post-import
ingest burst, not the import. A post-hoc backfill classified 5,355; the fixed import
produces 5,719 because the broken path had fuzzy-merged some distinct label values
into each other.)

Test

test_bank_import_classifies_label_entities: configure entity_labels, export,
delete, import, assert the label entity returns as entity_kind='label' and a
regular one as 'regular'. Verified it fails on main with 'regular' != 'label'.

test_document_transfer.py, test_admin_bank_transfer.py,
test_repair_bank_vector_indexes.py, test_hermes_templates_import.py,
test_entity_resolver.py and test_entity_labels.py pass; lint.sh + ty clean.

`import_bank_async` resolved the target bank's config before the archive's bank
row was restored. The bank cannot exist at that point — import refuses to write
into an existing bank — so the resolve saw only global + tenant config and never
the bank's own `entity_labels`, which arrives with the archive. Retain Phase 1
then classified every label entity as regular for the whole import.

That silently disabled #3208/#3214 on imported banks: the partial trigram index
`WHERE entity_kind != 'label'` excluded nothing, so every fuzzy probe kept paying
the recheck-discard cost the index removes. It also let the import fuzzy-merge
distinct label values, which the exact-match-only path (#3187) exists to prevent.
The migration backfill does not cover it — it runs once, and a bank imported
afterwards has nothing to correct it.

Measured on an 862-document production export whose bank has a free-text label
group: 5,374 label-shaped entities, of which the import marked 0 as labels.
Correcting the classification takes entity-resolution p50 from 263 ms to 37 ms
and a single fuzzy probe from 8.28 ms to 0.38 ms.

import_bank now takes a `resolve_config` callback and re-resolves once the bank
row is in place, replaying the documents with the bank's own config.
@nicoloboschi
nicoloboschi force-pushed the fix/import-label-entity-kind branch from 8f30aa1 to 1ce8f5b Compare August 7, 2026 10:42
@nicoloboschi
nicoloboschi merged commit 5394846 into main Aug 7, 2026
104 checks passed
@nicoloboschi
nicoloboschi deleted the fix/import-label-entity-kind branch August 7, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

import: bank import never classifies label entities, silently disabling the #3208 partial trigram index

1 participant