Skip to content

fix(validators): vision input hardening — 0-record, label-column, bbox/size geometry#314

Merged
shujaatTracebloc merged 7 commits into
developfrom
fix/vision-input-validators
Jun 18, 2026
Merged

fix(validators): vision input hardening — 0-record, label-column, bbox/size geometry#314
shujaatTracebloc merged 7 commits into
developfrom
fix/vision-input-validators

Conversation

@shujaatTracebloc

@shujaatTracebloc shujaatTracebloc commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Adversarial image-classification ingestion — re-verified against the merged develop validator chain (not the stale deployed image) — showed the #303 zero-record guard and #313 label-column guard were wired NLP-only, so vision still had the same two gaps. This extends both to the vision factories.

Gaps (confirmed on develop HEAD via the real map_validators chain)

Case Before Cause
header-only / empty CSV ✅ passes preflight → ingests 0 rows → orphan empty table + late "rows already in the database" no IngestableRecordsValidator
missing configured label column ✅ passes preflight → records clean to label=None → backend rejects each row HTTP 400 "label: may not be null" no LabelColumnValidator

(corrupt image, wrong resolution, single label, all-images-missing were already caught.)

Fix

  • New _zero_record_validator helper wires IngestableRecordsValidator(file_subdir="images") into all file-bearing vision categoriesimage_classification, object_detection, semantic_segmentation, keypoint_detection — so a zero-record vision manifest fails fast at preflight (no orphan table).
  • LabelColumnValidator added to image_classification only — it is the only vision category whose label is a CSV column. Object detection / segmentation / keypoint source labels from XML / masks / annotation files, so adding it there would wrongly reject every such dataset.
  • TextContentValidator stays NLP-only (it decodes UTF-8 text — meaningless for images).

Resulting image_classification chain: FileType → IngestableRecords → ImageResolution → LabelColumn → LabelDiversity → TableName → Duplicate.

Tests

  • Mapping: image gets both guards (exact-list updated); OD/seg/keypoint get the zero-record guard only and never LabelColumnValidator; updated the prior NLP-only content-hygiene assertion (TextContent stays NLP-only).
  • Behavioral (test_vision_input_validators.py): real chain on staged images — empty CSV → IngestableRecordsValidator reject; missing label column → LabelColumnValidator reject; valid → passes.
  • Full suite: 1209 passed, 1 xfailed, total coverage 96.7% (gate 95%). black-clean.

Follow-up to #303 / #313, extending the same hardening from NLP to vision.

🤖 Generated with Claude Code


Note

Medium Risk
Changes expand which datasets fail at preflight across all vision modalities and add coordinate/size logic in VOC and keypoint validators; behavior is well-tested but mis-tuned target_size or edge cases in best-effort image lookup could reject previously accepted datasets.

Overview
Extends NLP-style preflight guards to vision ingest: IngestableRecordsValidator (via _zero_record_validator) is wired into all file-bearing vision categories so header-only/empty manifests fail before creating orphan tables; LabelColumnValidator is added only to image_classification (other vision tasks don’t use a CSV label column).

Semantic segmentation adds a second ImageResolutionValidator with subdir="masks" so masks are readable and match target_size, not just images under images/.

Keypoint detection passes target_size into KeypointAnnotationValidator as expected_resolution to reject coords outside [0, W) × [0, H), with negative and out-of-bounds errors reported independently.

Object detection PascalVOCXMLValidator now rejects bboxes past declared <size>, cross-checks declared dimensions against the stem-paired file under SRC_PATH/images, and resolves images from SRC_PATH (not XML parent paths).

Tests cover mapping chains, end-to-end image-classification preflight, masks/keypoints/XML geometry, plus minor formatting in existing tests.

Reviewed by Cursor Bugbot for commit 2fa816c. Bugbot is set up for automated code reviews on this repo. Configure here.


Added: semantic segmentation + keypoint detection (vision sweep complete)

Semantic segmentation — masks were unvalidated (ImageResolutionValidator only scans <SRC>/images):

  • mask resolution ≠ image → passed (breaks pixel-wise labels); corrupt/unreadable mask → passed (only .png extension checked).
  • Fix: ImageResolutionValidator gains a subdir param; seg factory adds a subdir="masks" instance → masks must be readable + share the images' target resolution.

Keypoint detection — keypoint coords past the image edge passed (validator never saw image size; only non-negative was checked).

  • Fix: thread target_size into KeypointAnnotationValidator as expected_resolution; coords > width/height now rejected.

Not changed (pending product call): keypoint Visibility stays restricted to 0/1. Accepting COCO's 0/1/2 is a format decision (could admit data the runtime doesn't support), so it's flagged rather than silently relaxed.

Vision sweep now complete across all four categories (image, object detection, semantic segmentation, keypoint).

…gories

Adversarial image-classification ingestion (re-verified against the merged
develop chain) showed the #303 zero-record guard and #313 label-column guard
were wired NLP-only, so vision still had the same gaps:
- header-only / empty CSV -> 0 records -> orphan empty table + late "rows
  already in the database" failure (no IngestableRecordsValidator).
- missing configured label column -> records clean to label=None -> backend
  rejects each row with HTTP 400 "label: may not be null" (no LabelColumnValidator).

Fix:
- New `_zero_record_validator` helper wires IngestableRecordsValidator
  (file_subdir="images") into ALL file-bearing vision categories —
  image_classification, object_detection, semantic_segmentation,
  keypoint_detection — so a zero-record vision manifest fails fast at preflight.
- LabelColumnValidator added to image_classification ONLY: it is the only
  vision category whose label is a CSV column. Object detection / segmentation /
  keypoint source labels from XML / masks / annotation files, so adding it there
  would wrongly reject every such dataset.
- TextContentValidator stays NLP-only (it decodes UTF-8 text — meaningless for
  images).

Tests: mapping assertions (image gets both guards; OD/seg/keypoint get the
zero-record guard only, never LabelColumn); behavioral end-to-end proving image
empty-CSV -> IngestableRecords reject and missing-label-column -> LabelColumn
reject, valid passes; updated the prior NLP-only content-hygiene assertion.
Full suite: 1209 passed, 1 xfailed, total coverage 96.7% (gate 95%).

Follow-up to #303 / #313, extending the same hardening from NLP to vision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Collaborator

👋 Heads-up — Code review queue is at 35 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

…s (object detection)

Adversarial object-detection ingestion (against the latest chain) found two
PascalVOCXMLValidator gaps — both let malformed annotations through to training:

1. Bounding box outside the image: coords were checked non-negative and
   xmin<xmax / ymin<ymax, but NEVER bounded by the declared <size>. A box with
   xmax/ymax past the image edge (e.g. 9999 on a 64x64 image) passed.
2. Declared <size> never cross-checked against the ACTUAL image: an annotation
   declaring 128x128 for a real 64x64 image passed, silently corrupting the
   coordinate scaling training relies on (and letting a bbox "within" a
   too-large declared size sit outside the real image).

Fix:
- Thread the declared width/height from <size> into the bbox check; error when
  xmax > width or ymax > height.
- New `_validate_size_matches_image`: locate the image at
  <SRC>/images/<filename> (annotation's <filename>, else the xml stem +
  jpg/jpeg/png), read its real dimensions, and error on a mismatch with the
  declared <size>. Best-effort: a benign skip when the image is absent/unreadable
  or Pillow is missing (FilePairing/FileType own the missing-image case).

Tests: bbox exceeding declared width/height fails; within-size passes; declared
size matching the actual image passes; mismatch fails; absent image skips the
cross-check. Full suite: 1215 passed, 1 xfailed, coverage 96.6% (gate 95%);
xml_validator 99%.

Part of #314 — extends the vision input-validation hardening to OD annotations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@shujaatTracebloc shujaatTracebloc changed the title fix(validators): extend 0-record + label-column guards to vision categories fix(validators): vision input hardening — 0-record, label-column, bbox/size geometry Jun 18, 2026
Comment thread tracebloc_ingestor/validators/xml_validator.py
…314)

The size<->image cross-check resolved the image from the annotation's
<filename> element first. But the ingestor pairs images to annotations by STEM
(FilePairingValidator: annotations/<stem>.xml <-> images/<stem>.<ext>), so a
stale/wrong <filename> could point the check at a different on-disk image that
happens to match the declared <size> while the actually-paired image still
disagrees. Resolve the image by XML stem only (jpg/jpeg/png), matching the real
pairing. Tests updated to pair ann.xml with ann.jpg.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread tracebloc_ingestor/validators/xml_validator.py
shujaatTracebloc and others added 2 commits June 18, 2026 10:49
…parent (bugbot #314)

`_validate_size_matches_image` derived the images folder as
`file_path.parent.parent/images`, which only holds when the XML sits exactly at
`<root>/annotations/<stem>.xml`. `validate()` discovers XML via a recursive
`**/*.xml` glob, so a root-level or deeper-nested XML would resolve the wrong
images tree (or skip the check). Use `SRC_PATH/images` — the same root
FileTypeValidator(path="images") uses — for a consistent, correct lookup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…weep)

Adversarial testing of the remaining vision categories found three gaps where
malformed data reached training:

Semantic segmentation — masks were unvalidated (ImageResolutionValidator only
scans <SRC>/images):
1. a mask whose resolution differs from its image passed (breaks pixel-wise
   labels);
2. a corrupt / unreadable mask passed (only the .png extension was checked).
Fix: ImageResolutionValidator gains a `subdir` param; the seg factory adds a
second instance with subdir="masks" (name "Mask Resolution Validator"), so masks
must be readable and share the images' target resolution.

Keypoint detection:
3. keypoint coordinates past the image edge passed (the validator never saw the
   image size; coords were only checked non-negative).
Fix: thread the declared target_size into KeypointAnnotationValidator as
`expected_resolution`; coords above width/height are now rejected.

Not changed: keypoint Visibility is still restricted to 0/1. Whether to accept
COCO's 0/1/2 is a product/format decision (could let through data the runtime
doesn't support), so it's left pending confirmation rather than silently relaxed.

Tests: mask subdir validation (valid / wrong-res / corrupt), keypoint coord
bounds (within / out-of-bounds / no-resolution no-op), and factory wiring (seg
has images+masks resolution validators; keypoint annotation validator gets
target_size). Full suite: 1224 passed, 1 xfailed, coverage 96.6% (gate 95%).

Completes the vision input-validation sweep in this PR (image, object detection,
semantic segmentation, keypoint).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread tracebloc_ingestor/validators/keypoint_annotation_validator.py
The upper-bound coord check used `x > width`, so a coordinate exactly equal to
the width/height passed — but pixel indices run 0..W-1, so `x == W` is the first
index past the image (asymmetric with the inclusive-0 lower bound). Use
`x >= width` / `y >= height` (half-open [0, W) / [0, H)). Boundary test added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ba5cb07. Configure here.

Comment thread tracebloc_ingestor/validators/keypoint_annotation_validator.py
…ly (bugbot #314)

The bounds check was an `elif` after the negative-coordinate check, so a row like
(-1, 9999) reported only the negative x and hid the out-of-bounds y until a
re-ingest. Run the two checks independently so both surface in one pass. Test
added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@shujaatTracebloc shujaatTracebloc merged commit b677032 into develop Jun 18, 2026
4 checks passed
@shujaatTracebloc shujaatTracebloc deleted the fix/vision-input-validators branch June 18, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants