Skip to content

refactor(validators): compose common validators centrally (supersedes #316)#317

Merged
shujaatTracebloc merged 1 commit into
developfrom
refactor/centralize-common-validators
Jun 18, 2026
Merged

refactor(validators): compose common validators centrally (supersedes #316)#317
shujaatTracebloc merged 1 commit into
developfrom
refactor/centralize-common-validators

Conversation

@shujaatTracebloc

@shujaatTracebloc shujaatTracebloc commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

The 0-record guard, TableNameValidator, DuplicateValidator, and LabelDiversityValidator were copy-pasted across the per-category factoriesTableName+Duplicate in all 11, the 0-record guard wired three different ways (NLP _nlp_content_validators, vision _zero_record_validator, inline for tabular), label-diversity in each of the 6 classification factories. That repetition is exactly what let the tabular 0-record gap happen — a factory just forgot to add it.

This centralizes the universally-applicable validators in map_validators, driven by declarative ModalitySpec traits, so each factory returns only its category-specific validators.

Composition (one place)

[IngestableRecordsValidator(file_subdir=spec.file_subdir)]   # every category
  + spec.build_validators(options)                           # category-specific
  + [LabelDiversityValidator] if spec.is_classification
  + [TableNameValidator, DuplicateValidator]                 # every category

Changes

  • New ModalitySpec traits: file_subdir (images/texts/sequences/None) and is_classification.
  • map_validators builds the common frame around the factory output.
  • Factories drop the repeated lines; deleted _zero_record_validator; _nlp_content_validators_text_content_validator (text-content only); label_diversity_validator made public.

Behavior

Set-preserving — the resulting validator set per category is unchanged, except tabular_classification / tabular_regression / time_series_forecasting / time_to_event_prediction now correctly carry the 0-record guard. This closes the tabular gap centrally and supersedes #316. Only order changes: the 0-record guard now leads (fail-fast); pass/fail is order-independent.

Tests

Updated the order-sensitive mapping assertions; added test_common_validator_frame_composed_for_every_category that locks the frame (guard first, table/duplicate last, label-diversity for classification only). Full suite: 1226 passed, 1 xfailed, coverage 96.6% (gate 95%).

🤖 Generated with Claude Code


Note

Medium Risk
Touches the preflight validator chain for all 11 task categories and intentionally adds 0-record checks to tabular/time paths; regression risk is mitigated by broad mapping tests but empty-manifest failures will surface earlier for those categories.

Overview
Shared preflight validators (IngestableRecordsValidator, optional LabelDiversityValidator, TableNameValidator, DuplicateValidator) are no longer duplicated in each per-category factory. map_validators now wraps every ModalitySpec.build_validators output in a single frame, driven by new spec fields file_subdir and is_classification on ModalitySpec / registry.py.

Per-category factories in modalities/validators.py return only category-specific checks; _zero_record_validator is removed and NLP wiring is split so _text_content_validator handles UTF-8 content while the 0-record guard is built centrally from spec.file_subdir.

Behavior: Validator sets stay the same for most categories, but tabular / time-series families now get the 0-record guard (header-only / empty CSV), closing the gap called out in #316. Order changes so the guard runs first for fail-fast; pass/fail is otherwise order-independent.

Tests update expected ordering and add test_common_validator_frame_composed_for_every_category to lock guard-first, table/duplicate-last, and label-diversity only on classification categories.

Reviewed by Cursor Bugbot for commit e20467a. Bugbot is set up for automated code reviews on this repo. Configure here.

…tegory

The 0-record guard, table-name, duplicate, and label-diversity validators were
copy-pasted across the per-category factories: TableName + Duplicate in all 11,
the 0-record guard wired three different ways (NLP `_nlp_content_validators`,
vision `_zero_record_validator`, and inline for tabular), and label-diversity
called in each of the 6 classification factories. That repetition is what made
the tabular 0-record gap possible (a factory simply forgot to add it).

Centralize the universally-applicable validators in `map_validators`, driven by
declarative `ModalitySpec` traits, so each factory returns ONLY its
category-specific validators:

    [IngestableRecordsValidator(file_subdir=spec.file_subdir)]   # every category
      + spec.build_validators(options)                          # category-specific
      + [LabelDiversityValidator] if spec.is_classification
      + [TableNameValidator, DuplicateValidator]                # every category

- New `ModalitySpec` traits: `file_subdir` (images/texts/sequences/None) and
  `is_classification`.
- `map_validators` builds the common frame; factories drop the repeated lines.
- Deleted `_zero_record_validator`; `_nlp_content_validators` collapses to the
  text-content check (`_text_content_validator`); `label_diversity_validator`
  is now public (composed by map_validators).

Behavior-preserving — the resulting validator SET per category is unchanged,
EXCEPT tabular_classification / tabular_regression / time_series_forecasting /
time_to_event_prediction now correctly get the 0-record guard (closing the
tabular gap centrally; supersedes #316). Order changes only in that the 0-record
guard now leads (fail-fast) — pass/fail behavior is order-independent.

Tests: updated order-sensitive mapping assertions; added
`test_common_validator_frame_composed_for_every_category` locking the frame.
Full suite: 1226 passed, 1 xfailed, coverage 96.6% (gate 95%).
@shujaatTracebloc shujaatTracebloc merged commit 9b5bc41 into develop Jun 18, 2026
7 checks passed
@shujaatTracebloc shujaatTracebloc deleted the refactor/centralize-common-validators branch June 18, 2026 12:52
shujaatTracebloc added a commit that referenced this pull request Jun 19, 2026
Wire causal_language_modeling as a first-class supported modality across
every per-category dispatch site, mirroring how masked_language_modeling
(self-supervised) and token_classification (texts/ raw-text layout) are
already wired. The training-container side (tracebloc-client) already
supports it; this closes the ingestor-side gap (was 0 references).

Causal LM is self-supervised (only a `filename` column, no label) and NLP
(ships the #805 data-derived text profile for the contributor-tokenizer-fit
check). Each sample is one `.txt` of either plain text (pretraining) or a
tab-separated `prompt\tcompletion` pair (SFT) — both ordinary UTF-8 text, so
the centralized TextContentValidator is the whole content check (no bespoke
per-modality validator; honors the #317 centralization).

It stages raw text from `texts/`, NOT MLM's `sequences/` — the framework
reserves `sequences/` for pre-tokenized data. Decoder-only models tie
pad=eos, so there is no [MASK] requirement; the vocab+pad alignment check
lives on the training-client side and is fed by the is_nlp-gated text
profile here (the ingest-time tokenizer was dropped in #299).

Production wiring:
- constants: CAUSAL_LANGUAGE_MODELING + get_all_categories()
- registry: ModalitySpec (file-bearing, self-supervised, NLP, TEXT,
  file_subdir="texts")
- modalities/validators: causal_language_modeling factory (FileType +
  TextContentValidator + optional DataValidator; no label validators)
- modalities/transfer: causal_language_modeling -> text_transfer (texts/)
- cli/conventions: added to TEXT_CATEGORIES (.txt default, texts/ SRC_PATH)
- schema/ingest.v1.json: enum, "requires texts" rule, self-supervised
  "must not set label" rule (#213), texts description
- text_content_validator: docstrings

Template + example:
- templates/causal_language_modeling/ (script delegating to run_ingestion,
  README, labels CSV, 5 samples: 3 plain-text + 2 SFT tab pairs)
- examples/yaml/causal_language_modeling.yaml + root README table

Tests:
- updated every category enumeration (registry NLP set, template-category
  list, ALL_CATEGORIES, NLP/non-classification lists, text-profile params,
  equivalence CASES, e2e cases) + schema accept/reject
- new tests/test_causal_language_modeling.py: CLI run, validation boundaries
  (header-only / all-files-missing fail-fast, binary reject, clean pass),
  failure accounting

Full unit suite: 1250 passed, 1 xfailed.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
shujaatTracebloc added a commit that referenced this pull request Jun 25, 2026
* chore(release): bump version to 0.5.2

Lands the __version__ bump on develop. v0.5.1 was tagged to ship
causal_language_modeling (#318); develop had drifted to 0.4.0. Set
develop to 0.5.2 so it sits one patch ahead of the last released tag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: add seq2seq modality

Wire seq2seq (sequence-to-sequence / encoder-decoder) as a first-class
supported modality across every per-category dispatch site, mirroring the
causal_language_modeling enablement (#318). The training-container side
already supports it; this closes the ingestor-side gap (was 0 references).

seq2seq is self-supervised (only a `filename` column, no label) and NLP
(ships the #805 data-derived text profile for the contributor-tokenizer-fit
check). Each sample is one `.txt` holding a tab-separated `source\ttarget`
pair — the same on-disk shape as causal LM's `prompt\tcompletion`, ordinary
UTF-8 text — so it is wired identically to causal LM: it stages raw text from
`texts/` (not MLM's pre-tokenized `sequences/`), and the centralized
TextContentValidator is the whole content check (no bespoke per-modality
validator; honors the #317 centralization).

Production wiring:
- constants: SEQ2SEQ + get_all_categories()
- registry: ModalitySpec (file-bearing, self-supervised, NLP, TEXT,
  file_subdir="texts")
- modalities/validators: seq2seq factory (FileType + TextContentValidator +
  optional DataValidator; no label validators)
- modalities/transfer: seq2seq -> text_transfer (texts/)
- cli/conventions: added to TEXT_CATEGORIES (.txt default, texts/ SRC_PATH)
- schema/ingest.v1.json: enum, "requires texts" rule, self-supervised
  "must not set label" rule (#213), texts description
- text_content_validator: docstrings

Template + example:
- templates/seq2seq/ (script delegating to run_ingestion, README, labels CSV,
  5 source<TAB>target samples)
- examples/yaml/seq2seq.yaml + root README table

Tests:
- updated every category enumeration (registry NLP set, template-category
  list, ALL_CATEGORIES, NLP/non-classification lists, text-profile params,
  equivalence CASES, e2e cases) + schema accept/reject
- new tests/test_seq2seq.py: CLI run, validation boundaries (header-only /
  all-files-missing fail-fast, binary reject, clean pass), failure accounting

Full unit suite: 1273 passed, 1 xfailed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
shujaatTracebloc added a commit that referenced this pull request Jun 30, 2026
* feat: add embeddings (self-supervised contrastive) modality

Enable the NLP embeddings / contrastive use case end-to-end on the ingestor
side, mirroring how seq2seq is wired. The on-disk contract is one .txt per
sample under texts/, each a tab-separated anchor<TAB>positive pair OR
anchor<TAB>positive<TAB>negative triplet (no label column).

Wiring (mirrors seq2seq — closest sibling):
- utils/constants.py: TaskCategory.EMBEDDINGS.
- schema/ingest.v1.json: enum value, texts requirement, and the
  self-supervised no-label guard (#213).
- modalities/registry.py: ModalitySpec (file-bearing, self-supervised, NLP,
  texts/) — is_nlp gates the #805 data-derived text profile (the ingestor side
  of the federated tokenizer-alignment check).
- modalities/transfer.py + cli/conventions.py: raw-text staging from texts/.

What's distinct from seq2seq: the on-disk shape is STRUCTURED, so beyond the
shared TextContentValidator (UTF-8/binary hygiene) the embeddings factory adds
a centralized ContrastivePairsValidator that rejects any .txt that is not
exactly 2 or 3 non-empty tab fields (no plain prose, no empty fields, one
record per file).

Template fails loudly via the shared run_ingestion helper (no exit-0-on-failure);
shared validation lives in the centralized validators (#317).

Adds templates/embeddings (script + README + sample pairs/triplets),
examples/yaml/embeddings.yaml, and full test coverage (new + updated
enumerating tests, e2e params). Suite green; new validator at 100% coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: resolve embeddings sidecar paths via _safe_join (Bugbot)

ContrastivePairsValidator joined the manifest filename onto SRC_PATH/texts
with plain os.path.join, so an absolute / `..` manifest value could be
validated and read from outside texts/ even though text_transfer rejects it
(#239) — preflight could disagree with copy behavior. Resolve with _safe_join
and skip on ValueError, exactly as TextContentValidator does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
shujaatTracebloc added a commit that referenced this pull request Jul 1, 2026
* feat: add sentence_pair_classification modality

Enable the NLP sentence-pair classification use case end-to-end on the
ingestor side (Phase-4 cross-repo wiring for engine PR #301). The on-disk
contract is one .txt per sample under texts/, each a tab-separated
text_a<TAB>text_b sentence pair, plus a class label in the labels CSV.

sentence_pair is SUPERVISED text classification — the label travels in the
CSV exactly like text_classification (is_classification=True,
is_self_supervised=False, staged from texts/) — but its .txt has a STRUCTURED
shape (the text_classification layout with a tab separating the pair).

Wiring (mirrors text_classification; structural check mirrors embeddings):
- utils/constants.py: TaskCategory.SENTENCE_PAIR_CLASSIFICATION.
- schema/ingest.v1.json: enum value, texts requirement + description, and the
  "requires label" rule (supervised — NOT the self-supervised no-label guard).
- modalities/registry.py: ModalitySpec (file-bearing, classification, NLP,
  texts/) — is_nlp gates the #805 data-derived text profile (the ingestor side
  of the federated tokenizer-alignment check).
- modalities/validators.py + transfer.py + cli/conventions.py: same validator
  set + raw-text staging from texts/ as text_classification.

What's distinct from text_classification: the on-disk shape is STRUCTURED, so
beyond the shared TextContentValidator (UTF-8/binary hygiene) it adds a
structural check that rejects any .txt that isn't exactly 2 non-empty tab
fields (no plain prose, no empty side, one record per file).

Rather than clone ContrastivePairsValidator, extract the shared scaffold into
a new TabSeparatedRecordValidator base (centralized validators, #317);
ContrastivePairsValidator (2-or-3 fields) subclasses it with byte-identical
messages (its tests pass unchanged) and SentencePairValidator (exactly-2) is a
thin subclass. Template fails loudly via run_ingestion (no exit-0-on-failure).

Adds templates/sentence_pair_classification (script + README + sample pairs),
examples/yaml/sentence_pair_classification.yaml, and full test coverage (new +
updated enumerating tests, e2e params). Unit suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(release): bump version to 0.5.5

Ships the sentence_pair_classification modality; v0.5.5 will be the first
image to contain it. Bumping in this PR so the release tag can be cut
straight from develop after merge (no separate bump PR needed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants