Codebook-driven criteria, color, and colormap parsing with MLflow tracking#30
Merged
Merged
Conversation
…w tracking Parse machine-enforceable rules directly from the LARS codebook so labelling stays in sync with the annotator-facing spec: - criteria_from_codebook: extract hard pct/n_gates dBZ thresholds and reclassification targets from category descriptions. - color_criteria_from_codebook + COLOR_DBZ_RANGE: derive color presence/absence/exclusivity/dominance rules for validation. - colormap_from_codebook: read the Section 2.2 color-scale spec (colormap name, vmin, vmax); label_radar_data now sources vmin/vmax from the codebook when a codebook_path is given, falling back to DEFAULT_VMIN (-20) / DEFAULT_VMAX (60). - apply_criteria_to_labels: override LLM labels that violate codebook criteria, recording the original label and the rule that fired. - tracking.py: log runs, validation metrics, and a codebook hash to MLflow. - tests: cover the codebook parsers and validation metrics. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Parse machine-enforceable rules directly from the LARS codebook so labelling stays in sync with the annotator-facing spec, and add MLflow run tracking.
criteria_from_codebook— extract hardpct_gates_*/n_gates_*dBZ thresholds and reclassification targets from category descriptions.color_criteria_from_codebook(+COLOR_DBZ_RANGE) — derive color presence/absence/exclusivity/dominance rules for validation.colormap_from_codebook— read the Section 2.2 color-scale spec (colormap name,vmin,vmax).label_radar_datanow sourcesvmin/vmaxfrom the codebook when acodebook_pathis provided, falling back toDEFAULT_VMIN(-20) /DEFAULT_VMAX(60).apply_criteria_to_labels— override LLM labels that violate codebook criteria, recording the original label and the rule that fired.tracking.py— log params, validation metrics, and a codebook hash to MLflow for traceability.Motivation
The codebook's Section 2.2 color scale (
ChaseSpectral,vmin=-10,vmax=60) and the quantitative reclassification rules in each class description were previously only human-readable. This ingests them so the same spec drives inference and validation.Test plan
pytest tests/test_validation_tracking.py— 12 passed.🤖 Generated with Claude Code