Skip to content

[犀牛鸟-D1]:冻结基础模型xLatent Mixture(贾宗翰) - #239

Open
xizaoge-shuai wants to merge 1 commit into
Tencent:mainfrom
xizaoge-shuai:d1-final-offline-dinov3-latent
Open

[犀牛鸟-D1]:冻结基础模型xLatent Mixture(贾宗翰)#239
xizaoge-shuai wants to merge 1 commit into
Tencent:mainfrom
xizaoge-shuai:d1-final-offline-dinov3-latent

Conversation

@xizaoge-shuai

Copy link
Copy Markdown

Summary

  • Add deterministic offline multi-layer DINOv3 feature caching.
  • Train only LatentMixture-P3/P4/P5 and the Detect head without loading the teacher during training.
  • Add cache validation, GPU/CPU residency, trainability auditing, student-only checkpoint checks, experiments, ablations, and reproducibility evidence.

The complete experimental report is available at docs/d1/D1_FINAL_EXPERIMENT_REPORT.md.

Test evidence

  • D1 regression suite: 36 passed, 1 non-blocking warning.
  • COCO128 warm training cost reduction: 78.75%.
  • VisDrone500 cold cost reduction: 67.07%.

Ablations

  • Stream vs CPU preload vs GPU preload.
  • Auxiliary loss weights: 0, 0.01, 0.05, 0.1, and 0.2.
  • DINOv3 ViT-S/16 vs ViT-L/16 with three training seeds.

Limitations

  • Experiments use deterministic COCO128 and VisDrone500 subsets.
  • GPU residency trades additional VRAM for training throughput.
  • The multi-seed results do not establish statistically significant gains for aux=0.01 or ViT-L.

@isLinXu
isLinXu marked this pull request as ready for review August 28, 2026 08:03
@isLinXu

isLinXu commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the review. You're right that this PR carried too many files that don't belong in the merge. I audited all 27 changed files and will trim the branch so that only the D1 core lands.

Files to be removed (internal / process artifacts, not part of the feature)

These are local working artifacts from the admission-gate and experiment workflow. They contain machine-specific paths and one-off evidence that should not be merged into the repo:

  • docs/d1/evidence/ — all 14 files (run JSONs, SHA256 pin lists, commit/diff snapshots; e.g. p0-coco128-s0.json contains a local run path /data/users/zhjia/...)
  • docs/d1/824-admission-status.md — internal admission-gate status doc
  • docs/d1/p0-status.md — internal P0 status doc
  • docs/superpowers/plans/2026-08-25-d1-824-feature-cache.md — personal agent-workspace plan
  • docs/superpowers/specs/2026-08-25-d1-offline-dinov3-cache-design.md — personal agent-workspace spec

Files that stay (the D1 feature core)

  • ultralytics/data/foundation_cache.py — deterministic cache contracts (schema, identity, manifest validation)
  • ultralytics/data/preloaded_feature_cache.py — preloaded batched loader (GPU/CPU residency)
  • scripts/d1_build_feature_cache.py — offline DINOv3 multi-level cache builder
  • scripts/d1_train_cached_detector.py — cached-feature LatentMixture + Detect trainer
  • tests/test_d1_foundation_cache.py, tests/test_d1_feature_cache_builder.py, tests/test_d1_preloaded_feature_cache.py, tests/test_d1_cached_detector.py
  • docs/d1/D1_FINAL_EXPERIMENT_REPORT.md — consolidated experiment report (single doc replaces the scattered JSON evidence)

Plan

I'll rebase the branch to drop the files above and force-push. The experiment numbers quoted in the report stay unchanged. After the trimmed branch is up, please take another look.

@xizaoge-shuai
xizaoge-shuai force-pushed the d1-final-offline-dinov3-latent branch from 4b68efd to 82b4d95 Compare August 28, 2026 10:45
@xizaoge-shuai

Copy link
Copy Markdown
Author

Thanks for the review. I rebuilt the branch on the current upstream main and trimmed the PR to the D1 feature core only. Internal admission/evidence artifacts and personal workspace documents have been removed; the consolidated experiment report remains. The focused D1 tests pass. Please take another look when convenient.

@xizaoge-shuai

xizaoge-shuai commented Sep 9, 2026 via email

Copy link
Copy Markdown
Author

@xizaoge-shuai xizaoge-shuai changed the title [D1] Offline DINOv3 feature cache with LatentMixture detector [犀牛鸟-D1]:冻结基础模型xLatent Mixture(贾宗翰) Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants