Skip to content

feat(audio-alm): make ALM stages agent-ready - #2340

Open
shubhamNvidia wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
shubhamNvidia:agent/alm
Open

feat(audio-alm): make ALM stages agent-ready#2340
shubhamNvidia wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
shubhamNvidia:agent/alm

Conversation

@shubhamNvidia

Copy link
Copy Markdown
Contributor

Contracts only — no behavior change. Adds describe() + *_key params to the ALM data-builder / overlap stages and the pretrain extraction / io / planning stages.

Because window-building replaces the row rather than adding to it, the contract also declares preserves_upstream_keys=False and lists the fields it drops (removes_keys) — so the planner knows a key consumed upstream of ALM will not survive past it, instead of discovering that at runtime.

Source only — the tests/stages/audio/alm/ folder is untouched and identical to main.

Depends on the agent-ready foundation, #2332 — that must merge first (this branch imports nemo_curator/stages/audio/_agent/, so CI here stays red until it lands).

Adds describe() contracts to the ALM data-builder/overlap stages and the pretrain extraction/io/planning stages so a planner can tell what each reads and writes before running it.

Depends on the agent-ready foundation (nemo_curator/stages/audio/_agent/), which must merge first.

Signed-off-by: Shubham Bhawsar <shbhawsar@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@shubhamNvidia
shubhamNvidia marked this pull request as ready for review August 25, 2026 11:19
@shubhamNvidia
shubhamNvidia requested a review from a team as a code owner August 25, 2026 11:19
@shubhamNvidia
shubhamNvidia requested review from ayushdg and removed request for a team August 25, 2026 11:19
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds agent-readable contracts and configurable payload keys to ALM window-building and pretraining stages without intentionally changing their processing behavior.

  • Declares stage reads, writes, cardinality, metadata access, lifecycle effects, and row-independence constraints.
  • Marks the long-form manifest reader as a Ray fan-out stage.
  • Describes stable task identity and disk-output requirements for snippet extraction.
  • Retains existing finalization and tar-flush behavior with formatting and comment-only cleanup.

Confidence Score: 5/5

The contract changes appear safe to merge once the explicitly declared agent-ready foundation dependency is present.

No concrete changed-code defect remains beyond the acknowledged prerequisite dependency, and the reviewed declarations align with the visible stage behavior.

Important Files Changed

Filename Overview
nemo_curator/stages/audio/alm/alm_data_builder.py Adds configurable payload keys and a contract documenting window outputs, removed fields, row replacement, and per-row independence.
nemo_curator/stages/audio/alm/alm_data_overlap.py Makes window input and filtered-window output keys configurable and declares the corresponding per-row contract.
nemo_curator/stages/audio/alm/pretrain/extraction.py Declares snippet fan-out, file input, disk output, lifecycle, output-path, and stable-task-identity requirements; runtime extraction behavior is unchanged.
nemo_curator/stages/audio/alm/pretrain/io.py Adds contracts for manifest reading and output aggregation, static writer hints, and the Ray fan-out marker for the source reader.
nemo_curator/stages/audio/alm/pretrain/planning.py Adds contracts for segment filtering, snippet planning, and repetition filtering, including metadata and first-run internet requirements.
nemo_curator/stages/audio/alm/pretrain/finalize.py Contains formatting-only changes to output preparation, tar merging, reconciliation, and metrics patching.
nemo_curator/stages/audio/alm/pretrain/utils.py Contains only formatting changes to the invalid audio-path-resolution error construction.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Long-form manifest reader] --> B[Overlap filter]
    B --> C[Snippet cut planner]
    C --> D[Repetition filter]
    D --> E[Snippet extraction]
    E --> F[Manifest writer]
    E --> G[Audio tar shards]
    B --> H[Metrics aggregator]
    C --> H
    D --> H
    E --> H
    F --> I[Finalization]
    G --> I
    H --> I
Loading

Reviews (1): Last reviewed commit: "feat(audio-alm): make ALM stages agent-r..." | Re-trigger Greptile

@shubhamNvidia
shubhamNvidia requested review from ayushdg, mohammadaaftabv and sarahyurick and removed request for ayushdg August 25, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant