[kevchengcodes]

Type of Change

• Bug fix
• Small feature (agreed in an issue)
• New model (export recipe only)
• New model (export recipe and new runtime support)
• Documentation update

Motivation and Context

Issue #381 found that SAM3 Lite was not passing an attention mask to the detector, which led to inaccurate scores when compared against the HF reference. This is because without the attention mask, the detector was cross-attending to all the padding slots and averaging them into the scoring vector alongside the real prompt. So short prompts were getting drowned out by the padding.

While investigating, I also found that SAM3 Full's example_inputs in Torch was causing token truncation. Before, we had:

text_inputs = processor.tokenizer(["dummy"], return_tensors="pt")

Which would truncate any prompt to 3 tokens (i.e. max of 2 word prompts). This PR changes it to:

    text_inputs = processor.tokenizer(
        ["dummy"],
        padding="max_length",
        max_length=config.max_text_seq_len,
        return_tensors="pt",
    )

So that it properly allows prompts up to the max sequence length (32). Additionally, we add the attention mask to SAM3 Full as well.

Testing

• Tested locally
• Added/updated tests
• Added/updated docs

I reproduced the error using the script provided in the issue, and verified that these fixes address the problem.

[carinapeng]

Thanks for the fix!

[stikves]

This makes existing 0.2 runtimes incompatible with the new exports.

We should bump the metadata version number to 0.3, just like the recent batched LLM exports, and make runtime compatible with both (if needed), or explicitly reject 0.2

[kevchengcodes]

This makes existing 0.2 runtimes incompatible with the new exports.

We should bump the metadata version number to 0.3, just like the recent batched LLM exports, and make runtime compatible with both (if needed), or explicitly reject 0.2

@stikves I tested an old export with the new runtime and a new export with the old runtime. Both cases work, so I think we're okay keeping the same version.

[spacexm406-web]

workflows/repo-status

Constant improvement and performance daily expansion