Action channel layout + normalizers for egocentric (57D) and robot embodiments

[TQTQliu]

Thanks for the release. I'm running action-conditioned inference with Cosmos3-Nano via vLLM-Omni. The model card gives per-embodiment dims and says extra_params["action"] takes a normalized (T, D) array, but only agibotworld (FD) and av (ID) examples ship, and the exact channel layout + normalizers aren't documented. Could you share, ideally as a per-embodiment table:

1. Egocentric (57D): exact ordering — block order (ego 9D / left vs right hand / wrist 9D + grasp 15D), translation-vs-rotation order in each 9D pose, the 6D-rotation layout, the 5-fingertip order and which of the 21 keypoints they are, coordinate convention, and the domain_name string.
2. Robots (Franka 10D, dual Franka 20D, AgiBot 29D, UR/Google/WidowX 10D, UMI 9D): exact channel order (EE translation/rotation/gripper) — pose-delta or joint-angle based?
3. Normalizers: where the per-dimension stats live and how to apply them.

A pointer to the canonical file in cosmos-framework would be perfect. Thanks!

[Huaian666]

really impressive work from nvidia. the world model approach makes total sense for physical ai. looking forward to seeing where this goes.

[mli0603]

Thanks @TQTQliu for the question. We are working on releasing more examples similar to DROID that

• sample raw Lerobot dataset
• code to process the raw data
• normalization files
• update the Notebook to run these examples.

And we are working on both FD/ID. Will provide an update.

[Rohithmatham12]

I opened #212 as a small documentation follow-up related to this request. It does not attempt to document unreleased embodiment-specific layouts or normalizer files, but it clarifies that the current canonical references are the checked-in AV, DROID, and UMI action assets/notebooks, and points users to the DROID LeRobot sample metadata for the available state/action relationship.

That should make the current boundary clearer while NVIDIA prepares the additional examples and normalization files mentioned above.