GR00T N1An Open Foundation Model for Generalist Humanoid Robots

What the essential equation is doing

The real question is: How can the model learn a motor sequence when several movements could plausibly satisfy the same instruction? Instead of predicting one answer in a single jump, flow matching teaches the model to turn a noisy action chunk into a demonstrated one.

First the paper creates a partly corrupted action:

$$A_t^\tau=\tau A_t+(1-\tau)\epsilon.$$

What goes in:

• $A_t=[a_t,a_{t+1},\ldots,a_{t+H-1}]$ is the recorded chunk of future actions; the paper uses $H=16$.
• $\epsilon$ is random Gaussian noise with the same shape as that chunk.
• $\tau$ is a mixing position between 0 and 1.

The operation is a weighted blend. At $\tau=0$, the input is all noise. At $\tau=1$, it is the recorded action. For a one-number illustration, if a demonstrated joint command is $A_t=2$, the sampled noise is $\epsilon=-2$, and $\tau=0.75$, then $A_t^\tau=0.75(2)+0.25(-2)=1$. The training input sits between noise and the target.

The model is trained with:

$$\mathcal{L}_{\textit{fm}}(\theta)=\mathbb{E}_{\tau}\left[\left\|V_{\theta}(\phi_t,A_t^\tau,q_t)-(\epsilon-A_t)\right\|^2\right].$$

Here $\phi_t$ is the vision-language representation, $q_t$ is the robot state, and $V_\theta$ is the learned action model. The double bars square and add all coordinate-by-coordinate errors, producing one penalty. A lower loss means the model’s vector is closer to the training target. The expectation means this penalty is averaged over sampled noise levels and training examples.

At inference, the paper starts from noise and performs four updates:

$$A_t^{\tau+1/K}=A_t^\tau+\frac{1}{K}V_\theta(\phi_t,A_t^\tau,q_t),\qquad K=4.$$

Each update adds one quarter of the model’s current correction. More steps would make each move smaller and cost more computation; fewer steps would be cheaper but give the learned path fewer opportunities to refine the action. The report says four worked across its tested embodiments, not that four is optimal everywhere. Paper §2.1

There is a notation issue in the report as written. Because $A_t^\tau$ moves from $\epsilon$ at $\tau=0$ to $A_t$ at $\tau=1$, its direction with increasing $\tau$ is $A_t-\epsilon$. Equation 1 instead trains toward $\epsilon-A_t$, while the displayed inference update adds the predicted vector. Those signs point in opposite directions. The main visual communicates the intended denoising process described in the surrounding prose; reproducing the implementation would require checking the released code or an author correction.

Data ledger

The category totals overlap conceptually: Table 7 lists “total robot data” alongside simulation and neural subsets, while the grand total is the sum of human, simulation, neural, and remaining real/open robot data. The paper also reports that neural video expanded 88 hours of in-house teleoperation seeds to 827 hours, costing about 105,000 L40 GPU-hours. Paper §§2.2, 3 and Table 7

Evaluation tables

The “average” column in Table 2 is weighted by the number of tasks in each suite (24 RoboCasa, 9 DexMimicGen, and 24 GR-1 tasks), rather than taking the ordinary mean of the three displayed benchmark averages. The chart shows each benchmark separately so the different suites remain visible. Paper §§4.1, 4.4 and Table 2

Paper Table 3

Terms without the field shorthand

• Embodiment: the robot’s physical body, sensors, joints, and available controls.
• Vision-language-action model (VLA): a model that maps images and an instruction to robot actions.
• Proprioceptive state: the robot’s internal reading of its body, such as joint positions.
• Action chunking: predicting several near-future motor commands together instead of one at a time.
• Cross-attention: a learned way for action features to select useful information from image-and-language features.
• Latent action: a learned compact label for the change between two video frames when true motor commands are absent.
• Inverse dynamics model (IDM): a model that estimates which action could have caused an observed change.
• Post-training: adapting the general checkpoint with demonstrations for a target task.

What is inherited from earlier work

GR00T N1 explicitly uses Eagle 2 as its vision-language backbone, action chunking from the ACT line of work, latent-action labeling from LAPA, and a data mixture containing Open X-Embodiment. It adopts the broader VLA approach and flow-matching action generation used in recent robot policies, while joining the vision-language and action modules through cross-attention. These are direct dependencies or adoptions named in §§1–3 and §5, not relationships inferred from title similarity.

Source grounding and uncertainties

• Primary source: arXiv abstract page and full HTML, version 2.
• Status: NVIDIA technical report on arXiv; the source does not claim peer-reviewed conference publication.
• Checkpoint: GR00T-N1-2B has 2.2B parameters in total, including a 1.34B-parameter VLM. Paper §2
• Real-world uncertainty: most task cells are based on ten trials, so percentage differences may be sensitive to a small number of outcomes; partial scoring further complicates a binary interpretation.
• Baseline boundary: the main post-training comparisons use from-scratch baselines. They do not isolate which part of GR00T N1—data, backbone, architecture, or training recipe—causes each gain.
• Synthetic-data boundary: the authors report positive ablations but also acknowledge physical inconsistency and limited diversity. The study does not establish safe deployment from generated video.
• Generalist boundary: one checkpoint spans multiple bodies and tasks, but evaluation remains a finite collection of manipulation settings.

Questions for checking understanding

1. Why can human video form a wide data base but not directly command a robot?
2. What information passes from System 2 to System 1, and what remains body-specific?
3. In the noise-mixing equation, what do the cases $\tau=0$ and $\tau=1$ contain?
4. Which comparison most strongly supports the claim of data-efficient adaptation?
5. Why do the reported results not yet demonstrate a humanoid that can work generally in a home?
6. What does the apple handover example warn about during post-training?