FLARERobot Learning with Implicit World Modeling

Ruijie Zheng et al. · Conference on Robot Learning (CoRL) 2025 · Paper · Proceedings

FLARE teaches a robot to act while also forming a compact expectation of what it will see next. The expectation is not a generated picture. It is a small bundle of learned features that keeps the training signal focused on what matters for control.

A robot does not need to paint the future to prepare for it

A video-predicting policy may spend model capacity reproducing every future pixel: the mug’s texture, the tabletop grain, and the lighting. FLARE instead asks for a compact description of the future observation. The picture should make the trade-off visible: full-frame prediction preserves appearance, while a latent target can emphasize task-relevant changes such as whether the hand reaches the mug. [Paper §§1, 5]

This is a training claim, not proof that pixels never matter. The paper compares FLARE with UWM, a method that predicts future image latents and actions together, but it does not isolate compute or latency in a controlled benchmark. [Paper §§4.1, 6]

Thirty-two future tokens give the action model somewhere to “think ahead”

FLARE adds 32 learnable future tokens beside the robot-state and noisy-action tokens. Inside the diffusion transformer, all three streams exchange information. At an intermediate layer, the future-token states are compared with a 32-token encoding of the observation at the end of the action chunk. No future image is reconstructed. [Paper §§3.1–3.2, Appendix A]

The target encoder compresses 256 image tokens and 32 language tokens into those 32 target tokens. It is trained to retain information useful for action prediction, then updated slowly during downstream training so its targets can adapt without moving too abruptly. [Paper §3.2, Appendix A]

One training step rewards both a good action and a believable consequence

Imagine the robot sees a bottle and must produce a short sequence of arm movements. Training gives it two corrections at once: “make the action chunk closer to the demonstrated motion” and “make your hidden future tokens point toward the representation of what was actually seen after that motion.” The first correction teaches control; the second asks the same network to carry information about consequences. [Paper §§2–3.1]

The combined objective is:

$\mathcal{L}=\mathcal{L}_{fm}+\lambda\mathcal{L}_{align}.$

The inputs are the current observation, robot state, a noise-corrupted action chunk, the demonstrated clean action chunk, and the future observation. $\mathcal{L}_{fm}$ is the action-generation error. $\mathcal{L}_{align}$ is lower when predicted and observed future representations point in the same direction. The multiplier $\lambda$ controls the trade-off; the experiments use $\lambda=0.2$. For a toy pair of errors, action error $2$ and alignment error $3$, the total is $2+0.2\times3=2.6$. Raising $\lambda$ makes future agreement matter more, but the paper reports that too much emphasis can conflict with action learning. [Paper Eqs. 2–3, §4.4]

On two simulated suites, the future signal lifts the average

Across 24 RoboCasa kitchen tasks, FLARE reaches 70.1% average success, versus 61.9% for the same architecture trained only as a policy. Across 24 simulated GR-1 humanoid tasks, the comparison is 55.0% versus 44.0%. FLARE also exceeds UWM in both suites. All main methods receive 80,000 training steps except UWM, which receives 400,000 because it was still improving. [Paper Table 1, §4.1]

These are imitation-learning simulations. The authors evaluate 50 episodes per task and report the best result among the final five checkpoints, so the chart is strong comparative evidence under this protocol, not a guarantee for arbitrary robots or environments. [Paper §4.1]

Human head-camera video can teach the future without supplying robot actions

For five unfamiliar objects, the authors mix controlled head-mounted GoPro videos with a small number of teleoperated robot demonstrations. Human clips have no robot-action labels, so they update only the future-alignment objective. Robot demonstrations update both the action and future objectives. The shared target is what the scene becomes, which lets visually different human and robot motions contribute to the same training process. [Paper §4.3]

The study uses 150 human videos per object and up to 10 robot demonstrations per object. With one robot trajectory per object, the reported success is as high as 60%; with ten plus human video, it reaches 80%, roughly twice the action-only baseline. The setup is controlled and small: five objects, five reference starting poses per object, and partial credit of 0.5 for grasping without placing. [Paper §4.3]

The real robot result is promising—and still narrow

On four real GR-1 pick-and-place tasks, FLARE reaches up to 95.1% success and averages 14 percentage points above the policy-only baseline after post-training with 100 trajectories per task. The evaluation uses eight reference initial frames per task and four manipulated objects. The paper’s rollouts also show the robot moving around bottles and cans rather than knocking them over, but that behavior is qualitative evidence. [Paper §§4.2, Appendix E]

The result does not establish broad humanoid competence. The authors explicitly limit the real-robot study mainly to imitation-learned pick-and-place, still require expert demonstrations, and leave fine dexterity, reinforcement learning, and human videos from natural settings for future work. [Paper §6]

Research appendix

What the two losses compute

The action path begins by blending a demonstrated action chunk $A_t$ with Gaussian noise $\epsilon$:

$A_t^\tau=\tau A_t+(1-\tau)\epsilon.$

$\tau$ lies between 0 and 1. At $\tau=0$, the input is noise; at $\tau=1$, it is the demonstration. The network $V_\theta$ learns a direction that turns the noisy chunk toward a usable action:

$\mathcal{L}_{fm}(\theta)=\mathbb{E}_{\tau}\left[\left\|V_{\theta}(\phi_t,A_t^\tau,q_t)-(\epsilon-A_t)\right\|^2\right].$

Here $\phi_t$ encodes the current images and instruction, $q_t$ is the robot’s body state, and $\|\cdot\|^2$ squares and adds the prediction differences. A larger value means the predicted denoising direction is farther from the training target. At deployment, the paper starts from noise and applies four refinement steps. [Paper Eq. 1, §2]

The future path compares directions in representation space:

$\mathcal{L}_{align}(\theta)=-\mathbb{E}_{\tau}\left[\cos\left(f_\theta(\phi_t,A_t^\tau,q_t),g(\phi_{t+H})\right)\right].$

$f_\theta$ produces the $M=32$ predicted future-token states from an internal transformer layer. $g$ encodes the observation at the end of the action horizon, $\phi_{t+H}$, into 32 target tokens. Cosine similarity asks whether two feature vectors point in the same direction, largely ignoring their lengths. Better alignment raises the cosine and therefore lowers the negative loss. In implementation, the pseudocode uses the equivalent form $1-\text{cosine similarity}$; the constant changes the displayed value but not which parameters minimize it. [Paper Eq. 2, §3.1, Appendix D]

Simulation results

Benchmark or task group	FLARE	Policy only	UWM	GR00T N1 (scratch)	Diffusion Policy
RoboCasa: pick and place	53.2%	43.8%	35.6%	44.1%	29.2%
RoboCasa: doors and drawers	88.8%	78.7%	82.0%	80.0%	78.7%
RoboCasa: other	80.0%	75.2%	74.2%	69.6%	61.3%
RoboCasa: 24-task average	70.1%	61.9%	60.8%	60.6%	51.7%
GR-1: pick and place	58.2%	46.6%	30.1%	51.8%	40.4%
GR-1: articulated tasks	51.3%	47.4%	38.4%	42.8%	50.1%
GR-1: 24-task average	55.0%	44.0%	29.5%	45.1%	40.9%

Source: Paper Table 1. FLARE, policy only, GR00T N1 (scratch), and Diffusion Policy were trained for 80,000 steps; UWM was extended to 400,000. [Paper §4.1]

What mattered in the ablation

Future target	Success rate
No FLARE loss	43.9%
SigLIP2 tokens	49.6%
Average-pooled SigLIP2 tokens	50.9%
Action-aware embedding	55.0%

The alignment idea still helps with general-purpose SigLIP2 targets, while the compact action-aware target performs best in this experiment. The main model applies the alignment at layer 6 of 8 and uses $\lambda=0.2$. Earlier alignment at layer 4 performs worse in the reported sweep. [Paper Table 2, §4.4]

Pretraining scale and compute

The action-aware embedding pretraining mixture totals 169.5 million frames and 2,989.5 hours, including 1,742.6 hours of GR-1 simulation and seven Open X-Embodiment datasets plus an in-house GR-1 set. The prose rounds this to “approximately 2,000 hours of robotic data,” while Appendix Table 3 includes simulation and reports 2,989.5 total hours. Pretraining uses 256 H100 GPUs, batch size 8,192, and 150,000 steps; multitask FLARE training uses 32 H100 GPUs, batch size 1,024, and 80,000 steps. [Paper §3.2, Appendix Tables 3 and C]

Terms

Term	Meaning here
Latent representation	A learned compact feature description, not a human-readable image.
World model	A learned expectation of how observations change. FLARE’s version is implicit because it does not render the future.
VLA	A model that receives vision and language and produces robot actions.
Diffusion transformer / DiT	The network that repeatedly refines noisy action chunks.
Action chunk	Several consecutive robot commands predicted together.
Q-former	A learned bottleneck that compresses many visual-language tokens into 32 query tokens.
EMA	A slow-moving average used to update the target encoder.

Source grounding and publication status

The official title is “FLARE: Robot Learning with Implicit World Modeling.” The work appears in the Proceedings of the 9th Conference on Robot Learning, PMLR volume 305, pages 3952–3971 (2025). The primary text used here is arXiv:2505.15659; publication metadata is confirmed by the PMLR proceedings page.

Interpretive phrases in the main path—such as “somewhere to think ahead”—are teaching analogies. They do not mean the model has conscious foresight. Claims about results, protocols, and limitations are tied to the paper sections named beside them.

Verification questions

1. Why can predicting a compact future representation use a different kind of capacity from reconstructing future pixels?
2. What information enters the action loss, and what additional information enters the alignment loss?
3. What does increasing $\lambda$ change in the combined objective?
4. Which simulation comparison best isolates the value of future alignment?
5. Why can human video contribute without robot-action labels?
6. What parts of the real-robot evidence prevent a claim of general humanoid competence?