Posterior sampling
Sample a world model and follow its optimal policy.
Language as an Exploration Space
for VLA Reinforcement Learning
VLA RL can spend millions of steps without finding success.
Both methods start at 0%
× Lifts the mug “Close the microwave”
× Lifts the mug “Close the microwave”
“Open the drawer and put the bowl inside”
“put the green container on the bottom rack”
Loading the recorded prompt responses…
Watch. Reflect. Refine.
The VLM revises prompts from rollout feedback, and PPO updates the VLA.
Sample a world model and follow its optimal policy.
Sample a prompt and follow the VLA policy it induces.
Prompt search objective
Both commit to one policy for the episode and use feedback to guide the next sample.
On LIBERO-PRO, PDE is 1.7–2.5× as sample-efficient as PPO.
Below 10% success before RL
Replay a recorded behavior. Move the camera. Compare prompts.
Language as an Exploration Space for VLA Reinforcement Learning
@inproceedings{jiang2026promptdriven,
title={Prompt-Driven Exploration: Language as an Exploration Space for {VLA} Reinforcement Learning},
author={Jiang, Sunshine and Marangola, John and Zhang, David and Kowdeed, Raghuram and Luo, Ruiyang and Dashora, Nitish and Li, Richard and Agrawal, Pulkit and Hong, Zhang-Wei},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2026},
url={https://arxiv.org/abs/2607.08837}
}