Imagine, Verify, Act: Robot Planning with World Models and Vision-Language Evaluators

Humans and animals anticipate the consequences of their actions before acting, yet most robot policies remain reactive and limited to short horizons. In this talk, Dr. Gumbsch will sketch an approach to building embodied agents that learn to solve tasks by thinking ahead, rather than relying on expert demonstrations or extensive real-world interaction. Revisiting model-based reinforcement learning through the lens of recent advances in world modelling and foundation models, he will propose three ingredients. First, a compositional world model that represents scenes as objects and interactions as events, enabling selective imagination over multiple time scales. Second, an internal critic to judge predicted outcomes. Here, he will show how pre-trained Vision-Language Models can serve as zero-shot evaluators of behavior, both to guide exploration and to assess task success. Third, an imagined policy improvement loop that ties these together: the agent rehearses behaviors inside the world model, the VLM scores them, and the policy is updated from these synthetic rollouts. The dream is to build robots that have already mastered a task in imagination before attempting the first step.