FiS-VLA: Unifying Fast Robotic Manipulation with Slow VLM Reasoning (117Hz Control!)
Explore Fast-in-Slow (FiS-VLA), a groundbreaking unified dual-system Vision-Language-Action (VLA) foundation model designed to tackle the twin challenges of generalised policy and execution efficiency in robotics. Drawing inspiration from Kahneman's dual-system theory, which distinguishes between System 1 (fast, intuitive) and System 2 (slow, logical reasoning), FiS-VLA integrates these concepts to achieve state-of-the-art performance and exceptionally high control frequency.
*The Core Innovation: Embedding Fast Execution within Slow Reasoning*
Traditional dual-system VLA approaches maintain System 1 (action model) and System 2 (VLM-based reasoning) as separate entities, limiting System 1's access to the rich, pre-trained knowledge inherent in the large Vision-Language Model (VLM) used by System 2. FiS-VLA revolutionises this architecture by embedding the System 1 execution module directly within the VLM-based System 2, achieved by partially sharing parameters and repurposing the final transformer blocks of the LLM backbone (like LLaMA2). This unified paradigm ensures seamless coordination between reasoning and execution, enabling System 1 to inherit and leverage the VLM's extensive pretrained knowledge.
*Asynchronous and Heterogeneous Dual-System Design*
FiS-VLA strategically assigns fundamentally distinct roles to each system, supported by asynchronous operating frequencies and heterogeneous modality inputs.
*System 2 (Slow Reasoning):* Operates at a lower frequency, focusing on high-level semantic understanding, contextual reasoning, and processing 2D images and language instructions. It produces intermediate latent features that guide System 1’s subsequent actions over a future horizon.
*System 1 (Fast Execution):* Functions as a high-frequency, reactive action generation module. It processes the most recent, high-frequency inputs crucial for precision, including **3D point clouds**, **2D images**, and the **robot state**, ensuring geometry-aware, stable, and responsive actions, while being conditioned on System 2’s guidance.
*Achieving High-Frequency Control and SOTA Performance*
To jointly optimise both components, FiS-VLA employs a dual-aware co-training strategy. This approach combines a diffusion denoising objective for continuous action generation (System 1) with an autoregressive next-token prediction objective to preserve System 2’s high-level reasoning capabilities.
The result is a model that demonstrates exceptional execution speed. FiS-VLA achieves a *117.7 Hz control frequency* when the action chunk size is set to eight, significantly surpassing previous methods. In quantitative evaluations:
FiS-VLA achieves an *8% improvement* in average success rate in simulation benchmarks (RLBench).
It demonstrates an *11% improvement* in average success rate across real-world tasks, evaluated on dual-arm robots like the Agilex and AlphaBot, confirming robustness across different control paradigms (end-effector poses and joint positions).
The model exhibits strong generalisation capabilities, maintaining stable performance when confronted with unseen objects, complex backgrounds, and varying lighting conditions.
Watch the video to see FiS-VLA executing complex, high-precision tasks—from dual-arm coordination (like pouring water or folding a towel) to classic single-arm manipulations—all while benefiting from coordinated reasoning and lightning-fast execution!
---
*Project Links:*
Project Web Page: fast-in-slow.github.io
Paper (Arxiv): 2506.01953
Code: github.com/CHEN-H01/Fast-in-Slow
*Tags*
Robotics, Foundation Model, VLA, Vision-Language-Action, Fast-in-Slow, FiS-VLA, Dual System, Robotic Manipulation, High-Frequency Control, Real-Time Robotics, System 1, System 2, VLM, LLM, Diffusion Policy, Franka Panda, Dual-Arm Robot, Agilex, AlphaBot, SOTA, RLBench, Kahneman Theory, Asynchronous Control, 3D Point Cloud, End-to-End Robotics