Align-Then-stEer (ATE): Data-Efficient Adaptation for VLA Robotics (Cross-Embodiment & Cross-Task)
The Align-Then-stEer (ATE) framework is a novel, data-efficient, and plug-and-play solution designed to overcome the primary bottleneck in deploying pre-trained Vision-Language-Action (VLA) models: adapting them efficiently to new tasks and robotic embodiments. When a robot's hardware (embodiment) or the task differs from the pre-training data, the resulting action distribution mismatch demands extensive data and compute for fine-tuning.
ATE addresses this challenge through a two-stage strategy: Alignment and Steering.
*Stage 1: Alignment through Unified Action Latent Space*
In the first stage, ATE constructs a unified action latent space to bridge the domain gap between pre-training and adaptation actions. This is achieved by training two separate Variational Autoencoders (VAEs). A pre-training action VAE first captures the general visuomotor prior. A subsequent adaptation action VAE is trained on limited target domain data using a reverse KL divergence constraint, which ensures that the adaptation actions are embedded into a specific mode of the pre-training latent distribution. This process yields a unified, compact, and structured latent space.
*Stage 2: Steering Efficient Adaptation with Latent Guidance*
The second stage integrates a classifier guidance mechanism directly into the training objective of diffusion- or flow-based VLAs, such as Diffusion Policy (DP), RDT-1B, and $\pi_0$. This guidance function operates within the unified latent space, measuring the discrepancy between the model's generated actions and the ground-truth target actions. By using the gradient of this guidance function, ATE explicitly steers the policy fine-tuning process toward the target domain's action distribution, enabling rapid and precise adaptation without requiring additional data. The method prevents the VLA from deviating far from the structured latent manifold, thereby implicitly preserving valuable pre-trained visuomotor knowledge.
*Performance and Generalization:*
ATE consistently enhances performance across various tasks and architectures.
In simulation, across the RoboTwin 1.0 and ManiSkill3 benchmarks, ATE improves the average multi-task success rate by up to *9.8%* compared to direct fine-tuning.
In real-world cross-embodiment experiments on a dual-arm RealMan 7-DoF robot, ATE achieved a substantial *32% success rate gain* over the baseline in challenging tasks requiring long-horizon manipulation, bimanual coordination (like Cook Bun and *Make Sandwich*), and tool use (*Make Yogurt Bowl*).
ATE demonstrates superior robustness and generalization against real-world perturbations, including varying illumination (low/high/flickering), visual distractors (unseen items), spatial shifts (larger object displacement), and human disturbances.
An observed hidden advantage of ATE is the generation of **safer and smoother trajectories**, resulting in more consistent and stable force control compared to the baseline policy.
ATE is a lightweight solution that seamlessly integrates with existing score-based VLA architectures, enhancing the practicality of deploying these models to new platforms and tasks.
Project Website: https://align-then-steer.github.io/
Code Implementation: https://github.com/TeleHuman/Align-Th...
***
**Tags**:
AlignThenSteer, ATE, VLA, VisionLanguageAction, Robotics, RoboticManipulation, DataEfficient, PlugAndPlay, CrossEmbodiment, CrossTask, DiffusionModels, FlowMatching, VAE, LatentSpace, $\pi_0$, RDT-1B, DiffusionPolicy, RealMan, RoboTwin, ManiSkill3, AI, MachineLearning, RobotPolicy, Generalization, FineTuning