Ashok Elluswamy: Building Foundational Models for Robotics at Tesla
Ashok Elluswamy, head of the Tesla AI team, presents at Scaled ML 2026
Chapters:
0:00 - Introduction and Tesla AI Team Mission
0:20 - Tesla’s Mission: Producing Abundance
0:32 - Robotaxi Service in Austin, Driven by Cameras and AI
1:20 - Humanoid Robots and Automating Physical Work
2:05 - Full Self-Driving (FSD) Deployment and Safety
2:38 - FSD vs. Manual Driving Safety Metrics
3:30 - How the Software Works and Challenges
3:42 - End-to-End Driving System vs. Modular Approaches
4:34 - Why End-to-End Systems are Better for Robotics
5:17 - Example: Solving the Trolley Problem with a Puddle
6:06 - Example: Waiting for Chickens and Geese to Cross
7:20 - Advantages of Neural Networks: Latency and Scaling
8:01 - The Curse of Dimensionality
9:57 - The Role of Fleet Data and Collecting Interesting Scenarios
11:40 - Proactive Safety and Generalization to Rare Events
13:30 - Debugging and Interpretability in End-to-End Systems
14:26 - 3D Geometric Reasoning with Generative Gaussian Splatting
16:25 - Text-Based Reasoning for Situational Understanding
17:08 - Evaluation: The Long-Tail Problem and World Simulators
18:32 - Neural Network-Generated Simulation Video
19:34 - Using the Simulator for Policy Evaluation and Regression Testing
20:27 - Injecting Novel Issues and Adversarial Scenes
20:56 - Real-Time Simulation with Reduced Compute
21:48 - The Foundational Neural Network for All Robotics
22:06 - Generalization to Optimus Indoor Scenes and Manipulation
23:02 - Future Roadmap: Cybercabs and Optimus
24:05 - Join the Team
24:30 - Q&A: Cameras vs. Other Sensors
26:05 - Q&A: Rewards and Penalties in the World Simulator
26:24 - Q&A: Adding External Infrastructure
27:06 - Q&A: Control Command Frequency (36 Hz)
27:32 - Q&A: Human Voice Interaction
28:42 - Q&A: Improving Visual Perception and 3D Understanding
29:22 - Conclusion