Synthetic Data for Perception in Autonomous Driving
A broad variety of real-world scenarios require autonomous navigation systems to rely on machine learning-based perception algorithms. Such algorithms are knowingly data-dependent, yet data acquisition and labeling is a costly and tedious process. It is associated with manual labor, must handle rare "long tail" corner case events, and could be hard constrained by ethical aspects e.g. in case of near-accident scenarios.
One of the common alternatives to real data acquisition and annotation is represented by simulation and synthetic data. Simulation has a long history in driver assistance systems, but with the renaissance of neural networks the research community strengthened efforts in this direction and many synthetic datasets and simulation systems appeared.
Image synthesis driven by computer graphics achieved recently a remarkable realism. Yet synthetic image data generated in such a way reveals a significant domain gap with respect to real-world data. This is especially true in autonomous driving scenarios, representing a critical aspect to overcome to utilize synthetic data for the training of neural networks.
In this presentation, we discuss recent progress in the area of synthetic data for perception in autonomous driving, challenges associated with it, and methods to overcome the hurdles.
00:00 Intro
02:51 WHY TRAFFIC SCENE GENERATION/SIMULATION/SYNTHESIS? GENERATE VS REAL
06:14 SYNTHETIC DATASETS: SYNTHIA
08:54 VIRTUALKITTI
10:34 PARALLEL DOMAIN DATASET
14:06 WHAT IS COVARIATE SHIFT?
20:19 DOMAIN ADAPTATION. CYCADA
23:42 SEMANTICALLY CONSISTENT SYNTHTIC-TO-REAL DOMAIN ADAPTATION
27:05 OUR TAKE: CONTENT DISENTANGLEMENT
32:35 LOSS FUNCTION
34:19 QUALITATIVE COMPARISON
35:59 CONTENT SPACE EXPLORATION VIA TSNE
39:31 RESULTS
48:27 UNSUPERVISED TRAFFIC SCENE GENERATION SYNTHETIC 3D SCENE GRAPHS
51:03 SCENE GRAPH
52:22 GRAPH CONVOLUTIONAL NETWORK
59:07 EXAMPLES
01:04:26 CLASS MANIPULATION
01:08:15 DEMO and Q/A
[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]
Talk is based on the speaker's papers:
Content Disentanglement for Semantically Consistent Synthetic-to-Real Domain Adaptation (IEEE IROS'21)
https://arxiv.org/abs/2105.08704
Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs (IROS 2021)
Links will be added during August
Presenter BIO:
Artem Savkin studied Mathematics in Kaliningrad, Russia. He is currently a researcher at BMW and PhD candidate at TUM supervised by Federico Tombari. Artem's research focuses on sim2real domain adaptation for images and point clouds.
-------------------------
Find us at:
Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜ / 2d3dai
Discord server for, well, discord ➜ / discord
Blog ➜ https://2d3d.ai
AI consultancy Abelians ➜ https://abelians.com/