Canonicalization: Experiments in setting some chairs in order - Prof' Srinath Sridhar
In this talk, I will introduce the notions of invariance, equivariance, and 'object canonicalization' (i.e., mapping object properties to canonical states). I will demonstrate how canonicalization enables us to better solve tasks in computer vision and robotics including 6DoF object pose estimation and 3D reconstruction. I will discuss our work on fully supervised and weakly supervised canonicalization methods, but focus on self-supervised methods. Finally, I will discuss future directions including opportunities for using canonicalization to understand articulated and non-rigid objects.
00:00 Intro
05:09 Canonicalization
07:52 Canonicalization in 2D Computer Vision
10:55 Canonicalization in 3D Computer Vision
11:40 3D Datasets: Explicit Canonicalization
16:20 Related Work
17:58 Supervised Canonicalization
19:37 NOCS: Normalized Object Coordinate System
21:52 6 DoF Object Pose Estimation
22:51 NOCS Maps
25:25 Predicting NOCS Maps
27:56 Qualitative Results: Real Data
31:11 X-NOCS: Network Architecture
35:02 Pix2Surf: Single-View Single-Chart
36:45 NOCS Applications
37:45 Temporal NOCS (T-NOCS)
39:17 Canonicalization Results
41:21 NOCS: Normalized Object Coordinate Space
42:03 Articulated NOCS
43:39 Limitations
46:28 Self-Supervised Canonicalization
47:53 (1) Permutation Equivariance
49:05 (2) Rotation Equivariance
50:27 ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes
51:09 Rotation Canonicalization
53:10 Translation Equivariance
56:15 Summary
01:00:25 Neural Fields
01:03:05 Neural Fields in Visual Computing and Beyond https://neuralfields.cs.brown.edu/
01:04:07 Discussion
[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]
References:
Neural Fields https://neuralfields.cs.brown.edu/
Abstract: https://arxiv.org/abs/2111.11426
Slides
Theory behind canonicalization: https://arxiv.org/abs/1909.02533
The talk is based on the speaker's papers:
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation (CVPR 2019)
https://geometry.stanford.edu/project...
Multiview Aggregation for Learning Category-Specific Shape Reconstruction (NeurIPS 2019)
https://geometry.stanford.edu/project...
Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images (ECCV 2020)
https://geometry.stanford.edu/project...
CaSPR: Learning Canonical Spatiotemporal (NeurIPS 2020 Spotlight)
https://geometry.stanford.edu/project...
DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects
https://aadilmehdis.github.io/DRACO-P...
ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes
https://ivl.cs.brown.edu/ConDor/
Presenter Bio:
Srinath Sridhar (http://srinathsridhar.com/) is an assistant professor of computer science at Brown University. His research interests are in 3D computer vision and machine learning. Specifically, he focuses on visual understanding of 3D human physical interactions with applications ranging from robotics to mixed reality. He has won several fellowships (e.g., Google Research Scholar) and awards (e.g., Eurographics Best Paper Honorable Mention) for his work, and has previously spent time at Stanford, Max Planck Institute for Informatics, Microsoft Research Redmond, and Honda Research Institute.
-------------------------
Find us at:
Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜ / 2d3dai
Discord server for, well, discord ➜ / discord
Blog ➜ https://2d3d.ai