Maximizing Computer Vision's Field of View - 360° Computer Vision in Deep Learning (Dr. Marc Eder)
This talk introduces the emerging field of 360° computer vision, and provides an overview of the spherical distortion problem, highlighting how this distortion affects many of the highest profile problems in computer vision, from deep learning to structure-from-motion and SLAM. We surveyed some of the existing work on the topic, and identified 3 guiding principles that drive a general solution to the problem. Finally, we concluded with some opportunities for further research and some big picture takeaways from work thus far.
Dr. Marc Eder's talk is based on his CVPR 2020 paper 'Tangent Images for Mitigating Spherical Distortion'
Lecture references: / references_from_maximizing_computer_vision...
git: https://github.com/meder411/Tangent-I...
arxiv: https://arxiv.org/abs/1912.09390
00:00 Outline
03:14 What Is 360 Computer Vision?
04:14 Applications of 360 Computer Vision
07:37 The Spherical Distortion Problem
13:31 Geometry Estimation
14:58 Translational Equivariance
19:49 Dense Correspondence Matching
21:57 Sparse Correspondence Matching
24:39 Theorem Egregium
37:01 Quick Summary of Spherical Distortion
38:24 The Icosahedral Sphere
41:03 Subdivision and Spherical Resolution
45:42 Why don't they scale?
46:21 Why keep subdividing?
49:04 Tangent Images
50:43 Generating Tangent Images
53:32 Decouples Resolution from Subdivision Level
55:31 Semantic Segmentation
01:01:20 Tangent images mitigate distortion sufficiently to unlock wide FOV benefits!
01:04:59 Transferability
01:07:04 Spherical Keypoint Detection
01:09:58 Summary
01:14:46 Final Thoughts
[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]
Lecture abstract:
The advances in computer vision over the past decade are astounding when you compare to decades prior. If there is one shortcoming to the current engine of progress, it is that its field of view is still largely limited, in the most literal sense. Most vision algorithms are designed with undistorted, central-perspective images in mind. While this constraint is reflective of the prevalence of these types of cameras in circulation, this narrow field of view restricts progress to merely 30° - 60° crops of the world. Yet, not everything can be experienced, augmented, or understood from these small glimpses. This partial view cannot transport someone to another place, nor can it guarantee the context required to augment a scene or assist with a desired task. These applications require capturing a scene in its entirety: in all directions at once. With the advent and growth of commodity 360° cameras, it is now easy to obtain this type of imagery. However, these 360° images suffer from spherical distortion that is mathematically impossible to remove, and which has a powerful, deleterious effect on many algorithms' performance. As a result, it is imperative that we identify ways to reduce the impact of this distortion so that we may expand computer vision's field of view to the full 360°.
Presenter BIO:
Marc Eder recently completed his PhD in computer science at the University of North Carolina at Chapel Hill, where he was advised by Dr-Ing. Jan-Michael Frahm. Marc's research has primarily focused on computer vision problems involving 360° imaging. In particular, he has endeavored to identify new and improved representations for 360° images that can facilitate the seamless application of traditional central-perspective image algorithms. He has also employed this line of work for popular applications such as 3D indoor modeling. Recently, he co-organized the OmniCV Workshop at CVPR 2020, which brought together top computer vision researchers and engineers to discuss their work with omnidirectional images. Marc serves as a reviewer for CVPR, ICCV, and ECCV, among others, and most recently was acknowledged as a top reviewer for ECCV 2020. Before his doctorate in computer vision, Marc received a MS in electrical engineering at Boston University and a BA in history and Islamic & Middle Eastern Studies from Brandeis University. This fall, Marc will be joining Yembo, a San Diego-based startup leveraging computer vision to transform the home-services industry. More information about Marc can be found at www.marceder.com.
----------------------------------------
Find us at:
Newsletter for updates ➜ http://eepurl.com/gJ1t-D
Join our live online lectures ➜ https://www.meetup.com/2d3d-ai/
Subreddit for discussions ➜ / 2d3dai
Discord server for, well, discord ➜ / discord
Blog ➜ https://2d3d.ai
We are the people behind the AI consultancy Abelians ➜ https://abelians.com/