A Program to Build E(N)-Equivariant Steerable CNNs
In deep learning and computer vision, it is common for data to present certain symmetries. For instance, histopathological scans and satellite images can appear in any rotation. Examples in 3D include protein structures (which have arbitrary orientation) or natural scenes (where objects can freely rotate around their Z axis).
Equivariance is becoming an increasingly popular design choice to build data efficient neural networks by exploiting prior knowledge about the symmetries of the problem at hand. Euclidean steerable CNNs are one of the most common classes of equivariant networks. While the constraints these architectures need to satisfy are understood, no practical method to parametrize them generally has been described so far, with most existing approaches tailored to specific groups or classes of groups.
In this lecture, we will review the framework of Euclidean steerable CNNs and present a theoretical characterization of general steerable kernel spaces as well as a practical program to parameterize steerable filters. Our theory enables us to directly parameterize filters in terms of a band-limited basis on the Euclidean space, but also to easily implement steerable CNNs equivariant to a large number of groups. These include new architectures equivariant to, for example, the symmetries of the platonic solids or to 3D azimuthal symmetries (rotations around the Z axis).
00:00 Intro
06:27 Equivariance: CNN (rotation equivariance?)
08:29 Generalize Convolution
14:41 Steerable Filters
19:05 Feature Fields and Steerable CNNS
20:27 Rotation Equivariance
25:46 Steerable Filters
27:08 Solving the steerability constraint
30:53 Irreducible Representations (Irreps) of compact group G
33:10 Solving the steerability constraint
38:29 Building G-steerable basis for L² (IR^n)
40:40 General Program to implement G-equivariance
43:09 Benchmarking on MNIST variations
48:41 Benchmarking on rotated MNIST
49:24 Experiments on Natural Images
52:00 Conclusion and Discussion
[Chapters were auto-generated using our proprietary software - contact us if you are interested in access to the software]
Lecture slides: https://drive.google.com/file/d/1Qdz1...
The presentation is based on the speaker's two papers:
General E(2)-Equivariant Steerable CNNs (NeruIPS 2019)
https://arxiv.org/abs/1911.08251
https://github.com/QUVA-Lab/e2cnn
A Program to Build E(n)-Equivariant Steerable CNNs. (ICLR 2022)
https://openreview.net/forum?id=WE4qe...
Presenter Bio:
Gabriele Cesa is Research Associate at Qualcomm AI Research, Amsterdam and a PhD student at University of Amsterdam, under the supervision of Max Welling.
Gabriele's research focuses on augmenting machine learning methods with prior information about the geometry of a problem to achieve improved data efficiency and generalization. A particular emphasis has been given to equivariant neural networks, which can encode our knowledge about the symmetries in the data into the model's architecture.
Previously, Gabriele received a Master degree in Artificial Intelligence at the University of Amsterdam and a Bachelor degree in computer science at the University of Trento.
His github: https://github.com/Gabri95
-------------------------
Find us at:
Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜ / 2d3dai
Discord server for, well, discord ➜ / discord
Blog ➜ https://2d3d.ai