Structure-Aware Protein Self-Supervised Learning | Can (Sam) Chen
If you enjoyed this talk, consider joining the Molecular Modeling and Drug Discovery (M2D2) talks live: https://m2d2.io/talks/m2d2/about/
Also consider joining the M2D2 Slack: https://m2d2group.slack.com/join/shar...
Title: Structure-Aware Protein Self-Supervised Learning
Abstract: Protein representation learning methods have shown great potential to yield useful representation for many downstream tasks, especially on protein classification. Moreover, a few recent studies have shown great promise in addressing insufficient labels of proteins with self-supervised learning methods. However, existing protein language models are usually pretrained on protein sequences without considering the important protein structural information. To this end, we propose a novel structure-aware protein self-supervised learning method to effectively capture structural information of proteins. In particular, a well-designed graph neural network (GNN) model is pretrained to preserve the protein structural information with self-supervised tasks from a pairwise residue distance perspective and a dihedral angle perspective, respectively. Furthermore, we propose to leverage the available protein language model pretrained on protein sequences to enhance the self-supervised learning. Specifically, we identify the relation between the sequential information in the protein language model and the structural information in the specially designed GNN model via a novel pseudo bi-level optimization scheme. Experiments on several supervised downstream tasks verify the effectiveness of our proposed method.
Speaker: Can (Sam) Chen - https://mila.quebec/en/person/can-chen/
Twitter Prudencio: / tossouprudencio
Twitter Therence: / therence_mtl
Twitter Jonny: / hsu_jonny
Twitter Valence Discovery: / valence_ai
~
Chapters:
00:00 - Intro
06:46 - Method Overview
10:39 - SSL Distance Prediction
16:28 - Pseudo Bi-Level Optimization
20:06 - Experiments
26:00 - Q+A