Debugging Neural Networks
2021.08.11, Rishi P Gurnani, Georgia Institute of Technology
See table of contents below.
This video is part of NCN's Hands-on Data Science and Machine Learning Training Series which can be found at: https://nanohub.org/groups/ml/handson...
Imagine a program that did not pass along helpful descriptions of errors, let alone provide a notification that an error has occurred. How would one even begin to debug such a nightmare? Error-laden deep learning systems are susceptible to such "silent failures". The reason for silent failure in deep learning is that any code representing a mathematically valid expression will fly, even if the coded expression is not mathematically logical. Further, the capacity of a deep learning model may be large enough to compensate for illogical code. Thus, the performance of the model may look reasonable enough that one does not re-examine the code for errors. To mitigate chances of falling into this trap, we have collected and encoded a list of common-sense checks that deep learning systems should pass. Several checks are borrowed from a well-written blog post[1] by Andrej Karpathy, Tesla's Director of Artificial Intelligence, while some are original. The implementation of these checks is original work and is known as NetDebugger. Failure of any check results in an error along with a helpful error message. The presentation will start with an overview of deep learning theory to motivate the logic in NetDebugger and end with a hands-on NetDebugger tutorial involving PyTorch, RDKit, and polymer data (made publicly available by the Ramprasad Group at https://khazana.gatech.edu/). NetDebugger has saved me time in my research and, I hope, will be useful to the community.
The NetDebugger tool can be ran on nanoHUB at: https://nanohub.org/tools/netdebugger
This talk and additional downloads can be found on nanoHUB.org at: https://nanohub.org/resources/35419
Table of Contents:
00:00 Debugging Neural Networks
02:09 WHY DO WE NEED ML? … DESIGN CHALLENGES!
03:23 POLYMER CHEMICAL UNIVERSE
04:10 COMPLEX PROPERTIES
05:22 AGENDA
05:55 WHAT IS MACHINE LEARNING (ML)?
07:12 TRAINING A NEURAL NETWORK
07:32 TRAINING A NEURAL NETWORK
07:42 TRAINING A NEURAL NETWORK
07:43 TRAINING A NEURAL NETWORK: Part 1
09:21 TRAINING A NEURAL NETWORK: Part 2
10:59 TRAINING A NEURAL NETWORK: Part 3
11:43 DEBUGGING: "Do not go gentle into that good night"
13:03 DEBUGGING: "Do not go gentle into that good night"
15:00 NetDebugger
15:44 NetDebugger: Test #1, Output Shape
17:38 NetDebugger: Test #2, Input Independent Baseline
18:59 NetDebugger: Test #3, Overfit Small Batch
20:48 NetDebugger: Test #3, Overfit Small Batch
21:39 NetDebugger: Test #4, Chart Dependencies
23:51 NetDebugger: Test #5, Overfit Entire Training Data
24:17 Accessing NetDebugger tutorial on nanoHUB
48:08 SUMMARY
49:05 ACKNOWLEDGEMENTS