9. Faster RL Training: Profiling and Optimization
Presenter: Eric Undersander
Why focus on performance optimization for RL training? The goal is higher frame rates. A higher frame rate lets us scale our training to more steps, and it lets us iterate faster on our day-to-day experiments. In this short, accessible tutorial, we'll profile and optimize an example program in real time in a Colab notebook. We'll use one of Habitat's PointGoal navigation baselines as our example program, but the lessons here can be applied to any RL training. We'll walk through the process of capturing a profile, identifying candidates for optimizations, making improvements to our code, and evaluating speedup. We'll try some profiling tools including py-spy, speedscope, and Nsight Systems. We'll use a multithreaded trace to identify GPU-bound scenarios and opportunities for better parallelism. Follow along in the interactive Colab notebook at https://aihabitat.org/tutorial/2020/.