Estimating GPU Memory Consumption of Deep Learning Models (Video, ESEC/FSE 2020)
"Estimating GPU Memory Consumption of Deep Learning Models (Video, ESEC/FSE 2020)
Yanjie Gao, Yu Liu, Hongyu Zhang, Zhengxian Li, Yonghao Zhu, Haoxiang Lin, and Mao Yang
(Microsoft Research, China; Microsoft Research, China / National University of Singapore, Singapore; University of Newcastle, Australia; Microsoft Research, China; Microsoft Research, China; Microsoft Research, China; Microsoft Research, China)
Abstract: Deep learning (DL) has been increasingly adopted by a variety of software-intensive systems. Developers mainly use GPUs to accelerate the training, testing, and deployment of DL models. However, the GPU memory consumed by a DL model is often unknown to them before the DL job executes. Therefore, an improper choice of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail. Our recent empirical study has found that many DL job failures are due to the exhaustion of GPU memory. This leads to a horrendous waste of computing resources and a significant reduction in development productivity. In this paper, we propose DNNMem, an accurate estimation tool for GPU memory consumption of DL models. DNNMem employs an analytic estimation approach to systematically calculate the memory consumption of both the computation graph and the DL framework runtime. We have evaluated DNNMem on 5 real-world representative models with different hyperparameters under 3 mainstream frameworks (TensorFlow, PyTorch, and MXNet). Our extensive experiments show that DNNMem is effective in estimating GPU memory consumption.
Article: https://doi.org/10.1145/3368089.3417050
Submitted to the conference by Yanjie Gao on 2020-11-01
Video Tags: deep learning, memory consumption, estimation model, program analysis, fse20ind-p30-p, DOI: 10.1145/3368089.3417050
Presentation at the ESEC/FSE 2020 conference, November 8–13, 2020, https://2020.esec-fse.org/
Sponsored by ACM SIGSOFT, https://www.sigsoft.org/
Twitter: / fseconf
Reddit: / esecfse "
ISSTA 2021: Panels Celebrating 30 editions of ISSTA (statements from panel in timeband 1)
ISSTA 2021: Panels Celebrating 30 editions of ISSTA (statements from panel in timeband 3)