Neural network quantization with AdaRound
Qualcomm AI Research has been developing state-of-the-art quantization techniques that enable power-efficient fixed-point inference while preserving model accuracy. At NeurIPS 2019, we showed a demo with Data Free Quantization, and this year at NeurIPS 2020 we go a step further with AdaRound to achieve improved accuracy and make 4-bit quantization practical. AdaRound, which stands for Adaptive Rounding, is a post-training quantization technique, so it requires only minimal unlabeled data and no model fine-tuning. Rather than rounding to the nearest value during quantization, AdaRound automates finding the best rounding choice in order to retain model accuracy. Our demo shows two side-by-side examples of applying AdaRound versus a baseline quantization:
8-bit weight and 8-bit activation quantization on an object detection model
4-bit weight and 8-bit activation quantization on a semantic segmentation model
In both examples, the AdaRound quantized model is much more accurate than the baseline quantized model. In addition, the AdaRound quantized model is much more power efficient than the 32-bit floating point model. For example, 4-bit weight quantization provides greater up to 64X increase in performance per watt. AdaRound has been added to AI Model Efficiency Toolkit (AIMET), which is Qualcomm Innovation Center’s (QuIC) open-source project on GitHub for state-of-the-art model efficiency performance.
https://www.qualcomm.com/research/art...
https://github.com/quic/aimet