ControlNet: Running a 1.5B parameter generative AI model completely on device
In February 2023, we showcased the world’s first demo of Stable Diffusion running on an Android phone, which is an accepted demo at CVPR this year. Now, Qualcomm AI Research is demonstrating ControlNet, a 1.5 billion parameter image-to-image model, running entirely on a phone as well. ControlNet is a class of generative AI solutions known as language-vision models, or LVMs. It allows more precise control for generating images by conditioning on an input image and an input text description. In this demo, AI images are generated on the mobile device in under 12 seconds without requiring any cloud access, allowing an interactive user experience that is efficient, enjoyable, reliable, and private. The impressive performance was achieved with a suite of full-stack AI optimizations across the model architecture, AI software, and neural hardware accelerators. Our advanced AI tools and hardware used for this process include the AI Model Efficiency Toolkit (AIMET), the Qualcomm AI Stack, and the Qualcomm AI Engine.
Learn about our other CVPR 2023 activities: https://www.qualcomm.com/news/onq/202...
Visit the Qualcomm AI Research website: https://www.qualcomm.com/research/art...
Develop with the Qualcomm AI Stack
https://www.qualcomm.com/products/tec...