Stable Diffusion High resolution image synthesis with latent diffusion models
Stable diffusion is a model for synthetic image generation like Imagen, DALL.E-2 etc. It leverages diffusion models with classifier free guidance. It decomposes the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs). Using cross attention layers in the UNet it supports different forms of conditioning.
In this video, I will briefly provide an overview of the latent diffusion model and architecture of Stable Diffusion.
Here is the agenda:
00:00:00 What is Stable diffusion?
00:03:00 What are Latent Diffusion Models (LDMs)?
00:07:18 What is the architecture for LDMs?
00:10:50 Image Synthesis examples
00:11:50 Perceptual compression tradeoffs
00:16:40 Layout-to-image synthesis, Semantic synthesis, Super-resolution, Inpainting using Stable Diffusion.
For more details, please look at
https://github.com/CompVis/latent-dif...
https://arxiv.org/pdf/2112.10752.pdf
https://stability.ai/blog/stable-diff...
https://stability.ai/blog/stable-diff...
For better understanding, also look at these videos:
Diffusion model based training of GLIDE: • OpenAI GLIDE: Towards Photorealistic Image...
DALL·E 2: • OpenAI DALL·E 2: Hierarchical text conditi...
Imagen: • Imagen: Photorealistic Text to Image Diffu...
Rombach, Robin, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. "High-resolution image synthesis with latent diffusion models." In CVPR, pp. 10684-10695. 2022.