Tech Talk - Using Deep Learning to Generate Artwork with VQGAN-CLIP
One of the great promises of machine learning, deep learning, and AI is the possibility of increasing human capacity for creativity.
Generating unique art is challenging for humans and computers alike. Recent advances in deep learning technologies have allowed us to step closer than ever towards that goal of a truly creative and relatively independent machine.
PixRay is a project led by Github user Dribnet that leverages the strengths of two neural network architectures, CLIP and VQGAN, together to generate artwork that accurately reflects the meaning of an inputed prompt.
This art is unique and relies on the compatibility of its two constituent building blocks to produce a more reflective piece to the prompt with each training iteration. These artworks can take a variety of forms depending on the sub-drawer selected, so we are focusing on the PixelDraw script which uses a clip art medium.
Attendees can expect to leave this webinar with an understanding of how VQGAN-CLIP works to generate novel pixel art, and learn how to combine this innovative tool with the power of Gradient GPUs to rapidly generate artwork of their own.
This tech talk took place on 11/10/21.
Resources used in this presentation:
Key Papers:
Taming Transformers by Esser et al. - https://compvis.github.io/taming-tran...
Learning Transferable Visual Models From Natural Language Supervision by Radford et al. - https://openai.com/blog/clip/
An Introduction to Image Synthesis with Generative Adversarial Nets by Huang, Yu, and Wang
PixRay - https://arxiv.org/pdf/1803.04469.pdf
Resources for presentation & recommended further reading:
CLIP from OpenAI: what is it and how you can try it out yourself - https://habr.com/en/post/537334/
The Illustrated VQGAN - https://ljvmiranda921.github.io/noteb...
Understanding Taming Transformers for High-Resolution Image Synthesis - https://www.analyticsvidhya.com/blog/...
What the Heck are VAE-GANs? - https://towardsdatascience.com/what-t...
Understanding Variational Autoencoders (VAEs) - https://towardsdatascience.com/unders...