Discover how to build reliable, scalable LLM applications using Apache Airflow in Kaxil Naik's insightful talk. Explore design patterns that integrate proprietary enterprise data to create value-driven AI applications. From orchestrating structured and unstructured data to interacting with tools like OpenAI's GPT-4 and HuggingFace, learn how Airflow enables seamless machine learning workflows at scale.
Kaxil Naik, Senior Director of Engineering at Astronomer and a leader in data engineering, shares real-world examples and future enhancements for Airflow, including proposed new providers for LLM integration. Perfect for AI, data science, and engineering professionals seeking cutting-edge solutions for natural language processing (NLP) and AI-driven workflows.
You can view the slides for the session here: https://speakerdeck.com/kaxil/buildin...
#MachineLearning #AI #DataScience #NLP #ArtificialIntelligence #DeepLearning #LanguageProcessing #DataEngineering #BigData #LLM #TechTalk #ApacheAirflow #DataOrchestration #ODSC
Outline:
0:00 - Introduction
2:05 - Chat about Chatbots IRL
6:11 - Airflow for RAG
9:23 - Real Use Cases
21:57 - Running this in pr GitHub Code
Want to stay ahead on AI? Then subscribe to ODSC today 🚀🧠- https://www.youtube.com/c/OpenDataSci...
You can also follow ODSC on:
LinkedIn - / open-data-science
Twitter - / _odsc
Facebook - / opendatasci
Medium - / odsc