Modern data stack deploy dockerized data pipeline to Azure part 1
In this exciting tutorial, we guide you through the step-by-step process of deploying a data pipeline to Microsoft Azure. We start by dockerizing our data pipeline and setting it up on Azure using Azure Container Registry and Azure Container Instances. You'll learn how to mount Azure File Share for data storage and configure environment variables for connecting to your data warehouse. By the end of this video, you'll see your data pipeline running live in the cloud. Stay tuned for part two, where we will create a web app to display the dashboard. Don't miss out on this comprehensive guide to mastering data engineering in the cloud.
Previous video
• Modern data stack dockerize dagster, dlt...
Code repo
https://github.com/AIgineerAB/azure-c...
https://github.com/AIgineerAB/azure-c...
00:00 Introduction and Overview
00:57 Setting Up the Data Pipeline
08:16 Creating and Configuring Azure Storage
14:30 Deploying to Azure Container Registry
21:08 Creating Azure Container Instance
28:35 Final Steps and Testing
31:41 Conclusion and Next Steps