ADLS Dynamic Data Load from SQL Server Config Tables | SSMS | Azure Data Pipeline | Data Engineering
Create a Config table and load SQL data into Azure storage accounts parallelly based on SQL flags and configuration values from the table.
In this hands-on demo, you will learn how to create and insert configurations into an SQL server and create azure linked services to connect to Azure resources and query the data.
The pipeline will connect to the SQL server through the linked service dataset and query the config values. For every item in the config response, the pipeline will parallel execute the configuration values to query SQL tables and write to Azure blog storage via copy activity.
00:00 - Overview
00:44 - SQL config table creation
03:34 - ADF launch
03:45 - Linked service brief and creation
06:02 - ADF Config table lookup activity setup
10:07 - Azure runtime brief
10:27 - ADF forloop activity
14: 06 - ADF Copy Activity
17:15 - dynamic ADLS sink dataset
LET'S CONNECT!
📰 LinkedIn ➔ / jayachandra-sekhar-reddy
🐦 Twitter ➔ / reddyjaysekhar
📖Medium ➔ / jay-reddy
📲 Substack➔ https://databracket.substack.com
💁Fiverr ➔ https://www.fiverr.com/jayreddy9
#azure #azuredatafactory #azuredataengineer #azureservices #azuresql #azuretutorialforbeginners #azuretutorial #azuretutorials #pyspark #databricks #dataengineering #etl #data #bigdata #sql #sqlserver #sqlservermanagementstudio