Data Virtualization with Azure SQL Managed Instance
Data virtualization in Azure SQL Managed Instance allows you to query remote data in Parquet or CSV files stored in Azure Blob Storage or Azure Data Lake Services gen. 2.
This is achieved through OPENROWSET (best suited for quick data exploration) or External Tables (best suited for repeated queries, joins with local data, etc).
Typical use for Data virtualization includes:
• Reducing local storage consumption by querying less needed data from cheap blob storage in Azure
• Exploratory data analysis
• Creating a relational abstraction of raw data that is queried at the source and always up to date, avoiding ETL setups
• Relational abstraction of raw data that is always up to date, avoiding expensive ETL
• Easy distribution and consumption of shared data sets for multi-tenant applications
Data virtualization is an evolution of Polybase technology known in on premise SQL server. It caters for same use cases, but it is optimized for cloud SQL deployments and offers new performance oriented capabilities such as Partition Elimination and Predicate Pushdown (Parquet only).
Chapters:
0:0 Introduction and use cases
0:38 - Demo
0:55 - Openrowset
1:26 - Inferring data types
2:01 - Creation of External Table
2:39 - Creation of a View on top of External Table
2:58 - Adding the View data into Power BI
3:27 - External data driven insights in Power BI
3:44 - Difference to Polybase
Recommended content:
Data virtualization with Azure SQL Managed Instance: https://learn.microsoft.com/en-us/azu...
What is Azure SQL Managed Instance?: https://learn.microsoft.com/en-us/azu...
What's new in Azure SQL Managed Instance?: https://learn.microsoft.com/en-us/azu...
How to do more with less in Azure SQL Managed Instance?: https://techcommunity.microsoft.com/t...
2022 - a year of unparalleled innovation in Azure SQL Managed Instance: https://techcommunity.microsoft.com/t...