Microsoft Fabric for Power BI Users Tutorial 02 - Load data into a lakehouse using Python and Spark
In this short hands-on tutorial, we use Python and Spark to load an external CSV file with 28 million rows as a table in a lakehouse. We then quickly look at the table with a SQL SELECT query and build a Power BI report.
We use simple straight-forward Python code in this tutorial and explain every line. Python is my favourite method for analysing data in Fabric. If you are new to Python, I hope this will show how useful it is to learn some basics.
Timestamps
00:00 Introduction.
01:15 Load the sample CSV file into the Files area of the lakehouse.
02:35 Create a Python notebook.
03:10 Load the external data into a Spark dataframe and inspect the data.
06:05 Save the original data as a table.
07:10 Clean the data: remove columns, add column names, change column datatypes, add a column.
09:30 Save the clean data to a table.
10:20 Analyse the table with a SQL SELECT command.
10:35 Build a Power BI report.
11:45 Conclusion. Links to follow on.
If you want to follow along, you can download the original data file from the Land Registry site
https://www.gov.uk/government/statist...
The Python code is at
https://github.com/MarkWilcock/lbag-o... Fabric Series/Load PricePaid.py
We run a full range of data analysis and generative AI courses in London and online. See our courses at zomalex.co.uk