How to load reference data to database with Python ETL Pipeline | Excel to Postgres
In this video we will build on from the previous session where we developed a Python ETL pipeline. Today we will see how to design an ETL Pipeline with Python to extract reference data from Excel files and load this data in PostgreSQL. We often have There comes a time when you need to incorporate the reference data sitting in flat files in your data warehouse to help translate a table or map data. Today, we will learn how to iterate over a directory, read multiple flat files, and load them into our postgres database with Python.
The goal is to read data from a network share and then load it in the database. So, the reference data is available for analytics and reporting.
Link to medium article on this topic: / python-etl-pipeline-to-load-reference-data...
Link to GitHub repo: https://github.com/hnawaz007/pythonda...
Link to previous video: • How to build an ETL pipeline with Python |...
How to send emails with Python video: • Send Emails using Python | HTML Formatting...
Subscribe to our channel:
/ haqnawaz
---------------------------------------------
Follow me on social media!
Github: https://github.com/hnawaz007
Instagram: / bi_insights. .
LinkedIn: / haq-nawaz
---------------------------------------------
#ETL #Python #SQL
Topics covered in this video:
0:00 - Introduction to ETL
1:50 - The Setup
2:42 - Code ETL Pipeline
3:11 - Data Extract from Source Files
4:31 - Load data to PostgreSQL
6:11 - Test ETL Pipeline