28 Get Started with Delta Lake using Databricks | Benefits and Features of Delta Lake | Time Travel
Video explains - What is Delta Lake and why is it important? What are the Key Features of Delta Lake? How Delta table manages versions? How Delta table manages Time Travel? What is Schema Evolution in Delta Lake?
Chapters
00:00 - Introduction
00:22 - Key Features of Delta Lake
01:27 - Sign Up and Login into Databricks Community Edition for Free
03:22 - Get Started with Databricks basics
10:34 - Create Delta table using PySpark
12:20 - How Delta Table manages Metadata? What is Delta Log?
14:06 - How Versioning works in Delta Lake?
15:05 - DML Operations in Delta Table
17:38 - Time Travel in Delta Lake
20:49 - Schema Evolution in Delta Lake
24:25 - Convert Parquet to Delta Table
28:17 - Restoring to a version in Delta Lake
30:26 - Vacuum operations in Delta Lake
For Local PySpark Jupyter Lab setup just run the command - docker pull jupyter/pyspark-notebook
Python Basics - https://www.learnpython.org/
GitHub URL for code - https://github.com/subhamkharwal/pysp...
The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.
New video in every 3 days ❤️
#spark #pyspark #python #dataengineering