Storage Wars: The TRUTH with Kyle Weller
Apache XTable is an open-source project currently in the Apache Incubator that provides omnidirectional interoperability between different lakehouse table formats, such as Apache Hudi, Apache Iceberg, and Delta Lake. Its primary purpose is to enable seamless conversion and synchronization of metadata across these formats without needing to move or rewrite the underlying data files.
Chapters
• 0:00 - Introduction to Kyle Weller & Onehouse
• 0:50 - The Landscape of Open-Source Table Formats: Delta Lake, Iceberg, and Apache Hoodie
• 1:50 - The Motivation for Apache Xtable: Addressing Interoperability Challenges
• 3:00 - The Apache Xtable Project: Origin, Philosophy, and Community Growth
• 4:00 - Deep Dive: Understanding Apache Hoodie and Its Strengths
• 5:15 - Apache Hoodie's Technical Foundation: Blending Parquet and Avro for Low Latency
• 6:15 - Real-World Examples of Apache Hoodie's Low-Latency Applications
• 7:15 - Querying Apache Hoodie: Merge-on-Read vs. Read-Optimized Views
• 8:30 - The Evolution of Xtable: From OneTable to Apache Xtable and Governance
• 9:45 - How Apache Xtable Works: Metadata Translation, Not a New Table Format
• 10:45 - Apache Xtable's Independence and Operationalization
• 11:30 - Maintaining Format Specialties with Apache Xtable
• 12:45 - Apache Xtable vs. Delta Lake Uniform & Community Collaboration
• 13:45 - Getting Started with Apache Xtable: Resources and Demos
Documentation
Kyle Weller: / lakehouse
Apache Xtable: https://xtable.apache.org/
NextGenLakehouse Newsletter
#Databricks #DeltaLake #Delta #UnityCatalog #ETL #DataEngineering
Databricks DeltaLake Delta UnityCatalog ETL Data Engineering