Embedded ELT: Save your budget and simplify your data platform with Dagster Embedded ELT.
Most data pipelines require data movement steps. While elaborate 3rd party services like Fivetran, Airbyte, or Stitch have become popular plug-and-play solutions, their cost tends to balloon over time. Often the value of these services is limited, especially when dealing with simpler tasks such as database-to-database replication.
Pedram Navid, Head of Data Engineering and DevRel at Dagster Labs, explains the value of the newly released Embedded ELT feature in Dagster, which provides support for lightweight data movement libraries.
One organization saved $30,000 a year, moving their database-to-database sync off Fivetran and onto Embedded ELT.
This session will benefit heads of data engineering or ML looking to improve the performance and reduce the cost of their data platform.
More context on Dagster's Embedded ELT capability can be found in the Launch Week blogpost: https://dagster.io/blog/dagster-embed...
The demo in this session will use the following code: https://github.com/dagster-io/mdsfest...
00:00 Introduction and the challenges of ELT
01:23 The advent of data ingestion SaaS solutions (Fivetran, Stitch, Airbyte, etc.)
04:02 Why Embedded ELT - the value of doing data ingestion straight from the orchestrator
06:45 What is Dagster Embedded ELT? Use the Sling and dlt lightweight data movement libraries and enjoy the best of ingestion + orchestration
07:57 Example Architectures your CFO will love
09:44 Embedded ELT demo
15:06 Future roadmap
Try Dagster today with a 30-day free trial: https://dagster.io/lp/dagster-cloud-t...