Easier than DAGs: Data science automation and experimentation using Trel
Easily build reliable data science transformations using catalog-based dependency, a new approach to automation, and an alternative to DAGs.
PaaS Portal: https://trelcloud.com
Website: https://cumulativedata.com
Examples show how to
1. Migrate code from notebooks to jobs,
2. Build a large data mesh,
3. Parameterize your jobs
4. Do sophisticated A/B testing with time-travel
Chapters:
0:00 - Intro
1:12 - Catalog-based dependency
6:04 - Migrate code from notebooks to Trel jobs
8:40 - The registration file
12:14 - Building a data mesh
12:38 - Data mesh example
24:27 - Job parameterization
26:13 - Parameter A/B testing
27:56 - Transformation pipeline A/B testing
36:39 - Next steps