Understanding Transformer Pipelines
This video is part of a comprehensive course that covers the fundamentals of the StreamSets DataOps Platform. To enroll in this free course, follow this link: https://academy.streamsets.com/course...
In this lesson, we learn what a Transformer pipeline is, what situations they are used in, and why we need it to create successful streaming pipelines for data ingestion. The Transformer Engine is an execution engine that runs data processing pipelines on Apache Spark, taking advantage of Spark’s performance benefits.
We’ll give you a full walkthrough of building your first Transformer pipeline with advanced processing. In the next lesson, we’ll walk through the architecture of the Transformer Engine and more.
Learn more about StreamSets: https://streamsets.com/products/datao...
Try StreamSets now: https://streamsets.com/try-dataops/?u...