Keynote: The Vision and Practice of Stream and Batch Unification - LinkedIn's Story
Keynote: The Vision and Practice of Stream and Batch Unification - LinkedIn's Story
Becket Qin, Apache Flink® and Apache Kafka PMC Member and Streams Infrastructure at LinkedIn shares why LinkedIn chose Flink to handle their batch and streaming unification project.
Read more about how organizations use Apache Flink® and Ververica Platform for their stream processing needs. https://www.ververica.com/use-cases
Get all the recordings from Flink Forward Seattle 2023 free on-demand at Ververica Academy. https://www.ververica.academy/app/videos
Ververica is proud to host Flink Forward! Curious about Sponsoring future Flink Forward events? Subscribe for updates. https://www.flink-forward.org/contact
CHAPTERS
00:10: Intro: Streaming and Batch Unification
00:55: Data Applications
01:28: Data Infrastructure
02:39: Data Application Design
4:33: Stream and Batch API Unification
5:20: Technical Stack Unification
6:44: Data Allocation Design: a “new” design paradigm
7:10: How’s this different from Kappa?
9:00 “Batch is a special case of streaming”
11:04 Why should we care? Data Infra Cost Model
13:34 Example: Cost estimations vs. data Infra cost over time
15:55 LinkedIn’s Journey
17:15 Where is Stream and Batch Unification: Lifecycle Curve
18:10 Recap and Conclusion
About Ververica
Proud hosts of Flink Forward events, Ververica is the company founded by the original creators of the open source stream processing framework Apache Flink®. Ververica's mission is to power the core business of every company with cutting-edge real-time stream processing technology. In order to do that, the team at Ververica focuses on building the best technology available for stream processing, while at the same time creating a global and open community
around this technology.
We build and develop Ververica Platform, a stream processing platform that enables every enterprise to power their real-time business and use a production-grade streaming infrastructure while at the same time we actively contribute and participate in the open source Apache Flink community, the underlying technology framework of Ververica Platform itself.
https://www.ververica.com/
Follow Ververica on LinkedIn: / 5385380
Follow Ververica on X: / ververicadata
Learn more about Flink Forward: https://www.flink-forward.org/