WarpStream: A drop-in replacement for Kafka
In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Ryan and Richie, the founders of WarpStream. They discuss the architecture, benefits, and core functionalities of WarpStream, a drop-in replacement for Apache Kafka. The conversation covers their experience with Kafka, the design decisions behind WarpStream, and the operational challenges it addresses. They also delve into the seamless migration process, the scalability, and cost benefits, the integration with the Kafka ecosystem, and potential future features. This episode is a must-watch for developers and tech enthusiasts interested in modern, distributed data streaming solutions.
Chapters:
00:00 Introduction
02:27 Introducing Warpstream: A Kafka Replacement
11:07 Deep Dive into Warpstream's Architecture
35:42 Exploring Kafka's Ordering Guarantees
36:52 Handling Buffering and Compaction
38:44 Efficient Data Reading and File Caching
44:06 WarpStream's Flexibility and Cost Efficiency
01:06:59 Future Features
Links:
WarpStream : https://www.warpstream.com/
Blog: https://www.warpstream.com/blog
X:
Ryan: https://x.com/ryanworl
Richard Artoul: https://x.com/richardartoul
Kaivalya Apte: https://x.com/thegeeknarrator
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=...
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: • Write-ahead-logging
Popular playlists:
Realtime streaming systems: • Realtime Streaming Systems
Software Engineering: • Software Engineering
Distributed systems and databases: • Distributed Systems and Databases
Modern databases: • Modern Databases
Stay Curios! Keep Learning!
#distributedsystems #kafka #s3 #streaming