Learnings from building Open Source Distributed Systems with Kishore Gopalakrishna
In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companies evaluating real-time analytics solutions.
Don't miss this in-depth conversation packed with valuable insights for both seasoned developers and tech enthusiasts!
Chapters:
00:00 Introduction
03:13 Building Distributed Systems at LinkedIn
08:57 Testing and Challenges in Distributed Systems
30:50 Advantages of Columnar Storage
33:04 The Importance of Upserts
34:24 Building a Strong Open Source Community
41:10 Challenges and Lessons in System Design
51:35 Real-Time Analytics: Do You Need It?
StarTree: https://startree.ai/
Apache Pinot: https://pinot.apache.org/
Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.
https://app.codecrafters.io/join?via=...
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: • Write-ahead-logging
Popular playlists:
Realtime streaming systems: • Realtime Streaming Systems
Software Engineering: • Software Engineering
Distributed systems and databases: • Distributed Systems and Databases
Modern databases: • Modern Databases
Stay Curios! Keep Learning!
#distributedsystems #kafka #s3 #streaming #realtimeanalytics #database #pinot #startree