Building SaaS on Kafka Streams
Colt McNealy is re-imagining the future of microservices orchestration and he decided to build it entirely on Kafka Streams.
In this conversation we discuss how Kafka Streams provides the low latency, reliability, availability and elasticity that is needed for the next generation of microservices orchestration. Colt also shares the most exciting up and coming improvements in Kafka Streams community and the roadmap he'd dictate if he was the benevolent dictator of Kafka Streams.
Chapters:
00:00 intros
00:44 Why Kafka Streams?
04:10 Kafka Streams as a cache
05:20 Availability and state restoring
09:50 Chaos testing of Kafka Streams
10:55 Low latency deployments
16:00 Why there are no K8s Operators for Kafka Streams
19:00 Contributing to the Apache Kafka ecosystem
21:40 Exciting upcoming features
22:19 KIP-892 - Improved exactly once semantics
27:48 Using SpeeDB in Kafka Streams
35:29 KIP-925 - Rack aware assignment of streams partitions
36:36 Colt’s dream roadmap
44:10 Active-active deployments
47:40 Book recommendations
49:00 Pros and Cons of Terraform
51:47 Advice for founders