Building a Serverless Streaming Platform
Krishna Raman has many years of experience building platforms for developers. Now he's applying this experience at Delta Streams to build a serverless platform for stream processing in SQL.
In our conversation, we discussed the serverless developer experience, some of the secret sauce behind Delta Stream's Flink Operator, how to deliver a bring-your-own-account SaaS, K8s network policies and why they matter and when you shouldn't use K8s at all.
Chapters:
00:00 Welcome to the show
00:25 Career in platform engineering
05:00 What is DeltaStream
08:22 Serverless developer experience
11:10 What does the platform provide?
14:23 Dealing with mid-stream failures
16:23 The secret sauce behind Serverless Flink
17:38 Why and how to write a Flink operator?
20:00 How to write platforms on K8s
22:43 Network policies
24:05 Succeeding with bring-your-own-account
29:31 Moving from Calico to Cillium
36:00 SaaS doesn’t solve all your problems
36:57 When to use K8s and when to avoid it
42:00 Goodbye!