Inside AlgoX2: The Three-Layer Architecture That Replaces Kafka
Kafka is a scale-up architecture. AlgoX2 is a scale-out streaming OS — and the difference is everything.
In Episode 2 of Inside the Data Plane, AlgoX2 CEO Alexei Lebedev walks George Levin through the three-layer architecture behind AlgoX2: gateways, sequencers, and commits — independently scalable, separated across compute, storage, and network. This is the architectural choice that lets AlgoX2 hit 100M messages/sec per node at 50μs latency on commodity hardware, while Kafka brokers stay locked to a single machine.
Alexei breaks down why most people misunderstand multicast (and why it's actually the only way to get full line-rate fan-out from a 100-gig network), the "fat topics" problem that quietly destroys Kafka clusters past a few hundred topics, and how AlgoX2's sidecar processes give you content-based filtering and virtual streams without a downstream cluster.
What's covered:
• The three-layer data plane — gateways, sequencers, commits
• Why scale-out beats scale-up for streaming workloads
• Multicast as a hardware-replication strategy (and why it shouldn't be your problem)
• The fat topics problem — what breaks when you cross 1,000 topics in Kafka
• Sidecars, virtual streams, and inline filtering on the X2 substrate
• Why AlgoX2 calls itself a streaming operating system, not just a Kafka alternative
If you're running Kafka at scale or evaluating streaming infrastructure for AI and real-time workloads, this 15 minutes is worth your time.
🕐 Timestamps
0:00 Introduction
0:38 The Three-Layer Architecture
1:52 Why You Need Independent Scaling
2:52 Decoupled Compute and Storage
3:29 Kafka: A Scale-Up Architecture
4:48 What Scale-Out Actually Looks Like
5:41 The Multicast Problem
6:08 Why Alexei Loves Multicast
6:52 The Network Switch Is the Fastest Middleware
7:57 Why Single-Broker Fan-Out Is Blocking
9:05 How Multicast Unlocks Line Rate
9:44 Multicast Shouldn't Be Your Problem
11:06 The Fat Topics Problem
12:50 Why Many Small Files Kill Performance
13:06 How AlgoX2 Solves Fat Topics
13:31 Sidecars and Content-Based Filtering
15:12 Wrap
🔗 Links
AlgoX2: https://algox2.com
Episode 1 — The Origin Story: [add link]
#DataStreaming #SystemDesign #ApacheKafka