Kafka Deep Dive — Partitions, Groups & Exactly-Once
You know WHAT Kafka is — production Kafka is about the KNOBS: what does acks=all actually guarantee? What happens during a consumer-group rebalance? When exactly do you commit offsets, and what breaks if you get it wrong? These knobs are precisely what senior backend interviews probe.
You'll learn:
• The anatomy — topic → partitions (each an ordered log; the unit of parallelism AND ordering; same key → same partition), replicated with one leader + ISR followers
• The durability dial — acks=0 (silent loss), acks=1 (lost if the leader dies pre-replication), acks=all PAIRED with min.insync.replicas ≥ 2 (the real durability answer — unpaired, "all" is a false promise)
• Consumer groups — one partition → exactly one consumer per group; extra consumers idle (partitions cap parallelism); rebalances pause consumption (cooperative rebalancing softens); LAG is THE health metric
• Offset discipline — commit-before-processing = loss on crash (at-most-once); commit-after = reprocessing on crash (at-least-once → idempotent handlers); auto-commit can produce either failure mode — commit manually, after processing
• The advanced tier — idempotent producer (broker-side sequence dedup), transactions (consume→process→produce atomically = exactly-once WITHIN Kafka), compaction (latest-per-key changelogs), retention (the replay buffer). The gold sentence: "exactly-once inside Kafka; idempotency at the edges."
Ends with a talk-track you can repeat in the room.
📄 The full PDF (rebalance protocols, transactions detail, tuning cheatsheet) is part of the complete Codivora Backend series.
⏱️ Chapters
00:00 You know WHAT — now the knobs
00:50 Topic → partitions → ISR
01:50 The acks dial
02:50 Groups, rebalancing & lag
03:55 Offset commits
04:55 Exactly-once, compaction & retention
05:55 🎤 Explain it to the interviewer
👉 Subscribe to Codivora for the complete Backend & interview-prep library.
🔎 Also searched: microservices, kafka tutorial, system design, backend development