Go Kafka: How To Manually Commit Offsets The Right Way. Database Idempotency.
🔔 SUBSCRIBE for more content like this 👇: / @kcode_tech
-------
In this advanced Kafka tutorial, we implement manual offset management in Go to achieve exactly-once message processing! Learn how to build a fault-tolerant Kafka consumer using the High Water Mark strategy, manual offset commits, and idempotent database operations.
00:00 Intro
00:59 Visual Problem Statement
04:45 Init Kafka Setup Overview
06:51 Updating Kafka Consumer
10:13 DB setup with SQLX
17:13 Exactly-once implementation
23:30 Kafka Manual Offset Commit
32:45 Testing exactly-once delivery
-------
📂 FULL SERIES:
1. Kafka Producer & Consumer: • Go Kafka: Building Producer & Consumer wit...
2. Consumer offsets commit: • Go Kafka: How To Manually Commit Offsets T...
3. Consumer Partitions && scaling: • Go Kafka: What to do if API Scales? How to...
4. Outbox Pattern with Kafka: • Go Kafka: Transactional Outbox Pattern in ...
5. Inbox Pattern with Kafka: • Go Kafka: Inbox Pattern. Preventing Duplic...
6. CDC with Debezium Kafka: • Send Table Changes from DB to Kafka Automa...
📂 Kafka Playlist:
• Go Kafka: Inbox Pattern. Preventing Duplic...
📂 Debezium Live Streams:
• Golang learning - Debezium with multiple D...
📂 Folder Structure Rework Live Streams:
• [LIVE] Go Folder Structure
-------
🔗 Useful Links:
GitHub Repository: https://github.com/k-code-yt/golang-y...
Confluent Kafka Go Client: https://github.com/confluentinc/confl...
Apache Kafka Docs: https://kafka.apache.org/documentation/
📱 Social media links:
Twitch: / kcode_tech
#kcode #go #golang #kafka #exactlyonce #offset #concurrency #distributed #systems #messaging #idempotency #database #transactions #sync #mutex #programming #tutorial #advanced