Apache Kafka: Enterprise Use Cases
In this clip from the webinar "Harnessing Streaming Data with Apache Kafka," OpenLogic enterprise architects Connor Penhale and Joe Carder describe some of the challenges that they have encountered — and solved — for enterprise customers with Kafka deployments. Moderated by Javier Perez, Chief OSS Evangelist and Senior Director of Product Management at Perforce Software.
Watch the full webinar: https://ter.li/96796w
Transcript*
Javier: You work with our customers and they go and talk to you, and they need some help, right? Sometimes they need some help on around configurations, setting up, scaling up their deployments. Can you talk about a little bit about that? Maybe connectivity, maybe just help on the installation?
Joe: Yeah. We generally get engaged by our customers when they have some sort of specific issue that they're running into. One of our last engagements, they were having an issue where Kafka was losing communication with their ZooKeeper because of some resource and contention which was creating their ISR updates to get lost. And so you'd have some nodes thinking, “Oh, I'm the partition leader.” And another node is saying, “No, I'm the partition leader.” And it would just create this cascading failure across their whole cluster. So they came to us and said, “Hey, we need help figuring out what's going on here.” And we went through the environment survey, got an idea of what they were doing, and then we were able to narrow down through their logs what was happening. And this was a huge scale. They were doing 5.3 million messages per minute and with an end-to-end latency of less than five seconds.
And normally there's a trade-off there to where you're getting huge throughput, but your latency might be a little slow because you're prefetching a lot of stuff and you're not batching a lot of stuff. So that's going to affect your end-to-end latency. But these guys had like 76 brokers that they were running. So it’s just a massive installation. But every once in a while they'd hit a certain incident that would put them dead in the water. So that's the kind of thing that we've been helping with. And then a lot of people come to us because they want to get to that 5.3 million messages per minute. And then we offer them our advice there, too, which is always nice because that gives us more of a greenfield environment that we always like to work with.
Connor: Do it right the first time.
Javier: Connor, tell us a good use case. I know you want to share one.
Connor: Well, I love it when these Kafka instances get so big and handle such great data. And I worked with a company recently where it was just so cool to see how they did it. It was a large U.S. national cable and internet provider. Cable, internet over the top, the whole telco deal. So every time you get your remote, you press play and you mute or you push the volume up and down, this is going to generate a record for that set-top box and that's going to stream.
So if you have 150 million customers and some of those guys are watching $150 pay-per-view event, the big fight, the big open, whatever it is, if they're buffering, if they have quality of experience issues, they're going to catch that with complex event processing on that Kafka stream. If everyone's turning it up at the same time, are they excited about it or did the volume cut out? Is it a quality of experience issue where everyone's buffering at the same time or only certain regions? So you can start narrowing in on those problems at a dashboard level.
But the thing that I liked that they did is after this came in on the event stream, they have a customer experience team that's coming in to answer calls and say, “Hey, we heard you had a problem watching the big fight. How can we help you?” When they get that call, they already have records in place from all the complex event processing they did that said this customer with this account number had this problem with this pay- per-view event before they've even called in. And when they do call in, the customer rep, the person at that lowest tier one level has the ability to say, “We've already refunded you. You're good. We're here to help.” So when a cable company can increase their customer experience and make happy customers off of one of the only things that they really provide that’s differentiating…I mean, it’s classically known, right? No one loves their cable company right now. So anytime these guys can increase customer satisfaction, that's a huge win. And that's enabled by Kafka.
lightly edited for clarity