Apache Kafka Security Best Practices
https://cnfl.io/podcast-episode-228 | Security is a primary consideration for any system design, and Apache Kafka® is no exception. Out of the box, Kafka has relatively little security enabled. Rajini Sivaram (Principal Engineer, Confluent, and co-author of “Kafka: The Definitive Guide” ) discusses how Kafka has gone from a system that included no security to providing an extensible and flexible platform for any business to build a secure messaging system. She shares considerations, important best practices, and features Kafka provides to help you design a secure modern data streaming system.
In order to build a secure Kafka installation, you need to securely authenticate your users. Whether you are using Kerberos (SASL/GSSAPI), SASL/PLAIN, SCRAM, or OAUTH. Verifying your users can authenticate, and non-users can’t, is a primary requirement for any connected system.
But authentication is only one part of the security story. We also need to address other areas. Kafka added support for fine-grained access control using ACLs with a pluggable authorizer several years ago. Over time, this was extended to support prefixed ACLs to make ACLs more manageable in large organizations. Now on its second generation authorizer, Kafka is easily extendable to support other forms of authorization, like integrating with a corporate LDAP server to provide group or role-based access control.
Even if you’ve set up your system to use secure authentication and each user is authorized using a series of ACLs if the data is viewable by anyone listening, how secure is your system? That’s where encryption comes in. Using TLS Kafka can encrypt your data-in-transit.
Security has gone from a nice-to-have to being a requirement of any modern-day system. Kafka has followed a similar path from zero security to having a flexible and extensible system that helps companies of any size pick the right security path for them.
Be sure to also check out the newest Apache Kafka Security course on Confluent Developer for an in-depth explanation along with other recommendations.
EPISODE LINKS
► Kafka Security course: https://cnfl.io/kafka-security-101-ep...
► Kafka: The Definitive Guide v2: https://cnfl.io/kafka-the-definitive-...
► Security Overview: https://cnfl.io/security-overview-epi...
► Kris Jenkins’ Twitter: / krisajenkins
► Streaming Audio Playlist: • Streaming Audio Podcast | Apache Kafk...
► Join the Confluent Community: https://cnfl.io/join-confluent-commun...
► Learn more with Kafka tutorials, resources, and guides: https://cnfl.io/confluent-developer-e...
► Live demo: Intro to Event-Driven Microservices with Confluent: https://cnfl.io/event-driven-microser...
► Use PODCAST100 to get $100 of free Confluent Cloud usage: https://cnfl.io/try-cloud-episode-228
► Promo code details: https://cnfl.io/podcast100-details-ep...
TIMESTAMPS
0:00 - Intro
2:20 - Kafka: The Definitive Guide v2
3:41 - Kafka as a service
10:10 - Securing Kafka – the development process
12:56 - Authentication
18:23 - Authorization
29:09 - Audit logs
35:14 - Encryption
36:40 - It's a wrap
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#security #apachekafka #kafka #confluent