Event-driven applications: Apache Kafka and Python | DevNation Day 2021
What’s better than a pizza example to show how Python and Apache Kafka, a streaming platform, work together to enable reliable real-time data integration for your event-driven application? Let's dig into problems Kafka is solving, its Python libraries and prebuilt connectors together!
Code and data go together like tomato and basil; not many applications work without moving data in some way. As our applications modernize and evolve to become more event-driven, the requirements for data are changing. In this session we will explore Apache Kafka, a data streaming platform, to enable reliable real-time data integration for your applications. We will look at the types of problems that Kafka is best at solving, and show how to use them in your own applications. Whether you have a new application or are looking to upgrade an existing one, this session includes advice on adding Kafka using the Python libraries and includes code examples (with a bonus discussion of pizza toppings) to use. With Kafka in place, many things are possible so this session also introduces Kafka Connect, a selection of pre-built connectors that you can use to route events between systems and integrate with other tools. This session is recommended for engineers and architects whose applications are ready for next-level data abilities.
Apache Kafka is becoming the de-facto standard for streaming applications with many companies basing on Kafka their whole data pipeline. In micro-services contexts, Apache Kafka runs the fundamental role of decoupling information’s producers and consumers thus making sure data pipelines are scalable and reliable. I’m a Developer Advocate at Aiven.io, the leader in the managed Apache Kafka sector, spending most of my time in the developer’s shoes building reproducible examples of data pipelines. I’ve been blogging and speaking around the world since 2014 about various aspects of the Data lifecycle: from producing applications, analytics, visualization, and streaming platforms.
Speaker: Francesco Tisiot