DataXDay - The internals of stateful stream processing in Spark Structured Streaming
Let's talk about state management in Spark Structured Streaming. During this talk you will learn the streaming concepts that are particularly relevant for stateful stream processing in Structured Streaming, e.g. watermark and output modes, but also GroupState and GroupStateTimeout. We will be exploring simple stateful processing (with groupBy operator) and more advanced use cases with KeyValueGroupedDataset.mapGroupsWithState and the most advanced KeyValueGroupedDataset.flatMapGroupsWithState operator. In other words, you will learn how to use the stateful streaming API and understand the internals.
Par Jacek Laskowski - Freelance IT Consultant
DataXDay est une conférence organisée par Xebia et Publicis Sapient.
dataxday.fr
Suivez-nous sur Twitter : @dataxday