scala.bythebay.io: Neville Li, Scio - A Scala API for Google Cloud Dataflow
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Scio - A Scala API for Google Cloud Data... — https://devreal.ai/t/161111-scio-scal...
Speaker: Neville Li — https://devreal.ai/n/neville-li
Company: Spotify — https://devreal.ai/c/spotify
Project: Apache Beam — https://devreal.ai/p/apache-beam
Project: Google Cloud Dataflow — https://devreal.ai/p/google-cloud-dat...
Project: Scalding — https://devreal.ai/p/scalding
Project: Scio — https://devreal.ai/p/scio
Event: Scala by the Bay — https://devreal.ai/e/161111-scala-by-...
Explore these connections and related videos, photos, speakers’ blogs and podcasts, and other By the Bay networks.
-----
We will present Scio, a Scala API for Google Cloud Dataflow (incubated as Apache Beam). Apache Beam offers a simple, unified programming model for both batch and streaming data processing while Scio brings it much closer to other high level APIs many data engineers are familiar with, e.g. Spark and Scalding. We will cover design and implementation of the framework, including features like type safe BigQuery macros, REPL and serialization. There will also be a live coding demo.
Neville Li
Software engineer, Spotify
Neville is a software engineer at Spotify who works mainly on data infrastructure and tools for machine learning and advanced analytics. In the past few years he has been driving the adoption of Scala and new data tools for music recommendation, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.