Want to build a web or mobile app? Get in touch so we can help.

28. Scio: Moving Big Data to Google Cloud, a Spotify Story

October 23, 17:45

Stockholm, Sweden
Spotify Office, Birger Jarlsgatan 61 (11tr), Stockholm

External Registration

Open Registration Page

• 17.45: drink, socialize
• 18.00: first talk: Scio: Moving Big Data to Google Cloud, a Spotify Story
Abstract: We will talk about Spotify’s story of migrating our big data infrastructure to Google Cloud. Over the past year or so we moved away from maintaining our own 2500+ node Hadoop cluster to managed services in the cloud. We replaced two key components in our data processing stack, Hive and Scalding, with BigQuery and Scio and are able to iterate at a much faster speed. We will focus the technical aspect of Scio, a Scala API for Apache Beam and Google Cloud Dataflow and how it changed the way we process data.
Speaker: Neville Li - Neville is a software engineer at Spotify working on data and machine learning infrastructure. In the past few years he has been driving the adoption of Scala and new tools for data processing, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.
• 18.45: eat, drink, socialize  (more) 
• 19.00: TBA
Looking for a second speaker. Reach out to us if you are interested in holding a talk.
• 19.45: drink, socialize  (even more) 

Stockholm Hadoop User Group Stockholm Hadoop User Group

Propose talk to Stockholm Hadoop User Group