Scalable Collaborative Filtering on top of Apache Giraph
Featuring: Maja Kabiljo, Software Engineer at Facebook
Description: Apache Giraph is a highly performant distributed platform for doing graph and iterative computations. Collaborative filtering is a well known recommendation technique that is often solved with matrix-factorization based algorithms. This talk will detail our scalable implementation of SGD and ALS methods for collaborative filtering on top of Giraph. We will describe our novel methods for distributing the problem and the related Giraph extensions that allows us to scale to over a billion people and tens of millions of items. We will also review various additions required for handling Facebook’s data (for example, implicit and skewed item data). Finally, to complete our easy to use and holistic approach to scalable recommendations at Facebook, we detail our approach to quickly finding top-k recommendations per user.