Posterior Sampling by Autoregressive Generation: Daniel Russo
Speaker: Daniel Russo, Associate Professor, Decisions, Risk, and Operations Division, Columbia Business School
Title: Posterior Sampling by Autoregressive Generation
Abstract: Conventionally trained neural networks excel at prediction but often struggle to model uncertainty in their own predictions. We explore this challenge in the cold-start content exploration problem for recommendation systems. We present a scalable approach to Bayesian uncertainty quantification by posing it as a problem of autoregressive generative modeling. First, we pre-train a generative model to predict the next user’s response to a recommended item based on that item’s features and previous recommendation responses for the item from other users. At inference time, our algorithm makes item recommendations based on limited previous responses and autoregressively generated hypothetical future responses. Far from a heuristic, we synthesize insights from the literature to show our method is a novel implementation of Thompson (posterior) sampling, a prominent bandit algorithm. We prove that the algorithm has low regret whenever the pre-trained autoregressive model has near optimal prediction loss. We then empirically demonstrate the scalability of our approach on a news recommendation problem where text features are required for the best performance.
Bio: Daniel Russo is a Philip H. Geier Jr. Associate Professor in the Decision, Risk, and Operations division of Columbia Business School. His research lies at the intersection of statistical machine learning and online decision making, mostly falling under the broad umbrella of reinforcement learning. His work has been recognized by the Frederick W. Lanchester Prize, an INFORMS Junior Faculty Interest Group Best Paper Award, and first place in the George Nicholson Student Paper Competition. Daniel serves as an associate editor at the journals Operations Research, Management Science, and Stochastic Systems. Outside academia, he works with Spotify’s to apply reinforcement learning and large language models in audio recommendations.