Making LLM Inference Affordable // Daniel Campos // LLMs in Production Conference Part 2
// Abstract
The impressive reasoning abilities of LLMs can be an attractive proposition for many businesses, but using foundational models and APIs can be slow and full of bumpy API latency windows. Self-hosting models can be an attractive alternative, but how do you choose what model to use, and if you have a latency or inference budget, how do you make it fit? We will discuss how pseudo-labeling, knowledge distillation, pruning, and quantization can ensure the highest efficiency possible.
// Bio
Hailing from Mexico Daniel started his NLP journey with his BS in CS from RPI. He then worked at Microsoft on Ranking at Bing with LLM(back when they had 2 commas) and helped build out popular datasets like MSMARCO and TREC Deep Learning. While at Microsoft he got his MS in Computational Linguistics from the University of Washington with a focus on Curriculum Learning for Language Models. Most recently, he has been pursuing his Ph.D. at the University of Illinois Urbana Champaign focusing on efficient inference for LLMs and robust dense retrieval. During his Ph.D., he worked for companies like Neural Magic, Walmart, Qualtrics, and Mendel.AI and now works on bringing LLMs to search at Neeva.