How to build LLM-based product
Dr. JunLing Hu, founder and CEO of Coach.ai
[No audio until the start of presentation at 5:50]
ABSTRACT: If you are considering building an LLM-powered product, such as customer support, internal document search, or a conversational assistant, how do you implement them? In this talk, I will review the basic components and the product pipeline. I will review the options of building in-house (using Llama 2) or using third-party solutions (such as ChatGPT). Another major component is semantic search or document retrieval. I will review both in house and third-party to this problem. By the end of this talk, you will gain a clear overview of LLM product pipeline, and understand where your skill can be best applied.
This talk describes a pipeline that can use different tools. For example, one can use a file system for saving embedding, or Pinecone for embedding remote hosting, or Weaviate for internal embedding databases. I don’t recommend Langchain as it is overly complicated.
Speaker Bio:
Dr. Junling Hu is the founder and CEO of Coach.ai, which provides LLM-powered conversational AI platform. Prior to founding the company, Junling was the Director of Applied AI at Live Person, where she led a team building LLM-based customer support solutions. Junling is the author of the book The Evolution of Artificial Intelligence. Junling received her Ph.D. in Computer Science from the University of Michigan at Ann Arbor, with her Ph.D. thesis focused on reinforcement learning.
https://www.meetup.com/sf-bay-acm/eve...
0:00 Chapter Intro (no audio)
4:26 Speaker Intro (no audio)
5:50 Presentation (audio)
5:53 What is LLM (Large Language Model)?
9:04 Architecture of LLM
11:36 The development of LLM (2018-2023)
16:10 Traditional ML Products
17:15 LLM-based Products
20:03 The LLM-product pipeline
22:01 Prompt engineering
22:42 The LLM-product pipeline (2)
26:45 The LLM-product pipeline (3)
28:26 Representing words
30:53 Representation of a document
34:40 Retrieval-Augmented Generation
51:35 Vector databases
1:05:27 Types of finetuning
1:12:01 Finetuning with RLHF (Reinforcement Learning with Human Feedback)
1:13:25 Efficient finetuning: LoRA (Low Rank Adaptation)
1:14:55 Demo
1:17:32 Contact
1:20:55 SFBayACM possible workshop