Fine-Tuning Text Embeddings For Domain-specific Search (w/ Python)
🤝 Work with me: https://aibuilder.academy/yt/hOLBrIjRAj4
🚀 Ship AI apps in weeks, not months: https://aibuilder.academy/courses/yt/...
In this video, I walk through how to fine-tune a text embedding model for domain adaptation using the Sentence Transformers Python library.
Resources:
📰 Blog: https://shawhin.medium.com/fine-tunin...
💻 GitHub Repo: https://github.com/ShawhinT/YouTube-B...
🤗 Model: https://huggingface.co/shawhin/distil...
💿 Dataset: https://huggingface.co/datasets/shawh...
References:
[1] • How to Improve LLMs with RAG (Overview + P...
[2] • Text Embeddings, Classification, and Seman...
[3] • Fine-Tuning BERT for Text Classification (...
[4] https://sbert.net/docs/sentence_trans...
[5] https://sbert.net/docs/sentence_trans...
[6] https://sbert.net/docs/sentence_trans...
[7] https://sbert.net/docs/package_refere...
Intro - 0:00
RAG - 0:48
Problem with Vector Search - 2:25
Fine-tuning - 3:49
Why fine-tune? - 4:43
5 Steps for Fine-tuning Embeddings - 6:23
Example: Fine-tuning Embeddings on AI Jobs - 6:55
Step 1: Gather Positive (and Negative) Pairs - 7:53
Step 2: Pick a Pre-trained Model - 12:50
Step 3: Pick a Loss Function - 14:18
Step 4: Fine-tune the Model - 15:57
Step 5: Evaluate the Model - 18:00
What's Next? - 19:13