Improving RAG Retrieval by 60% with Fine-Tuned Embeddings
Actually worked better than I thought lol
Resources:
Code: https://github.com/ALucek/ft-modernbe...
Model: https://huggingface.co/AdamLucek/Mode...
Dataset: https://huggingface.co/datasets/AdamL...
Philipp Schmid’s Blog: https://www.philschmid.de/fine-tune-e...
Matryoshka Representation Learning Blog: https://huggingface.co/blog/matryoshka
MRL Paper: https://arxiv.org/pdf/2205.13147
Chapters:
00:00 - Why Care About Embedding Models
02:41 - Setting the Scene
04:33 - Synthetic Dataset Creation
06:09 - Triplets
08:05 - Formatting our Dataset
08:53 - Choosing a Base Model
10:14 - Evaluation Dataset Prep
12:44 - Matryoshka Representation Learning
15:51 - Creating the Sequence Evaluator
17:00 - Evaluation Metric Breakdown
21:08 - Base Model Evaluation
22:04 - Loading the Model for Training
22:51 - Loss Function Selection
25:05 - Trainer Arguments
26:08 - Training the Model!
26:57 - Comparing Base vs Fine Tune Metrics
28:28 - Using The Fine Tuned Model
#ai #datascience #machinelearning