From Bytes to Borsch - Fine-tuning Gemma LLM to Speak Ukrainian
0:00 - Introduction of Artur Kiulian and topic of fine-tuning large language models for low resource underrepresented languages
1:30 - Original goal of preserving indigenous languages using LLMs
2:10 - UNLP conference challenge to fine-tune a model for Ukrainian
3:00 - Challenges with existing open-source models being English-biased
4:10 - Testing foundation models on Ukrainian language queries
5:25 - Issues with model performance on structured Ukrainian Q&A
6:20 - Overview of the landscape of fine-tuning approaches
7:50 - Selecting fine-tuning for adjusting model output format
8:10 - Challenges faced: lack of quality data, hyperparameter tuning, evaluation difficulties
9:30 - Benchmarking fine-tuned models against foundation models
10:20 - Phenomenon of linguistic code-switching Azirivka observed
11:20 - Next steps: extending datasets, refining benchmarks, pre-training models further
12:15 - Peer-reviewed paper on the research experience
12:40 - Details on fine-tuning process and data used
13:40 - Advantages of using the Gemma model for multilingual support