Topic 8: Fine-Tuning an LLM

Learning Goals

• Understand different varieties of fine-tuning

• Know when fine-tuning is more appropriate than RAG

• Know how to run a fine-tuning pipeline using Tinker.

Tasks

1. Find your partner on this sheet - let me know if your assigned partner is not here

2. Install Tinker from thinkingmachines.ai following these instructions.

   1. Install the Cookbook too.

   2. Sign into the console using your UVA email address.

   3. Set TINKER_API_KEY in your shell environment.

3. Read through the Introduction to Fine-Tuning LLMs

4. Complete the Fine-Tuning Exercise: Text-to-SQL with Tinker

Resources

• sql_create_context_v4.json - the data used to fine-tune Llama to convert text to SQL.

• sql_matches.py - code for testing if two SQL queries are semantically the same.

• Instructions for installing Tinker

• Lecture slides

Portfolio

Create a subdirectory in your GitHub portfolio named Topic8FineTuning and save your programs, each modified version named to indicate its task number/letter and purpose. Create appropriately named text files saving the outputs from your terminal sessions running the programs. Create README.md with a table of contents of the directory and a summary and discussion of what you discovered.