Google Cloud Run Hackathon Solution | AdaptML
This project, the "AdaptML: Production-Grade LoRA Fine-Tuning Platform," is a serverless application built for the Google Cloud Run Hackathon. It makes fine-tuning large language models (LLMs) like Google's Gemma-2B incredibly fast, cost-effective, and accessible.
The core of the solution is its ability to perform fine-tuning using QLoRA (Quantized Low-Rank Adaptation). This technique reduces memory usage by 75% by using 4-bit quantization, allowing large models to be trained on cost-effective NVIDIA L4 GPUs instead of more expensive, high-VRAM hardware.
The entire platform is serverless and scales to zero, meaning it costs nothing when idle.
System Architecture
The solution uses a decoupled, three-part architecture running entirely on Google Cloud Run:
Streamlit Dashboard (Cloud Run Service): A user-friendly web interface where users can submit training jobs with custom parameters (like LoRA rank and epochs) and monitor their status in real-time.
Flask API (Cloud Run Service): A REST API backend that orchestrates the entire process. It receives requests from the dashboard, triggers training jobs, and provides status updates.
Training Worker (Cloud Run Job with GPU): The heavy-lifting component. When triggered by the API, this job provisions an NVIDIA L4 GPU, fine-tunes the Gemma model using QLoRA, and then saves the resulting adapter files to a Google Cloud Storage bucket.
Key Features & Metrics
Cost: A typical training run costs under $1.00.
Speed: Fine-tuning on a sample dataset completes in just 2-3 minutes.
Efficiency: Instead of producing a multi-gigabyte model, the platform outputs a lightweight 35MB adapter file.
Scalability: As a serverless application, it automatically scales to handle multiple jobs and, most importantly, scales to zero when inactive, eliminating idle costs.