Fine-tuning Large Language Models: A Practical Guide
Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case. Includes LoRA, QLoRA, and full fine-tuning techniques.
Key takeaways
- Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case.
- Includes LoRA, QLoRA, and full fine-tuning techniques.
On this page
Fine-tuning Large Language Models: A Practical Guide#
Fine-tuning allows you to adapt pre-trained language models to your specific domain or task. This guide covers practical techniques for fine-tuning modern LLMs.
Why Fine-tune?#
- Domain Adaptation: Make models understand your specific domain
- Task Specialization: Improve performance on specific tasks
- Cost Efficiency: Smaller fine-tuned models can outperform larger base models
- Data Privacy: Keep sensitive data on-premises
Fine-tuning Methods#
1. Full Fine-tuning#
Updates all model parameters. Most effective but resource-intensive.
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset
model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Load your dataset
dataset = load_dataset("your-dataset")
# Tokenize
def tokenize_function(examples):
return tokenizer(examples["text"], truncation=True, max_length=512)
tokenized_dataset = dataset.map(tokenize_function, batched=True)
# Training arguments
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
learning_rate=2e-5,
fp16=True,
logging_steps=10,
save_steps=500,
)
# Trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset["train"],
)
trainer.train()
2. LoRA (Low-Rank Adaptation)#
Efficient fine-tuning that only updates a small number of parameters.
from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
# LoRA configuration
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16, # Rank
lora_alpha=32,
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
lora_dropout=0.05,
)
# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # See how many parameters are trainable
# Train as normal
trainer = Trainer(model=model, ...)
trainer.train()
3. QLoRA (Quantized LoRA)#
Combines quantization with LoRA for even more efficiency.
from transformers import BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
# 4-bit quantization
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4"
)
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-2-7b-hf",
quantization_config=quantization_config,
device_map="auto"
)
# Apply LoRA
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)
Hardware Requirements#
| Model Size | Full Fine-tuning | LoRA | QLoRA |
|---|---|---|---|
| 7B | 4x A100 (80GB) | 1x A100 (40GB) | 1x RTX 3090 (24GB) |
| 13B | 8x A100 (80GB) | 2x A100 (40GB) | 1x A100 (40GB) |
| 70B | 16x A100 (80GB) | 4x A100 (40GB) | 2x A100 (40GB) |
Data Preparation#
Format Your Data#
# Instruction-following format
data = [
{
"instruction": "Explain quantum computing",
"input": "",
"output": "Quantum computing uses quantum mechanical phenomena..."
},
{
"instruction": "Translate to French",
"input": "Hello, how are you?",
"output": "Bonjour, comment allez-vous?"
}
]
# Convert to training format
def format_prompt(example):
if example["input"]:
return f"### Instruction:\n{example['instruction']}\n\n### Input:\n{example['input']}\n\n### Response:\n{example['output']}"
return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"
Training Best Practices#
1. Learning Rate Scheduling#
training_args = TrainingArguments(
learning_rate=2e-5,
lr_scheduler_type="cosine",
warmup_steps=100,
warmup_ratio=0.1,
)
2. Gradient Checkpointing#
training_args = TrainingArguments(
gradient_checkpointing=True, # Saves memory
)
3. Mixed Precision Training#
training_args = TrainingArguments(
fp16=True, # or bf16=True for newer GPUs
)
Evaluation#
from transformers import pipeline
# Load fine-tuned model
model = AutoModelForCausalLM.from_pretrained("./results/checkpoint-1000")
tokenizer = AutoTokenizer.from_pretrained("./results/checkpoint-1000")
# Create pipeline
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
# Test
prompt = "Explain Kubernetes in simple terms:"
result = generator(prompt, max_length=200, temperature=0.7)
print(result[0]["generated_text"])
Deployment#
Export for Production#
# Merge LoRA weights
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained("./merged-model")
Serve with vLLM#
# Install vLLM
pip install vllm
# Start server
python -m vllm.entrypoints.openai.api_server \
--model ./merged-model \
--port 8000
Common Pitfalls#
- Overfitting: Use validation set and early stopping
- Catastrophic Forgetting: Use LoRA or adapter methods
- Data Quality: Ensure high-quality, diverse training data
- Learning Rate: Start with 2e-5 and adjust
- Batch Size: Balance memory and training stability
Conclusion#
Fine-tuning LLMs has become more accessible with techniques like LoRA and QLoRA. Start with QLoRA for resource-constrained environments, and use full fine-tuning when you have the resources and need maximum performance.
Get the DevOps Troubleshooting Cheat Sheet
Subscribe and get our free one-page reference for the errors that eat an afternoon — CrashLoopBackOff, OOMKilled, Terraform state locks, and more — plus new guides as we publish them.
Infrastructure as Code: Terraform vs Pulumi vs Ansible
Compare Terraform, Pulumi, and Ansible for Infrastructure as Code. Learn when to use each tool and how they complement each other in modern DevOps workflows.
Practical Guide: Infrastructure Drift Detection Workflow
We had three months of slow drift between our Terraform code and AWS reality. Here's the daily-cron + Slack workflow that closed the gap.
More from AI
Explore more articles in this category
Claude Code Mods: Programmable, Unsandboxed, and Your Problem in CI
Mods make Claude Code a real runtime you can program. They also run with your full native permissions, no sandbox, so a bad one is a bad CI step.
Gemini 4 Argon: A Frontier Model You Mostly Can't Use Yet
Google shipped a 1M-token output ceiling and strong cyber-defense benchmarks, then handed the keys to almost nobody. Here is what that gating pattern tells you.
GPT-6.1 Sol's 80% Price Cut Rewrites Agentic Coding Math
OpenAI priced GPT-6.1 Sol at a fifth of GPT-6 Astra and called it nearly as good at agentic coding. That claim is now a routing decision, not a headline.
You might have missed
Evergreen posts worth revisiting.
