Fine-tuning Large Language Models: A Practical Guide
Last reviewed . Tool versions and defaults may have changed since, so check the commands against current docs.
Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case. Includes LoRA, QLoRA, and full fine-tuning techniques.
On this page
Fine-tuning allows you to adapt pre-trained language models to your specific domain or task. This guide covers practical techniques for fine-tuning modern LLMs.
Why Fine-tune?
- Domain Adaptation: Make models understand your specific domain
- Task Specialization: Improve performance on specific tasks
- Cost Efficiency: Smaller fine-tuned models can outperform larger base models
- Data Privacy: Keep sensitive data on-premises
Fine-tuning Methods
1. Full Fine-tuning
Updates all model parameters. Most effective but resource-intensive.
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset
model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Load your dataset
dataset = load_dataset("your-dataset")
# Tokenize
def tokenize_function(examples):
return tokenizer(examples["text"], truncation=True, max_length=512)
tokenized_dataset = dataset.map(tokenize_function, batched=True)
# Training arguments
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
learning_rate=2e-5,
fp16=True,
logging_steps=10,
save_steps=500,
)
# Trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset["train"],
)
trainer.train()
2. LoRA (Low-Rank Adaptation)
Efficient fine-tuning that only updates a small number of parameters.
from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
# LoRA configuration
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16, # Rank
lora_alpha=32,
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
lora_dropout=0.05,
)
# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # See how many parameters are trainable
# Train as normal
trainer = Trainer(model=model, ...)
trainer.train()
3. QLoRA (Quantized LoRA)
Combines quantization with LoRA for even more efficiency.
from transformers import BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
# 4-bit quantization
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4"
)
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-2-7b-hf",
quantization_config=quantization_config,
device_map="auto"
)
# Apply LoRA
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)
Hardware Requirements
| Model Size | Full Fine-tuning | LoRA | QLoRA |
|---|---|---|---|
| 7B | 4x A100 (80GB) | 1x A100 (40GB) | 1x RTX 3090 (24GB) |
| 13B | 8x A100 (80GB) | 2x A100 (40GB) | 1x A100 (40GB) |
| 70B | 16x A100 (80GB) | 4x A100 (40GB) | 2x A100 (40GB) |
Data Preparation
Format Your Data
# Instruction-following format
data = [
{
"instruction": "Explain quantum computing",
"input": "",
"output": "Quantum computing uses quantum mechanical phenomena..."
},
{
"instruction": "Translate to French",
"input": "Hello, how are you?",
"output": "Bonjour, comment allez-vous?"
}
]
# Convert to training format
def format_prompt(example):
if example["input"]:
return f"### Instruction:\n{example['instruction']}\n\n### Input:\n{example['input']}\n\n### Response:\n{example['output']}"
return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"
Training Best Practices
1. Learning Rate Scheduling
training_args = TrainingArguments(
learning_rate=2e-5,
lr_scheduler_type="cosine",
warmup_steps=100,
warmup_ratio=0.1,
)
2. Gradient Checkpointing
training_args = TrainingArguments(
gradient_checkpointing=True, # Saves memory
)
3. Mixed Precision Training
training_args = TrainingArguments(
fp16=True, # or bf16=True for newer GPUs
)
Evaluation
from transformers import pipeline
# Load fine-tuned model
model = AutoModelForCausalLM.from_pretrained("./results/checkpoint-1000")
tokenizer = AutoTokenizer.from_pretrained("./results/checkpoint-1000")
# Create pipeline
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
# Test
prompt = "Explain Kubernetes in simple terms:"
result = generator(prompt, max_length=200, temperature=0.7)
print(result[0]["generated_text"])
Deployment
Export for Production
# Merge LoRA weights
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained("./merged-model")
Serve with vLLM
# Install vLLM
pip install vllm
# Start server
python -m vllm.entrypoints.openai.api_server \
--model ./merged-model \
--port 8000
Common Pitfalls
- Overfitting: Use validation set and early stopping
- Catastrophic Forgetting: Use LoRA or adapter methods
- Data Quality: Ensure high-quality, diverse training data
- Learning Rate: Start with 2e-5 and adjust
- Batch Size: Balance memory and training stability
Conclusion
Fine-tuning LLMs has become more accessible with techniques like LoRA and QLoRA. Start with QLoRA for resource-constrained environments, and use full fine-tuning when you have the resources and need maximum performance.
Cite this article
Kiril Urbonas (2024). Fine-tuning Large Language Models: A Practical Guide. DevOpsNess. https://www.devopsness.com/blog/fine-tuning-large-language-models-practical-guide
[Fine-tuning Large Language Models: A Practical Guide](https://www.devopsness.com/blog/fine-tuning-large-language-models-practical-guide) (Kiril Urbonas, DevOpsNess, 2024-02-12)
More from AI
Explore more articles in this category
MLOps: Model Registry vs MLflow Tracking, And When You Need Both
Tracking experiments and shipping models are different problems. The MLOps tooling assumes one solution; production splits them. The patterns we use.
What Is the Model Context Protocol (MCP)?
MCP is an open standard that gives AI agents one consistent way to reach external tools and data instead of bespoke glue.
OpenAI vs Anthropic vs Gemini API: Cost and Quality Compared
A practitioner's guide to picking between the three frontier LLM APIs based on task, price, latency, and enterprise terms.
You might have missed
Evergreen posts worth revisiting.