Skip to main content
Fine-tuning Large Language Models: A Practical Guide

Fine-tuning Large Language Models: A Practical Guide

KU
· Founder & Chief Editor
• 4 min read•

Last reviewed . Tool versions and defaults may have changed since, so check the commands against current docs.

Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case. Includes LoRA, QLoRA, and full fine-tuning techniques.

Fine-tuning allows you to adapt pre-trained language models to your specific domain or task. This guide covers practical techniques for fine-tuning modern LLMs.

Why Fine-tune?

  • Domain Adaptation: Make models understand your specific domain
  • Task Specialization: Improve performance on specific tasks
  • Cost Efficiency: Smaller fine-tuned models can outperform larger base models
  • Data Privacy: Keep sensitive data on-premises

Fine-tuning Methods

1. Full Fine-tuning

Updates all model parameters. Most effective but resource-intensive.

Python
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset

model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Load your dataset
dataset = load_dataset("your-dataset")

# Tokenize
def tokenize_function(examples):
    return tokenizer(examples["text"], truncation=True, max_length=512)

tokenized_dataset = dataset.map(tokenize_function, batched=True)

# Training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-5,
    fp16=True,
    logging_steps=10,
    save_steps=500,
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
)

trainer.train()

2. LoRA (Low-Rank Adaptation)

Efficient fine-tuning that only updates a small number of parameters.

Python
from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")

# LoRA configuration
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,  # Rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
)

# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # See how many parameters are trainable

# Train as normal
trainer = Trainer(model=model, ...)
trainer.train()

3. QLoRA (Quantized LoRA)

Combines quantization with LoRA for even more efficiency.

Python
from transformers import BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

# 4-bit quantization
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=quantization_config,
    device_map="auto"
)

# Apply LoRA
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)

Hardware Requirements

Model SizeFull Fine-tuningLoRAQLoRA
7B4x A100 (80GB)1x A100 (40GB)1x RTX 3090 (24GB)
13B8x A100 (80GB)2x A100 (40GB)1x A100 (40GB)
70B16x A100 (80GB)4x A100 (40GB)2x A100 (40GB)

Data Preparation

Format Your Data

Python
# Instruction-following format
data = [
    {
        "instruction": "Explain quantum computing",
        "input": "",
        "output": "Quantum computing uses quantum mechanical phenomena..."
    },
    {
        "instruction": "Translate to French",
        "input": "Hello, how are you?",
        "output": "Bonjour, comment allez-vous?"
    }
]

# Convert to training format
def format_prompt(example):
    if example["input"]:
        return f"### Instruction:\n{example['instruction']}\n\n### Input:\n{example['input']}\n\n### Response:\n{example['output']}"
    return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"

Training Best Practices

1. Learning Rate Scheduling

Python
training_args = TrainingArguments(
    learning_rate=2e-5,
    lr_scheduler_type="cosine",
    warmup_steps=100,
    warmup_ratio=0.1,
)

2. Gradient Checkpointing

Python
training_args = TrainingArguments(
    gradient_checkpointing=True,  # Saves memory
)

3. Mixed Precision Training

Python
training_args = TrainingArguments(
    fp16=True,  # or bf16=True for newer GPUs
)

Evaluation

Python
from transformers import pipeline

# Load fine-tuned model
model = AutoModelForCausalLM.from_pretrained("./results/checkpoint-1000")
tokenizer = AutoTokenizer.from_pretrained("./results/checkpoint-1000")

# Create pipeline
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

# Test
prompt = "Explain Kubernetes in simple terms:"
result = generator(prompt, max_length=200, temperature=0.7)
print(result[0]["generated_text"])

Deployment

Export for Production

Python
# Merge LoRA weights
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained("./merged-model")

Serve with vLLM

Bash
# Install vLLM
pip install vllm

# Start server
python -m vllm.entrypoints.openai.api_server \
  --model ./merged-model \
  --port 8000

Common Pitfalls

  1. Overfitting: Use validation set and early stopping
  2. Catastrophic Forgetting: Use LoRA or adapter methods
  3. Data Quality: Ensure high-quality, diverse training data
  4. Learning Rate: Start with 2e-5 and adjust
  5. Batch Size: Balance memory and training stability

Conclusion

Fine-tuning LLMs has become more accessible with techniques like LoRA and QLoRA. Start with QLoRA for resource-constrained environments, and use full fine-tuning when you have the resources and need maximum performance.

Explore topics:LLMAI
React
Share this post
Cite this article
Plain text

Kiril Urbonas (2024). Fine-tuning Large Language Models: A Practical Guide. DevOpsNess. https://www.devopsness.com/blog/fine-tuning-large-language-models-practical-guide

Markdown

[Fine-tuning Large Language Models: A Practical Guide](https://www.devopsness.com/blog/fine-tuning-large-language-models-practical-guide) (Kiril Urbonas, DevOpsNess, 2024-02-12)

KU

Kiril Urbonas

Founder & Chief Editor

608 articles
View all articles by Kiril Urbonas

You might have missed

Evergreen posts worth revisiting.