Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case. Includes LoRA, QLoRA, and full fine-tuning techniques.

Key takeaways

Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case.
Includes LoRA, QLoRA, and full fine-tuning techniques.

Fine-tuning Large Language Models: A Practical Guide

Fine-tuning allows you to adapt pre-trained language models to your specific domain or task. This guide covers practical techniques for fine-tuning modern LLMs.

Why Fine-tune?#

Domain Adaptation: Make models understand your specific domain
Task Specialization: Improve performance on specific tasks
Cost Efficiency: Smaller fine-tuned models can outperform larger base models
Data Privacy: Keep sensitive data on-premises

Fine-tuning Methods #

1. Full Fine-tuning #

Updates all model parameters. Most effective but resource-intensive.

python.python

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset

model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Load your dataset
dataset = load_dataset("your-dataset")

# Tokenize
def tokenize_function(examples):
    return tokenizer(examples["text"], truncation=True, max_length=512)

tokenized_dataset = dataset.map(tokenize_function, batched=True)

# Training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-5,
    fp16=True,
    logging_steps=10,
    save_steps=500,
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
)

trainer.train()

2. LoRA (Low-Rank Adaptation)#

Efficient fine-tuning that only updates a small number of parameters.

python.python

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")

# LoRA configuration
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,  # Rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
)

# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # See how many parameters are trainable

# Train as normal
trainer = Trainer(model=model, ...)
trainer.train()

3. QLoRA (Quantized LoRA)#

Combines quantization with LoRA for even more efficiency.

python.python

from transformers import BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

# 4-bit quantization
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=quantization_config,
    device_map="auto"
)

# Apply LoRA
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)

Hardware Requirements #

Model Size	Full Fine-tuning	LoRA	QLoRA
7B	4x A100 (80GB)	1x A100 (40GB)	1x RTX 3090 (24GB)
13B	8x A100 (80GB)	2x A100 (40GB)	1x A100 (40GB)
70B	16x A100 (80GB)	4x A100 (40GB)	2x A100 (40GB)

Data Preparation #

Format Your Data #

python.python

# Instruction-following format
data = [
    {
        "instruction": "Explain quantum computing",
        "input": "",
        "output": "Quantum computing uses quantum mechanical phenomena..."
    },
    {
        "instruction": "Translate to French",
        "input": "Hello, how are you?",
        "output": "Bonjour, comment allez-vous?"
    }
]

# Convert to training format
def format_prompt(example):
    if example["input"]:
        return f"### Instruction:\n{example['instruction']}\n\n### Input:\n{example['input']}\n\n### Response:\n{example['output']}"
    return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"

Training Best Practices #

1. Learning Rate Scheduling #

python.python

training_args = TrainingArguments(
    learning_rate=2e-5,
    lr_scheduler_type="cosine",
    warmup_steps=100,
    warmup_ratio=0.1,
)

2. Gradient Checkpointing #

python.python

training_args = TrainingArguments(
    gradient_checkpointing=True,  # Saves memory
)

3. Mixed Precision Training #

python.python

training_args = TrainingArguments(
    fp16=True,  # or bf16=True for newer GPUs
)

Evaluation #

python.python

from transformers import pipeline

# Load fine-tuned model
model = AutoModelForCausalLM.from_pretrained("./results/checkpoint-1000")
tokenizer = AutoTokenizer.from_pretrained("./results/checkpoint-1000")

# Create pipeline
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

# Test
prompt = "Explain Kubernetes in simple terms:"
result = generator(prompt, max_length=200, temperature=0.7)
print(result[0]["generated_text"])

Deployment #

Export for Production #

python.python

# Merge LoRA weights
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained("./merged-model")

Serve with vLLM #

bash.bash

# Install vLLM
pip install vllm

# Start server
python -m vllm.entrypoints.openai.api_server \
  --model ./merged-model \
  --port 8000

Common Pitfalls #

Overfitting: Use validation set and early stopping
Catastrophic Forgetting: Use LoRA or adapter methods
Data Quality: Ensure high-quality, diverse training data
Learning Rate: Start with 2e-5 and adjust
Batch Size: Balance memory and training stability

Fine-tuning LLMs has become more accessible with techniques like LoRA and QLoRA. Start with QLoRA for resource-constrained environments, and use full fine-tuning when you have the resources and need maximum performance.

Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case. Includes LoRA, QLoRA, and full fine-tuning techniques.

Key takeaways

Learn how to fine-tune LLMs like Llama 2, Mistral, and GPT models for your specific use case.
Includes LoRA, QLoRA, and full fine-tuning techniques.

Fine-tuning Large Language Models: A Practical Guide

Fine-tuning allows you to adapt pre-trained language models to your specific domain or task. This guide covers practical techniques for fine-tuning modern LLMs.

Why Fine-tune?#

Domain Adaptation: Make models understand your specific domain
Task Specialization: Improve performance on specific tasks
Cost Efficiency: Smaller fine-tuned models can outperform larger base models
Data Privacy: Keep sensitive data on-premises

Fine-tuning Methods #

1. Full Fine-tuning #

Updates all model parameters. Most effective but resource-intensive.

python.python

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset

model_name = "meta-llama/Llama-2-7b-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Load your dataset
dataset = load_dataset("your-dataset")

# Tokenize
def tokenize_function(examples):
    return tokenizer(examples["text"], truncation=True, max_length=512)

tokenized_dataset = dataset.map(tokenize_function, batched=True)

# Training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-5,
    fp16=True,
    logging_steps=10,
    save_steps=500,
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
)

trainer.train()

2. LoRA (Low-Rank Adaptation)#

Efficient fine-tuning that only updates a small number of parameters.

python.python

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")

# LoRA configuration
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,  # Rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
)

# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # See how many parameters are trainable

# Train as normal
trainer = Trainer(model=model, ...)
trainer.train()

3. QLoRA (Quantized LoRA)#

Combines quantization with LoRA for even more efficiency.

python.python

from transformers import BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

# 4-bit quantization
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=quantization_config,
    device_map="auto"
)

# Apply LoRA
lora_config = LoraConfig(...)
model = get_peft_model(model, lora_config)

Hardware Requirements #

Model Size	Full Fine-tuning	LoRA	QLoRA
7B	4x A100 (80GB)	1x A100 (40GB)	1x RTX 3090 (24GB)
13B	8x A100 (80GB)	2x A100 (40GB)	1x A100 (40GB)
70B	16x A100 (80GB)	4x A100 (40GB)	2x A100 (40GB)

Data Preparation #

Format Your Data #

python.python

# Instruction-following format
data = [
    {
        "instruction": "Explain quantum computing",
        "input": "",
        "output": "Quantum computing uses quantum mechanical phenomena..."
    },
    {
        "instruction": "Translate to French",
        "input": "Hello, how are you?",
        "output": "Bonjour, comment allez-vous?"
    }
]

# Convert to training format
def format_prompt(example):
    if example["input"]:
        return f"### Instruction:\n{example['instruction']}\n\n### Input:\n{example['input']}\n\n### Response:\n{example['output']}"
    return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"

Training Best Practices #

1. Learning Rate Scheduling #

python.python

training_args = TrainingArguments(
    learning_rate=2e-5,
    lr_scheduler_type="cosine",
    warmup_steps=100,
    warmup_ratio=0.1,
)

2. Gradient Checkpointing #

python.python

training_args = TrainingArguments(
    gradient_checkpointing=True,  # Saves memory
)

3. Mixed Precision Training #

python.python

training_args = TrainingArguments(
    fp16=True,  # or bf16=True for newer GPUs
)

Evaluation #

python.python

from transformers import pipeline

# Load fine-tuned model
model = AutoModelForCausalLM.from_pretrained("./results/checkpoint-1000")
tokenizer = AutoTokenizer.from_pretrained("./results/checkpoint-1000")

# Create pipeline
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

# Test
prompt = "Explain Kubernetes in simple terms:"
result = generator(prompt, max_length=200, temperature=0.7)
print(result[0]["generated_text"])

Deployment #

Export for Production #

python.python

# Merge LoRA weights
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = lora_model.merge_and_unload()
merged_model.save_pretrained("./merged-model")

Serve with vLLM #

bash.bash

# Install vLLM
pip install vllm

# Start server
python -m vllm.entrypoints.openai.api_server \
  --model ./merged-model \
  --port 8000

Common Pitfalls #

Overfitting: Use validation set and early stopping
Catastrophic Forgetting: Use LoRA or adapter methods
Data Quality: Ensure high-quality, diverse training data
Learning Rate: Start with 2e-5 and adjust
Batch Size: Balance memory and training stability

Key takeaways

Fine-tuning Large Language Models: A Practical Guide

Stay Updated

Infrastructure as Code: Terraform vs Pulumi vs Ansible

Practical Guide: Infrastructure Drift Detection Workflow

About Kiril Urbonas

More from AI

AI Pair Programming: Tips to Get Better Code

Best AI Coding Assistants in 2026 — Compared by Use Case

GitHub Copilot Alternatives Worth Trying in 2026

You might have missed

A Pragmatic Multi-Region Strategy for Small Teams

What We Learned Running Weekly Game Days on Our CI/CD Pipeline

GitOps with Argo CD: Best Practices for 2026

Key takeaways

Fine-tuning Large Language Models: A Practical Guide

Stay Updated

Infrastructure as Code: Terraform vs Pulumi vs Ansible

Practical Guide: Infrastructure Drift Detection Workflow

About Kiril Urbonas

More from AI

AI Pair Programming: Tips to Get Better Code

Best AI Coding Assistants in 2026 — Compared by Use Case

GitHub Copilot Alternatives Worth Trying in 2026

You might have missed

A Pragmatic Multi-Region Strategy for Small Teams

What We Learned Running Weekly Game Days on Our CI/CD Pipeline

GitOps with Argo CD: Best Practices for 2026