Intermediate

Fine-tuning LLMs

Learn when and how to fine-tune LLMs - from full fine-tuning to parameter-efficient methods like LoRA and QLoRA.

Full Fine-tuning vs PEFT

Approach	Parameters Updated	GPU Memory	Best For
Full Fine-tuning	All (100%)	Very high (2-3x model size)	Maximum quality, large compute budgets
LoRA	~0.1-1%	Low (model size + small overhead)	Most use cases, good quality/cost balance
QLoRA	~0.1-1%	Very low (quantized model)	Consumer GPUs, budget-constrained
Adapters	~1-5%	Low	Multi-task learning, modular approaches

LoRA (Low-Rank Adaptation)

LoRA freezes the pre-trained model weights and injects trainable low-rank decomposition matrices into each Transformer layer. Instead of updating a weight matrix W of size d x d, LoRA learns two smaller matrices A (d x r) and B (r x d) where r is much smaller than d.

Python - Fine-tuning with LoRA using PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer

# Load base model
model_name = "meta-llama/Meta-Llama-3-8B"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Configure LoRA
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,                        # Rank (lower = fewer params, higher = more capacity)
    lora_alpha=32,               # Scaling factor
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # Which layers to adapt
)

# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Output: trainable params: 4,194,304 || all params: 8,030,261,248 || trainable%: 0.0522

# Train
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    args=TrainingArguments(
        output_dir="./lora-output",
        num_train_epochs=3,
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        learning_rate=2e-4,
        bf16=True,
    ),
)
trainer.train()

QLoRA

QLoRA combines LoRA with 4-bit quantization, making it possible to fine-tune a 70B model on a single 48GB GPU:

Python - QLoRA fine-tuning

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

# 4-bit quantization config
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",           # NormalFloat4 quantization
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,       # Nested quantization for extra savings
)

# Load model in 4-bit
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3-8B",
    quantization_config=bnb_config,
    device_map="auto",
)

# Apply LoRA on top of quantized model (same as before)
model = get_peft_model(model, lora_config)

Instruction Tuning

Fine-tune a model to follow instructions by training on instruction-response pairs:

JSON - Training data format (Alpaca-style)

[
  {
    "instruction": "Summarize the following article in 3 bullet points.",
    "input": "Machine learning is a subset of AI that enables...",
    "output": "- ML enables computers to learn from data without explicit programming\n- It includes supervised, unsupervised, and reinforcement learning\n- Applications span healthcare, finance, and technology"
  },
  {
    "instruction": "Write a Python function to calculate factorial.",
    "input": "",
    "output": "def factorial(n):\n    if n <= 1:\n        return 1\n    return n * factorial(n - 1)"
  }
]

Unsloth for Fast Fine-tuning

Python - Unsloth (2x faster fine-tuning)

from unsloth import FastLanguageModel

# Load model with Unsloth optimizations (2x faster, 60% less memory)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B",
    max_seq_length=2048,
    load_in_4bit=True,
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                     "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
)

# Train with standard HuggingFace Trainer
# (Unsloth patches run automatically for speed)

When to Fine-tune vs Prompt vs RAG

Approach	Best When	Cost	Latency
Prompt Engineering	Model already knows how, just needs guidance	Lowest	Higher (long prompts)
RAG	Need current or private knowledge	Medium	Medium (retrieval step)
Fine-tuning	Need new behavior, style, or domain expertise	Highest	Lowest (no long prompts)

✅

Decision framework: Start with prompt engineering. If that's insufficient, try RAG for knowledge-heavy tasks. Only fine-tune when you need to change the model's behavior, style, or teach it a specialized format that prompting can't achieve. Fine-tuning is best for teaching "how to respond" while RAG is best for teaching "what to know."

Evaluation

After fine-tuning, evaluate your model against the base model:

Task-specific metrics: Accuracy, F1, BLEU/ROUGE for your specific task.
Human evaluation: Have humans rate response quality, helpfulness, and accuracy.
A/B testing: Compare fine-tuned vs base model on real user traffic.
Regression testing: Ensure fine-tuning didn't degrade general capabilities.

← Previous Training LLMs Next → Open vs Closed Models

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka

📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp

🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX