LLM Fine-Tuning – LoRA, QLoRA and Parameter-Efficient Fine-Tuning (PEFT)

The proliferation of open-weight large language models — Llama 3, Mistral, Qwen, Gemma — has created a new core skill for data scientists: fine-tuning a pre-trained LLM for a specific domain or task. Full fine-tuning of a 7 billion parameter model updates all 7 billion weights simultaneously, requiring a gradient and optimizer state for every parameter in memory alongside the activations — a peak memory footprint of 100GB or more for a 7B model with Adam. This is inaccessible to almost all individual practitioners and expensive even for well-resourced teams. Parameter-Efficient Fine-Tuning (PEFT) methods reduce this barrier by orders of magnitude: LoRA and QLoRA enable fine-tuning of 7B models on a single consumer GPU, fine-tuning of 70B models on a single A100, and fine-tuning of 400B+ models on a small cluster — at a fraction of the cost. This guide covers the full PEFT toolkit: LoRA mathematics and implementation, QLoRA for 4-bit quantisation, prompt tuning and prefix tuning as alternatives, instruction dataset preparation, training with the HuggingFace PEFT library and Axolotl, evaluation, and deployment strategies for fine-tuned models.

LoRA — Low-Rank Adaptation Mathematics and Implementation

LoRA is motivated by the observation that the weight updates during full fine-tuning have a low intrinsic rank: despite having millions of parameters, the change in weights ΔW = W_fine_tuned − W_pretrained lives in a low-dimensional subspace of the full parameter space. If the intrinsic dimensionality of the task-relevant weight change is low, then we can represent ΔW accurately as a product of two small matrices: ΔW = B × A, where A ∈ ℝ^{r×k} and B ∈ ℝ^{d×r}, with rank r much smaller than both d and k. During forward pass, the output of a LoRA-augmented layer is: h = W₀x + ΔWx = W₀x + BAx, where W₀ is the frozen pre-trained weight and BA is the trainable low-rank update. A is initialised from a Gaussian distribution; B is initialised to zero so that ΔW = 0 at the start of training and the fine-tuned model starts identically to the pre-trained model. The scaling factor α/r (where α is a fixed scalar hyperparameter) controls the magnitude of the update: setting α = r makes the update scale-invariant to the choice of r.

The linear algebra interpretation is elegant: LoRA constrains the weight update to lie in the column space of B (a d-dimensional subspace of rank r), which is also the row space of A. The rank r controls the expressiveness of the update — higher rank allows more complex weight changes, lower rank restricts updates to simpler transformations. The connection to dimensionality reduction (specifically SVD) is exact: if we were to compute the full weight update ΔW and take its SVD, LoRA approximates the top-r singular value components of that update. In practice, r=8 captures the task-relevant update for most instruction-following and domain adaptation tasks; r=64 is used for complex reasoning or creative tasks. The total trainable parameters for a LoRA adapter on a single attention layer with d=4096, k=4096, r=16 is 2 × 4096 × 16 = 131,072 — compared to 4096 × 4096 = 16,777,216 for the full layer weight, a 128× compression.

The HuggingFace PEFT library implements LoRA with three lines of configuration:

from peft import LoraConfig, get_peft_model, TaskType

config = LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"],
    task_type=TaskType.CAUSAL_LM
)
model = get_peft_model(base_model, config)
model.print_trainable_parameters()
# trainable params: 41,943,040 / 7,241,732,096 (0.58%)

After fine-tuning, the adapter weights are saved separately from the base model (a few hundred MB vs. the full 14GB of a 7B model in bf16). The adapter can be loaded onto any copy of the base model without re-downloading the base, enabling efficient distribution and version control of fine-tuned models. The connection to MLflow model registry is direct: LoRA adapters are small artefacts that fit naturally in the MLflow artefact store, with the base model referenced as a dependency rather than bundled into each run.

QLoRA — 4-Bit Quantisation and Double Quantisation

QLoRA (Dettmers et al., 2023) extends LoRA with two quantisation innovations that halve the memory footprint of the base model. First, NormalFloat4 (NF4) quantisation stores each weight in 4 bits using a data type optimised for normally distributed weights (as most pre-trained model weights approximately are): it divides the normal distribution into 16 equal-probability bins and assigns each bin an integer index 0–15. This is the information-theoretically optimal quantisation for normally distributed values — it minimises quantisation error under the assumption that weights follow a normal distribution, outperforming uniform 4-bit quantisation. Second, double quantisation further reduces memory by quantising the quantisation constants themselves: the scaling factors used to convert 4-bit integers back to bfloat16 values are themselves stored in 8-bit instead of 32-bit, saving an additional ~0.5 GB for a 7B model. During the forward pass, the 4-bit weights are dequantised to bfloat16 for matrix multiplication; the LoRA adapters are always stored in bfloat16 and trained with full precision gradients. The result is that a 7B model that normally requires 14GB in bfloat16 fits in ~5GB in 4-bit NF4, and with the LoRA adapters and optimizer states, the total VRAM requirement is around 10GB — within reach of an RTX 3090 or 4090.

The model compression perspective is illuminating: QLoRA is a form of post-training quantisation (PTQ) applied to the base model before fine-tuning, rather than the quantisation-aware training (QAT) used in production deployment. The base model is quantised once and frozen; all gradient flow occurs through the bfloat16 LoRA adapters. After fine-tuning, the adapters can be merged into the quantised base model or kept separate for efficient serving. For deployment, the merged model can be re-quantised with GPTQ or AWQ (more accurate quantisation algorithms than NF4 for inference) to get the best inference speed and quality. The connection to model deployment is direct: quantised models deployed with llama.cpp, vLLM, or TGI (Text Generation Inference) achieve 2-4× throughput improvement over bfloat16 serving at only a small quality degradation.

Instruction Datasets and Fine-Tuning Workflow

The most important decision in LLM fine-tuning is not the PEFT method or hyperparameters — it is the quality of the instruction dataset. A fine-tuned LLM will faithfully reproduce the patterns in its training data: a dataset of inconsistent, low-quality instructions produces an inconsistent, unreliable model; a dataset of clear, diverse, high-quality instruction-response pairs produces a model that generalises well to new instructions. Dataset curation principles: (1) Diversity — cover the full range of tasks the model will encounter in production; (2) Quality — each response should be the best possible answer to the instruction, written at the level of an expert in the domain; (3) Format consistency — use a single chat template (Alpaca, ChatML, or Llama-3 format) throughout; (4) No leakage — evaluation prompts must not appear in training data. The connection to RLHF and preference data is sequential: instruction fine-tuning is typically the first stage (SFT — supervised fine-tuning), and RLHF or DPO (Direct Preference Optimisation) is the second stage that aligns the model’s outputs with human preferences.

The standard fine-tuning toolkit in 2026: HuggingFace PEFT + TRL (Transformer Reinforcement Learning library) + Axolotl for orchestration, with MLflow for experiment tracking. Training on domain-specific data — medical literature, legal documents, financial reports, code in a specific framework — consistently produces models that outperform general-purpose models (even much larger ones) on the target domain. This domain fine-tuning strategy is the practical realisation of transfer learning for LLMs: the vast world knowledge of the pre-trained base model is retained (because LoRA updates are small), while domain-specific instruction-following and knowledge are added via the adapters. The resulting domain-fine-tuned models are evaluated with domain-specific benchmarks and compared against the base model and against prompt-engineered versions of larger models — a systematic comparison that belongs in the MLflow experiment registry.

✦ SUMMARIZE THIS ARTICLE WITH AI

ChatGPT Perplexity Claude Grok

The Transformer architecture that LoRA adapters are injected into is in our Transformers guide. RLHF and DPO — the alignment stages that follow SFT fine-tuning — are in our RLHF guide. Model compression (quantisation, distillation) for efficient LLM inference after fine-tuning is in our Model Compression guide. Prompt engineering as the no-fine-tuning alternative for LLM task adaptation is in our Prompt Engineering guide. Generative AI system design — RAG, agents, tool use — that fine-tuned models power is in our Generative AI guide. MLflow experiment tracking for managing fine-tuning runs and adapter registry is in our MLflow guide.