How it works
QLoRA (Quantized LoRA) combines two techniques: quantization and LoRA. The base model is loaded in reduced precision (4 bits, using the NF4 format), and fine tuning happens through LoRA matrices in higher precision (16 bits). Only the LoRA matrices are trained; the base model stays frozen and quantized.
This drastically reduces video memory (VRAM) consumption, allowing you to train models with tens of billions of parameters on a single consumer GPU (for example, 24 GB). 4 bit quantization keeps most of the model's quality, and the LoRA matrices recover the precision lost during adaptation.
Practical example
Training a 7 billion parameter model on a 16 GB GPU would be impossible in full precision. With QLoRA, you load the model in 4 bits and train the LoRA matrices. The result is a small adapter that, combined with the quantized base model, reaches quality close to full fine tuning.
Libraries like bitsandbytes and PEFT (Hugging Face) implement QLoRA. The typical code:
from transformers import BitsAndBytesConfig
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")What it's for
QLoRA is used to democratize fine tuning of large models. Small teams and researchers can adapt powerful models without GPU clusters. It is the most used technique for creating specialized models (in code, medicine, law) from open base models, with affordable cost.