90 — LoRA / QLoRA Fine-Tuning

Concept overview

LoRA trains low-rank update matrices for selected model layers while keeping the base weights frozen. This can reduce trainable parameter and optimizer-state requirements compared with full fine-tuning. QLoRA combines low-rank adaptation with a quantized frozen base to reduce training memory further; it does not mean that all model parameters are trained in low-bit form.

Adapters are dependent artifacts. Their meaning depends on the base model revision, target modules, tokenizer, and training configuration. Fine-tuning changes behavior and task fit; it is usually a poor substitute for retrieval when the main requirement is frequently updated factual knowledge. Adapter training still requires governed data and careful held-out evaluation.

Why it matters in production

Proof of concept

A prototype compares prompting and retrieval baselines with a small adapter trained on clean task examples.

Production

Production versions data, base weights, adapter settings, and evaluation results, then validates the exact serving configuration.

Enterprise scale

Enterprise scale handles many adapters, tenant isolation, loading latency, reproducible training, and rollback of both behavior and deployment artifacts.

Architecture components

Training dataset

Contains reviewed task demonstrations with rights, provenance, and leakage controls.

Frozen base model

Provides the pretrained representation under a pinned revision.

Low-rank adapter trainer

Optimizes selected update matrices with recorded rank and target modules.

Evaluation suite

Measures task quality, safety behavior, and regressions beyond training examples.

Adapter registry and server

Stores compatible artifacts and loads or merges them according to supported runtime behavior.

The architecture, mapped

flowchart TD
 B["Pinned frozen base model"] --> T["Adapter training"]
 D["Reviewed task examples"] --> T
 Q["Optional quantized base for QLoRA"] --> T
 T --> A["Versioned low-rank adapter"]
 A --> E["Held-out and regression evaluation"]
 B --> S["Compatible serving bundle"]
 E --> S
 S --> C["Canary release"]

stateDiagram-v2
 [*] --> Training
 Training --> Candidate: Artifact recorded
 Candidate --> Evaluating
 Evaluating --> Rejected: Critical regression
 Evaluating --> Canary: Gates pass
 Canary --> Approved: Live checks pass
 Canary --> RolledBack: Regression detected
 Approved --> [*]
 Rejected --> [*]
 RolledBack --> [*]

Runtime flow

1. 1

   Confirm adaptation need

   Show that prompting, retrieval, or schema controls do not adequately solve the behavior gap.

2. 2

   Prepare governed examples

   Clean demonstrations and split evaluation by source or time to avoid leakage.

3. 3

   Configure the adapter

   Choose target modules, rank, precision, and training settings within the hardware budget.

4. 4

   Train reproducibly

   Record base revision, tokenizer, data lineage, and checkpoints.

5. 5

   Evaluate the served artifact

   Test the adapter with the exact quantization and runtime intended for deployment.

6. 6

   Promote gradually

   Canary the model-adapter pair and retain the previous approved pair for rollback.

Adapting a correspondence extraction style

Document processing · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

A business-services team needs consistent extraction of a specialized correspondence schema.

The problem

Prompt examples improve results but remain inconsistent on recurring domain phrasing.

Agent requirement

Learn the extraction behavior while keeping changing reference facts in retrieval.

Architecture

A LoRA adapter trains on reviewed examples and is evaluated against unseen source organizations.

Runtime

• The training pipeline records base and dataset revisions.
• The served adapter is tested for exact schema validity and field accuracy before a small rollout.

Failure scenario

If the adapter improves common fields but harms rare exception fields, promotion stops until the data or training configuration is corrected.

Operational impact

Measure field-level accuracy, invalid-output rate, and total training plus serving cost; fewer trainable parameters do not remove evaluation obligations.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Using fine-tuning to refresh volatile facts

Use retrieval for changing knowledge.

Evaluating only training loss

Measure held-out task and regression performance.

Assuming adapter portability

Validate base, target modules, precision, and runtime compatibility.

When to choose this pattern

Choose it when

• Use for stable task behavior and specialized output patterns.
• Use when clean demonstrations are available.
• Use when full fine-tuning exceeds the justified resource budget.

Avoid it when

• Avoid adaptation for facts that change frequently.
• Do not train on unreviewed sensitive feedback automatically.
• Avoid deployment without exact serving-bundle evaluation.

Architecture checklist

• Establish prompting and retrieval baselines.
• Record dataset rights and provenance.
• Pin base model and tokenizer.
• Version rank and target modules.
• Evaluate the final serving precision.
• Retain a complete rollback bundle.

Related concepts

• 79 — Embedding Model Fine-Tuning
• 82 — Small Language Models (SLMs) Deployment
• 84 — Model Quantization (AWQ/GGUF)
• 93 — Model Versioning and A/B Testing
• 95 — Distributed Memory Architectures
• 100 — Asynchronous Fine-Tuning Pipelines

Primary references

• LoRA: Low-Rank Adaptation of Large Language Models
• QLoRA: Efficient Finetuning of Quantized LLMs
• Hugging Face PEFT quantization guide