Fine-tuning

Fine-tuning continues training from a pretrained model for a task, domain or interaction pattern. ULMFiT studied language-model adaptation for text classification.^[1] Full updates and parameter-efficient approaches change different parameter sets; LoRA uses trainable low-rank additions while freezing the large original weights.^[2]

Repeated classification criteria, output formats and domain expressions can motivate training. First test clear prompting and examples. Frequently changing facts still need refreshed evidence or structured lookup; RAG has a different role.^[3] Fine-tuning does not automatically learn this week's deadline.

A fictional business classifies inquiries as delivery, return or product usage. Define labels consistently, then hold out evaluation examples. Variants from one customer or inquiry can leak across a naive split. Inspect grouping, near-duplicates and labels before training.

Reevaluate the target task, existing core capabilities and safety conditions. Include preparation, training, serving, retries and review in cost comparisons. A smaller adapter does not make incorrect labels harmless. Save versions, settings, failure types and a working rollback configuration.

A fictional held-out set contains 90 delivery and 10 return inquiries. Model A correctly classifies 80 delivery and six return cases: 86% overall. Fine-tuned B gets 88 delivery and two return cases correct: 90% overall. The total improves while return recall falls from 60% to 20%. Recall among actual return cases differs from precision among predicted returns. These are illustrative results, not model performance.

Record per-label successes, failures and business consequences before choosing release criteria. Overall accuracy alone cannot establish an improvement when a rare class matters. Connect evaluation composition, model reporting and evaluation under intended conditions.

The ULMFiT paper ↗ describes adapting a pretrained language model to tasks. The LoRA paper ↗ describes parameter-efficient adaptation with frozen base weights.^[1][2] In either case, separate the intervention from evaluation conditions.

Save a baseline with fixed instructions, examples and sources. Compare before and after on the same evaluation set, recording formatting, label errors, unsupported answers, time and total cost. Do not finish evaluation using only questions repeatedly inspected during development. Check evaluation leakage and training-data versions.

For an adapter, retain base identifier and version, adapter version, tokenizer, instruction format and application order as one executable configuration. A changed base does not guarantee unchanged adapter quality. Preserve training settings, evaluation results and an earlier configuration that can actually be restored.

When failures increase, compare changed labels, data, models and lookup policies. Update retrieved sources for changing deadlines and examine training examples for repeated classification behavior. Fine-tuning is not a universal data-refresh mechanism.

Dettmers et al., QLoRA section 3, describe storing the frozen base model mainly in four bits, then dequantizing weights to BFloat16 for computation while training LoRA adapters.^[4] This does not make every activation, gradient or matrix operation four-bit. The method combines quantization with adapters; it does not define all fine-tuning.

This is an ORBiS Tree experiment proposal. Fix data, splits, base model and instruction format. Report searched settings and resource budgets for full fine-tuning, LoRA and QLoRA separately. Leaving only one method at defaults confounds training method with tuning effort. Across multiple initializations, report target-task, language and rare-label results with peak memory, time and total cost. A single score difference does not establish general superiority. Preserve failures, constraints and the recovery configuration in data records and model evaluations.