ORBiS TREE · ENGLISH DOCUMENT
Concepts →LoRA
Low-Rank Adaptation, a parameter-efficient technique for adapting large neural networks.
한국어
English
This English page is a curated translation layer linked to the Korean source article. Community changes are currently made on the Korean source, where the full revision history and anonymous edit trail are preserved.
LoRA, or Low-Rank Adaptation, is a parameter-efficient fine-tuning method that keeps the original model weights fixed and learns smaller low-rank update matrices for selected layers.
How it works
Instead of updating every parameter in a large model, LoRA represents weight changes with a lower-dimensional decomposition. This can substantially reduce the number of trainable parameters and the memory required during adaptation.
Why it matters
LoRA makes it practical to maintain multiple specialized adaptations of the same base model. It is widely used when full Fine-tuning would be too expensive, although the best target layers and rank settings depend on the task and model.
Related concepts
SOURCES
Sources
-
LoRA: Low-Rank Adaptation of Large Language ModelsMicrosoft / arXivOpen source ↗
KNOWLEDGE LINKS
Continue from here
BACKLINKS