ORBiS Tree
KO

ORBiS TREE · ENGLISH DOCUMENT

Concepts →

Quantization

A technique that represents model weights or activations with lower numerical precision to reduce resource use.

한국어 English
#Inference#LLM#Model Compression#Quantization
Korean source Edit source
English updated 8월 29, 2026 Source article updated 8월 29, 2026 1 sources
This English page is a curated translation layer linked to the Korean source article. Community changes are currently made on the Korean source, where the full revision history and anonymous edit trail are preserved.
Quantization reduces the precision used to store or compute model values, for example moving from higher-precision floating point to lower-bit integer or floating-point formats.

How it works

Lower precision can reduce memory footprint, bandwidth, and inference cost. Methods differ in whether they quantize weights, activations, or both, and whether quantization is applied after training or incorporated during training.

Why it matters

Quantization is important for deploying large models on constrained hardware and for improving throughput, but aggressive compression can reduce accuracy or create task-specific regressions that must be measured.

Related concepts

See also Inference, GPU, LoRA, and Latency and Throughput.

SOURCES

Sources

  1. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained TransformersFrantar et al. / arXiv
    Open source ↗

KNOWLEDGE LINKS

GPT → Large Language Model (LLM) →

BACKLINKS

Knowledge Distillation Concepts

KEEP EXPLORING

Related articles

Latency and ThroughputTwo core serving metrics describing response delay and the amount of work a system completes over time.GPUA highly parallel processor widely used to train and run modern machine-learning models.Context WindowThe amount of tokenized input and generated text a model can consider within one inference context.KV CacheA cache of attention keys and values used to avoid recomputing earlier context during autoregressive generation.InferenceThe process of running a trained model on new input to produce predictions, embeddings, or generated output.Knowledge DistillationA training method in which a smaller student model learns from the outputs or representations of a larger teacher model.
ORBiS Tree English · source-linked translation
Korean source & revision history · Report an issue · Editorial policy