SEARCH · ENGLISH EDITION
Find an AI concept.
Search titles, summaries, article text, source-language titles, and known aliases.
Context Window
The amount of tokenized input and generated text a model can consider within one inference context.
GPU
A highly parallel processor widely used to train and run modern machine-learning models.
Inference
The process of running a trained model on new input to produce predictions, embeddings, or generated output.
KV Cache
A cache of attention keys and values used to avoid recomputing earlier context during autoregressive generation.
Knowledge Distillation
A training method in which a smaller student model learns from the outputs or representations of a larger teacher model.
Latency and Throughput
Two core serving metrics describing response delay and the amount of work a system completes over time.
Mixture of Experts (MoE)
A model architecture that routes each input to a subset of specialized parameter blocks called experts.
Quantization
A technique that represents model weights or activations with lower numerical precision to reduce resource use.