SEARCH · ENGLISH EDITION
Find an AI concept.
Search titles, summaries, article text, source-language titles, and known aliases.
EN
Attention
A mechanism that lets a neural network weigh the relevance of different positions when computing a representation.
EN
Context Window
The amount of tokenized input and generated text a model can consider within one inference context.
EN
Inference
The process of running a trained model on new input to produce predictions, embeddings, or generated output.
EN
KV Cache
A cache of attention keys and values used to avoid recomputing earlier context during autoregressive generation.
EN
Latency and Throughput
Two core serving metrics describing response delay and the amount of work a system completes over time.
EN
Transformer
A neural network architecture that processes sequences primarily through attention mechanisms.