SEARCH · ENGLISH EDITION
Find an AI concept.
Search titles, summaries, article text, source-language titles, and known aliases.
Attention
A mechanism that lets a neural network weigh the relevance of different positions when computing a representation.
BERT
Bidirectional Encoder Representations from Transformers, an encoder-based language representation model.
Foundation Model
A broadly trained model that can be adapted to many downstream tasks and applications.
GPT
Generative Pre-trained Transformer, a family of autoregressive language models associated with OpenAI.
GPU
A highly parallel processor widely used to train and run modern machine-learning models.
Google DeepMind
Google's AI research organization, formed from the combination of DeepMind and Google Brain teams.
KV Cache
A cache of attention keys and values used to avoid recomputing earlier context during autoregressive generation.
Large Language Model (LLM)
A general-purpose language model trained on large datasets and usually built at substantial scale.
Mixture of Experts (MoE)
A model architecture that routes each input to a subset of specialized parameter blocks called experts.
Quantization
A technique that represents model weights or activations with lower numerical precision to reduce resource use.
Transformer
A neural network architecture that processes sequences primarily through attention mechanisms.