ORBiS Tree
KO

ORBiS TREE · ENGLISH DOCUMENT

Concepts →

Transformer

A neural network architecture that processes sequences primarily through attention mechanisms.

한국어 English
#Attention#NLP#Transformer
Korean source Edit source
English updated 8월 29, 2026 Source article updated 8월 28, 2026 1 sources
This English page is a curated translation layer linked to the Korean source article. Community changes are currently made on the Korean source, where the full revision history and anonymous edit trail are preserved.
The Transformer was introduced in the 2017 paper “Attention Is All You Need.” It replaced recurrent sequence processing with attention-centered blocks, making it easier to train models in parallel and to capture relationships between tokens over long ranges.[1]

How it works

Transformer blocks combine attention, feed-forward layers, residual connections, and normalization. Encoder-style models such as BERT focus on representation learning, while decoder-style models such as GPT generate tokens autoregressively. Many systems use variations of these original designs.

Why it matters

The architecture became a core building block of modern language models and later expanded into vision, audio, and multimodal systems. Understanding Transformers helps explain concepts such as Attention, KV Cache, and Context Window.

Related concepts

See also Attention, BERT, GPT, and Large Language Model (LLM).

SOURCES

Sources

  1. Attention Is All You NeedNeurIPS / arXiv
    Open source ↗

KNOWLEDGE LINKS

GPT → Large Language Model (LLM) →

BACKLINKS

Attention ConceptsBERT ModelsChatGPT ProductsGPT ModelsGoogle DeepMind OrganizationsKV Cache ConceptsMixture of Experts (MoE) ConceptsLarge Language Model (LLM) Models

KEEP EXPLORING

Related articles

Latency and ThroughputTwo core serving metrics describing response delay and the amount of work a system completes over time.GPUA highly parallel processor widely used to train and run modern machine-learning models.Context WindowThe amount of tokenized input and generated text a model can consider within one inference context.KV CacheA cache of attention keys and values used to avoid recomputing earlier context during autoregressive generation.InferenceThe process of running a trained model on new input to produce predictions, embeddings, or generated output.Knowledge DistillationA training method in which a smaller student model learns from the outputs or representations of a larger teacher model.
ORBiS Tree English · source-linked translation
Korean source & revision history · Report an issue · Editorial policy