ORBiS Tree

SEARCH · ENGLISH EDITION

Find an AI concept.

Search titles, summaries, article text, source-language titles, and known aliases.

Results for “Tokenization”
EN

BERT

Bidirectional Encoder Representations from Transformers, an encoder-based language representation model.

EN

Claude

A family of general-purpose AI models and conversational products developed by Anthropic.

EN

Context Window

The amount of tokenized input and generated text a model can consider within one inference context.

EN

Embedding

A numerical vector representation used to encode semantic or structural properties of data.

EN

Gemini

A family of multimodal AI models and products developed by Google.

EN

Large Language Model (LLM)

A general-purpose language model trained on large datasets and usually built at substantial scale.

EN

Tokenization

The process of converting text or other input into discrete units that a model can process.

EN

Transformer

A neural network architecture that processes sequences primarily through attention mechanisms.