ORBiS TREE · ENGLISH DOCUMENT

Large Language Model (LLM)

A general-purpose language model trained on large datasets and usually built at substantial scale.

한국어 English

English updated 10월 12, 2026 Source article updated 10월 12, 2026 3 sources

This English page is a curated translation layer linked to the Korean source article. Community changes are currently made on the Korean source, where the full revision history and anonymous edit trail are preserved.

Overview

A large language model (LLM) is a broad category of neural language models trained on extensive text and many parameters to learn statistical and structural patterns in language. Many use Transformer architectures, but LLM is not the name of one architecture.^[1]

How training works

Pretraining learns general language patterns. Autoregressive GPT-style models predict the next token from preceding tokens. Instruction, preference or domain-specific training may follow. Data, architecture and additional training can produce different behavior within the same broad category.

Prompting and in-context learning

GPT-3 research systematically evaluated zero-, one- and few-shot tasks using instructions or demonstrations in the prompt without updating model weights.^[2] Such in-context adaptation lets a user describe a task through input rather than carrying out a separate training run.

Useful tasks and verification limits

LLMs can support summarization, classification, question answering, drafting and coding. Generated text is not inherently a database lookup. Scale does not establish accuracy, freshness or safety; consequential factual claims need evidence and independent checking.

Related concepts

GPT is a model family. ChatGPT is a service through which people can use models conversationally. Distinguish the broad model category, a particular model family and an actual product.

Worked example: classification versus drafting

This is an ORBiS Tree teaching scenario. Classifying 'My delivery is late' as shipping differs from drafting an apology and checking procedure. Specify category rules for the former and response requirements for the latter. Describing tasks through instructions and examples connects to in-context learning.^[2]

Separate the model, service and tools

LLM: the model category that computes language representations or generated text.
Service: a product can combine models with a conversation interface, file handling or search. Verify actual features for the specific service and date.
Retrieval tool: finds external material and supplies it to the model; finding information differs from generating wording.
Agent: a system that selects actions and uses tools toward a goal. Permissions and processing steps can change outcomes even with the same model.

When another tool fits the operation

Use a calculator or database for rule-defined totals and let the LLM explain the result. Compare rule-based processing for fixed decisions. Current product descriptions or internal documents call for external evidence retrieval.^[3] The steps of a workflow can use different mechanisms.

Building evaluation questions

A fictional customer-support tool can be tested on normal questions, similar product names, missing answers and conflicting sources. Record wording quality, factual accuracy, format compliance and latency separately. A public benchmark does not establish identical benefits on the actual documents and questions.

Why scale alone is insufficient

Instructions, retrieved material, output constraints, postprocessing and operating conditions also affect service behavior. Compare calls and review work per completed task, alongside per-call cost. Treat undisclosed properties as unverified comparison inputs rather than assumed strengths.

Probability example: likely text versus factual correctness

In a fictional autoregressive calculation, the first token has conditional probability 0.5 and the second, given that first token, has probability 0.25. Their sequence probability is 0.5×0.25 = 0.125. Mean negative natural-log probability per token is -(ln(0.5) + ln(0.25))/2. Defining perplexity as its exponential gives √8, approximately 2.83. This is not a measured model score or service accuracy.

Likely next tokens do not establish true statements. Perplexities are not directly comparable without fixed tokenizer, data distribution, length and evaluation rules. Distinguish token splitting from factual evidence. Fluent wording is not an automatically verified answer.

Separate prompt demonstrations from weight training

The GPT-3 study ↗ evaluates tasks using prompt examples without additional weight training.^[2] Fictional mappings such as 'change delivery address → shipping' and 'refund request → returns' describe a task in that input. Define label rules and exceptions for negation, such as 'I am not requesting a refund', rather than classifying by keyword presence alone.

Fix the model and question set when comparing prompts with and without examples. Do not reuse questions employed to select examples as independent final evaluation. Changing only the prompt differs from weight-changing training. Service storage and retrieval is another function; writing information in a conversation alone does not establish permanent learning.

Worked example: format 90%, factuality 85%, joint success 75%

Among twenty fictional answers, eighteen have valid JSON format, seventeen are factually correct and fifteen satisfy both. Format compliance is 90%, factuality 85% and joint success 75%. The three format-only and two fact-only successes reveal different failure types.

Define required fields, data types, units, evidence and permitted actions for the actual task. Passing JSON parsing does not authorize saving or tool execution. Join outcomes by question ID and record other quality measures alongside workplace completion. These numbers are not product benchmarks or universal acceptance thresholds.

Worked example: separate calculation, retrieval and explanation

Three fictional amounts, 12,000, 7,500 and 4,500 won, total 24,000 won by a fixed rule. A calculator or verified code can establish the number while a model explains which items were included. Current rules require retrieving the applicable version and checking its scope and exceptions. Calculation and factual retrieval serve different purposes.

The original RAG study ↗ evaluates combining retrieved material with generation.^[3] When evidence is missing or conflicts, retain uncertainty and identify needed material instead of filling the gap with invented facts. Connect source and date to the actual retrieval/tool configuration to distinguish language capability from end-to-end service success.

SOURCES

Sources

On the Opportunities and Risks of Foundation ModelsStanford CRFM
Open source ↗
Language Models are Few-Shot LearnersOpenAI
Open source ↗
Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksLewis et al. / NeurIPS
Open source ↗

KNOWLEDGE LINKS

Continue from here

AI Agent → AI Memory → ChatGPT → Data Provenance → Fine-tuning → GPT → HELM → Hallucination in AI → Retrieval-Augmented Generation (RAG) → System Card → Tokenization → Transformer →