Foundation Model

The Stanford CRFM report introduced the term foundation model in 2021 for models trained on broad data at scale and adapted to multiple downstream tasks.^[1] The term also appears in Korean as a reusable base model.

A foundation model is not the name of one neural architecture, such as Transformer, or one company's product. Its defining role is to provide a common pretrained base that can be adapted through prompts, fine-tuning or additional components.^[1]

Sharing one base can accelerate downstream applications. Errors, bias and safety weaknesses can also propagate into multiple dependent systems. The original report discusses this concentration under the concept of homogenization and examines opportunities alongside risks.^[1]

Inspect training data and objectives, input/output modalities, adaptation methods and disclosed evaluations and limitations, alongside parameter count. Applications using the same base can still differ in performance and risk because their deployment arrangements differ.

LLMs are an important example within foundation-model discussions, but the terms have different scope. LLM emphasizes language modeling; foundation model emphasizes reuse and ecosystem effects across downstream applications. The latter can also include vision and robotics.

The Stanford CRFM report ↗ describes a reusable base trained on broad data and adapted downstream.^[1] Transformer names an architecture, LLM a language-model category and foundation model an application role. One model can fit all three descriptions without making the terms synonymous.

Distinguish weights from an API or conversation service. BERT-style input judgment and GPT-style sequential generation have different output goals; some bases cover multiple input modalities. A name does not establish supported inputs, external retrieval, tool permissions or weight availability. Separate verified specifications from unknown properties.

Suppose connecting and initially validating a common base takes 24 hours, and adapting each of three applications takes four hours. Sharing setup gives 24 + 3×4 = 36 hours. Repeating that setup separately gives 3×(24 + 4) = 84 hours. The 48-hour difference represents assumed duplicate work, not actual development or pretraining time or guaranteed savings.

A shared base still requires integration, maintenance, version changes and regression checks. Its errors can spread across applications, so assess common improvements separately from application quality. Adaptation training, retrieval and instructions change different parts and can be combined for a defined purpose.

Reusing a fictional base for summarization, classification and document questions involves different answer keys and failure criteria. Check omitted source conditions in summaries, important-class recall in classification, and evidence availability or conflict in questions. One task's score cannot be transferred to another.

Datasheets ↗ proposes documenting data motivation, composition, collection and uses.^[2] Record languages, collection dates, task distributions and overlap in training/evaluation material. Keep undisclosed properties unknown. Link provenance and evaluation overlap to compare broad-training claims with the actual verified data scope.

Model Cards ↗ proposes reporting intended uses, performance and limits under relevant conditions.^[3] Connect model records to the deployed configuration, distinguishing model, tokenizer, retrieved material, instructions and tool versions. A new base requires checking whether prior application results still apply.

A small fictional document-question pilot can separate questions with evidence, missing evidence and conflicting sources. Define expected answers, stopping conditions and failure records. Check that Korean/English wording preserves the answer key. This example is not a standard prescribing test counts or release thresholds. A reusable base's value depends on evaluation within the actual application.