Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation connects a generative model with an external retrieval system. The original RAG research combined parameterized generation with an external document index.^[1] Operational pipelines prepare documents, retrieve relevant passages and supply them as generation context. Embeddings and vector search are common components; retrieval need not be exclusively vector-based.

For a short one-off source, start by providing the original and clear instructions. For a changing collection, retrieval adds source selection and update operations. Fine-tuning changes learned behavior rather than automatically refreshing facts. Methods can be combined if their responsibilities are evaluated separately.

In a fictional manual, product A is indoor-only and product B has different outdoor conditions. An answer about placing A in rain must use A's evidence. Retrieving B and producing fluent advice is a system failure. Retain product identity, document version and source location.

Check authorized source availability, retrieved passages, split conditions, grounded generation and handling of unknown answers. Similarity-search design can affect retrieval behavior.^[2] A small reviewed question set is a starting example, not a required sample size or performance guarantee. Record both stage failures and final outcomes.

This is a hypothetical evaluation. Of 20 questions with known supporting evidence, 16 retrieve it and 12 of those produce grounded correct answers. Retrieval success is 16/20 = 80%; answer success conditional on retrieval is 12/16 = 75%; end-to-end answer success is 12/20 = 60%. Do not treat the final percentage as the performance of both stages. This simplified example does not count missing-evidence cases as success; record unsupported lucky guesses separately in practice.

Inspect indexes, chunking and query wording for the four retrieval failures. For the four failures despite retrieved evidence, inspect context assembly, condition application and generation. The RAG paper ↗ introduces combining retrieval with generation.^[1] Separate retrieval, delivered context and unsupported output to choose an intervention.

If refund policies v1 and v2 are retrieved together, similarity alone does not identify the applicable version. Check effective dates, products and regions. Record missing evidence, conflicting applicability and denied access separately. Do not reveal unauthorized content to resolve an access failure.

Compare source versions with the effective time of stored decisions. A citation link does not prove that the sentence applies the cited conditions correctly.

More material does not guarantee repair of omitted conditions or stale versions. Lost in the Middle ↗ found position-sensitive behavior in the evaluated models and tasks.^[3] This is not a claim that all current models fail to the same extent.

Compare a known-evidence-only condition, actual retrieval results and changed evidence positions. If even the first condition fails, retrieving more passages is insufficient. When combining RAG with learned behavior, evaluate retrieval, answers and version application separately. Include questions whose answer cannot be verified.

Lewis et al. (NeurIPS 2020), sections 2.1 and 2.4, distinguish RAG-Sequence from RAG-Token by their marginalization over latent documents. They keep the document encoder and index fixed while jointly fine-tuning the query encoder and BART generator.^[1] The practical retrieval/training distinction above does not mean RAG cannot be trained. The Wikipedia-based evaluation does not establish present Korean workplace performance.

This is an ORBiS Tree evaluation proposal. Hold questions, model and output settings constant when comparing no retrieval, supplied correct evidence and actual retrieval. Record corpus, retriever, segmentation and query versions plus documents actually supplied; inspect evaluation leakage. Changing retrieval and generation together obscures attribution. Report source versions, question counts by failure type, failures and latency and cost. Operational feasibility and correctness require separate judgments.