ORBiS TREE · ENGLISH DOCUMENT
Research →Datasheets for Datasets
A documentation framework for describing how datasets were created, composed, maintained, and intended to be used.
한국어
English
This English page is a curated translation layer linked to the Korean source article. Community changes are currently made on the Korean source, where the full revision history and anonymous edit trail are preserved.
Overview
Datasheets for Datasets is a documentation approach for recording a dataset's motivation, composition, collection, preprocessing, distribution, recommended uses and maintenance. Gebru and colleagues adapted the idea of engineering component datasheets into questions that dataset creators can answer.[1]
Why context matters
A name and download URL do not explain represented populations and periods, collection decisions, exclusions or unsuitable uses. Missing context makes reproduction and interpretation of model results harder. Documentation helps readers assess fitness for a purpose; it does not make every dataset risk-free or grant legal permission.
Example: documenting customer inquiries
This ORBiS Tree example is hypothetical. For delivery and return classification, record the purpose, channels and dates, excluded inquiries, labeling rules and disagreements, privacy handling, deduplication and transformations. Identify absent products, languages and recent policies as limits. Include a maintainer, version and change history. These are examples, not a fixed number of required questions or a certification standard. Unknown fields should remain unknown.
Related reporting documents
A model card documents model purposes, evaluation conditions and limitations.[2] A system card describes a system's components and interactions. Data provenance tracks origins and transformations. A datasheet focuses on the dataset itself. These documents can be connected, but one does not substitute for the others.
Use a datasheet in evaluation
Check whether evaluation data represents actual work conditions. Interpret missing periods, rare categories and changes in collection separately. Publication dates and composition can inform contamination investigations. In scientific workflows, dataset context helps identify the conditions under which an experiment's result was obtained.
Worked example: a datasheet with inspectable counts
This is a fictional classification dataset. Suppose 1,200 collected rows become 1,000 after removing 200 duplicates. Counts of 600 delivery, 300 return and 100 usage questions give 60%, 30% and 10%. Record counts before and after cleaning, and define whether duplication means the same customer, inquiry or text. One thousand rows do not automatically represent one thousand independent inquiries.
The Datasheets paper ↗ motivates questions spanning creation, collection, use and maintenance.[1] This example records dates, channels, inclusion and exclusion conditions, labeling rules, deduplication, owner and version. Unchecked usage permission remains unknown; it should not be relabeled as either granted or absent.
Why an 800/200 split does not guarantee valid evaluation
Splitting the fictional 1,000 rows into 800 training and 200 evaluation rows may still produce an easy test if variants of an inquiry or a customer's conversation appear in both. Define the independent unit and group by customer, inquiry or period as appropriate to the actual task. Inspect class and channel coverage as well as the overall split.
Evaluation contamination also depends on publication timing and access during model development. This grouping example does not eliminate every form of contamination. As with separating experimental conditions in scientific work, choose evaluation units for the intended generalization claim.
A dataset revision changes comparison conditions
Suppose v1 labels cancellation as return, while v2 creates a separate label. A score change can mix model improvement with a changed answer key. Record labeling criteria, modified and excluded rows, and file versions. Compare models on the same data and criteria, or report the effect of the revision separately.
Connect originals and transformations through data lineage, and deployed-model effects to model evaluation scope. The Model Cards paper ↗ provides a distinct reporting approach for model use and evaluation.[2] Dataset description and model-performance reporting are different documents.
Turn a review into concrete questions
- Can sources, collection periods and transformations reproduce the result?
- Are missing groups, languages, recent policies and rare labels documented?
- Are privacy processing and usage conditions supported by evidence or still unknown?
- Are update ownership, change history and reproduction of older versions recorded?
- Does the recommended purpose match the actual model and workflow?
Documentation alone does not guarantee performance or usage rights. Link system composition and evaluation under actual conditions to track where the data was used.
SOURCES
Sources
-
Datasheets for DatasetsMicrosoft Research / arXivOpen source ↗
-
Model Cards for Model ReportingMitchell et al. / FAT*Open source ↗
KNOWLEDGE LINKS
Continue from here
BACKLINKS