HELM

HELM is an evaluation framework developed to make language-model comparisons more systematic across tasks, metrics, and deployment concerns. It emphasizes that a single accuracy number does not capture all relevant behavior.

The framework evaluates models across scenarios and can consider dimensions such as accuracy, calibration, robustness, fairness, efficiency, and other properties depending on the release and configuration.

HELM is useful as an example of multi-dimensional evaluation. It reinforces the need to document prompts, model versions, datasets, and metrics when comparing general-purpose models.

See also MMLU, Safety Evaluation, Model Card, and Benchmark Data Contamination.