Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

NeurIPSSpotlight2025

Authors
David Heineman, Valentin Hofmann, Ian Magnusson, Yuling Gu, Noah A. Smith, Hannaneh Hajishirzi, Kyle Lo, Jesse Dodge
Venue
NeurIPS 2025
Track
Spotlight

TL;DR

Developing large language models is expensive and often involves making decisions with small experiments, typically by evaluating on large, multi-task evaluation suites…

Opening excerpt from the authors’ abstract. source

Read the paper

Topics

large language model language model uncertainty evaluation

Related NeurIPS papers

← All NeurIPS 2025 Spotlight papers · Browse the whole archive