Measuring risk preferences in large language models using adaptive binary search

Abstract

Large language models (LLMs) increasingly support decision-making in high-stakes domains, where a model's risk preference, its tendency to favor lower-risk or higher-risk options under uncertainty, can shape the decisions it supports. Prior studies measuring these preferences in LLMs have produced mixed results. Several used fixed designs, such as choice lists, which locate the point where a preference changes only as finely as their preset values allow. Adaptive procedures, which adjust each query to earlier responses, concentrate measurement near that point and are established for eliciting human risk preferences. We present such a procedure for LLMs: it poses a choice between a lower-risk and a higher-risk option, varies the expected value of the higher-risk option across steps, locates by binary search the model's tipping point, the point at which its outputs shift to favor the higher-risk option, and reports it as an index of risk preference under one probability structure. We applied the procedure to eight LLMs from four providers (OpenAI, San Francisco, CA, USA; Anthropic, San Francisco, CA, USA; Google, Mountain View, CA, USA; xAI, Palo Alto, CA, USA), using emergency and disaster management vignettes, and tested whether presenting each option's expected value explicitly, rather than leaving it to be inferred, changed the measured preference. Median tipping points for six of the eight models fell near the point of equal expected value under both presentations. Claude Sonnet 4.6's tipping point was higher when the expected value was left to be inferred, with a shift whose bootstrap interval excludes 0; Claude Haiku 4.5's was higher as a point estimate, but its shift interval includes 0. In individual queries, deviations from expected-value maximization concentrated in four models: those of the two Anthropic models all toward the lower-risk option and those of the two Google models almost all toward the higher-risk option. Over the range queried in both presentations, presenting the expected value explicitly reduced these deviations in all four models. Because the two presentations convey the same expected values, the reduction indicates that these models did not respond to them as equivalent. The procedure applies to any model accessible via a text interface and is a reusable tool for comparing and screening models for risk-related decision support.

The authors' abstract, as published at the source. Frontiers in Artificial Intelligence, 2026 · DOI ↗