The ChatGPT source cue shapes answer acceptance in a reasoning task

Abstract

Introduction As generative AI systems increasingly provide answers in everyday and academic contexts, understanding how source attribution influences evaluative judgment has become critical. Methods In a within-subjects design, 103 participants evaluated 30 pre-generated reflective-correct, intuitive-incorrect, and arithmetic control-correct answers labeled ChatGPT or computer. Assignment of two parallel batteries to source labels and trial presentation order were randomized, and displays were visually matched. An independent unlabeled pretest with 35 participants supported battery comparability. Results Participant-level analysis showed greater overall acceptance under ChatGPT (68.0% versus 63.4%, p = 0.002). A trial-level logistic mixed model with crossed participant and parallel-problem-pair random intercepts showed a source-by-category interaction ( p < 0.001): acceptance was higher under ChatGPT for arithmetic controls (OR = 6.08, 95% CI [2.70, 13.69]), while source contrasts within the two CRT categories were not significant. Judgment accuracy is reported descriptively. Signal-detection analyses of CRT answers detected no significant source difference in sensitivity ( p = 0.930) or criterion ( p = 0.082). Evaluation times were longer under ChatGPT in the raw-time analysis ( p = 0.034), but the source difference was not significant after log transformation ( p = 0.180). Exploratory correlations detected no significant associations between source-related differences and trust, AI attitudes, or critical-thinking disposition. Discussion These findings distinguish overall source-related endorsement, particularly for arithmetic controls, from successful discrimination between correct and incorrect CRT answers.

The authors' abstract, as published at the source. Frontiers in Psychology, 2026 · DOI ↗