{"id":"df64ec6f-7c92-4773-b630-567f7df3c2e2","arxiv_id":"2506.00582","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LLM confidence is less sensitive to task difficulty than human confidence and bends to persona stereotypes, and separating confidence prompts from answer prompts (AFCE) improves calibration on hard tasks.","lead":"This paper shows that large language models express confidence differently from humans: their confidence is less tied to task difficulty and shifts with persona and demographic cues even when accuracy stays the same. The authors propose a simple two-step prompting method, Answer-Free Confidence Estimation, that separates confidence reports from answer generation and reduces overconfidence on hard questions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that AFCE is more human-like is not established: the human regression line comes from different Moore–Healy questions, and the slope effect appears only for GPT-4o, so the central comparison is confounded and overstated.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the human comparison is confounded by task mismatch. This is the most serious threat to the central claim because the 'more human-like sensitivity' conclusion depends on the slope comparison in Figure 3, and the paper explicitly acknowledges it cannot use the original Moore–Healy questions. I considered whether the ECE comparison is also suspect due to AFCE's coarse 0-10 grouped confidence versus per-question baselines, but the comparison against the Quiz-Like baseline, which is also block-level and uses the same response scale, suggests the overconfidence-reduction result is not purely an artifact of grouping. The unresolved issue is specifically the human slope overlay. I therefore agree with the reader's conditional verdict: the empirical findings are plausible and the AFCE method is useful, but the human-like claim requires either matched human data or at least a range-controlled reanalysis. No change to the conditional verdict is needed.","tokens_in":21798,"tokens_out":9135,"duration_ms":98264,"concrete_test":"Restrict the confidence–accuracy regression in Figure 3 to the subtasks whose accuracy falls within the range observed in Moore and Healy (2008), then re-fit the vanilla and AFCE slopes for GPT-4o. If AFCE's slope no longer differs from vanilla's, or no longer approaches the human slope, the 'more human-like sensitivity' claim is an artifact of comparing across non-overlapping difficulty ranges. A stronger version would collect human confidence judgments on a random subset of the same MMLU/GPQA questions and compare slopes on identical items.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim is that AFCE 'delivers more human-like sensitivity to task difficulty.' This rests on overlaying LLM confidence–accuracy regression slopes from MMLU/GPQA with human slopes from Moore and Healy (2008). The paper itself notes in the Section 3 footnote that it cannot repeat the original Moore–Healy questions. Because the human and LLM data come from different question sets, formats, and difficulty distributions, any difference in regression slope could reflect task mismatch rather than a genuine difference in confidence behavior. Moore–Healy's hard/easy tasks were general-knowledge items with a particular difficulty spread; MMLU/GPQA cover different subjects and a different accuracy range. A regression slope is not invariant to the range and distribution of the predictor, so the overlay in Figure 3 does not establish that AFCE is closer to human calibration. The claim is further weakened because the steeper-slope result is reported for GPT-4o only; for Llama-3-70B and Claude-3, Figure 3 and Section 4.2 show relatively flat confidence curves even with AFCE. Thus the headline 'more human-like sensitivity' generalizes a single-model, cross-dataset comparison. The overconfidence-reduction claim may still hold, but the human-like part of the central claim is not supported without matched or otherwise controlled human data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies confidence calibration in three LLMs (Llama-3-70B, Claude-3-Sonnet, GPT-4o) on MMLU and GPQA, comparing observed confidence–accuracy patterns with human over/underconfidence findings from Moore and Healy (2008). The authors propose Answer-Free Confidence Estimation (AFCE), a two-stage prompting method that elicits confidence scores separately from answers, and report that AFCE reduces Expected Calibration Error (ECE) on expert-level tasks, especially for GPT-4o, and yields a steeper calibration slope that they interpret as more human-like sensitivity to task difficulty. The paper also reports that persona prompts induce overplacement for expert personas and underplacement for layman personas, and that demographic personas produce stereotypical confidence shifts without corresponding accuracy changes. The manuscript includes prompt templates, dataset statistics, and a code repository.","tokens_in":22080,"tokens_out":4549,"duration_ms":45879,"significance":"If the empirical claims hold, the paper makes a useful practical contribution: AFCE is a simple, black-box-compatible alternative to verbalized confidence that appears to reduce overconfidence on difficult multiple-choice QA, and the persona/demographic findings are relevant to the growing use of LLMs in computational social science and role-playing applications. The paper is generally careful to provide prompt templates, dataset splits, and appendix results across seven models, which aids reproducibility. However, the central 'human-like sensitivity' claim rests on an uncontrolled cross-dataset comparison, and the 'significantly reduces overconfidence' claim is not backed by any statistical inference; the observed effects are plausible but currently supported only by point estimates.","major_comments":[{"comment":"The headline claim that AFCE 'delivers more human-like sensitivity to task difficulty' is not established by the evidence presented. The human regression slopes in Figure 3 come from Moore and Healy (2008), whose items and difficulty distributions differ from MMLU and GPQA; the paper itself notes in the §3 footnote that the original questions were not available. A regression slope is not invariant to the range and distribution of the predictor, so differences between LLM slopes and human slopes could reflect task mismatch rather than a genuine difference in confidence behavior. The claim is further weakened because the steeper-slope effect is reported only for GPT-4o; for Llama-3-70B and Claude-3, Section 4.2 and Figure 3 show relatively flat confidence curves even with AFCE. To support the human-like claim, the authors should either collect human calibration data on the same item sets, reanalyze Moore–Healy data on a matched difficulty range, or explicitly reframe the conclusion as improved sensitivity to task difficulty without claiming human equivalence.","section":"§4.1, §4.2; Table 1"},{"comment":"The word 'significantly' in the abstract and in Section 4.2 is not supported by any statistical test, confidence interval, or bootstrap. ECE is computed from a small number of 10-question blocks per condition: for example, the 170-question high-school physics condition yields only 17 blocks, and 10-bin ECE over such a small sample will have large variance. The observed differences in Table 1, such as GPT-4o expert-physics ECE of 16.1 (AFCE) versus 21.1 (top-k), may be real, but the paper provides no measure of uncertainty. The authors should add error bars or significance tests (e.g., bootstrap over question blocks) before claiming that AFCE 'significantly reduces overconfidence'.","section":"§4.2, Appendix Table 5"},{"comment":"The claim that AFCE 'consistently outperforms' all baselines is contradicted by the appendix results on additional models. For example, in Table 5, Mixtral-8x7B high-school physics shows AFCE ECE 48.7 versus Top-K 47.7 and Vanilla 49.5, and LLaMA-3-8B high-school chemistry shows AFCE ECE 39.3 versus Vanilla 31.7. The main-text Table 1 also shows cases where AFCE is not the best on easier subtasks (e.g., GPT-4o high-school biology: AFCE ECE 6.1 versus Vanilla 2.9). The authors should restrict the superiority claim to expert-level tasks or provide a quantitative summary (e.g., mean rank with significance) rather than 'consistent' across all conditions.","section":"§7, Table 10"},{"comment":"The ablation section states that 'no significant differences' were observed under variations in question order and group size, but no significance test is reported. Table 10 shows point estimates such as GPQA-biology ECE changing from 10.0 with 10 questions to 7.1 with 5 questions, and high-school physics ECE changing from 14.7 to 22.0 under 5 questions. Without confidence intervals or paired tests, the robustness claim is unsupported. The authors should either provide the statistical evidence or soften the wording to 'no large differences were observed in these point estimates'.","section":"§10"}],"minor_comments":[{"comment":"References Hase et al. 2024a and 2024b are the same paper and should be merged; this duplication is a presentation issue that should be fixed in revision.","section":"References"},{"comment":"The header structure of Table 1 is difficult to parse: the 'AvE' column appears to be placed under 'High School Physics' in the text rendering, and the caption describes it as 'Average ECE' but the row values are repeated. The table should be reformatted so that each difficulty level has clearly labeled Acc/ECE subcolumns and the average ECE column is visually separated.","section":"Table 1"},{"comment":"The overplacement formula in the Figure 4 caption uses terms such as 'Confidence Estimate Others' and 'Accuracy Others' that are not formally defined in the text; adding a short definition of these quantities in Section 5.1 would improve reproducibility.","section":"Figure 4 caption"},{"comment":"The footnote stating that the original Moore and Healy (2008) questions are unavailable is a crucial caveat for the central comparison; it should be moved into the main text and discussed explicitly in the limitations section, which currently only mentions ECE limitations and prompt dependence.","section":"§3"},{"comment":"The statement that GPT-4o 'demonstrates balanced confidence estimations across races, ages, and genders' is stronger than Figure 5 shows; GPT-4o still shows nonzero D_Demographic values for some groups. A more cautious wording, such as 'smaller disparities than the other models,' would match the figure.","section":"§6.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical study of a practical calibration method, but the 'human-like' framing is the weakest part and should be either supported by matched human data or removed. The lack of any statistical inference is the main technical gap; with point estimates only, the central claims are not yet at the standard of a journal paper. The paper may be better suited to a venue that accepts empirical workshop-style contributions, or the authors should substantially strengthen the quantitative evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the AFCE result is real and worth knowing; the 'human-like sensitivity' framing is the part to distrust. Separating confidence elicitation from answer generation is a clean idea, and it does reduce overconfidence on expert-level QA across the three main models — the ECE gains are consistent on hard tasks, and the robustness checks (open-ended QA, question order, group size) add credibility. The persona and demographic findings — confidence moves with the role while accuracy stays flat — are a useful empirical contribution to the role-playing literature.\n\nThe soft spot is the comparison with Moore and Healy. The paper acknowledges it cannot repeat the human questions, and then overlays human regression slopes from a different task set onto LLM slopes from MMLU/GPQA. That is not a controlled comparison; slope differences can be produced by the range and distribution of difficulty. Worse, the steeper calibration slope appears mainly for GPT-4o. Llama-3-70B and Claude-3 show relatively flat confidence curves even with AFCE. So the abstract's 'more human-like sensitivity to task difficulty' is not established. What is established is that AFCE lowers ECE on hard tasks for these models.\n\nThere are also no error bars or significance tests. ECE is computed from small numbers of 10-question blocks with 10 bins, so a few percentage points of improvement is within plausible noise. And there is a prompt confound in the overplacement experiment: self-estimates use a 'you' prompt with no role, while other-estimates use third-person wording ('A is an expert...'), which alone could shift confidence. These issues are fixable, but they currently bound what the paper can claim.\n\nNet: solid method, interesting persona results, but the central human-comparison headline overreaches. A serious referee should engage with this — the AFCE idea is simple and reproducible, and the persona effects matter for anyone using LLMs in computational social science. With matched tasks, significance testing, and a more careful scope, it would be a worthwhile publication. I'd want to see the revision before signing off, but it's not a desk reject.","headline":"AFCE cuts overconfidence on hard QA, but the 'human-like sensitivity' claim rests on a confounded cross-task comparison and only shows up clearly in GPT-4o.","tokens_in":22600,"tokens_out":2921,"would_cite":true,"duration_ms":28513,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage prompting method that asks LLMs for confidence before answers reduces overconfidence and makes confidence track task difficulty more closely, especially for GPT-4o.","keywords":["confidence calibration","LLM overconfidence","answer-free confidence estimation","verbalized confidence","persona bias","task difficulty sensitivity","expected calibration error","role-playing LLMs"],"falsifier":"Have humans and the same model answer an identically matched set of items, compute the confidence-accuracy regression slope under vanilla prompting and AFCE, and compare slopes; if AFCE's slope is not steeper, or if the gap to human slopes does not close on the matched set, the 'more human-like sensitivity' claim fails.","tokens_in":21599,"feed_emoji":"🎯","tokens_out":6713,"duration_ms":54496,"temperature":0.7,"pith_summary":"Large language models, like humans, tend to be overconfident on hard questions and underconfident on easy ones, but the paper argues that the resemblance stops there: model confidence is far less sensitive to task difficulty, and it shifts with the persona the model is asked to adopt even when accuracy barely changes. The central proposal is Answer-Free Confidence Estimation (AFCE), which asks the model first to estimate how many of ten questions it can answer correctly and only later asks for the answers. On expert-level science questions from MMLU and GPQA, AFCE lowers Expected Calibration Error across three models and, for GPT-4o, produces a steeper calibration slope that the authors read as closer to human calibration patterns. The stakes are practical: if verbalized confidence can be detached from the act of answering, then calibration can be improved without model-internal access, and persona-driven confidence biases become a measurable risk for role-playing applications.","feed_headline":"Splitting answers from confidence curbs LLM overconfidence","feed_subtitle":"Asking a model to rate itself before answering produces better calibration on hard questions and tracks difficulty more like humans.","key_machinery":"The load-bearing device is Answer-Free Confidence Estimation (AFCE), a two-stage prompting scheme. In the confidence stage the model reads ten questions and estimates how many it can answer correctly, choosing an integer from 0 to 10, without producing answers; in the accuracy stage it answers the same questions by option letter only. The paper hypothesizes that generating factual answers is an epistemically intensive process that dominates reasoning and defaults to a high baseline confidence, so removing answer generation lets the model attend to confidence and task difficulty itself. The evaluation machinery is Expected Calibration Error (ECE) with ten bins, used to compare AFCE against vanilla verbalized confidence, top-k prompting, quiz-like prompting, sampling-based confidence, and first-token probability.","core_discovery":"The paper's central claim is that answer generation and confidence estimation are mediated by different mechanisms in LLMs, and that fusing them together is what produces overconfidence. AFCE separates the two: one prompt asks the model to read a set of questions and estimate how many (from 0 to 10) it can answer correctly; a separate prompt asks only for the option letters. Across LLaMA-3-70B, Claude-3-Sonnet, and GPT-4o on high-school, college, and expert-level physics, chemistry, and biology questions, this separation lowers Expected Calibration Error on the hardest tasks and, for GPT-4o, yields a regression slope closer to the ideal calibration line and to human slopes reported in the psychology literature. The paper also claims that models overplace confidence toward expert personas, underplace it toward layman personas, and express stereotypically lower confidence for demographic groups such as female, Asian, or older personas, while their accuracy stays roughly constant.","pith_inferences":["Beyond the paper: if decoupling confidence from execution is the mechanism, the same separation should reduce overconfidence in agentic or tool-use settings where the model both plans and acts; this is testable by comparing joint and split confidence elicitation in multi-step tasks.","Beyond the paper: the 'more human-like' slope claim depends on the comparability of human calibration data collected on different questions; a direct matched-question study with the same item set for humans and models would either strengthen or dissolve that comparison.","Beyond the paper: the demographic confidence biases likely mirror stereotypes in training text rather than real group differences; if so, demographic-aware calibration or counterfactual prompting could be evaluated by checking whether confidence gaps persist when the model is told the persona has identical knowledge.","Beyond the paper: the paper's ECE improvement on hard tasks might come partly from AFCE lowering confidence overall rather than from sharper discrimination between easy and hard items; inspecting item-level regression slopes rather than aggregate ECE would distinguish these."],"forward_implications":["Verbalized confidence methods should stop assuming human-like behavior; if answer generation and confidence estimation are decoupled, calibration should be treated as a separate stage rather than bundled into one prompt.","AFCE gives black-box API models a prompt-only route to better calibration on hard tasks: on GPT-4o's expert-level questions average ECE drops substantially relative to vanilla, quiz-like, and sampling baselines.","Persona-based simulation studies inherit the models' stereotyped confidence biases: an 'expert' persona inflates confidence and a 'layman' persona deflates it with little change in accuracy, so downstream social-science simulations should separate role-played confidence from role-played performance.","The method extends beyond multiple-choice questions: on open-ended NQ-open and SimpleQA sets, AFCE also lowers ECE relative to baselines, and it is robust to question order and group size.","Models like LLaMA-3-70B and Claude-3 show a relatively flat confidence curve across difficulty, suggesting they fall back on a standard confidence level; this flatness would be a useful diagnostic for when verbalized confidence cannot be trusted."],"supporting_citations":[{"why":"Supplies the human calibration patterns and the difficulty-based over- and underconfidence model that the paper compares LLM confidence against.","marker":"Moore and Healy (2008)"},{"why":"Defines Expected Calibration Error, the evaluation metric used for all calibration comparisons.","marker":"(Guo et al., 2017b)"},{"why":"Provides the MMLU dataset used for the high-school and college difficulty subtasks.","marker":"(Hendrycks et al., 2021)"},{"why":"Provides the GPQA expert-level dataset used for the hardest-difficulty calibration tests.","marker":"(Rein et al., 2023)"},{"why":"Provides the sampling-based confidence baseline and the evidence that LLMs tend to be overconfident.","marker":"(Xiong et al., 2024)"},{"why":"Provides the top-k verbalized confidence baseline and the finding that GPT-4o aligns confidence with token probability.","marker":"(Tian et al., 2023)"},{"why":"Provides the vanilla verbalized confidence baseline used as the main comparison.","marker":"(Lin et al., 2022)"},{"why":"Provides the first-token probability baseline used as a white-box confidence estimate.","marker":"(Wang et al., 2024a)"},{"why":"Provides the role-playing and simulation methodology that the persona and demographic experiments build on.","marker":"(Aher et al., 2023b)"}],"fun_headline_variants":["Separating confidence from answers trims LLM overconfidence","Ask LLMs to self-assess before answering—better calibration follows","AFCE: letting LLMs rate themselves without answers improves calibration","Two-step prompting: rate first, then answer—cuts LLM overconfidence","Why LLMs are overconfident—and a two-prompt method that helps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that AFCE is more human-like assumes that human calibration slopes from a different set of questions can be fairly compared with model slopes on MMLU and GPQA, even though the original human questions were not available.","fun_headline_variants_meta":{"raw":{"variants":["Separating confidence from answers trims LLM overconfidence","Ask LLMs to self-assess before answering—better calibration follows","AFCE: letting LLMs rate themselves without answers improves calibration","Two-step prompting: rate first, then answer—cuts LLM overconfidence","Why LLMs are overconfident—and a two-prompt method that helps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000758,"raw_usage":{"total_tokens":3366,"prompt_tokens":944,"completion_tokens":2422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":2343}},"tokens_in":560,"tokens_out":2422,"duration_ms":18990,"temperature":1.0,"reasoning_tokens":2343,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:01:44.357504+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have humans and the same model answer an identically matched set of items, compute the confidence-accuracy regression slope under vanilla prompting and AFCE, and compare slopes; if AFCE's slope is not steeper, or if the gap to human slopes does not close on the matched set, the 'more human-like sensitivity' claim fails.","supporting_citations":[],"review_version":1}