{"id":"0225f1dd-db92-4f3a-81f5-2e1e1a496915","arxiv_id":"2608.10008","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Verbalized confidence from four zero-shot LLM recommenders is systematically under-confident and cannot separate correct items from catalog hallucinations, so confidence-gated abstention barely reduces hallucination.","lead":"This paper checks whether AI product and movie recommenders know when they invent titles that are not in the catalog. It finds they usually do not: their confidence scores are badly miscalibrated, and using those scores to filter out bad guesses mostly removes good ones.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fuzzy-match threshold at 90 silently reclassifies near-miss hallucinations as catalog hits; without a sensitivity analysis the under-confidence and abstention-ceiling claims may be artifacts of the label oracle.","rationale":"The reader's weakest assumption is the hand-set fuzzy-matching rule in Section 3.3, and I agree that this is the most load-bearing point. Every calibration metric in Table 2, every reliability curve in Figure 1, and every abstention result in Table 3 is computed against labels produced by that rule. A threshold error does not merely add noise; it systematically shifts the accuracy denominator in the direction that creates the paper's headline pattern. Under-confidence appears because accuracy is high relative to confidence; if accuracy is inflated by lenient matching, the gap is spurious. The abstention ceiling appears because hallucinations are not being removed; if hallucinated titles are labeled in-catalog, their removal is invisible. The paper does have independent support: consistency across four vendor models, the Llama/Yelp logprob check, and the prompt-perturbation ablations all point in the same direction. But those checks share the same label oracle, so they do not de-risk the fuzzy-match assumption. The proposed test is cheap and would settle the question: report exact-match-only results and a manual audit of fuzzy-only positives. Because the concern is real but currently unquantified, the correct disposition is conditional acceptance pending this sensitivity analysis, which is exactly the reader's verdict, so no adjustment is needed.","tokens_in":10429,"tokens_out":4626,"duration_ms":53003,"concrete_test":"Recompute OOD@10, ECE, Brier, and the Table 3 conformal sweep under three label oracles: exact match only, token_set_ratio >= 80, and token_set_ratio >= 95. Report the fraction of recommendations that pass fuzzy but fail exact, stratified by catalog and model, and manually review 100 random fuzzy-only positives per catalog to estimate the false-match rate. If fuzzy-only positives are below 0.5% of recommendations and the manual false-match rate is low, the central claims are unchanged; if not, re-estimate the under-confidence gap and abstention ceiling with the stricter oracle.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claims—systematic under-confidence and a 0.7 pp abstention ceiling—depend entirely on the binary catalog-membership labels used to compute accuracy and hallucination rate. Section 3.3 assigns in-catalog status after exact-match failure when token_set_ratio >= 90 on lowercased, punctuation-stripped strings. The paper reports no counts of fuzzy-only matches, no manual error-rate estimate for that rule, and no threshold sensitivity analysis. If many near-miss hallucinations pass the fuzzy rule, measured accuracy is inflated and OOD is deflated. That would exaggerate the under-confidence gap (confidence 67–86 vs. accuracy 92–100) and could even reverse it if the inflation is large. It would also corrupt the abstention analysis: if hallucinated titles are silently labeled correct, then a filter that removes them is recorded as removing correct items, which is exactly the observed 0.7 pp ceiling. The MovieLens near-zero OOD result is especially vulnerable because the fuzzy rule is likely doing most of the matching there. The fuzzy threshold is not implausibly lenient, so the concern may fail, but it is load-bearing because every headline number in the paper passes through this one label oracle.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript audits the joint distribution of catalog faithfulness and verbalized confidence for four zero-shot LLM recommenders (Mistral Large, Llama-3.3-70B, GPT-OSS-120B, Claude Sonnet 4.6) across three catalogs (MovieLens-25M, Amazon Toys, Yelp), with popularity-stratified calibration metrics. The main claims are (i) OOD@10 is catalog-dependent (0-0.2% on MovieLens, 4.5-8.3% on Amazon, 2.2-8.4% on Yelp); (ii) all four models are systematically under-confident across all twelve model-catalog cells, verbalizing mean confidence 67-86 on items that are correct 92-100% of the time; and (iii) a split-conformal abstention threshold over verbalized confidence reduces hallucination by at most 0.7 pp at 4-21 pp coverage cost, showing that the confidence channel, not the conformal procedure, is the binding constraint. The paper also runs prompt-perturbation ablations and a logprob check, and it explicitly disclaims that the conformal score-quantile guarantee transfers to hallucination-rate control.","tokens_in":10626,"tokens_out":10323,"duration_ms":107460,"significance":"The manuscript addresses a genuine gap: prior generative-recommender evaluations report binary OOD rates without asking whether the model's confidence separates correct items from hallucinations. The cross-vendor replication (four independent model families), the exact prompt templates, the model identifiers and access dates, and the explicit statement of the conformal guarantee's limits are strengths. If the under-confidence and 0.7 pp ceiling findings survive a robustness audit of the label oracle, they have clear operational consequences for abstention policies. The main weakness is that every headline quantity passes through a single hand-set fuzzy-matching threshold, and calibration metrics are reported without uncertainty intervals; both issues are addressable in revision.","major_comments":[{"comment":"The in-catalog label oracle is a single hand-set rule: after exact match fails, a title is counted as in-catalog if token_set_ratio >= 90 on lowercased, punctuation-stripped strings. The manuscript reports no counts of fuzzy-only matches per catalog, no manual error-rate estimate for that rule, no sensitivity analysis around the threshold, and no exact definition of the fuzzy metric. Because both accuracy (Figure 1, Table 2) and OOD@10 (Table 1) are computed from this binary label, a lenient threshold would silently reclassify near-miss hallucinations as catalog hits, inflating accuracy and deflating OOD; this could exaggerate or reverse the under-confidence gap and would directly contaminate the 0.7 pp abstention ceiling, since a filter that removes hallucinations would be recorded as removing correct items. Please add exact-only versus fuzzy-only OOD rates, a threshold sweep (for example 80, 85, 90, 95, and exact-only), and a hand-audited sample of fuzzy-only matches, with particular attention to the near-zero MovieLens result where the fuzzy rule may be doing most of the matching.","section":"§3.3, Tables 1 and 3"},{"comment":"Table 2 reports ECE and Brier to three decimals without any uncertainty intervals or significance tests, in contrast to Table 1, which has bootstrap CIs. The cross-model and head-tail comparisons in Section 4.3 (for example, \"Claude has the highest ECE in every (dataset, stratum) cell\" and \"ECE-tail exceeds ECE-head on Yelp for every LLM\") are therefore not accompanied by evidence about sampling variation. Given the 300 users per audit set and bin counts as small as 21 items in Figure 1, many of these differences may be within noise. Please provide cluster-bootstrap or percentile intervals for ECE/Brier and for the head-tail differences, and state the effective per-cell sample sizes.","section":"Table 2 and §4.3"},{"comment":"The text says that \"MovieLens tail samples (7-26 items per cell) are too thin for stratified inference,\" yet Table 2 still reports ECE-tail and Brier-tail for MovieLens for all four models. Metrics computed on 7-26 items cannot support the quantitative comparisons made in Section 4.2, and the reported three-decimal values are misleading. Please either remove the MovieLens tail columns or flag them clearly as not interpretable, and exclude them from any cross-cell summary claims.","section":"§4.3 and Table 2"},{"comment":"The conformal calibration description is ambiguous about the calibration set composition: it says the empirical quantile is computed \"over in-catalog calibration items.\" If the quantile is estimated only from in-catalog items, the exchangeability assumption with test items that include hallucinations does not hold, and the stated marginal score-quantile guarantee does not follow. Moreover, Table 3 reports only hallucination rates and no actual coverage values, so the text's claim that coverage tracks 1-alpha within ±0.07 cannot be verified from the table. Please clarify whether calibration scores are computed on all calibration items or only in-catalog ones, and report the coverage column in Table 3.","section":"§3.5 and Table 3"}],"minor_comments":[{"comment":"The abstract states that models verbalize a mean confidence of 67-86 on items with 92-100% accuracy, but these numbers do not appear in any table; Table 2 shows ECE and Brier only. Please add a table or text block reporting per-cell mean confidence and accuracy.","section":"Abstract and Table 2"},{"comment":"The paper says per-cell audit counts are 285-300 users after dropping errored responses, but it does not give exact counts per cell; please add a footnote or small table with final N per model-dataset cell.","section":"§3.1"},{"comment":"The two prompt-perturbation ablations (Llama Yelp ECE 0.224 -> 0.141; Claude Yelp -> 0.434) are reported without sample sizes or uncertainty information. Please state the per-ablation N and whether the same audit users are used.","section":"§4.2"},{"comment":"The caption says marker size is proportional to bin count, but the legend does not give the scale; adding a size legend or numeric bin-count annotations would improve interpretability.","section":"Figure 1"},{"comment":"The text says \"the median reduction is below 0.1 pp,\" but no median or distribution of reductions is shown in Table 3; please add a summary statistic or a small table of all 32 reductions.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the empirical protocol is valuable and largely reproducible in principle. The main risk is the label-oracle sensitivity, which is load-bearing because every headline number passes through the fuzzy-match rule; I do not think this is fatal because it can be addressed with additional analysis. If the authors provide the fuzzy-match sensitivity analysis, calibration uncertainty intervals, and the conformal-set clarification, I would support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a real finding — systematic under-confidence on catalog faithfulness across four independent models — and a protocol worth copying. But every headline number passes through a single fuzzy-match oracle that is not sensitivity-analyzed, so treat the point estimates as provisional until that is fixed.\n\nWhat is new: this is the first joint audit of OOD@K and probabilistic calibration (ECE, Brier, reliability) for LLM recommenders, using catalog membership as the oracle instead of ranking correctness. The under-confidence direction replicates in all twelve cells across four vendors, which is solid evidence. The conformal sweep showing that verbalized confidence cannot support abstention (max 0.7 pp reduction, sometimes negative) is a clean operational result. The paper also deserves credit for explicitly stating that the conformal guarantee covers the score quantile, not hallucination rate, and for flagging that MovieLens tail strata are too thin for stratified inference. That is honest discipline.\n\nSoft spots, in proportion: (1) The fuzzy-match rule (token_set_ratio >= 90) is load-bearing for every in-catalog label, yet the paper reports no counts of fuzzy-only matches, no manual error-rate estimate, and no threshold sensitivity analysis. The stress-test concern is fair: if near-miss hallucinations pass the rule, OOD is deflated and the abstention ceiling could be partly an artifact. I don't think the threshold is obviously wrong — 90 is reasonably strict — but \"reasonably strict\" is not a substitute for a measured error rate. This is fixable and should be fixed before the results are treated as a benchmark. (2) Calibration metrics have no error bars or significance tests. Table 1 gives bootstrap CIs for OOD, but ECE and Brier are point estimates. Given the 12-cell consistency, I doubt the direction flips, but the magnitudes — especially the 0.7 pp ceiling — need uncertainty bounds. (3) Minor: the released code is not linked with a commit hash. The floating model alias for Mistral is disclosed, which is good.\n\nWho this is for: anyone doing production audits of LLM recommenders, and anyone working on selective prediction or calibration for generative systems. It deserves a serious referee — the protocol and the under-confidence finding are worth engaging. My recommendation: conditional acceptance, requiring a sensitivity analysis of the fuzzy-match oracle and error bars on calibration metrics.","headline":"A genuinely useful audit with a real under-confidence result across four vendors, but the fuzzy-match label oracle needs a sensitivity analysis before the headline numbers harden.","tokens_in":11155,"tokens_out":2174,"would_cite":true,"duration_ms":23425,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLM recommenders are systematically under-confident about catalog faithfulness, so verbalized-confidence abstention cannot separate correct items from hallucinations.","keywords":["LLM recommenders","hallucination","confidence calibration","catalog faithfulness","abstention","conformal prediction","OOD@10","verbalized confidence"],"falsifier":"Re-run the audit on MovieLens with exact-match-only coding and with fuzzy-match thresholds at 85 and 95; if OOD@10 rises above 0.2% or the conformal sweep reduces hallucination by more than 0.7 percentage points, then the lenient matching threshold is material to the abstention-ceiling claim.","tokens_in":10204,"feed_emoji":"🎯","tokens_out":4802,"duration_ms":43404,"temperature":0.7,"pith_summary":"This paper tries to establish that the confidence LLM recommenders attach to their top-K suggestions is calibrated against generic recommendation quality, not against whether the suggested item actually exists in the target catalog. It audits four zero-shot LLMs from different vendors across three catalogs and finds that all of them are systematically under-confident: they verbalize mean confidence of 67–86 on items that are in-catalog 92–100% of the time. Because the confidence channel cannot tell a real catalog entry from a hallucinated one, a conformal abstention threshold reduces hallucination by at most 0.7 percentage points while sacrificing 4–21 points of coverage. The paper argues that audits of LLM recommenders should report calibration alongside hallucination rate, and that confidence elicitation should be anchored to catalog membership rather than left generic.","feed_headline":"Under-confident LLMs can't abstain away hallucinations","feed_subtitle":"Verbalized confidence runs 67–86 on 92–100% accurate picks; conformal gating trims hallucinations by at most 0.7 points.","key_machinery":"The central object is the joint distribution of the catalog-membership indicator (whether a generated title exists in the catalog, coded by exact match plus a fuzzy token_set_ratio >= 90 fallback) and the verbalized confidence elicited by the \"Just Ask\" template, measured with ECE, Brier, and reliability diagrams. A split-conformal quantile over the nonconformity score $s = 1 - c$ sets the abstention threshold. The mismatch between what the prompt asks for (recommendation quality) and what the audit measures (catalog existence) is the mechanism that clusters nearly all scores in the band $[0.05, 0.55]$, so quantile thresholds remove correct items faster than they remove hallucinations.","core_discovery":"Across all four LLMs, all three catalogs, and both popularity strata, verbalized confidence is systematically under-confident with respect to catalog faithfulness: models report mean confidence in the 67–86 range while their actual in-catalog accuracy is 92–100%. This is the opposite direction from the over-confidence usually reported in LLM-hallucination work, and it holds even on MovieLens where the hallucination rate is near zero. The under-confidence is best explained as an elicitation mismatch: the \"Just Ask\" prompt elicits a generic recommendation-quality rating, not a probability that the title exists in the catalog. As a result, a split-conformal abstention threshold built on the nonconformity score $s = 1 - c$ fails to separate correct items from hallucinations, reducing hallucination by at most 0.7 percentage points across the alpha sweep at 4–21 points of coverage cost.","pith_inferences":["The near-zero MovieLens hallucination rate may be inflated by the lenient fuzzy-match rule; re-coding with exact-match-only or a stricter threshold could reveal a nonzero OOD rate, which would weaken the \"calibration matters even when hallucination is zero\" framing.","If the elicitation mismatch is the cause, then training-time calibration objectives that target catalog membership directly (e.g., rewarding doubt on out-of-catalog items) might restore a usable abstention signal, but the paper does not test this.","The logprob check on Llama times Yelp is also under-confident, so the finding is not purely a verbalization artifact; this suggests the under-confidence is a property of the model's internal probability estimates for this task, not just of the prompt.","The paper's twelve-cell design supports a strong cross-vendor claim, but a larger sample of models and catalogs would tell whether under-confidence is a general zero-shot phenomenon or specific to this prompt-and-catalog configuration."],"forward_implications":["Model selection changes when calibration is reported alongside OOD: GPT-OSS-120B is worst on hallucination rate on long-tail catalogs but best-calibrated on Yelp, so choosing on one axis alone trades off the other.","Conformal abstention composed on verbalized confidence is not a free safety net for zero-shot LLM recommenders; the finite-sample guarantee holds on score coverage but not on hallucination rate.","Catalog-anchored confidence prompts can materially move calibration, but in opposite directions by vendor, so prompt design is part of the calibration problem, not a neutral measurement choice.","Audits of grounded, fine-tuned, or retrieval-constrained recommenders should measure the same joint distribution, since the zero-shot results may not characterize those production configurations."],"supporting_citations":[{"why":"Supplies the \"Just Ask\" verbalized-confidence elicitation template used for all main results.","marker":"[20]"},{"why":"Supplies the split-conformal abstention procedure and its marginal score-quantile coverage guarantee.","marker":"[1]"},{"why":"Supplies the MovieLens-25M catalog used as the canonical-entertainment audit set.","marker":"[8]"},{"why":"Supplies the Amazon Reviews 2023 Toys & Games catalog used as the long-tail e-commerce audit set.","marker":"[9]"},{"why":"Supplies the Yelp Open Dataset used as the regional-services audit set.","marker":"[23]"},{"why":"Closest neighbor, decomposing uncertainty against ranking correctness rather than catalog membership; the paper contrasts its oracle with this one.","marker":"[15]"},{"why":"Grounding baseline (RecLM) that the paper contrasts with inference-time abstention as a hallucination-mitigation strategy.","marker":"[17]"},{"why":"Documents over-confidence in LLM confidence elicitation on QA tasks, the pattern the paper's under-confidence flips.","marker":"[22]"}],"fun_headline_variants":["Catalog hallucination audits miss miscalibrated confidence","Abstention thresholds fail on under-confident recommender LLMs","Even with zero hallucinations, LLM confidence is miscalibrated","LLM recommenders: under-confident and unable to abstain well"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported hallucination rates rest on a hand-set fuzzy-matching rule that counts near-miss titles as in-catalog once token_set_ratio reaches 90; if that rule is too lenient, hallucination is undercounted and the abstention ceiling is measured against the wrong baseline.","fun_headline_variants_meta":{"raw":{"variants":["Catalog hallucination audits miss miscalibrated confidence","Abstention thresholds fail on under-confident recommender LLMs","Even with zero hallucinations, LLM confidence is miscalibrated","LLM recommenders: under-confident and unable to abstain well"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000347,"raw_usage":{"total_tokens":1997,"prompt_tokens":1141,"completion_tokens":856,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":757,"completion_tokens_details":{"reasoning_tokens":783}},"tokens_in":757,"tokens_out":856,"duration_ms":8017,"temperature":1.0,"reasoning_tokens":783,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:47:47.897096+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the audit on MovieLens with exact-match-only coding and with fuzzy-match thresholds at 85 and 95; if OOD@10 rises above 0.2% or the conformal sweep reduces hallucination by more than 0.7 percentage points, then the lenient matching threshold is material to the abstention-ceiling claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the \"Just Ask\" verbalized-confidence elicitation template used for all main results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Yelp Open Dataset used as the regional-services audit set."},{"cited_title":"Uncertainty Quantification and Decomposition for LLM-based Recommendation","cited_arxiv_id":"2501.17630","evidence_quote":"Closest neighbor, decomposing uncertainty against ranking correctness rather than catalog membership; the paper contrasts its oracle with this one."}],"review_version":1}