{"id":"3ff4689a-a202-4333-abc6-211ecada7364","arxiv_id":"2509.05878","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An LLM Jury of ten models evaluates whether clinician-defined key facts appear in AI-generated discharge summaries, matching a seven-physician majority as well as a single physician (kappa 0.81 vs 0.67).","lead":"A Stanford team tested whether a jury of ten language models can check whether AI-written hospital discharge summaries include the key facts doctors care about. The jury agreed with a seven-physician panel almost as well as a single doctor does, and the authors also describe a workflow for generating more factual summaries.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The jury is validated only on MedAgentBrief summaries but is then used to score Single-Prompt summaries; without evidence that its kappa transfers, the reported MedAgentBrief advantage may be an artifact of the validation distribution.","rationale":"The paper is a solid empirical study: the seven-physician majority gold standard, leave-one-out human baseline, and bootstrapped confidence intervals are sensible, and the code is open source. The strongest claim is that the LLM Jury can substitute for physician review when checking clinician-defined key facts in AI-generated discharge summaries. For that claim to support the paper's headline comparison of generation strategies, the jury must be reliable not only on the two MedAgentBrief systems where it was validated, but also on the Single-Prompt outputs that Figure 1 compares. The reader's weakest assumption points exactly here, and I agree with it. Section 4.2 applies the validated jury to Single-Prompt summaries without any human validation on that distribution; this is a distribution-shift problem rather than an internal inconsistency. I also note a secondary statistical reporting issue: the DeepSeek-based non-inferiority result reports a kappa difference of 2% with a 95% CI of -11% to 15% and P<0.001, which cannot all be simultaneously true under standard bootstrap methods; this should be corrected, though the primary GPT-4o non-inferiority appears internally consistent. Overall, the central validation is plausible but the unvalidated transfer to Single-Prompt outputs undermines the general utility claim and the MedAgentBrief advantage, so conditional acceptance with a request for transfer validation remains the appropriate verdict.","tokens_in":8387,"tokens_out":10770,"duration_ms":94514,"concrete_test":"Collect physician labels on a random sample of 20 Single-Prompt summaries generated by GPT-4o (the model with the reported 17-point improvement), using the same 90 key facts and seven-physician majority protocol. Compute the LLM Jury's kappa against this gold standard and the jury-minus-physician kappa difference with the same bootstrap CI. If the one-sided 95% CI for the difference crosses the -10% non-inferiority margin, or if the jury kappa is significantly below the 81% measured on MedAgentBrief summaries, the Figure 1 benchmarking conclusion is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The meta-evaluation in §3.4 builds the gold standard and measures jury agreement only on 60 MedAgentBrief summaries from two systems (GPT-4o and DeepSeek-R1). The central validation claim—kappa=81% and non-inferiority to a single physician (kappa=67%)—is therefore established only for that distribution. In §4.2 and Figure 1, the same jury is applied to Single-Prompt summaries from multiple models, and the paper concludes that MedAgentBrief improves factual presence (e.g., GPT-4o from 48% to 65%). This inference requires the jury's decision behavior to transfer from MedAgentBrief-style outputs to Single-Prompt outputs, where summary length, formatting, and error profiles may differ. If the jury's false-positive or false-negative rates shift on Single-Prompt summaries, the performance ordering in Figure 1 could reflect the jury's distribution-specific behavior rather than true factual differences. No human labels are reported for Single-Prompt summaries, so this transfer is unvalidated. The kappa=81% result itself is not impugned for the distribution on which it was measured; the problem is the unmeasured generalization to the benchmark that motivates the framework.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MedFactEval, a framework in which clinicians define high-salience key facts and a ten-LLM jury votes on whether those facts are present or contradicted in AI-generated discharge summaries. It also introduces MedAgentBrief, a multi-step, model-agnostic workflow with iterative refinement and provenance tagging, and benchmarks it against a single-prompt baseline. The central validation is a meta-evaluation on 60 summaries generated by two MedAgentBrief systems (GPT-4o and DeepSeek-R1), where the LLM Jury's fact-presence judgments are compared with a seven-physician majority-vote gold standard. The jury achieves Cohen's kappa of 81% (95% CI 66–92%) on the GPT-4o summaries and is statistically non-inferior to a single physician (kappa difference 15%, 95% CI -1% to 29%, P<0.001). The same jury is then used in Section 4.2 to score single-prompt summaries across multiple models, leading to the claim that MedAgentBrief improves factual presence (e.g., GPT-4o from 48% to 65%).","tokens_in":8535,"tokens_out":1789,"duration_ms":17010,"significance":"If the reported meta-evaluation holds, the paper makes a useful practical contribution: an LLM jury with pre-specified clinician facts can approximate physician fact-checking at scale, and the leave-one-out human baseline, bootstrap confidence intervals, and pre-specified non-inferiority margin are appropriate methodological choices. The open-source code and the concrete demonstration of a performance-cost trade-off are also strengths. The central risk is that the jury is validated only on MedAgentBrief-style outputs and then applied without direct validation to single-prompt outputs, so the headline benchmarking result could reflect distribution-specific jury behavior. The contradiction-assessment component, also part of MedFactEval, receives no meta-evaluation at all.","major_comments":[{"comment":"The jury's agreement with the seven-physician gold standard is measured only on 60 MedAgentBrief-generated summaries (from GPT-4o and DeepSeek-R1), but the same jury is then used to score Single-Prompt summaries in §4.2 and Figure 1. This transfer is unvalidated: single-prompt outputs differ in format, length, and error profile, and the jury's false-positive or false-negative rates could shift on that distribution. The reported advantage of MedAgentBrief (e.g., GPT-4o from 48% to 65%) is therefore not directly supported by the meta-evaluation. The authors should either provide human labels (or at least a sample-based reliability check) for single-prompt summaries, or explicitly re-frame the §4.2 results as exploratory and conditional on the unverified transfer assumption.","section":"§3.4, §4.2, Figure 1"},{"comment":"The contradiction-assessment component of the LLM Jury is never meta-evaluated against human judgment. The manuscript reports contradiction rates of 10–25% and claims high sensitivity in detecting subtle inconsistencies, but no gold-standard comparison or kappa is reported for the contradiction task. Since contradictions are a central safety concern and a stated part of the framework, the absence of validation should be acknowledged as a limitation, or the contradiction results should be presented as purely illustrative rather than as validated framework output.","section":"§3.3, Step 2; §4.2, paragraph 'Analysis of Factual Errors'"},{"comment":"Key facts were optionally suggested by a Task-Assisting LLM (GPT-4o or Gemini 2.0 Flash) and then reviewed by physicians. Because the same or overlapping model families later serve as jury members, there is a potential information leak: a key fact phrased by GPT-4o may be easier for a GPT-4o-based judge to recognize than facts independently authored by physicians. This does not invalidate the external-gold-standard design, but a sensitivity analysis separating LLM-suggested from physician-authored key facts would strengthen the claim that the jury is measuring clinician-defined facts rather than reproducing its own phrasing.","section":"§3.3, Step 1; §4.1"},{"comment":"The non-inferiority test uses a pre-specified margin of 10% on the kappa difference, but the manuscript does not state whether the one-sided P-value was computed against this margin with a bootstrap percentile, a normal approximation, or another method. The reported values (P<0.001 for a difference of 15% with a 95% CI of -1% to 29%) are plausible, but the exact test should be described to allow replication. This is a local methodological clarity issue rather than a fatal flaw.","section":"§3.4, Statistical Analysis"}],"minor_comments":[{"comment":"The caption states that the blue shaded area represents the 95% CI for a single physician's agreement, but the figure panels are not all referenced with sufficient precision in the text; please ensure each panel (A–D) is explicitly tied to the GPT-4o or DeepSeek-R1 workflow in the caption.","section":"Figure 2"},{"comment":"The discussion of automated fact-checking would benefit from citing a few more recent LLM-as-judge and clinical factuality evaluations beyond VeriFact, MedHallBench, and FactEHR, particularly work that validates LLM judges against clinician panels for generation tasks, to better position the contribution.","section":"§2, Related Work"},{"comment":"The cohort description states that 30 patients were randomly sampled, but the sampling frame and inclusion criteria (LOS 2–14 days, Division of Hospital Medicine) could be described more precisely, including whether the 30 cases are all from a single site and how seasonality was handled.","section":"§3.1"},{"comment":"The key-fact categorization sums to 90, but the category percentages (51% + 40% + 6% + 3% = 100%) are rounded such that the counts (46+36+5+3=90) are not exactly proportional to the displayed percentages; this is a trivial rounding issue but should be reconciled for a journal version.","section":"§4.1, Table 1"},{"comment":"The URLs for Supplementary Data 1 and 2 are not embedded as clickable anchors in the text; please format them as standard hyperlinks and verify that the anonymized samples load without authentication.","section":"Supplementary Data links"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid meta-evaluation of an LLM jury for fact-presence checking, and the core non-inferiority result is well designed. My concern is scope: the abstract and Section 4.2 present the MedAgentBrief advantage as an established benchmark result, but the jury's reliability was only measured on MedAgentBrief outputs. This is fixable by adding a small human-labeled sample for single-prompt summaries or by softening the claim. I also recommend asking the authors to clearly separate the validated fact-presence component from the unvalidated contradiction component. If the authors can provide evidence that the jury transfers across generation strategies, I would support acceptance; without that, the central benchmarking claim remains conditional."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this paper gives clinical NLP a genuinely useful evaluation tool — a ten-model jury checking clinician-defined key facts — and backs it with a real meta-evaluation. The kappa of 0.81 against a seven-physician panel, non-inferior to a single physician, is a meaningful result. It deserves a serious referee.\n\nWhat is new is the package: key facts defined by clinicians, a multi-LLM majority vote, and a validation design that compares against an external gold standard rather than the jury's own outputs. The leave-one-out human baseline and bootstrapped confidence intervals are done properly. MedAgentBrief with provenance tagging is a reasonable workflow, and the performance-cost trade-off in Figure 1 is a practical contribution. Code is open source, which is a plus.\n\nSoft spots, in order of size. First, the jury is validated only on 60 MedAgentBrief summaries (two systems), then used to score Single-Prompt summaries in the benchmark. The transfer is unvalidated: no human labels exist on Single-Prompt outputs. The conclusion that MedAgentBrief improves factual presence could be an artifact of the jury's behavior on that distribution. This does not undercut the central kappa result, but it does weaken the benchmarking claim. The authors should either collect human labels on a sample of Single-Prompt summaries or soften the claim.\n\nSecond, there is a statistical inconsistency in the DeepSeek comparison: they report a kappa difference of 2% (95% CI -11% to 15%) with P<0.001 for non-inferiority at a 10% margin. A 95% CI that crosses -10% should not yield P<0.001. Either the CI is mislabeled (should be 90%) or the p-value is wrong. That needs fixing.\n\nThird, the Small LLM Jury looks post-hoc: the three models were chosen after seeing their individual performance, with no separate validation. Minor, but they should acknowledge it as exploratory.\n\nFourth, the benchmark is small and single-center, and the key facts are subjective (even with optional LLM assistance). They acknowledge these in limitations, so this is minor.\n\nThe citation pattern is fine; self-citations to VeriFact are relevant. The central validation holds up. The transfer issue is the main thing to fix.\n\nWho should read this: anyone building automated evaluation for clinical summarization. It is a useful framework even with the caveats. I would send it to peer review, with a request to address the transfer concern and the DeepSeek stats.","headline":"A solid, useful evaluation framework with a real validation result, but the jury's transfer to single-prompt summaries is unvalidated and one reported p-value looks inconsistent with its confidence interval.","tokens_in":9236,"tokens_out":2964,"would_cite":true,"duration_ms":24658,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that an LLM jury—ten language models voting on whether clinician-specified key facts appear in a draft summary—agrees with a seven-physician gold standard almost perfectly (Cohen's kappa 81%) and is statistically no…","keywords":["clinical text summarization","factuality evaluation","LLM jury","discharge summaries","key fact extraction","Cohen's kappa","non-inferiority testing","clinical AI quality assurance"],"falsifier":"Take a holdout set of single-prompt summaries, have a seven-physician panel label the same key facts, and compare the untuned LLM jury's majority vote to that gold standard; if the kappa falls below the pre-specified non-inferiority margin of 10 percentage points relative to a single physician, then the jury's benchmark scores cannot be trusted for the comparison that drives the paper's main practical claim.","tokens_in":8097,"feed_emoji":"🩺","tokens_out":6742,"duration_ms":53165,"temperature":0.7,"pith_summary":"The paper introduces MedFactEval, a framework for fact-grounded evaluation of AI-generated clinical text, and MedAgentBrief, a multi-step workflow for generating discharge summaries. It claims that a ten-model LLM jury, taking a majority vote on whether clinician-defined key facts are present in a summary, agrees with a seven-physician gold standard with Cohen's kappa of 81%, statistically non-inferior to a single physician reviewer (kappa 67%, P<0.001). The same jury is then used to benchmark generation strategies, showing that MedAgentBrief consistently improves factual presence over a single-prompt baseline at higher inference cost. The intended payoff is a scalable, automated proxy for physician review in continuous quality assurance of clinical AI.","feed_headline":"Ten-LLM jury matches physician fact-checking on clinical summaries","feed_subtitle":"A 10-model majority vote matched a 7-physician panel on key facts, making automated checks viable.","key_machinery":"The load-bearing mechanism is the LLM Jury: ten distinct language models each answer a binary 'is fact X present in this summary?' question, and the majority vote decides the verdict, smoothing the errors of individual judges. The other component is the clinician-defined key fact: instead of scoring a summary globally, physicians specify three high-salience facts per case, converting 'is this summary good?' into concrete, verifiable questions. Agreement is measured with Cohen's kappa against a seven-physician majority gold standard, with non-inferiority tested against a pre-specified margin of 10 percentage points.","core_discovery":"The central discovery is that an ensemble of ten LLMs, each giving a binary verdict on whether a clinician-specified key fact is present in a generated summary, can reproduce physician judgment reliably. On 60 discharge summaries produced by two MedAgentBrief systems, the jury's majority vote agreed with the seven-physician majority at kappa = 81% (95% CI 66–92%), and the authors establish statistical non-inferiority to the average single physician (kappa = 67%, P<0.001). The paper also reports that MedAgentBrief, which refines a draft note-by-note with provenance tagging and a verification pass, raises the fact-presence score of a representative foundation model from 48% to 65%, and that errors of omission dominate over contradictions across all systems tested.","pith_inferences":["If the jury's reliability transfers beyond the 60 MedAgentBrief summaries on which it was validated, the method could replace single-reviewer spot checks in deployment; the paper itself does not yet demonstrate that transfer for single-prompt outputs.","The key-fact framing suggests a testable extension: feed omission reports back to the generation model as few-shot corrections and measure whether factual presence rises across iterations, closing the improvement loop the paper implies but does not run.","A likely boundary condition is that the jury's agreement depends on the clarity of key facts and the summary distribution; institutions with different note styles or patient populations should revalidate locally before relying on the reported kappa."],"forward_implications":["Clinical teams could run continuous quality assurance on LLM-generated drafts by defining a few key facts per case and letting an LLM jury flag omissions and contradictions, rather than paying physicians to review every summary.","The benchmarking result implies that multi-step generation workflows with provenance tagging and verification improve factuality over single-prompt generation for every foundation model tested, at a predictable cost in inference dollars and latency.","Because omissions dominate over contradictions, future development effort should focus on recall of high-salience facts rather than hallucination reduction alone.","The open-source code and benchmark let other institutions build their own local key-fact benchmarks; the authors state the framework generalizes to other clinical information synthesis tasks once key facts are redefined."],"supporting_citations":[{"why":"Shows LLMs can produce clinically usable discharge summaries, establishing the generation capability that needs fact-checking.","marker":"[2]"},{"why":"Captures the regulatory and safety concern about LLM factual errors that motivates a scalable evaluation method.","marker":"[4]"},{"why":"Describes omission and commission errors in AI-generated discharge summaries, the error types MedFactEval measures.","marker":"[5]"},{"why":"Describes prior automated fact-checking against the full record, which MedFactEval reframes by using clinician-defined key facts.","marker":"[9]"},{"why":"Demonstrates that LLMs disagree on automatic fact decomposition, motivating clinician-specified facts rather than automatic ones.","marker":"[10]"},{"why":"Supplies the LLM-as-judge evidence underlying the multi-model jury design.","marker":"[12]"},{"why":"Provides the Cohen's kappa framework used to measure agreement and set the non-inferiority margin.","marker":"[14]"}],"fun_headline_variants":["Ten-LLM jury agrees with 7-physician panel at 81% kappa","Automated clinical fact-checking matches expert judgment","LLM vote non-inferior to doctor on summary accuracy","MedAgentBrief workflow boosts factual completeness by 17 pts","Multi-LLM jury: reliable fact-check without expert review"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The jury's agreement with physicians was measured only on 60 MedAgentBrief-generated summaries, yet the same jury is then used to score single-prompt summaries, so the reported MedAgentBrief advantage depends on the jury's reliability transferring to a different generation strategy without direct validation on that distribution.","fun_headline_variants_meta":{"raw":{"variants":["Ten-LLM jury agrees with 7-physician panel at 81% kappa","Automated clinical fact-checking matches expert judgment","LLM vote non-inferior to doctor on summary accuracy","MedAgentBrief workflow boosts factual completeness by 17 pts","Multi-LLM jury: reliable fact-check without expert review"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000911,"raw_usage":{"total_tokens":3907,"prompt_tokens":931,"completion_tokens":2976,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":2887}},"tokens_in":547,"tokens_out":2976,"duration_ms":18283,"temperature":1.0,"reasoning_tokens":2887,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:20:29.325715+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a holdout set of single-prompt summaries, have a seven-physician panel label the same key facts, and compare the untuned LLM jury's majority vote to that gold standard; if the kappa falls below the pre-specified non-inferiority margin of 10 percentage points relative to a single physician, then the jury's benchmark scores cannot be trusted for the comparison that drives the paper's main practical claim.","supporting_citations":[{"cited_title":"Van Veen, C","cited_arxiv_id":null,"evidence_quote":"Shows LLMs can produce clinically usable discharge summaries, establishing the generation capability that needs fact-checking."},{"cited_title":"Mesk´ o and E","cited_arxiv_id":null,"evidence_quote":"Captures the regulatory and safety concern about LLM factual errors that motivates a scalable evaluation method."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes omission and commission errors in AI-generated discharge summaries, the error types MedFactEval measures."},{"cited_title":"Munnangi, A","cited_arxiv_id":null,"evidence_quote":"Demonstrates that LLMs disagree on automatic fact decomposition, motivating clinician-specified facts rather than automatic ones."},{"cited_title":"Zheng, W.-L","cited_arxiv_id":null,"evidence_quote":"Supplies the LLM-as-judge evidence underlying the multi-model jury design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Cohen's kappa framework used to measure agreement and set the non-inferiority margin."}],"review_version":2}