Pith.

Machine Learning · arXiv:2610.00381 v1 · submitted 30 Sep 2026 · 123 references

OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

Luyao Tang, Cheng Chen

The paper says

“Medical models are judged not only on correctness, but on whether reported confidence matches actual accuracy.”

more of the abstract

Generalist multimodal medical models have expanded what a single model can perceive, yet they still express bounded decisions such as diagnoses, findings or cell counts as generated text, so the reported probability reflects the next token rather than the decision itself. Motivated by decision-native interfaces such as Jev, we introduce OmniMed-Jev, which represents each medical decision as a Choice, Noul or Score decision over a runtime-supplied candidate set and returns a full distribution over that set: mutually exclusive classes, binary presence of a finding, or a bounded ordered value. The design is omni in three respects: it accepts diverse imaging modalities, covers different prediction tasks, and expresses them through one candidate-conditioned probability model, so heterogeneous outputs become comparable probabilities rather than task-specific strings. In an interface-controlled comparison against a generative baseline trained on the same backbone, data and schedule, OmniMed-Jev's reported probabilities track observed correctness far more closely, reducing calibration error by up to an order of magnitude and reliability error by up to two, while point-prediction performance remains comparable; counting is the one family where the generative baseline stays ahead. Making the decision distribution the model's output is not a format change but what turns reported numbers into probabilities that mean what they say. These results support explicit decision modeling as a way to make reported confidence meaningful within the evaluated tasks, and they are not evidence of clinical readiness: the comparison cannot separate the interface from associated training differences, which we state alongside the results. Code is available at github.com/lytang63/OmniMed-Jev.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 3 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. An unusually honest, well-statisted interface-controlled study whose native calibration numbers are credible, but whose headline 9x–126x gains over a generative baseline rest on an inference-time reconstruction whose length bias is never quantified. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 123 references say.

The paper says, at Section 4.4 / Appendix F.2-F.3, Eqs. (5) and (9)

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The headline reductions (3x-26x ECE, 9x-126x reliability) are computed against a baseline distribution reconstructed as a softmax of summed token log-likelihoods. As the authors state in F.3, nothing in the baseline's training makes these log-likelihoods sum to one over the candidate set; the normalization is the authors'.

more of objection 1

The summed score is length-biased: candidates with fewer tokens (short class labels, single-digit counts) systematically receive higher normalized scores than multi-token candidates (multi-digit regression values) irrespective of decision likelihood. The paper discloses the pseudo-distribution status and calls the reconstruction 'generous,' but disclosure does not bound the effect: no sensitivity analysis reports how the 9x-126x gaps change under a length-normalized alternative such as mean per-token log-likelihood, and the claimed favorability of the chosen reconstruction is asserted rather than demonstrated. Without such a re-run, only OmniMed-Jev's native ECE values (0.0806/0.0114/0.0408/0.0913) support the hedged claim; the magnitude figures in the abstract and Section 5.2 are not established.

The paper says, at Table 2 / Section 5.2 / Figure 3, counting row

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.87):

“Two caveats on these numbers (i) The counting cell of the generative baseline is diagnostic, not decisive.On the counting family the top-1 candidate of the reconstructed distribution never equals the gold count:0of126rows.”

Pith says: Objection 2 of 3 · major

The 44x counting reliability reduction and the baseline's resolution=0 / AURC=1.0 in the counting cell are mechanically forced: the reconstructed distribution never places the gold count at top-1 (0 of 126 rows), as the authors acknowledge in F.3 (b516-b518) and Section 5.3.

more of objection 2

Yet the main text (Section 5.2 and Figure 3) still presents '44x on counting' as a headline reliability gain without the caveat inline. The counting calibration cell should either be removed from the headline claims or carry the 'diagnostic, not decisive' caveat at the point of use, since the number is an artifact of the reconstruction on a 33-way grid rather than a property of the interface.

The paper says, at Appendix F.2, multi-label family

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

In the multi-label family — the source of the largest reported gain (126x reliability, 20x AURC) — the baseline's output is rewritten into per-finding binary prompts ('Is <finding> present in this image? Answer Yes or No.'), which is a different interface than the baseline was trained with (semicolon-joined finding strings).

more of objection 3

The paper does not separate the effect of this per-finding rewriting from the reconstruction itself. Because the interface is the treatment under comparison, evaluating the baseline under a prompt structure it never trained on is a correctness risk for the largest gap in the paper; the authors should either justify the binary elicitation as the natural generative query or run a sensitivity check that isolates it.

4 minor objections

The paper says, at Appendix G.2

Pith says: minor objection 1 of 4

The counting block header is rendered verbatim as '4 Choice decisions' while the suite assigns counting a Score decision over an ordered grid (Table 3); the authors note the discrepancy but a one-sentence footnote in the main text (Section 3.2) would prevent reader confusion about the counting task's type assignment.

The paper says, at Table 4 / Section 5.5

Pith says: minor objection 2 of 4

The matched 126-row counting comparison shows OmniMed-Jev's total-count top-1 accuracy is only 0.1508 (vs 0.3234 in the family aggregate) and MAE 2.5159 vs the baseline's 2.1429; this appendix table is more informative than the main-text claim that counting is the one family favoring the baseline, and a cross-reference from Section 5.5 would strengthen the boundary-case discussion.

show 2 more

The paper says, at Title / Abstract

Pith says: minor objection 3 of 4

The phrase 'System One' invokes Kahneman (2011) but the connection is never developed in the method or discussion; either a sentence connecting the decision-interface design to the fast/effortless framing or a removal of the term would tighten the presentation.

The paper says, at Section 4.4

Pith says: minor objection 4 of 4

The sentence calling the reconstruction 'generous' because the baseline receives the same candidate set and prompt conflates two distinct claims: giving the baseline the candidate set is favorable, but the summed-log-likelihood scoring is an unvalidated choice; the two should be stated separately.

What would settle it

Recompute the generative baseline's reconstructed distribution in Table 2 using mean per-token log-likelihood (sum of log-likelihoods divided by the candidate's token count) in place of the raw sum in Eq. (9), holding everything else fixed, and recompute ECE, reliability, and AURC for classification, multi-label, and regression. If multi-label reliability error for Gen-SFT (currently 0.1294) drops below, say, 0.05, or if the 126× reliability gap vs OmniMed-Jev narrows to under 10×, the headline comparison is substantially an artifact of the cumulative-length normalization; if the gaps remain within a factor of two of the reported values, the concern is dismissed.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 3 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 28 s · 33,611 tokens read, 4,288 written
  3. Editor wrote the verdict above 27 s · 34,543 tokens read, 4,020 written
  4. Referee major revision: 3 major objections, 4 minor 44 s · 33,901 tokens read, 6,866 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Two things to know up front. First, this is an honest paper in a way that's rare for this area: it reports its own failed probes (the add_none error signal runs backwards, the coordinate head collapsed), and it states its confounds in the results rather than burying them in a compliance paragraph. Second, the headline 9x–126x calibration gaps are real in the tables but rest on a reconstruction that is likely length-biased, so the magnitude is not established.

What's actually new: a controlled interface comparison for medical multimodal decision modeling — Choice/Noul/Score over a runtime candidate set, one head, one objective — with calibration, Murphy reliability/resolution, and selective risk as the yardsticks instead of accuracy alone. The design genuinely holds backbone, images, and schedule fixed; the one config row that differs is a max_length increase that follows from packing multiple decisions. The stats are done right: paired bootstrap, McNemar/sign tests, Wilson intervals, and a full per-checkpoint appendix. That's competent, careful work, and the claim is scoped correctly — bounded decisions only, no clinical readiness, retrospective data.

The soft spot is the central comparison, and the stress test lands on it. The generative baseline is scored by summed token log-likelihoods, then softmaxed (Eq. 5; Eqs. 9–10). Candidate lengths vary within families — multi-digit regression values, short class labels — and a summed score penalizes longer candidates by construction, independent of decision content. The authors call the result a pseudo-distribution and note the reconstruction is generous, but there is no sensitivity analysis with a length-normalized alternative (mean per-token log-likelihood). So the 3x–26x ECE and 9x–126x reliability reductions could be inflated by the artifact. The counting cell is degenerate by construction (0/126 top-1, forcing resolution 0 and AURC 1.0), which the authors acknowledge; the 44x figure there can't carry weight. The multi-label rewrite into per-finding binary prompts is another interface mismatch the paper records but doesn't quantify. The headline counting table also mixes 201 versus 126 rows before the appendix correction, which they disclose.

None of this kills the hedged claim, which the paper states carefully: decision modeling as implemented gives calibrated, auditable probabilities for bounded decisions at comparable accuracy. That survives on OmniMed-Jev's own native numbers — ECE 0.01–0.09 across families — measured on the trained distribution, not a reconstruction. What's fragile is the magnitude of superiority over the generative baseline and any causal reading about the interface; the paper declines both, explicitly.

Minor: small N (200 and 126), self-constructed splits, comparison at the best-validation checkpoint. Regression is the reminder of why that matters — native ECE is 0.09 at step 2424 but 0.19 at 3685.

Referee it seriously. The reconstruction concern is fixable with one length-normalized sensitivity analysis, and the honesty holds up under scrutiny. I'd cite it for the native calibration result, not for the gap magnitudes.

The strongest objection

The central comparison (Table 2, Eqs. 5 and 9) scores the generative baseline's candidates by the raw sum of token log-likelihoods, then takes a softmax over candidates. Candidate lengths vary substantially within each family (class labels of 1–3 tokens, counts of 1–2 tokens, multi-digit regression values), so the summed score is dominated by length rather than by decision likelihood. Because the baseline was never trained to output a normalized distribution, its ECE, reliability, and AURC under this reconstruction measure an inference-time artifact, not the model's confidence. The paper discloses this in F.3 (calling it a pseudo-distribution) and calls the reconstruction generous, but disclosure does not bound the effect: no sensitivity analysis shows how the 9×–126× gaps change under a length-normalized alternative such as mean per-token log-likelihood. The multi-label family, where the 126× reliability gain is largest, also rewrites the baseline's output into per-finding binary prompts, a different interface than the baseline was trained with, compounding the unfairness. The counting cell is degenerate by construction (0 of 126 top-1 matches, forcing resolution=0 and AURC=1.0), as the authors acknowledge, so the 44× counting gap is not informative. Without a length-normalized re-run, the headline magnitude is not established; only OmniMed-Jev's own native ECE (0.08/0.01/0.04/0.09) stands as evidence for the hedged claim.

The reader's weakest assumption—that the generative baseline's calibration is measurable via the Eq. (5)/(9) reconstruction—is exactly the load-bearing point. I agree, and I refine it: the raw-sum normalization over differing candidate lengths is a concrete, testable flaw that the paper's disclosure ('pseudo-distribution', F.3) does not bound. The paper is unusually honest about the confound of denser supervision and the extra head (Appendix I, b5), and the central hedged claim—that decision modeling as implemented produces calibrated probabilities for bounded decisions—is supported by OmniMed-Jev's own native ECE, independent of the baseline. That part is solid. But the magnitude numbers in the abstract and Section 5.2 ('up to 10× calibration, 126× reliability') rest on comparing a trained distribution to a single chosen reconstruction of an untrained one. No sensitivity analysis over the reconstruction's normalization is reported; the failed elicitation attempts in Table 10 are not the same as testing the chosen estimator's robustness. A length-normalized alternative is the minimal check. If it moves the baseline's reliability by an order of magnitude, the headline claim needs re-scoping; if not, the CONDITIONAL verdict stands with the same conditions. Hence I recommend UNCHANGED and add the test to the reader's condition list. Agreement: agree, since the reader already identified the reconstruction as the weakest assumption; I am specifying the mechanism and a settlement test.

The referee's report

The paper is valuable for three reasons even setting aside the headline: within the stated protocol the statistics are careful (paired bootstrap intervals, Wilson intervals, per-row paired tests, root-level checkpoint reporting); the absolute calibration of OmniMed-Jev's native distribution (ECE 0.01-0.09 across families) is directly established and does not depend on the baseline reconstruction; and the writing is unusually honest, disclosing the pseudo-distribution status of the baseline scores, the degenerate counting cell, the supervision-density confound (155,729 vs 23,558 targets), the failure of an add_none probe, and the removal of an unsuccessful span/coordinate head. If the interface effect is real, the paper provides a controlled demonstration that decision-native outputs make medical model confidence audit-friendly, which is directly relevant to selective prediction and deferral policies. The main risk is that the headline magnitude of the comparison (3x-26x ECE, 9x-126x reliability reductions) rests on a reconstruction whose fairness to the generative baseline is neither validated nor bounded by sensitivity analysis.

Pith's reading of the paper (a machine's summary, not the authors')

A medical model can be accurate and still be unreliable for triage if the confidence it reports does not track how often it is actually right. This paper argues the fault lies in the output interface itself: a model that writes its diagnosis, findings, or cell count as text is reporting the probability of a token, not the probability of the decision. OmniMed-Jev trains the same multimodal backbone to return a full distribution over a candidate set supplied at runtime, and reports that the native probabilities track observed correctness far more closely, cutting expected calibration error by 3x to 26x and Murphy reliability error by 9x to 126x in three of four task families while point accuracy stays statistically indistinguishable from a matched generative baseline. Counting is the one family where the generative baseline stays ahead, which the authors attribute to folding a distribution over a wide ordered grid into an expectation. The authors state plainly that the comparison cannot separate the interface from the denser supervision and additional decision head that come with it, and that the results are not evidence of clinical readiness.

What carries the argument. The central object is the candidate-conditioned decision distribution: each decision is a tuple $(Q, T, C)$ whose candidate set $C = \{c_1, \ldots, c_K\}$ is part of the input, and a shared scoring head consumes hidden states at marked candidate positions to emit a normalized vector $p_\theta(c \mid S, D_j)$ over that set. Three decision types map medical tasks — Choice for mutually exclusive classes, Noul for binary presence of a finding, and Score for a bounded ordered grid folded to an expectation — and all three are trained by one cross-entropy objective. Because candidates are input semantics rather than classifier indices, the decision space can be manipulated at inference time without retraining: options can be permuted, the set restricted, or the annotated candidate removed. This shared probability model is what makes heterogeneous medical outputs expressible, comparable, and directly trainable as probabilities rather than repairable after the fact.

Core claim. The central claim, in the authors' own words, is that "the same backbone, trained to output the decision distribution instead of the answer string, reports numbers that mean what they say." Over four task families — classification, multi-label chest-radiograph findings, cell counting, and ejection-fraction regression — expected calibration error falls from 0.2589/0.2981/0.4204/0.1267 to 0.0806/0.0114/0.0408/0.0913, and Murphy reliability falls from 0.1002/0.1294/0.1824/0.0258 to 0.0110/0.0010/0.0041/0.0205. Point-prediction performance is essentially unchanged: only the counting MAE differs distinguishably from zero at 95% confidence, and that difference favours the generative baseline. The one family where the generative baseline wins is counting, which the authors attribute to a specific mechanism of the Score representation rather than a fundamental limit.

The premise it rests on. The headline gaps measure OmniMed-Jev's trained decision distribution against a pseudo-distribution reconstructed from the generative baseline's summed token log-likelihoods — a quantity the baseline was never trained to produce, and one the authors acknowledge is the more favourable reconstruction they could have chosen.

If the paper is right (Pith's inferences, not the authors')

Escalation and deferral rules can consume per-finding probabilities that behave as specified: on multi-label chest-radiograph findings, reliability error falls from 0.1294 to 0.0010 and selective-risk area from 0.2474 to 0.0126, a twenty-fold reduction in the error a deferral policy carries at every coverage level.

The reliability-versus-resolution split flips: in classification, multi-label, and counting the generative baseline's reliability error exceeds its resolution, while all four OmniMed-Jev families sit on the resolution side of the boundary — the baseline orders cases usefully but in numbers that overstate how often it is right.

Calibration becomes measurable and trainable from the training objective itself rather than recovered from token log-likelihoods after the fact, which makes post-hoc calibration a complementary repair rather than the only option.

Counting degrades because folding a wide ordered-grid distribution to its expectation regresses extreme counts toward the middle; the authors state this as a specific mechanism that is fixable rather than fundamental.

If the interface effect survives the supervision confound, the same candidate-scoring mechanism should carry calibrated confidence for any bounded medical decision beyond the four families tested — a claim the paper does not make, but its omni-decision framing invites.

The cleanest separating experiment the authors did not run is an ablation that downsamples the 139,537 multi-label decision targets to the baseline's 23,558 answer-level signals; that alone would tell whether the calibration gains come from the interface or from simply seeing more supervision per image.

The failed 'none' probe — p(none) is higher on correct decisions than wrong ones and its AUC as an error predictor is reversed at 0.13 — suggests the architecture has no handle for expressing 'no candidate fits', which a calibration-aware objective that rewards null mass on mistakes might supply.

A medical model that must produce narrative output stays generative for that part; the boundary worth probing is where a bounded decision nests inside a free-form report, since the interface's gains apply only where a candidate set exists.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Medical models are judged not only on correctness, but on whether reported confidence matches actual accuracy. Generalist multimodal medical models have expanded what a single model can perceive, yet they still express bounded decisions such as diagnoses, findings or cell counts as generated text, so the reported probability reflects the next token rather than the decision itself. Motivated by decision-native interfaces such as Jev, we introduce OmniMed-Jev, which represents each medical decision as a Choice, Noul or Score decision over a runtime-supplied candidate set and returns a full distribution over that set: mutually exclusive classes, binary presence of a finding, or a bounded ordered value. The design is omni in three respects: it accepts diverse imaging modalities, covers different prediction tasks, and expresses them through one candidate-conditioned probability model, so heterogeneous outputs become comparable probabilities rather than task-specific strings. In an interface-controlled comparison against a generative baseline trained on the same backbone, data and schedule, OmniMed-Jev's reported probabilities track observed correctness far more closely, reducing calibration error by up to an order of magnitude and reliability error by up to two, while point-prediction performance remains comparable; counting is the one family where the generative baseline stays ahead. Making the decision distribution the model's output is not a format change but what turns reported numbers into probabilities that mean what they say. These results support explicit decision modeling as a way to make reported confidence meaningful within the evaluated tasks, and they are not evidence of clinical readiness: the comparison cannot separate the interface from associated training differences, which we state alongside the results. Code is available at github.com/lytang63/OmniMed-Jev.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-02

The authors' main claim, as the classifier picked it out 0.77

“These results support explicit decision modeling as a way to make reported confidence meaningful within the evaluated tasks, and they are not evidence of clinical readiness: the comparison cannot separate the interface from associated training differences, which we state alongside the results.”

How the main claim is stated: asserted 0.61

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.98 · human participants 0.13 · controlled comparison 0.97 · randomized assignment 0.02 · states quantitative result 0.96 · compares with prior results 0.97 · mentions code or data availability 0.99

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Medical models are judged not only on correctness, but on whether reported confidence matches actual accuracy. background 1.00 0.05 0.05 asserted 1.00
Generalist multimodal medical models have expanded what a single model can perceive, yet they still express bounded decisions such as diagnoses, findings or cell counts as generated text, so the reported probability reflects the next token rather than the decision itself. background 0.57 0.12 0.03 asserted 1.00
Motivated by decision-native interfaces such as Jev, we introduce OmniMed-Jev, which represents each medical decision as a Choice, Noul or Score decision over a runtime-supplied candidate set and returns a full distribution over that set: mutually exclusive classes, binary presence of a finding, or a bounded ordered value. contribution 0.62 0.82 0.07 asserted 0.99
The design is omni in three respects: it accepts diverse imaging modalities, covers different prediction tasks, and expresses them through one candidate-conditioned probability model, so heterogeneous outputs become comparable probabilities rather than task-specific strings. method 0.64 0.70 0.05 asserted 1.00
In an interface-controlled comparison against a generative baseline trained on the same backbone, data and schedule, OmniMed-Jev's reported probabilities track observed correctness far more closely, reducing calibration error by up to an order of magnitude and reliability error by up to two, while point-prediction performance remains comparable; counting is the one family where the generative baseline stays ahead. result 1.00 0.96 0.50 asserted 1.00
Making the decision distribution the model's output is not a format change but what turns reported numbers into probabilities that mean what they say. interpretation 0.97 0.33 0.61 asserted 1.00
These results support explicit decision modeling as a way to make reported confidence meaningful within the evaluated tasks, and they are not evidence of clinical readiness: the comparison cannot separate the interface from associated training differences, which we state alongside the results. limitation 0.82 0.74 0.52 asserted 0.61
Code is available at github.com/lytang63/OmniMed-Jev. contribution 0.59 0.32 0.02 asserted 0.92

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-02: 1 request, 6,523 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. OmniMed-Jev rethinks medical MLLMs as System One decision models. (a) Generalist medical MLLMs already span heterogeneous modalities and tasks, yet bounded predictions with fundamentally different semantics are serialized through the same autoregressive text interface. Classification, multi-label recognition, counting, and regression therefore share the same generative output mechanism despite requiring distinct decision semantics. (b) OmniMed-Jev replaces unnecessary generation with typed probabilistic decisions over runtime-specified candidate spaces. The shared medical backbone…
Figure 2 of the paper
Figure 2. Learning dynamics across four task families. All ten matched test checkpoints are shown; the dashed line and enlarged markers mark the comparison step. Counting and regression use logarithmic MAE axes to retain early outliers without hiding later differences. The interfaces differ in trajectory as well as in endpoint: the generative baseline leads early in classification and is overtaken after step 1616, while in regression it passes through a large mid-training excursion that OmniMed-Jev does not exhibit. candidate set, p˜(cjk) = expP t log pLM y (jk) t | y (jk) <t , S, Dj  P l…
Figure 3 of the paper
Figure 3. Break-even boundary for top-label probability skill. The dashed boundary is resolution = reliability in the Murphy decomposition; green marks resolution > reliability and pale rust the opposite. All four OmniMed-Jev families are on the green side; the generative baseline crosses to the rust side in classification, multi-label recognition and counting. The boundary is relative to each model’s own top-label correctness rate, not a clinical decision threshold. question a clinician asks of a reported probability: when the model says it is right with probability p, is it right about p…
Figure 4 of the paper
Figure 4. Selective prediction at reported coverage levels. Points show top-label error when the model answers the most confident 50%, 80%, 90%, 95% and 100% of decisions; lines connect only those reported aggregates, not a reconstructed full curve. Panel AURC pairs are OmniMed-Jev / Gen-SFT. For counting the baseline’s reconstruction orders no decision correctly, so its curve is flat at an error of 1.0, which reflects the reconstruction of Section 4.4 rather than generative confidence in general. The multi-label family is where the interface pays off most, because there the decisions are…

Figures are the authors', reproduced from the paper.

References (123)
  1. [1] Sellergren, Andrew and Kazemzadeh, Sahar and Jaroensri, Tiam and Kiraly, Atilla and Traverse, Madeleine and Kohlberger, Timo and Xu, Shawn and Jamil, Fayaz and others , year =. 2507.05201 , archivePrefix = on PithPith's reviewarXiv
  2. [2] 2026 , howpublished = on Pith
  3. [3] Team, Gemma and Kamath, Aishwarya and Ferret, Johan and Pathak, Shreya and others , year =. 2503.19786 , archivePrefix = on PithPith's reviewarXiv
  4. [4] Singhal, Karan and Azizi, Shekoofeh and Tu, Tao and Mahdavi, S. Sara and Wei, Jason and Chung, Hyung Won and Scales, Nathan and Tanwani, Ajay and Cole-Lewis, Heather and Pfohl, Stephen and Payne, Perry and Seneviratne, Martin and Gamble, Paul and Kelly, Chris and Babiker, Abub... on Pith
  5. [5] Singhal, Karan and Tu, Tao and Gottweis, Juraj and Sayres, Rory and Wulczyn, Ellery and Amin, Mohamed and Hou, Le and Clark, Kevin and Pfohl, Stephen R. and Cole-Lewis, Heather and Neal, Darlene and Rashid, Qazi Mamunur and Schaekermann, Mike and Wang, Amy and Dash, Dev and Ch... on Pith
  6. [6] Li, Chunyuan and Wong, Cliff and Zhang, Sheng and Usuyama, Naoto and Liu, Haotian and Yang, Jianwei and Naumann, Tristan and Poon, Hoifung and Gao, Jianfeng , booktitle =. 2023 , url = on Pith
  7. [7] Moor, Michael and Huang, Qian and Wu, Shirley and Yasunaga, Michihiro and Dalmia, Yash and Leskovec, Jure and Zakka, Cyril and Reis, Eduardo Pontes and Rajpurkar, Pranav , booktitle =. 2023 , pages = on Pith
  8. [8] Zhang, Kai and Zhou, Rong and Adhikarla, Eashan and Yan, Zhiling and Liu, Yixin and Yu, Jun and Liu, Zhengliang and Chen, Xun and Davison, Brian D. and Ren, Hui and Huang, Jing and Chen, Chen and Zhou, Yuyin and Fu, Sunyang and Liu, Wei and Liu, Tianming and Li, Xiang and Chen... on Pith
  9. [9] Chen, Zhihong and Varma, Maya and Xu, Justin and Paschali, Magdalini and Veen, Dave Van and Johnston, Andrew and others , year =. 2401.12208 , archivePrefix = on PithPith's reviewarXiv
  10. [10] Bannur, Shruthi and Bouzid, Kenza and Castro, Daniel C. and Schwaighofer, Anton and Thieme, Anja and Bond-Taylor, Sam and others , year =. 2406.04449 , archivePrefix = on PithPith's reviewarXiv
  11. [11] Zambrano Chaves, Juan Manuel and Huang, Shih-Cheng and Xu, Yanbo and Xu, Hanwen and Usuyama, Naoto and Zhang, Sheng and Wang, Fei and Xie, Yujia and Khademi, Mahmoud and Yang, Ziyi and Awadalla, Hany and Gong, Julia and Hu, Houdong and Yang, Jianwei and Li, Chunyuan and Gao, J... on Pith
  12. [12] Deitke, Matt and Clark, Christopher and Lee, Sangho and Tripathi, Rohun and Yang, Yue and Park, Jae Sung and others , year =. 2409.17146 , archivePrefix = on PithPith's reviewarXiv
  13. [13] Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and others , year =. 2409.12191 , archivePrefix = on PithPith's reviewarXiv
  14. [14] Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae , booktitle =. 2023 , url = on Pith
  15. [15] Pan, Jiazhen and Liu, Che and Wu, Junde and Liu, Fenglin and Zhu, Jiayuan and Li, Hongwei Bran and Chen, Chen and Ouyang, Cheng and Rueckert, Daniel , year =. 2502.19634 , archivePrefix = on PithPith's reviewarXiv
  16. [16] Yang, Jiancheng and Shi, Rui and Wei, Donglai and Liu, Zequan and Zhao, Lin and Ke, Bilian and Pfister, Hanspeter and Ni, Bingbing , journal = on Pith
  17. [17] Liu, Jingyu and Lian, Jie and Yu, Yizhou , year =. 2006.10550 , archivePrefix = on PithPith's reviewarXiv
  18. [18] Gan, Lu and Li, Xi and Wang, Xichun , journal = on Pith
  19. [19] Leclerc, Sarah and Smistad, Erik and Pedrosa, Joao and Ostvik, Andreas and Cervenansky, Frederic and Espinosa, Florian and Espeland, Torvald and Berg, Erik Andreas Rye and Jodoin, Pierre-Marc and Grenier, Thomas and Lartizien, Carole and Dhooge, Jan and Lovstakken, Lasse and B... on Pith
  20. [20] 2025 , howpublished = on Pith
show 80 more
  1. [21] Liu, Bo and Zhan, Li-Ming and Xu, Li and Ma, Lin and Yang, Yan and Wu, Xiao-Ming , booktitle =. 2021 , pages = on Pith
  2. [22] Lau, Jason J. and Gayen, Soumya and Ben Abacha, Asma and Demner-Fushman, Dina , journal = on Pith
  3. [23] Pal, Ankit and Umapathi, Logesh Kumar and Sankarasubbu, Malaikannan , booktitle =. 2022 , pages = on Pith
  4. [24] Jin, Di and Pan, Eileen and Oufattole, Nassim and Weng, Wei-Hung and Fang, Hanyi and Szolovits, Peter , journal = on Pith
  5. [25] Arora, Rahul K. and Wei, Jason and Soskin Hicks, Rebecca and Bowman, Preston and Qui\ nonero-Candela, Joaquin and Tsimpourlas, Foivos and Sharman, Michael and Shah, Meghan and Vallone, Andrea and Beutel, Alex and Heidecke, Johannes and Singhal, Karan , year =. 2505.08775 , arc... on PithPith's reviewarXiv
  6. [26] Guo, Chuan and Pleiss, Geoff and Sun, Yu and Weinberger, Kilian Q. , booktitle =. 2017 , pages = on Pith
  7. [27] BRIER, GLENN W. , journal = on Pith
  8. [28] Murphy, Allan H. , journal = on Pith
  9. [29] Geifman, Yonatan and El-Yaniv, Ran , booktitle =. 2017 , url = on Pith
  10. [30] Chow, C. , journal = on Pith
  11. [31] Angelopoulos, Anastasios N. and Bates, Stephen , journal = on Pith
  12. [32] Kadavath, Saurav and Conerly, Tom and Askell, Amanda and Henighan, Tom and Drain, Dawn and Perez, Ethan and others , year =. 2207.05221 , archivePrefix = on PithPith's reviewarXiv
  13. [33] Lin, Stephanie and Hilton, Jacob and Evans, Owain , journal =. 2022 , url = on Pith
  14. [34] Tian, Katherine and Mitchell, Eric and Zhou, Allan and Sharma, Archit and Rafailov, Rafael and Yao, Huaxiu and Finn, Chelsea and Manning, Christopher D. , booktitle =. 2023 , pages = on Pith
  15. [35] Xiong, Miao and Hu, Zhiyuan and Lu, Xinyang and Li, Yifei and Fu, Jie and He, Junxian and Hooi, Bryan , booktitle =. 2024 , url = on Pith
  16. [36] Zhao, Tony Z. and Wallace, Eric and Feng, Shi and Klein, Dan and Singh, Sameer , booktitle =. 2021 , pages = on Pith
  17. [37] Kuhn, Lorenz and Gal, Yarin and Farquhar, Sebastian , booktitle =. 2023 , eprint = on Pith
  18. [38] Mozannar, Hussein and Sontag, David , booktitle =. 2020 , pages = on Pith
  19. [39] Verma, Rajeev and Barrejon, Daniel and Nalisnick, Eric , booktitle =. 2023 , pages = on Pith
  20. [40] Malinin, Andrey and Gales, Mark , booktitle =. 2021 , url = on Pith
  21. [41] Leng, Jixuan and Huang, Chengsong and Zhu, Banghua and Huang, Jiaxin , booktitle =. 2025 , eprint = on Pith
  22. [42] Xie, Johnathan and Chen, Annie S. and Lee, Yoonho and Mitchell, Eric , booktitle =. 2024 , pages = on Pith
  23. [43] Kapoor, Sanyam and Gruver, Nate and Roberts, Manley and Collins, Katherine and Pal, Arka and Bhatt, Umang and Weller, Adrian and Dooley, Samuel and Goldblum, Micah and Wilson, Andrew Gordon , booktitle =. 2024 , url = on Pith
  24. [44] Xiao, Jiancong and Hou, Bojian and Wang, Zhanliang and Jin, Ruochen and Long, Qi and Su, Weijie J. and Shen, Li , booktitle =. 2025 , pages = on Pith
  25. [45] Wang, Ziming and Shi, Zeyu and Zhou, Haoyi and Gao, Shiqi and Sun, Qingyun and Li, Jianxin , booktitle =. 2025 , pages = on Pith
  26. [46] Hu, Tiancheng and Minixhofer, Benjamin and Collier, Nigel , booktitle =. 2026 , pages = on Pith
  27. [47] Sanz-Guerrero, Mario and Mager, Manuel and von der Wense, Katharina , booktitle =. 2026 , pages = on Pith
  28. [48] Zhou, Kaitlyn and Hwang, Jena D. and Ren, Xiang and Sap, Maarten , booktitle =. 2024 , pages = on Pith
  29. [49] Chhikara, Prateek , journal =. 2025 , url = on Pith
  30. [50] Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =. 2022 , eprint = on Pith
  31. [51] Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , booktitle =. 2023 , url = on Pith
  32. [52] Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and As... on Pith
  33. [53] Damani, Mehul and Puri, Isha and Slocum, Stewart and Shenfeld, Idan and Choshen, Leshem and Kim, Yoon and Andreas, Jacob , booktitle =. 2026 , eprint = on Pith
  34. [54] Bani-Harouni, David and Pellegrini, Chantal and Stangel, Paul and \"Ozsoy, Ege and Zaripova, Kamilia and Navab, Nassir and Keicher, Matthias , booktitle =. 2026 , eprint = on Pith
  35. [55] Ma, Zhengzhao and Wen, Xueru and Cao, Boxi and Lu, Yaojie and Lin, Hongyu and Yang, Jinglin and He, Min and Han, Xianpei and Sun, Le , booktitle =. 2026 , eprint = on Pith
  36. [56] Wang, Ziqi and Lou, Xingzhou and Wu, Meiqi and Wen, Zhengqi and Zhang, Junge , booktitle =. 2026 , pages = on Pith
  37. [57] Li, Yibo and Xiong, Miao and Wu, Jiaying and Hooi, Bryan , booktitle =. 2025 , pages = on Pith
  38. [58] Tan, Chee Heng and Lin, Zhuoyi and Motani, Mehul and Lee, Wee Sun , year =. 2607.04332 , archivePrefix = on PithPith's reviewarXiv
  39. [59] Singh, Sadanand and Reddy, Allam and Chopra, Manan , year =. 2607.00164 , archivePrefix = on PithPith's reviewarXiv
  40. [60] Yaldiz, Duygu Nur and Spiliopoulou, Evangelia and Qi, Zheng and Varia, Siddharth and Doss, Srikanth and Pappas, Nikolaos , booktitle =. 2026 , pages = on Pith
  41. [61] Savage, Thomas and Wang, John and Gallo, Robert and Boukil, Abdessalem and Patel, Vishwesh and Safavi-Naini, Seyed Amir Ahmad and Soroush, Ali and Chen, Jonathan H , journal = on Pith
  42. [62] Boie, Sebastian Daniel and Reis, Florian and Frey, Nicolas and Gr\"unewald, Elias and Balzer, Felix , journal = on Pith
  43. [63] Bentegeac, Rapha\"el and Le Guellec, Bastien and Kuchcinski, Gr\'egory and Amouyel, Philippe and Hamroun, Aghiles , journal = on Pith
  44. [64] Omar, Mahmud and Agbareia, Reem and Glicksberg, Benjamin S and Nadkarni, Girish N and Klang, Eyal , journal = on Pith
  45. [65] Naderi, Nariman and Safavi-Naini, Seyed Amir Ahmad and Savage, Thomas and Khalafi, Mohammad Amin and Lewis, Peter R. and Atf, Zahra and Nadkarni, Girish and Soroush, Ali , journal = on Pith
  46. [66] Griot, Maxime and Hemptinne, Coralie and Vanderdonckt, Jean and Yuksel, Demet , journal = on Pith
  47. [67] Steyvers, Mark and Tejeda, Heliodoro and Kumar, Aakriti and Belem, Catarina and Karny, Sheer and Hu, Xinyue and Mayer, Lukas W. and Smyth, Padhraic , journal = on Pith
  48. [68] Bedi, Suhana and Liu, Yutong and Orr-Ewing, Lucy and Dash, Dev and Koyejo, Sanmi and Callahan, Alison and Fries, Jason A. and Wornow, Michael and Swaminathan, Akshay and Lehmann, Lisa Soleymani and Hong, Hyo Jung and Kashyap, Mehr and Chaurasia, Akash R. and Shah, Nirav R. and... on Pith
  49. [69] Zhang, Li and W\"olflein, Georg and Ferber, Dyke and Liang, Junhao and Carrero, Zunamys I. and Wu, Xuewei and Vibert, Julien and Clusmann, Jan and M\"ohrmann, Lino and M\"ohrmann, Elena E. and Wichmann, Catharina and Wolf, Fabian and Lenz, Tim and Kather, Jakob Nikolas , journal = on Pith
  50. [70] Kim, Young-Tak and Kim, Hyunji and Bahl, Manisha and Lev, Michael H. and Gonz\'alez, Ramon Gilberto and Gee, Michael S. and Do, Synho , journal = on Pith
  51. [71] Van Calster, Ben and Collins, Gary S and Vickers, Andrew J and Wynants, Laure and Kerr, Kathleen F and Barre\ nada, Lasai and Varoquaux, Gael and Singh, Karandeep and Moons, Karel GM and Hernandez-Boussard, Tina and Timmerman, Dirk and McLernon, David J and van Smeden, Maarten... on Pith
  52. [72] Banerji, Christopher R. S. and Chakraborti, Tapabrata and Harbron, Chris and MacArthur, Ben D. , journal = on Pith
  53. [73] Tahermazandarani, Maryam and Mahmood, Adnan and Islam, Fahmida and Sheng, Quan Z. , year =. 2608.09080 , archivePrefix = on PithPith's reviewarXiv
  54. [74] Kim, Yubin and Jeong, Hyewon and Chen, Shan and Li, Shuyue Stella and Park, Chanwoo and Lu, Mingyu and others , year =. 2503.05777 , archivePrefix = on PitharXiv
  55. [75] Vazhentsev, Artem and Sviridov, Ivan and Barseghyan, Alvard and Kuzmin, Gleb and Panchenko, Alexander and Nesterov, Aleksandr and Shelmanov, Artem and Panov, Maxim , journal = on Pith
  56. [76] Testoni, Alberto and Calixto, Iacer , booktitle =. 2026 , pages = on Pith
  57. [77] Du, Yuetian and Wang, Yucheng and Kong, Ming and Liang, Tian and Long, Qiang and Chen, Bingdi and Zhu, Qiang , booktitle =. 2025 , pages = on Pith
  58. [78] Byun, Ji Young and Park, Young-Jin and Corbeil, Jean-Philippe and Ben Abacha, Asma , year =. 2604.02543 , archivePrefix = on PithPith's reviewarXiv
  59. [79] Sharifdeen, Ashshak and Ahamed, Shihab Aaqil and Khan, Ufaq and Munir, Muhammad Akhtar and Ibrahim, Sujair and Ahamed, Mohamed Rafeek Mareer and Xie, Yutong and Razzak, Imran and Khan, Muhammad Haris , year =. 2608.27004 , archivePrefix = on PithPith's reviewarXiv
  60. [80] Guo, Kevin H. and Yan, Chao and Baidya, Avinash and Brown, Katherine and Gao, Xiang and Xiong, Juming and Yin, Zhijun and Malin, Bradley A. , year =. 2605.01011 , archivePrefix = on PithPith's reviewarXiv
  61. [81] Sadanandan, Binesh and Behzadan, Vahid , year =. 2603.20985 , archivePrefix = on PitharXiv
  62. [82] Jabbour, Sarah and Fouhey, David and Banovic, Nikola and Shepard, Stephanie D. and Kazerooni, Ella and Sjoding, Michael W. and Wiens, Jenna , year =. 2508.07617 , archivePrefix = on PithPith's reviewarXiv
  63. [83] Machcha, Sravanthi and Yerra, Sushrita and Gupta, Sahil and Sahoo, Aishwarya and Sultana, Sharmin and Yu, Hong and Yao, Zonghai , booktitle =. 2026 , pages = on Pith
  64. [84] Cocchieri, Alessio and Ragazzi, Luca and Tagliavini, Giuseppe and Moro, Gianluca , booktitle =. 2026 , pages = on Pith
  65. [85] Fanconi, Claudio and van der Schaar, Mihaela , year =. 2506.11887 , archivePrefix = on PitharXiv
  66. [86] Lechuga Lopez, L. Julian and Shamout, Farah E. and Rudner, Tim G. J. , booktitle =. 2026 , pages = on Pith
  67. [87] Mehrtens, Hendrik and Bucher, Tabea and Brinker, Titus J. , booktitle =. 2023 , pages =. 2506.18162 , archivePrefix = on PithPith's reviewarXiv
  68. [88] Strong, Joshua and Men, Qianhui and Noble, J. Alison , booktitle =. 2025 , pages = on Pith
  69. [89] Sun, Emma and Strong, Joshua and Noble, Alison , year =. 2609.26384 , archivePrefix = on PithPith's reviewarXiv
  70. [90] Wong, Andrew and Otles, Erkin and Donnelly, John P. and Krumm, Andrew and McCullough, Jeffrey and DeTroyer-Cooley, Olivia and Pestrue, Justin and Phillips, Marie and Konye, Judy and Penoza, Carleen and Ghous, Muhammad and Singh, Karandeep , journal = on Pith
  71. [91] Qazi, Ihsan Ayyub and Ali, Ayesha and Khawaja, Asad Ullah and Akhtar, Muhammad Junaid and Sheikh, Ali Zafar and Alizai, Muhammad Hamad , journal = on Pith
  72. [92] Jabbour, Sarah and Fouhey, David and Shepard, Stephanie and Valley, Thomas S. and Kazerooni, Ella A. and Banovic, Nikola and Wiens, Jenna and Sjoding, Michael W. , journal = on Pith
  73. [93] Dratsch, Thomas and Chen, Xue and Rezazade Mehrizi, Mohammad and Kloeckner, Roman and M\"ahringer-Kunz, Aline and P\"usken, Michael and Bae. Radiology , year = 2023, volume = on Pith
  74. [94] Shekar, Shruthi and Pataranutaporn, Pat and Sarabu, Chethan and Cecchi, Guillermo A. and Maes, Pattie , journal = on Pith
  75. [95] Xu, Xuhai 'Orson' and Hu, Haoyu and Zhang, Haoran and Wang, Will Ke and Wang, Reina and Soenksen, Luis R. and Badri, Omar and Jafry, Sheharbano and Burger, Elise and Nwandu, Lotanna and Mehta, Apoorva and Duhaime, Erik P. and Qasim, Asif and Lin, Hause and Pereira, Janis Karle... on Pith
  76. [96] Nicolson, Angus and Bradburn, Elizabeth and Gal, Yarin and Papageorghiou, Aris T. and Noble, J. Alison , journal = on Pith
  77. [97] Kim, Sunnie S. Y. and Liao, Q. Vera and Vorvoreanu, Mihaela and Ballard, Stephanie and Vaughan, Jennifer Wortman , booktitle =. 2024 , pages = on Pith
  78. [98] 2024 , howpublished = on Pith
  79. [99] 2024 , howpublished =. doi:10.6028/NIST.AI.600-1 , url = on Pithdoi:10.6028/nist.ai.600-1
  80. [100] Gallifant, Jack and Afshar, Majid and Ameen, Saleem and Aphinyanaphongs, Yindalon and Chen, Shan and Cacciamani, Giovanni and Demner-Fushman, Dina and Dligach, Dmitriy and Daneshjou, Roxana and Fernandes, Chrystinne and Hansen, Lasse Hyldig and Landman, Adam and Lehmann, Lisa ... on Pith

The first 100 of 123 references are listed here. All of them are in this paper's citation record.

Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment