{"id":"b71f0510-7710-458d-9270-be73efd440a9","arxiv_id":"2607.19624","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Using pathologists' eye movements as a training signal improves prostate pathology report generation and makes model attention more human-like.","lead":"Pathologists' visual attention, recorded while they examine prostate slides, is used as extra training supervision for AI report generators. This nudges the models' attention toward the same regions and improves report quality, though the evidence comes from small test sets without released data or code.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WSI-LLaVA evaluation uses 11 test examples; reported accuracy gains reflect single-case changes, leaving the central claim of consistent improvement statistically unsupported.","rationale":"The reader's verdict (CONDITIONAL) is appropriate, and the reader's rationale already lists small test size and unvalidated LLM metric among its concerns. However, the reader's stated weakest_assumption focuses on the fidelity of the human saliency map (Section 2), not on the reliability of the evaluation. In my view, the most load-bearing issue is the empirical evidence itself: without confidence intervals or significance tests, the reported improvements—especially for WSI-LLaVA with n=11—cannot be distinguished from noise. Even if the human attention proxy were perfect, a statistically fragile evaluation would prevent any strong conclusion about the alignment loss's benefit. Conversely, a statistically robust evaluation might still rescue the claim even if the saliency map has imperfections, as long as the proxy is informative. Therefore, the evaluation reliability is the first gate for the central claim. I therefore partially agree with the reader: the saliency fidelity is a concern, but the small-sample evaluation is more immediately decisive. The verdict remains CONDITIONAL: the paper should require significance testing, metric validation, and ideally a larger test set before acceptance. No change to the reader's verdict is needed, but the conditions should emphasize these statistical requirements.","tokens_in":8134,"tokens_out":5909,"duration_ms":62777,"concrete_test":"Run a bootstrap resampling on the WSI-LLaVA test set (n=11) to compute 95% confidence intervals for ACC-G, ACC-PNI, ACC-EPE, and BLEU-4 differences between 'w/o alignment' and 'Ours'. If intervals overlap zero (or differences are within one example), the reported improvements are not significant. Independently, have a pathologist manually score Gleason, PNI, EPE, and margin presence on a random sample of 20 generated and ground-truth reports and compare to ChatGPT-5.2 extractions; if agreement is poor (e.g., Cohen's kappa below 0.6), the accuracy columns in Table 1 are not reliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that attention alignment with pathologist gaze yields the best performance on both HistGen and WSI-LLaVA (Table 1)—rests on empirical comparisons that are not statistically reliable. For WSI-LLaVA, report-quality metrics are computed on only 11 test examples (§4.1). In Table 1, ACC-G rises from 54.5% to 63.6% (6/11 to 7/11 correct) and ACC-PNI from 81.8% to 90.9% (9/11 to 10/11); each improvement is a single example. The claimed '19.3% average accuracy gain' is thus driven by one or two positive cases. No confidence intervals or significance tests accompany any result. Moreover, component accuracy is extracted by ChatGPT-5.2, an LLM-based metric with no reported validation against pathologist annotations; if Chinese extraction is noisy, exact-match scores may be meaningless. NLP metrics on the same 11 samples are similarly high-variance. Since the improvements are within the noise floor for such small samples, the empirical claim that the alignment loss causes the gains is not currently supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an auxiliary attention-alignment loss for pathology report generation. The authors collect a multimodal pathologist-attention dataset on 121 TCGA-PRAD prostate WSIs, with synchronized viewport trajectories and verbal descriptions. Viewport centers are converted into multi-scale, Gaussian-smoothed saliency maps, and a KL-divergence loss (Eq. 4) regularizes model attention over image tokens to match these human attention maps. The loss is added to the next-token objective for two models with different fusion mechanisms: HistGen (cross-attention) and WSI-LLaVA (self-attention). Experiments report gains in NLP-based metrics, component-level accuracy extracted by an LLM, and better agreement between inference-time model attention and pathologist attention. The central claim is that aligning model attention to expert gaze improves both report quality and interpretable visual grounding.","tokens_in":8421,"tokens_out":5268,"duration_ms":52866,"significance":"If the empirical claims held, this would be a useful and low-cost contribution: expert gaze would act as an auxiliary supervision signal for pathology vision-language models, and inference-time attention maps would provide interpretable visual evidence. The collected dataset of synchronized viewport trajectories and verbal descriptions is a novel resource, and the proposed loss is simple and architecture-agnostic. The paper also evaluates two common fusion designs, which strengthens the generality of the proposed method. However, the current evidence is not yet statistically convincing: the WSI-LLaVA evaluation rests on 11 test examples, the LLM-based metric extractor is unvalidated, and the visual-support evaluation partly measures the same quantity optimized by the training loss. The underlying idea is plausible and the issues are addressable, but the strength of the claims must be revised to match the evidence.","major_comments":[{"comment":"Report-quality results for WSI-LLaVA are computed on 11 test examples. In Table 1, ACC-G improves from 54.5% to 63.6%, ACC-PNI from 81.8% to 90.9%, and ACC-EPE from 45.5% to 63.6% — corresponding to 1, 1, and 2 corrected samples, respectively. The abstract's 19.3% average accuracy gain is thus driven by a handful of cases. No confidence intervals, bootstrap, or significance tests accompany any metric. This is insufficient support for the claim that attention alignment consistently improves report quality. Please either evaluate on a substantially larger test set, provide uncertainty estimates (e.g., binomial confidence intervals), or temper the claims accordingly.","section":"§4.1, Table 1"},{"comment":"The component exact-match scores (ACC-G, ACC-PNI, ACC-EPE, ACC-S) are extracted by ChatGPT-5.2 from generated and ground-truth reports, but no validation of this extractor is reported. Extraction errors propagate directly into the main quantitative results. Please validate the ChatGPT-5.2 extractor against manual or pathologist-annotated labels on a held-out subset, or report an agreement metric, and state the prompt/version used.","section":"§4.1 Metrics"},{"comment":"The human attention vector a_human is derived from 20 Hz viewport-center sampling, duplicate removal, positive-label filtering, Gaussian blur, and multi-magnification merging. The Gaussian sigma/kernel size and magnification-merge weights are not specified, and the transformation from scanpath to a faithful token-level target distribution is assumed rather than validated. Because Eq. (4) trains the model to match this distribution, any bias in the saliency proxy is directly learned. Please report the exact construction parameters and provide a sensitivity analysis (e.g., varying sigma and merge weights) on a validation criterion such as component accuracy.","section":"§2, §3.2 Eq. (4)"},{"comment":"The NSS and KL improvements in Table 2 measure agreement between model attention and the same pathologist-attention maps used to define L_KL. Consequently, the NSS/KL gains are, to a substantial degree, an expected consequence of the training objective rather than independent evidence of better visual grounding. The ROC-AUC against Gleason segmentation is the non-circular signal, but the observed gains (0.500→0.537 for HistGen; 0.598→0.615 for WSI-LLaVA) are small and unaccompanied by uncertainty estimates. Please present ROC-AUC as the primary visual-support evidence and add confidence intervals or significance tests.","section":"§3.2, Table 2"},{"comment":"The abstract and introduction claim gains 'across five clinically relevant report components,' but §4.1 states that intraductal carcinoma is omitted because it does not appear in ground-truth reports, and Table 1 does not report ACC-S for WSI-LLaVA. In fact, component accuracies are reported for four components on HistGen and three on WSI-LLaVA. Please restate the claims to match the components actually evaluated, or provide additional annotations to cover the missing components.","section":"Abstract, §4.1"}],"minor_comments":[{"comment":"Tables 3 and 4 are identical in content and title ('Ablation on attention-extraction layer'). Please remove the duplicate and refer to a single table.","section":"Table 3 / Table 4"},{"comment":"The column header 'ACC-EDE' is inconsistent with the text's 'ACC-EPE' (extraprostatic extension). Please correct.","section":"Table 1"},{"comment":"The parenthetical '(Section 3.2)' for the saliency smoothing procedure should refer to Section 2, where human attention map construction is described.","section":"§3.3"},{"comment":"The text does not state whether the dataset, code, or trained models will be released. Since the dataset is a central contribution, a data-availability statement would aid reproducibility.","section":"§4.1"},{"comment":"The use of ChatGPT-5.2 as a metric extractor is unusual and appears without any reference or version identifier. If it is a proprietary model, please provide the exact model identifier and date of use.","section":"§4.1 Metrics"}],"recommendation":"major_revision","confidential_remarks":"The proposed idea is plausible and the dataset is potentially valuable, but the empirical support is currently too weak for the strength of the claims. The 11-sample WSI-LLaVA evaluation, the unvalidated LLM-based metric extractor, and the partially circular visual-support evaluation together make the central claim of consistent improvement unsupported as written. These issues are addressable by collecting more test data, adding uncertainty quantification, validating the extractor, and softening the claims. I would not reject the paper, but I would require these revisions before publication. The duplicated Table 3/Table 4 appears to be an editing error rather than a substantive issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick heads-up: the new thing here is real. This is the first paper, by its own lit review, to align a report-generation model's attention with pathologist viewport trajectories. The dataset—121 prostate WSIs with multi-scale viewports, verbal descriptions, cursor data, about 7 hours of expert annotation—is a genuine resource, even though it is not yet released. The method is clean and cheap: add a KL alignment loss between model attention and a human saliency map, at training time only. Testing on two models with different fusion mechanisms is a reasonable way to show it generalizes.\n\nThe stress-test note lands. On WSI-LLaVA, report quality is evaluated on 11 test slides. ACC-G goes from 6/11 to 7/11, ACC-PNI from 9/11 to 10/11. The headline 19.3% average accuracy gain is driven by one or two cases. There are no confidence intervals and no significance tests. The component metrics come from ChatGPT-5.2 extraction with no validation against pathologist-annotated reports. The abstract says five clinically relevant components, but Table 1 reports four accuracy columns because intraductal carcinoma was dropped post hoc. Tables 3 and 4 are identical, which looks like an editing slip. The visual-support numbers are less convincing than they appear: the loss optimizes the same quantity being measured, so NSS and KL improving is somewhat expected. The human saliency construction also has underspecified parameters (Gaussian bandwidth, magnification merge weights), and nobody checks whether the proxy is faithful.\n\nNone of this makes the central idea wrong. The mechanism is plausible: constrain where the model looks and it will describe more of what matters. HistGen's test set is larger, and the directional results there are more credible, though still lacking error bars. The math is straightforward, the citations are relevant, and the authors are honest about some limitations. But the paper consistently overstates the strength of the evidence, especially for WSI-LLaVA.\n\nI would send this to peer review, not desk reject it. The idea and dataset are worth referee time, and the authors can fix the main weaknesses with more test data, validated extraction, and a claims rewrite. If the dataset and code become available, I would cite it. As it stands, I would be cautious about relying on the numbers.","headline":"First use of pathologist gaze to supervise pathology report generation, with a real dataset, but the evaluation is too small to back the headline gains.","tokens_in":8968,"tokens_out":4640,"would_cite":false,"duration_ms":51610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Using recorded pathologist gaze as an auxiliary training signal, this paper shows that aligning a report-generation model's attention over image patches to human attention improves both report quality and clinical accuracy, and makes the mo","keywords":["Pathologist attention","Report generation","Whole-slide imaging","Attention alignment","Prostate cancer","Vision-language model","Explainability","Gaze supervision"],"falsifier":"A reader could rebuild the human saliency maps from the described pipeline and test whether they predict held-out pathologist fixations or Gleason-positive pixels better than chance (e.g., ROC-AUC > 0.5). If the maps carry no signal, or if swapping the human map for a random map with identical spatial statistics still yields the reported gains, the effect is not due to the semantic content of pathologist attention.","tokens_in":8008,"feed_emoji":"🔬","tokens_out":4714,"duration_ms":39731,"temperature":0.7,"pith_summary":"This paper proposes that the gaze patterns of pathologists reading whole-slide images can be used as a training signal for AI report generators. The authors built a dataset of 121 prostate cancer slides with synchronized viewport trajectories, verbal descriptions, and cursor movements, and used it to fine-tune two vision-language models with an extra loss that pulls the model's attention over image patches toward the pathologist's attention. They report that this attention-alignment loss improves both standard NLP report-quality metrics (by about 10.9% on average) and accuracy on five clinically relevant components (by about 19.3%), across two architectures with different attention mechanisms. The inference-time attention maps also align better with pathologist gaze, giving generated reports visual grounding in the regions the human expert actually inspected.","feed_headline":"Pathologist gaze training lifts AI prostate report accuracy by 19%","feed_subtitle":"Using recorded expert gaze as supervision, two pathology AI models write better reports and highlight the true diagnostic regions.","key_machinery":"The load-bearing mechanism is the attention-alignment loss L_KL = KL(softmax(a_human) || softmax(a_model)), where a_human is built from multi-magnification viewport trajectories (sampled at 20 Hz, centers Gaussian-blurred, merged across 2x-20x magnifications) and a_model is the mean attention the model assigns to each image token when generating the report. The loss is added to the standard next-token cross-entropy objective with weight lambda=1.0, using only last-layer attention, selected over all-layer alignment by ablation.","core_discovery":"The paper's central claim is that aligning model attention to pathologist attention during training improves report generation. Concretely, they define a human saliency vector per image-token position by projecting recorded 20 Hz viewport-center scanpaths onto the patch grid and Gaussian-blurring them, and add a KL-divergence loss between softmax of this human vector and softmax of the model's last-layer attention over image tokens. Across HistGen (cross-attention) and WSI-LLaVA (self-attention), this yields consistent gains in BLEU, ROUGE-L, METEOR and in exact-match accuracy for Gleason pattern, PNI, EPE, and surgical margin, and produces attention heatmaps that correlate better with human","pith_inferences":["Extrapolating from the paper's mechanism, the same alignment loss could regularize any token-based vision-language model, and might reduce hallucinated findings by suppressing attention to regions the pathologist never examined—a testable extension using negative cases.","The paper leaves the multi-scale saliency merge weights and Gaussian blur bandwidth unspecified; these choices determine the target distribution, so the method's sensitivity to them is an open empirical question.","If human gaze is a strong prior for diagnostic relevance, the collected maps could serve as weak labels for weakly-supervised detection or grading of Gleason patterns, beyond report generation.","The reported 10.9%/19.3% gains are averages over components; the per-component spread (8%-34% accuracy gains) suggests that gaze alignment helps most on components with clear visual signatures, a pattern the paper does not analyze."],"forward_implications":["Experts' gaze can serve as a plug-and-play auxiliary supervision for pathology report generation, with no extra inputs or overhead at inference.","Training with attention alignment prevents overfitting in low-data fine-tuning: WSI-LLaVA without alignment drops below zero-shot, while alignment lifts it above.","The attention maps produced at inference are more human-aligned and more consistent with Gleason-region ground truth, strengthening interpretability of generated reports.","The modest annotation load (about 7 hours total) suggests the approach can transfer to other cancer types and report-generation models."],"fun_headline_variants":["Gaze-guided AI pens better prostate cancer reports","Pathologist gaze steers AI to sharper prostate reports","Aligning AI gaze to doctors lifts prostate report accuracy 19%","AI pathology reports improve when trained on expert gaze","Attention match to pathologist gaze boosts prostate AI reports"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the human saliency vector built from viewport trajectories is a faithful, token-level target distribution; if the recorded gaze is noisy, miscalibrated, or mis-synchronized with the verbal descriptions, the KL loss trains the model toward a flawed proxy rather than genuine diagnostic attention.","fun_headline_variants_meta":{"raw":{"variants":["Gaze-guided AI pens better prostate cancer reports","Pathologist gaze steers AI to sharper prostate reports","Aligning AI gaze to doctors lifts prostate report accuracy 19%","AI pathology reports improve when trained on expert gaze","Attention match to pathologist gaze boosts prostate AI reports"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1430,"prompt_tokens":779,"completion_tokens":651,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":574}},"tokens_in":523,"tokens_out":651,"duration_ms":6106,"temperature":1.0,"reasoning_tokens":574,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:11:33.361892+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could rebuild the human saliency maps from the described pipeline and test whether they predict held-out pathologist fixations or Gleason-positive pixels better than chance (e.g., ROC-AUC > 0.5). If the maps carry no signal, or if swapping the human map for a random map with identical spatial statistics still yields the reported gains, the effect is not due to the semantic content of pathologist attention.","supporting_citations":[],"review_version":1}