{"id":"02de84b7-99c2-464d-a546-ea6408731800","arxiv_id":"2507.12591","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CT-ScanGaze, the first public eye-tracking dataset on CT volumes, contains 909 scans with radiologist gaze, reports, and findings, and CT-Searcher, a 3D scanpath model, beats adapted 2D baselines on it.","lead":"Researchers recorded eye movements of two radiologists reading 909 CT scans, producing the first public 3D gaze dataset for computed tomography. They also built CT-Searcher, a model that predicts radiologist-like fixation sequences in three dimensions, and a heuristic that converts 2D X-ray gaze data into synthetic 3D pretraining data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-radiologist evidence for general 'radiologist-like' scanpaths is invalid because both radiologists appear in every training fold, and Table 4's CC intervals actually do not overlap; the claim should be downgraded to reproducing the two familiar readers.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the cross-radiologist evaluation in Sec. 5.4 does not measure generalization to an unseen radiologist because the 5-fold cross-validation is over CT volumes, not over radiologists. I agree with that diagnosis, and I add a concrete internal inconsistency: the paper states that per-radiologist score differences are insignificant, yet Table 4 shows non-overlapping 95% confidence intervals for CC (0.1901 +/- 0.017 vs 0.1503 +/- 0.014), which by the paper's own reported intervals indicates a significant difference. This makes the Sec. 5.4 conclusion both logically unsupported and empirically contradicted on at least one metric. I considered other possible concerns: the absence of a trivial center-bias baseline, the unvalidated 2D-to-3D pretraining conversion, and the unverifiable full data release. These are real limitations and worth addressing, but they are not the single most load-bearing issue for the paper's broadest claim. The dataset contribution and the benchmark numbers for the two participating radiologists remain valuable; the problem is the overreach from 'fits these two readers' to 'general radiologist-like scanpath patterns.' A leave-one-radiologist-out retraining experiment would settle the question directly. Since the reader already issued CONDITIONAL on this same concern, my stress-test does not change the verdict.","tokens_in":24599,"tokens_out":7381,"duration_ms":92060,"concrete_test":"Retrain CT-Searcher twice with leave-one-radiologist-out: hold out all volumes read by radiologist #1 in one run and all volumes read by radiologist #2 in another run (no volumes from that reader in any training fold), then evaluate each model on the held-out reader. If the held-out reader's ScanMatch and CC are comparable to the 'Both' column in Tables 1/3/4, the generalizability claim survives; if they drop materially or the two readers' CC intervals remain non-overlapping, Sec. 5.4's conclusion fails. Also recompute the per-radiologist comparison with a paired test over the 5 folds rather than interval overlap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CT-Searcher produces 'radiologist-like' scanpaths (Sec. 5.2) rests partly on the cross-radiologist evaluation in Sec. 5.4. That evaluation cannot support the conclusion drawn. The checkpoints come from a 5-fold cross-validation split over CT volumes, not over radiologists; consequently every training fold contains gaze data from both participants, and the 'Radiologist #1' / 'Radiologist #2' test subsets are only held-out volumes for familiar readers. This tests fit to each reader's style, not generalization to an unseen radiologist. Moreover, the text claims 'differences in scores between the two radiologists are insignificant,' but Table 4's 95% CIs for CC are 0.1901 +/- 0.017 ([0.173, 0.207]) vs 0.1503 +/- 0.014 ([0.136, 0.164]), which do not overlap; by the displayed intervals the two readers differ significantly on the spatial saliency metric. The dataset and the benchmark numbers for the two study radiologists are not invalidated, but the general 'radiologist-like scanpath' claim is overreaching. The paper should either provide leave-one-radiologist-out training or explicitly limit the claim to reproducing the two participating readers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CT-ScanGaze, a new dataset of eye-tracking data from two experienced radiologists reading 909 chest and abdomen CT volumes, including scanpaths, radiology reports, and extracted findings. The authors also propose CT-Searcher, a transformer-based 3D scanpath prediction model that combines a Swin UNETR feature extractor, 3D positional encoding, and separate spatial and duration prediction heads. A pretraining pipeline converts existing 2D CXR gaze datasets (EGD and REFLACX) into synthetic 3D gaze data. Experiments compare CT-Searcher with 3D-adapted versions of PathGAN, HAT, Gazeformer, and GazeformerISP using scanpath and saliency metrics, and ablation studies assess the contributions of the 3D positional encoding and pretraining. The paper also reports a cross-radiologist evaluation in Sec. 5.4.","tokens_in":24889,"tokens_out":6599,"duration_ms":72240,"significance":"If released as promised, CT-ScanGaze is the first public eye-gaze dataset for CT volumes, and it combines four modalities (CT, gaze, report, findings), filling a clear gap in medical imaging and 3D scanpath prediction. The CT-Searcher baseline is a reasonable first 3D scanpath model and outperforms the adapted 2D baselines on the reported metrics; the ablations support the design choices, and the release of code and data is a concrete reproducibility contribution. However, the broader claim that the model learns general radiologist scanpath patterns is not supported: the cross-radiologist evaluation in Sec. 5.4 includes both radiologists in every training fold, so it only tests held-out volumes for familiar readers. The dataset and benchmark for the two participating radiologists are valuable, but the generalization claim needs either a true leave-one-radiologist-out experiment or explicit re-scoping to the two study readers.","major_comments":[{"comment":"The cross-radiologist evaluation cannot support the conclusion that CT-Searcher 'successfully learns general scanpath patterns rather than overfitting to an individual radiologist's style.' The checkpoints come from a 5-fold cross-validation over CT volumes, so both radiologists' gaze data appear in every training fold. The test subsets containing only Radiologist #1 or Radiologist #2 are held-out volumes for those same radiologists, not unseen readers. This conflates generalization across volumes with generalization across radiologists. The claim should be explicitly limited to reproducing the two participating readers, or the authors should run a leave-one-radiologist-out experiment to substantiate generalization to an unseen radiologist.","section":"Sec. 5.4"},{"comment":"The statement that 'the differences in scores between the two radiologists are insignificant' is contradicted by the reported 95% confidence intervals for CC. Radiologist #1 has 0.1901 ± 0.017 ([0.173, 0.207]) and Radiologist #2 has 0.1503 ± 0.014 ([0.136, 0.164]); these intervals do not overlap, suggesting a statistically significant difference in how well the model reproduces each reader's spatial saliency. The authors need to either support the insignificance claim with explicit statistical tests (e.g., permutation tests or overlap-corrected intervals) or remove the claim and temper the interpretation.","section":"Sec. 5.4, Table 4"}],"minor_comments":[{"comment":"The protocol for handling ground truth scanpaths longer than the model's fixed output length N=400 is unspecified. Simplified ground truths have up to 1,507 fixations (Sec. 2.3). The paper should state whether ground truths are truncated to N=400 in training and evaluation, or describe how the metrics accommodate variable-length sequences.","section":"Sec. 5.1 / Appendix D"},{"comment":"The 2D-to-3D conversion of gaze data uses ad hoc rules (flipping x, fixing a coordinate to 0.5, and mapping the remaining axis to the slice dimension) that are not validated against real 3D gaze data. Since the pretraining step is shown to improve performance in Table 5, the authors should discuss the rationale for these choices and their potential domain gap.","section":"Sec. 4.6"},{"comment":"The frame-to-DICOM mapping relies on OCR with manual correction, but the extent of manual correction is not quantified. Reporting the percentage of frames that required correction would improve reproducibility and help estimate the accuracy of the gaze-slice alignment.","section":"Sec. 2.2"},{"comment":"The Duration column for HAT is reported as '–' with no explanation. Please clarify whether HAT does not predict durations, or whether the metric was not computed, and how this affects the comparison.","section":"Table 1"},{"comment":"The phrase 'radiologist-like 3D fixation sequences' should be qualified as 'like the two participating radiologists' in light of the dataset's two-reader design and the limitations of the cross-radiologist evaluation.","section":"Abstract / Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The dataset release is the main scientific contribution and appears valuable to the community. The cross-radiologist overclaim is a central issue that can be fixed by re-scoping the claims or adding a leave-one-radiologist-out experiment. The paper is otherwise well-organized and the experimental work is solid for a benchmark paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: CT-ScanGaze is a real contribution. It is the first public eye-tracking dataset on CT volumes, with 909 scans, gaze, reports, and findings from two experienced radiologists. That fills a clear gap for research on explainable CAD, radiology training, and 3D scanpath modeling. CT-Searcher is a sensible first 3D scanpath baseline, and the ablations (3D PE, pretraining) are informative. The 2D-to-3D pretraining conversion is a clever idea, though ultimately heuristic.\n\nThe main soft spot is Sec. 5.4. The cross-radiologist evaluation uses checkpoints from 5-fold cross-validation over volumes, not over radiologists. Since every training fold contains data from both radiologists, the per-radiologist test splits only tell you how well the model fits held-out volumes from familiar readers. That does not measure generalization to an unseen radiologist. The text also says the differences are insignificant, but Table 4's 95% CIs for CC are [0.173, 0.207] vs [0.136, 0.164], which do not overlap. That is a straightforward error.\n\nTwo smaller issues. First, there is no trivial baseline—e.g., center-bias or saliency-only scanpath—so the absolute scores (ScanMatch 0.1466) are hard to interpret. Second, the data release is promised but not verifiable from the preprint; only a single example is shown. The pretraining synthetic data also has an arbitrary flavor (flip x, z=0.5), though the ablations do show it helps.\n\nNone of this sinks the dataset. The collection pipeline is careful, the simplification analysis is honest, and the benchmark itself is a service. The citation pattern looks fine, building on EGD, REFLACX, and existing 2D scanpath work. But the claim in Sec. 5.2 that CT-Searcher produces 'radiologist-like scanpaths' in a general sense is broader than the evidence supports; at best, it reproduces the two participating readers.\n\nIf this comes to peer review, I would send it out with a request for major revision: redo or relabel the cross-radiologist analysis, add trivial baselines, and make the data actually accessible. The dataset deserves referee time.","headline":"First public CT gaze dataset, but the generalization claim in Sec. 5.4 does not hold; worth a serious referee after major revision.","tokens_in":25489,"tokens_out":3446,"would_cite":true,"duration_ms":35775,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new public dataset of radiologist gaze on CT scans and a 3D scanpath model that mimics expert visual search.","keywords":["CT-ScanGaze","3D scanpath prediction","eye tracking","radiologist gaze","computed tomography","volumetric medical imaging","transformer","explainable AI"],"falsifier":"Record eye gaze from a third, never-seen radiologist on a subset of the same CTs and run CT-Searcher's existing 5-fold checkpoints on them; if its ScanMatch and CC scores collapse to near the baseline range, the cross-radiologist generalization claim is falsified.","tokens_in":24405,"feed_emoji":"🩻","tokens_out":9335,"duration_ms":94598,"temperature":0.7,"pith_summary":"CT-ScanGaze is the first publicly released eye-tracking dataset of CT reading, recording where two expert radiologists look while interpreting chest and abdomen volumes. The paper's central claim is that its transformer model, CT-Searcher, can generate radiologist-like 3D scanpaths from a CT volume, capturing both where fixations land inside a slice and how gaze moves back and forth across slices. On the authors' benchmark, CT-Searcher beats four 2D scanpath models adapted to 3D on every reported metric, with ScanMatch 0.1466 versus 0.0828 for the best baseline. Every CT in the dataset is paired with a speech-to-text radiology report and extracted findings, so the gaze data can be linked to diagnostic reasoning. The broader point is to give volumetric medical imaging a public testbed for modeling expert visual search, a task that previously had no CT-specific dataset.","feed_headline":"First CT gaze dataset fuels a 3D scanpath predictor","feed_subtitle":"A public 909-scan dataset plus a transformer model that beats 2D baselines at imitating expert search through volumes","key_machinery":"The mechanism that carries the argument is CT-Searcher's transformer architecture with explicit depth encoding. A frozen Swin UNETR encoder converts the CT volume into voxel features, which are projected into tokens along with a learned stop token and given 3D sinusoidal positional encodings so slice position is preserved through the network. A transformer decoder attends to these features using learnable gaze queries and feeds two prediction heads: a spatial head produces a softmax distribution over all voxels plus the stop token, and a duration head regresses fixation time using a reparameterization trick. Before fine-tuning on CT-ScanGaze, the model is pretrained on synthetic 3D gaze built from chest X-ray eye-tracking datasets (EGD and REFLACX) by flipping the x-coordinate, fixing slice position at the middle, and mapping y to depth; this pretraining is what lets a small real dataset support a stable scanpath model.","core_discovery":"The central discovery is that a 3D-aware predictor of eye movements can imitate expert volumetric search: it lands fixations on diagnostically relevant regions within each axial slice and reproduces the non-linear, back-and-forth slice navigation radiologists use when they revisit suspicious findings. The paper establishes this by introducing CT-ScanGaze (909 CT volumes, 131,618 slices, about 4,772 minutes of gaze), simplifying the dense raw fixation sequences to an average of 222 fixations with the MultiMatch toolbox, and by benchmarking CT-Searcher against PathGAN, HAT, Gazeformer, and GazeformerISP adapted to 3D inputs. CT-Searcher is the only method whose qualitative temporal slice navigation matches the ground-truth revisiting pattern, and it achieves the best scores on scanpath metrics (ScanMatch, MultiMatch, SED) and spatial metrics (CC, KLDiv, NSS). The authors take the small difference between per-radiologist scores as evidence that the model learned shared search patterns rather than one reader's personal habit.","pith_inferences":["The paper's cross-radiologist check still trains on both readers' data, so it does not yet prove the model would predict an unfamiliar radiologist's scanpath; a clean held-out radiologist test is the natural next experiment.","The synthetic pretraining heuristic fixes every fixation at the middle slice, which may bias the model toward a center-slice prior; randomizing slice depth during conversion would test whether that bias matters.","With more participating radiologists, this dataset could support models that separate shared expert search strategies from individual reading styles, which would sharpen the radiologist-like claim.","The same encoder-and-pretraining design could transfer to other volumetric search tasks, such as industrial CT inspection or geological core reading, where expert gaze is equally hard to collect."],"forward_implications":["CT-ScanGaze gives the medical imaging community a public benchmark for 3D scanpath prediction, with both original and simplified gaze sequences released.","The paper's pretraining recipe means future volumetric gaze models can be seeded with 2D gaze data instead of requiring a new large eye-tracking study for every 3D domain.","Predicted radiologist-like scanpaths can be used as attention priors for explainable classification and report generation on CT, connecting visual search to diagnostic output.","The released reports and findings let researchers study which gaze patterns precede which diagnostic statements, not just where eyes land.","Because the model reproduces revisits to suspicious regions, its outputs could support radiology training tools that show learners where experienced readers look and look again."],"supporting_citations":[{"why":"Supplies chest X-ray gaze data converted into synthetic 3D pretraining samples.","marker":"[38]"},{"why":"Second source of chest X-ray gaze data used to build the synthetic 3D pretraining set.","marker":"[6]"},{"why":"CXR-to-CT reconstruction method used to turn 2D X-ray images into CT-like volumes for pretraining.","marker":"[40]"},{"why":"Swin UNETR encoder that extracts 3D voxel features for CT-Searcher, initialized from a CT-pretrained checkpoint.","marker":"[29]"},{"why":"Gazeformer, a 2D scanpath transformer adapted to 3D, is one of the baselines CT-Searcher outperforms.","marker":"[50]"},{"why":"GazeformerISP is the strongest baseline on ScanMatch and is adapted to 3D for comparison.","marker":"[13]"},{"why":"HAT, a state-of-the-art 2D scanpath method, is adapted to 3D and serves as a baseline.","marker":"[79]"},{"why":"PathGAN, an early GAN-based scanpath model adapted to 3D, is the weakest baseline in the comparison.","marker":"[4]"},{"why":"MultiMatch supplies both the simplification algorithm that trims dense gaze sequences and the similarity metric used in evaluation.","marker":"[18]"},{"why":"ScanMatch provides the main scanpath-similarity score reported in the comparison tables.","marker":"[16]"}],"fun_headline_variants":["First public CT gaze dataset and 3D scanpath predictor","CT-Searcher: a 3D model that predicts radiologist eye movement","Dataset plus AI to model how radiologists read CT scans","3D scanpath prediction on CT with first public gaze data","New CT gaze dataset trains AI to mimic expert search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the two participating radiologists being representative of expert CT readers, and their shared visual style dominating the learned signal.","fun_headline_variants_meta":{"raw":{"variants":["First public CT gaze dataset and 3D scanpath predictor","CT-Searcher: a 3D model that predicts radiologist eye movement","Dataset plus AI to model how radiologists read CT scans","3D scanpath prediction on CT with first public gaze data","New CT gaze dataset trains AI to mimic expert search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000738,"raw_usage":{"total_tokens":3294,"prompt_tokens":943,"completion_tokens":2351,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":2265}},"tokens_in":559,"tokens_out":2351,"duration_ms":17723,"temperature":1.0,"reasoning_tokens":2265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:44:11.879752+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record eye gaze from a third, never-seen radiologist on a subset of the same CTs and run CT-Searcher's existing 5-fold checkpoints on them; if its ScanMatch and CC scores collapse to near the baseline range, the cross-radiologist generalization claim is falsified.","supporting_citations":[{"cited_title":"Perspective projection-based 3d ct recon- struction from biplanar x-rays","cited_arxiv_id":null,"evidence_quote":"CXR-to-CT reconstruction method used to turn 2D X-ray images into CT-like volumes for pretraining."},{"cited_title":"Creation and validation of a chest x-ray dataset with eye-tracking and report dictation for ai development","cited_arxiv_id":null,"evidence_quote":"Supplies chest X-ray gaze data converted into synthetic 3D pretraining samples."},{"cited_title":"Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images","cited_arxiv_id":null,"evidence_quote":"Swin UNETR encoder that extracts 3D voxel features for CT-Searcher, initialized from a CT-pretrained checkpoint."},{"cited_title":"Gazeformer: Scalable, effective and fast prediction of goal-directed hu- man attention","cited_arxiv_id":null,"evidence_quote":"Gazeformer, a 2D scanpath transformer adapted to 3D, is one of the baselines CT-Searcher outperforms."},{"cited_title":"Beyond average: Individualized visual scanpath prediction","cited_arxiv_id":null,"evidence_quote":"GazeformerISP is the strongest baseline on ScanMatch and is adapted to 3D for comparison."},{"cited_title":"Unify- ing top-down and bottom-up scanpath prediction using trans- formers","cited_arxiv_id":null,"evidence_quote":"HAT, a state-of-the-art 2D scanpath method, is adapted to 3D and serves as a baseline."},{"cited_title":"It de- pends on how you look at it: Scanpath comparison in mul- tiple dimensions with multimatch, a vector-based approach","cited_arxiv_id":null,"evidence_quote":"MultiMatch supplies both the simplification algorithm that trims dense gaze sequences and the similarity metric used in evaluation."},{"cited_title":"Scanmatch: A novel method for comparing fixation sequences","cited_arxiv_id":null,"evidence_quote":"ScanMatch provides the main scanpath-similarity score reported in the comparison tables."}],"review_version":1}