REVIEW 4 major objections 4 minor 24 references
Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that the order in which a physician's gaze moves across an image is a learnable supervisory signal that brings weakly supervised segmentation within 0.33 Dice of fully supervised U-Net.
desk verdict A novel reverse-truncation mechanism for gaze-based segmentation, but the evaluation is confounded by a U-Net/nnU-Net backbone inconsistency and test-set-tuned threshold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the reversed, truncated gaze track. Given fixation points with start timestamps, GradTrack constructs the gaze track $T$, then produces $T_{50\%}$, $T_{75\%}$, $T_{100\%}$ by removing the earliest fixations in reverse order. For each truncated track it computes a distance map $D^r(\mathbf{p}) = \min_{t\in T^r}\|\mathbf{p}-t\|_2$ and converts it to a soft attention map $G^r = e^{-D^r/\beta}\cdot \mathbb{I}(e^{-D^r/\beta}>\tau)$. These maps supervise predictions at decoder blocks 2, 4, and 6 through a weighted cross-entropy loss, providing staged prior information so that later decoder stages are guided by fuller gaze context. The Track Attention module fuses the predicted foreground map into the decoder features with a stop-gradient, letting gaze track knowledge shape segmentation without backpropagating through the fusion.
What would settle it
Run GradTrack's training protocol on a public gaze dataset with fixation timestamps shuffled but all other signals unchanged; if Dice does not drop appreciably below the reported 81.01, the temporal-order claim is not doing the work attributed to it. Equivalently, retrain the 'w/o Sequential' variant on NCI-ISBI and check whether the gap generalises beyond Kvasir-SEG.
Extended reading notes
Core claim
GradTrack's central claim is that the temporal order of gaze fixations is a usable supervisory signal for weakly supervised segmentation, and that reversing that order during training stabilises learning. The Gaze Track Map Generation module converts raw gaze data into a continuous track, then forms three track attention maps by keeping the last 50%, 75%, and 100% of the fixation sequence. A distance-based exponential decay turns each truncated track into a soft attention map that supervises the decoder; the Track Attention modules inject the learned maps back into the feature stream. The paper reports that this scheme outperforms GazeMedSeg by 3.21% Dice on Kvasir-SEG and 2.61% on NCI-ISBI, and comes within 1.11% and 0.33% of a fully supervised U-Net on the two datasets.
Load-bearing premise
The method presumes that every training image has gaze annotations with reliable start timestamps, and that the comparison numbers inherited from GazeMedSeg were produced on the same annotations and data split.
Editorial extensions
If this is right
- Gaze-based weakly supervised segmentation can close most of the gap to full supervision: on NCI-ISBI the gap to U-Net is 0.33 Dice, so gaze annotation may serve as a practical substitute for pixel-level labels.
- Multi-level supervision from truncated tracks gives increasing benefit at deeper decoder stages; placing guidance in the encoder hurts, so decoder-only integration is the design to keep.
- Reverse truncation is intentionally conservative: it biases predictions toward under-activation, trading recall of uncertain regions for precision against noise.
- The component ablation on Kvasir-SEG shows a cumulative climb from 63.63 Dice with only the under-activated VAM to 81.01 for the full design, so each module adds measurable value.
- The field-of-view threshold is a sensitive knob: 0.25 works best, small values inject redundant noise and large values starve the model, so the method needs calibration per dataset.
Reading between the lines
- Beyond the paper, the same ordering signal might transfer to other sequential human annotations, such as mouse-trace or touch-screen scribbles with timestamps, turning any time-stamped annotation trace into this style of staged supervision.
- The 0.73 Dice gain of the full design over the non-sequential variant (81.01 vs 80.28) suggests that temporal order contributes only modestly on this benchmark; a larger gain would need datasets where diagnostic scanning order varies more between images.
- A testable extension would replace the hand-set truncation ratios of 50/75/100% with learned or per-image ratios, letting the model decide how much of a physician's early scanning to trust.
- If the gaze annotations were released, re-running the exact protocol on a second institution's data would clarify whether the 3.21% and 2.61% gains are tied to the specific annotators or generalise across gaze patterns.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GradTrack, a gaze-based weakly supervised segmentation framework that augments a U-Net-style decoder with gaze track attention. The GTMG module converts fixation points and start timestamps into three reverse-truncated track attention maps (Eqs. 1-2), and TA modules predict and fuse these maps into decoder features (Eqs. 3-4), with total loss Eq. 5. On Kvasir-SEG and NCI-ISBI, GradTrack reports Dice 81.01/80.25, versus 77.80/77.64 for the inherited GazeMedSeg baseline, and 0.33 points below fully supervised U-Net on NCI-ISBI. The paper claims that temporal order of gaze fixations is the key information dividend.
Significance. If the empirical claims are valid, the paper makes a useful contribution by incorporating temporal order into gaze-based WSSS, and the module design is concrete and well-specified. The ablation shows that each component improves over the under-activation baseline, and the multi-dataset evaluation is a strength. However, the headline comparison is not currently architecture-matched, and the gaze annotations are not described or released in sufficient detail; the contribution is therefore conditional on a controlled re-evaluation.
major comments (4)
- [Sec. 2.1 / Sec. 3.2 / Table 1] Section 2.1 states that GradTrack adopts a U-Net model as the backbone, while Section 3.2 states that nnU-Net is the backbone architecture. These are not interchangeable: Table 1 reports fully supervised Dice of 82.12/80.58 for U-Net versus 88.41/82.43 for nnU-Net. Because the GazeMedSeg baseline is inherited from [24] and its backbone is not specified, the reported 3.21/2.61 Dice gains and the '0.33 gap to fully supervised U-Net' could stem from the backbone or training recipe rather than from GTMG and TA. Please reconcile the backbone statements and rerun GazeMedSeg under GradTrack's exact training schedule and backbone.
- [Sec. 3.3 / Sec. 3.1] Section 3.3 states that the PointSup, AGMM, Boxinst, BoxTeacher, and GazeMedSeg results are inherited from [24]. Section 3.1 says only that training annotations were 'meticulously generated by annotators,' without specifying the gaze-tracking device, fixation/timestamp format, or whether these annotations match the gaze data used by [24]. Without a shared gaze-annotation protocol and data split, the comparison is not controlled; please provide the gaze data or a precise specification and rerun the inherited baseline under the same conditions.
- [Fig. 4(b) / Sec. 3.2] The field-of-view threshold tau=0.25 is chosen on Kvasir-SEG by scanning test-set Dice across tau values (0.05-0.5) and selecting the maximum (81.01 at tau=0.25). Since tau appears directly in Eq. (2) and thresholds the attention maps used as supervision, selecting it on the test set can optimistically bias the reported results. Please use a validation split or report the selection procedure with held-out test evaluation.
- [Table 2] The case for reverse truncation is weak as reported. The full model achieves 81.01 +/- 0.66 while the 'w/o Sequential' variant achieves 80.28 +/- 0.02, a 0.73-point difference with overlapping standard deviations; the central claim that temporal order of fixation points is the 'information dividend' would benefit from statistical testing or a more direct comparison isolating temporal order from the GTMG/TA construction.
minor comments (4)
- [Eq. (2)] Equation (2) contains the typo 'can be is defined as'; replace with 'is defined as'.
- [Abstract / Table 1 / Sec. 3.2] The abstract writes 'nnUNet' while Table 1 and Section 3.2 write 'nnU-Net'; use one consistent notation throughout.
- [Sec. 2.4] The phrase 'basic overly under-activation hard label supervision' is confusing; consider 'basic under-activation hard-label supervision' or define what 'overly' modifies.
- [Eq. (5)] In Eq. (5), both L50 and L75 are weighted by the same lambda_1; if the authors intend different weights at different decoder depths, this should be stated explicitly.
Circularity Check
No circularity: gaze track maps are constructed directly from gaze annotations, the losses use those maps as supervision, and the reported Dice gains rest on external empirical comparisons rather than on any fitted or self-referential term.
full rationale
The derivation chain is self-contained. The GTMG module builds a gaze track T from fixation points and start timestamps, computes distance maps via Eq. (1), and converts them into attention maps via Eq. (2). The TA module predicts these maps from decoder features, and Eq. (3) supervises those predictions with the precomputed G_r maps. The total loss Eq. (5) is a weighted sum of these gaze-track supervision terms plus an under-activated VAM loss. Nothing in this chain is defined in terms of the test Dice, the evaluation ground truth, or the reported final numbers. The hyperparameters tau, beta, lambda_1, and lambda_2 are hand-set empirical choices, with a sensitivity analysis for tau (Fig. 4b); they are not fitted to the evaluation labels, so the reported improvements are not forced by construction. The comparisons to GazeMedSeg and other baselines are inherited from external prior work [24], not from the current authors, and the backbone inconsistency (U-Net in Sec. 2.1 vs nnU-Net in Sec. 3.2) is a legitimate experimental-comparability concern, but it does not make the central claim reduce to its own inputs. The only self-citation, ref. [21], appears in a list of scribble-based methods and is not load-bearing for any claim. Thus there is no significant circularity.
Assumptions & free parameters
free parameters (5)
- Field-of-view threshold tau =
0.25
- Decay rate beta =
10 (Kvasir-SEG), 15 (NCI-ISBI)
- Under-activation threshold =
0.7
- Balancing coefficients lambda1, lambda2 =
0.5 each
- Reverse truncation ratios =
50%, 75%, 100%
assumptions (4)
- domain assumption Temporal order of gaze fixations encodes progressive diagnostic reasoning
- domain assumption Gaze annotations for the training images exist, are accurate, and include start timestamps
- standard math The D-CRF-refined visual attention map from [12] is a sound basis for hard pseudo labels
- domain assumption nnU-Net preprocessed at 224x224 retains enough detail for these segmentation tasks
Cite this review
Pith. "Pith review of Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation." pith.science (2026). https://pith.science/paper/3I6WKR7N
@misc{pith2026250522230,
author = {Pith},
title = {Pith review of: Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/3I6WKR7N}},
note = {Machine review of arXiv:2505.22230}
}
read the original abstract
Weakly supervised semantic segmentation (WSSS) in medical imaging struggles with effectively using sparse annotations. One promising direction for WSSS leverages gaze annotations, captured via eye trackers that record regions of interest during diagnostic procedures. However, existing gaze-based methods, such as GazeMedSeg, do not fully exploit the rich information embedded in gaze data. In this paper, we propose GradTrack, a framework that utilizes physicians' gaze track, including fixation points, durations, and temporal order, to enhance WSSS performance. GradTrack comprises two key components: Gaze Track Map Generation and Track Attention, which collaboratively enable progressive feature refinement through multi-level gaze supervision during the decoding process. Experiments on the Kvasir-SEG and NCI-ISBI datasets demonstrate that GradTrack consistently outperforms existing gaze-based methods, achieving Dice score improvements of 3.21\% and 2.61\%, respectively. Moreover, GradTrack significantly narrows the performance gap with fully supervised models such as nnUNet.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[24]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Zhong, Y., Tang, C., Yang, Y., Qi, R., Zhou, K., Gong, Y., Heng, P.A., Hsiao, J.H., Dou, Q.: Weakly-supervised medical image segmentation with gaze annotations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 530–540. Springer (2024)
work page 2024
-
[1]
Image and Vision Computing144, 104961 (2024)
Bisogni, C., Nappi, M., Tortora, G., Del Bimbo, A.: Gaze analysis: A survey on its applications. Image and Vision Computing144, 104961 (2024)
work page 2024
-
[2]
The Cancer Imaging Archive (2015)
Bloch, N., Madabhushi, A., Huisman, H., Freymann, J., Kirby, J., Grauer, M., Enquobahrie, A., Jaffe, C., Clarke, L., Farahani, K.: Nci-isbi 2013 challenge: Au- tomated segmentation of prostate structures. The Cancer Imaging Archive (2015). https://doi.org/http://doi.org/10.7937/K9/TCIA.2015.zF0vlOPv
-
[3]
IEEE Transactions on Image Processing (2025)
Chen, J., Huang, W., Zhang, J., Debattista, K., Han, J.: Addressing inconsistent labeling with cross image matching for scribble-based medical image segmentation. IEEE Transactions on Image Processing (2025)
work page 2025
-
[4]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition
Cheng, B., Parkhi, O., Kirillov, A.: Pointly-supervised instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 2617–2626 (2022)
work page 2022
-
[5]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Cheng, T., Wang, X., Chen, S., Zhang, Q., Liu, W.: Boxteacher: Exploring high- quality pseudo labels for weakly supervised instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3145–3154 (2023)
work page 2023
-
[6]
IEEE Transactions on Pattern Analysis and Machine Intelligence46(1), 61–84 (2023)
Ghosh, S., Dhall, A., Hayat, M., Knibbe, J., Ji, Q.: Automatic gaze analysis: A survey of deep learning based approaches. IEEE Transactions on Pattern Analysis and Machine Intelligence46(1), 61–84 (2023)
work page 2023
-
[7]
IEEE Journal of Biomedical and Health Infor- matics (2024)
Ibragimov, B., Mello-Thoms, C.: The use of machine learning in eye tracking stud- ies in medical imaging: A review. IEEE Journal of Biomedical and Health Infor- matics (2024)
work page 2024
Show all 24 references
-
[8]
Nature methods18(2), 203–211 (2021)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods18(2), 203–211 (2021)
2021
-
[9]
In: MultiMedia modeling: 26th international conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, proceedings, part II 26
Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., De Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: MultiMedia modeling: 26th international conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, proceedings, part II 26. pp. 451...
2020
-
[10]
IEEE Journal of Biomedical and Health Informatics (2024)
Jiang, H., Gao, M., Liu, Z., Tang, C., Zhang, X., Jiang, S., Yuan, W., Liu, J.: Glanceseg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy. IEEE Journal of Biomedical and Health Informatics (2024)
2024
-
[11]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Kong,Y.,Wang,S.,Cai,J.,Zhao,Z.,Shen,Z.,Li,Y.,Fei,M.,Wang,Q.:Gaze-detr: Using expert gaze to reduce false positives in vulvovaginal candidiasis screening. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 133–143. Springer (2024)
2024
-
[12]
Advances in neural information processing systems24(2011)
Krähenbühl, P., Koltun, V.: Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems24(2011)
2011
-
[13]
IEEE Transactions on Medical Imaging (2024)
Li, Z., Zheng, Y., Shan, D., Yang, S., Li, Q., Wang, B., Zhang, Y., Hong, Q., Shen, D.: Scribformer: Transformer makes cnn work better for scribble-based medical image segmentation. IEEE Transactions on Medical Imaging (2024)
2024
-
[14]
Pattern recognition122, 108341 (2022)
Liu, X., Yuan, Q., Gao, Y., He, K., Wang, S., Tang, X., Tang, J., Shen, D.: Weakly supervised segmentation of covid19 infection with scribble annotation on ct images. Pattern recognition122, 108341 (2022)
2022
-
[15]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Luo, X., Hu, M., Liao, W., Zhai, S., Song, T., Wang, G., Zhang, S.: Scribble- supervised medical image segmentation via dual-branch network and dynamically mixed pseudo labels supervision. In: International Conference on Medical Image Computing and Computer-Assisted Interventi...
2022
-
[16]
In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, Oc- tober 5-9, 2015, proceedings, part III 18
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, Oc- tober 5-9, 2015, proceedings, part III 18....
2015
-
[17]
In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion
Tang, F., Xu, Z., Qu, Z., Feng, W., Jiang, X., Ge, Z.: Hunting attributes: Context prototype-aware learning for weakly supervised semantic segmentation. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. pp. 3324–3334 (2024)
2024
-
[18]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Tian, Z., Shen, C., Wang, X., Chen, H.: Boxinst: High-performance instance seg- mentation with box annotations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5443–5452 (2021)
2021
-
[19]
IEEE Transactions on Medical Imaging 40(8), 1990–2001 (2021)
Valvano, G., Leo, A., Tsaftaris, S.A.: Learning to segment from scribbles using multi-scale adversarial attention gates. IEEE Transactions on Medical Imaging 40(8), 1990–2001 (2021)
2021
-
[20]
In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
Wang, B., Pan, H., Aboah, A., Zhang, Z., Keles, E., Torigian, D., Turkbey, B., Krupinski, E., Udupa, J., Bagci, U.: Gazegnn: A gaze-guided graph neural network for chest x-ray classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision....
2024
-
[21]
arXiv preprint arXiv:2408.12814 (2024)
Wang, Z., Ye, Y., Chen, Z., Shu, M., Xia, Y.: From few to more: Scribble-based medical image segmentation via masked context modeling and continuous pseudo labels. arXiv preprint arXiv:2408.12814 (2024)
2024 arXiv
-
[22]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Wu, L., Zhong, Z., Fang, L., He, X., Liu, Q., Ma, J., Chen, H.: Sparsely annotated semantic segmentation with adaptive gaussian mixtures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15454– 15464 (2023)
2023
-
[23]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Zhao, X., Tang, F., Wang, X., Xiao, J.: Sfc: Shared feature calibration in weakly supervised semantic segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 7525–7533 (2024)
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.