{"id":"b9546ec1-3f59-4575-8109-7f46ffef2b05","arxiv_id":"2412.12890","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SUGE improves gaze estimation accuracy by estimating image and label quality from consistency among real, predicted, and neighbor-derived labels and using those scores to reweight and correct training samples.","lead":"This paper introduces a training recipe, SUGE, that scores each gaze-training image and its label for reliability and then down-weights or corrects the unreliable ones. The practical payoff is a small but consistent accuracy gain on standard gaze datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 4's 'neighboring label' is not necessarily a weighted average: the reconstruction weights from Eq. 2 are unconstrained and can be negative, so Yhat^n can extrapolate outside the local label manifold and bias every downstream uncertainty metric.","rationale":"The reader's weakest assumption points to the neighboring-label module, and I agree that this is the most fragile part of the pipeline. However, the most load-bearing defect is more elementary than smoothness: as written, Eq. 2 does not enforce nonnegativity, so Eq. 4 is not a weighted average of neighbor labels. This is checkable directly from the equations, independent of any empirical noise model. If the signed weights frequently go negative, the 'neighboring label' is an extrapolation, and all of TupleMD/TripleMD (Eqs. 8-9) inherit a systematic distortion; the claimed mechanism of 'local label consistency' is then not what is being optimized. The empirical gains are small (5.05 vs 5.46 on EyeDiap for the Gaze360 backbone, and 0.05 deg for the reconstruction weighting in Table 3), so even a modest sensitivity to this implementation detail could change the conclusion. I would keep the reader's CONDITIONAL verdict, but add this check as an explicit condition before accepting the paper's interpretation. I do not see a reason to escalate to REJECT: the method is specified in enough detail that the test can be run, and the direction of the reported gains is coherent with a (possibly different) denoising mechanism. The check is cheap and conclusive.","tokens_in":11388,"tokens_out":7253,"duration_ms":74555,"concrete_test":"Instrument the released implementation (or re-implement Algorithm 1) on EyeDiap after the warm-up phase: for each training sample compute A_i via Eq. 2 and record (a) the fraction of negative A_ij and (b) the fraction of Yhat^n_i falling outside the convex hull / observable yaw-pitch range of the K neighbor labels. Then rerun the full EyeDiap training with A_i replaced by the projection of the Eq. 2 solution onto the probability simplex (nonnegative, sum 1) while keeping all other hyperparameters fixed, and compare with Table 3's SUGE row. If the error moves by more than ~0.1 deg, the claimed mechanism depends on signed extrapolation; if it stays within ~0.05 deg, negative weights are not load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the construction of the neighboring label Yhat^n_i in Eq. 4. Eq. 1 minimizes ||E(X_i) - sum_j A_ij E(X_ij)||^2 + lambda ||A_i||^2 subject only to sum_j A_ij = 1; there is no nonnegativity constraint on A_ij. The closed form in Eq. 2 is therefore a signed linear reconstruction, not a convex combination. In a 16-d feature space with K=4 same-person neighbors, the optimal weights can be negative whenever X_i lies outside the convex hull of its neighbors, which is a generic situation rather than an edge case. Consequently, Yhat^n_i = sum_j Y_ij A_ij can fall outside the range of observed gaze angles. Since Dng in Eqs. 8-9 is the angular distance between GT and this Yhat^n, TupleMD and TripleMD no longer measure local label agreement; they measure agreement with a signed extrapolation. The paper repeatedly calls Yhat^n a 'neighboring label' and a 'weighted average' (Abstract, Fig. 3, Section 'Neighboring Labeling Module'), and the ablation 'w/o reconstruction weighting' (5.10 vs 5.05 in Table 3) only tests replacement by a plain average, not by a constrained nonnegative average. If negative weights are frequent, the reported gain could come from an artifact of extrapolation rather than from the claimed local smoothness. This is more specific than the reader's smoothness concern: even when labels are locally smooth, Eq. 4 does not compute what the paper claims to compute.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SUGE, a training-time method for gaze estimation that estimates two kinds of uncertainty: image quality and label correctness. For each training sample, it constructs a \"neighboring label\" by linearly reconstructing the sample's feature vector from K same-person feature neighbors and applying the same weights to their ground-truth labels. The neighboring label, together with the model's pseudo label and the ground-truth label, defines two uncertainty metrics (TupleMD and TripleMD), which are fed to a bimodal GMM to obtain label and image confidences. These confidences are then used to truncate unreliable samples, correct labels via a weighted combination of ground-truth, pseudo, and neighboring labels, and weight the training loss. Two networks are trained simultaneously in a co-training style, exchanging corrected labels and weights. Experiments on EyeDiap, MPIIFaceGaze, and Gaze360 with Gaze360 and GazeTR backbones report consistent improvements over the authors' re-implemented baselines, plus ablations and parameter sensitivity analyses on EyeDiap.","tokens_in":11732,"tokens_out":3643,"duration_ms":34704,"significance":"If the method is sound, the paper addresses a real and under-explored issue in gaze estimation: training-set contamination by low-quality images and imprecise annotations. The proposed triplet-label consistency is a novel mechanism that could generalize to other regression tasks with similarly structured uncertainty. The manuscript is also usefully concrete: it re-implements two strong baselines, provides ablations that isolate each module, and uses held-out benchmark splits for evaluation, so the empirical claim is not circular. However, the central mechanism relies on an unconstrained linear reconstruction whose interpretation as a \"weighted average\" of neighbor labels is mathematically incorrect, and the reported gains are small and lack statistical support. These issues affect the paper's main claims and require substantial revision.","major_comments":[{"comment":"Eq. (1) minimizes ||E(X_i) - sum_j A_ij E(X_ij)||^2 + lambda ||A_i||^2 subject only to sum_j A_ij = 1, with no nonnegativity constraint on A_ij. The closed form in Eq. (2) can therefore contain negative weights, and Yhat^n_i = sum_j Y_ij A_ij in Eq. (4) is not a weighted average or a convex combination; it can extrapolate outside the range of the neighbors' gaze labels. Since TupleMD (Eq. 8) and TripleMD (Eq. 9) measure angular distances to this Yhat^n, the uncertainty metrics no longer measure local agreement with a smooth label manifold whenever a feature vector lies outside the convex hull of its neighbors, which is a generic situation in a 16-dimensional space with K=4. The ablation 'w/o reconstruction weighting' in Table 3 compares only against a plain average, not against a nonnegative-constrained reconstruction, so it does not test whether the reported gains arise from the claimed local-smoothness prior or from a signed extrapolation artifact. The paper should either impose a nonnegativity constraint (e.g., with a simplex projection) and re-run all experiments, or provide empirical statistics on the frequency and magnitude of negative weights and show that the results are insensitive to them.","section":"Neighboring Labeling Module, Eqs. (1)-(4)"},{"comment":"The empirical claims are based on single runs without error bars, confidence intervals, or significance tests. The largest reported gain over a re-implemented baseline is 0.41 degrees, but on MPIIFaceGaze SUGE (GazeTR) reports 4.01 vs GazeTR† 4.00, i.e., a slight degradation, and on Gaze360 the GazeTR-based gain is 0.10 degrees. Given the small magnitude of these differences, the phrase 'significantly suggest' in the text is not justified. The authors should report repeated-run statistics (mean and standard deviation over multiple seeds) and, where possible, a paired significance test across the same folds, so that the reader can judge whether the gains are within run-to-run noise.","section":"Tables 1-2 and Section 'Comparison with SOTA Gaze Estimation Methods'"},{"comment":"The abstract and conclusion claim 'state-of-the-art performance,' but Table 1 does not support this claim in a straightforward way: SUGE (GazeTR) does not improve over the original GazeTR on MPIIFaceGaze (4.01 vs 4.00), and the comparison set omits several recent gaze estimation methods that report strong results on these benchmarks. The claim should be narrowed to 'improvements over two re-implemented baselines when SUGE is applied,' or the paper should include a comparison against the current state-of-the-art on the same protocols.","section":"Abstract and Table 1"}],"minor_comments":[{"comment":"The loss term is written as ||ˆYi - f(E(Xi)||_1; the norm's closing delimiter is missing and the notation is inconsistent with the rest of the paper. Please use a correctly delimited L1 norm.","section":"Eq. (15)"},{"comment":"The citation '(Friend, Reising, and Cook 1993)' appears to be an education-policy reference on co-teaching, not the machine-learning co-training algorithm; the description in the text (two networks exchanging corrected labels) more closely matches other sources such as Blum and Mitchell (1998) or the co-teaching approach of Han et al. (2018). The authors should replace or supplement the citation appropriately.","section":"Co-training Strategy and References"},{"comment":"The statement 'we reduce the feature dimension to 16 at the final layer of the encoder' is not motivated. Since the neighboring label construction is sensitive to the feature space in which neighbors are found, the choice of 16 should be justified or included in the parameter sensitivity analysis.","section":"Implementation Details"},{"comment":"The threshold tau is set to 0.5 for both label and image confidence, but the paper does not discuss whether the optimal tau differs between the two quantities or whether the same threshold is appropriate across datasets. A brief discussion of the truncation operation's effect would improve clarity.","section":"Eq. (10) and Algorithm 1"},{"comment":"Figures 2 and 4 are qualitative; adding quantitative evidence (e.g., the fraction of samples with confidence truncated to zero, or a comparison of corrected versus original labels for a few examples) would strengthen the claim that the uncertainty estimates are meaningful.","section":"Visualization Results"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computer vision venue and the problem is well motivated. The core technical issue is the unconstrained signed weights in the neighboring-label construction, which undermines the paper's interpretation of its own mechanism. This is fixable by constraining the weights or by providing empirical evidence that signed extrapolation does not drive the results. The lack of statistical significance testing is also a serious concern given the small effect sizes. I would not reject the paper outright, but it needs substantial revision before the claims can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper proposes SUGE, a training-time method for gaze estimation that estimates both image and label uncertainty by comparing three labels: ground truth, the model's pseudo label, and a 'neighboring label' built from feature-space neighbors. The triplet consistency idea is genuinely new, and the method is clearly and completely specified, with ablations for each component. The reported gains, 0.10–0.41 degrees on EyeDiap and Gaze360 over reimplemented baselines, are consistent with the mechanism, and the ablation study suggests each piece contributes something.\n\nThat said, there are three soft spots. First, the neighboring label in Eq. 4 is called a weighted average, but the reconstruction weights from Eq. 2 are not constrained to be nonnegative. The closed-form solution only enforces sum-to-one, so the weights can be negative and the 'neighboring label' can lie outside the range of the neighbors' labels. The stress-test note is correct: this is more than a semantic quibble. If negative weights are frequent, the uncertainty metrics measure agreement with a signed extrapolation, not local smoothness. The authors should either add a nonnegativity constraint (and solve the constrained optimization) or analyze the actual weight distribution on their benchmarks. Second, there are no error bars or significance tests, and one result (SUGE GazeTR on MPII, 4.01 vs 4.00) is worse than its own baseline. Third, hyperparameters are selected on the same benchmark used for evaluation, which inflates the apparent gains.\n\nThe central argument is not fatally undermined. The method is straightforward to reproduce, the gains are in a plausible range, and the evaluation includes held-out splits. The main weakness is the mismatch between the description and the actual construction of the neighboring label, plus the lack of statistical rigor. I'd send this to a serious referee: it's a concrete, novel technique that could be fixed with a constraint and a better evaluation. It would be a good reading-group discussion too.","headline":"A novel triplet-label consistency method for gaze estimation with small but consistent gains; the neighboring-label construction needs a nonnegativity fix and more rigorous evaluation.","tokens_in":12278,"tokens_out":2174,"would_cite":true,"duration_ms":19055,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By comparing three labels per image—ground truth, prediction, and neighbor-derived label—SUGE detects unreliable samples, then weights and corrects them, improving gaze estimation.","keywords":["gaze estimation","uncertainty estimation","label noise","triplet-label consistency","neighboring labels","sample weighting","label correction","co-training"],"falsifier":"Re-annotate a random subset of EyeDiap or Gaze360 training images independently and check whether samples flagged with low label confidence are precisely those whose original labels disagree with the re-annotation, and whether the corrected labels land closer to the re-annotation; if they do not, the triplet-label uncertainty is not measuring real label noise.","tokens_in":11159,"feed_emoji":"👁","tokens_out":10945,"duration_ms":88435,"temperature":0.7,"pith_summary":"Gaze datasets collected in real settings contain two kinds of bad training material: images that are occluded, blurred, or even show no face at all, and labels that do not line up with where the person actually looked. Training on such examples pulls gaze models away from accurate predictions. The paper proposes SUGE, which compares three labels for each training image—the ground-truth annotation, the model's own predicted pseudo label, and a neighboring label assembled from similar images' labels—and turns the pairwise angular disagreements into confidence scores for both the image and the label. Low image confidence down-weights the sample in the loss; low label confidence triggers a label correction that blends the original annotation with pseudo and neighboring labels. This is worth caring about because annotation and image noise are unavoidable in gaze datasets, so a training procedure that detects and suppresses them can improve gaze estimation without any extra supervision.","feed_headline":"A three-way label check cuts gaze error to 5.05° on EyeDiap","feed_subtitle":"The method weighs and corrects unreliable samples, beating noisy-label baselines on standard gaze benchmarks.","key_machinery":"The mechanism that carries the argument is the neighboring label, defined as the reconstruction-weighted average of ground-truth labels of the K nearest neighbors of an image in the encoder's feature space, restricted to the same person. It gives each sample a third label that is independent of both the possibly wrong annotation and the model's own prediction. Around this, the paper defines two uncertainty statistics: TupleMD, $\\min(D_{pg}, D_{ng})/(D_{pn} + \\epsilon)$, which scores label quality by how far the ground-truth label stands from both other labels, and TripleMD, $\\min(D_{pg}, D_{pn}, D_{ng})$, which scores image quality by the largest disagreement among any pair of the three labels. A two-component GMM converts these statistics into label and image confidences; sample weighting puts the image confidence into the loss $L = \\sum_i \\hat{W}_i \\|\\hat{Y}_i - f(E(X_i))\\|_1$, and label correction blends ground-truth, pseudo, neighboring, and flipped labels according to label confidence. A co-training loop lets two networks exchange their corrected labels and weights to avoid overconfidence in self-generated confidences.","core_discovery":"The central claim is that a triplet-label consistency measure can detect which gaze training samples are unreliable and then suppress the damage they cause, without needing a clean external set. For each sample, the method builds a neighboring label $\\hat{Y}^n_i = \\sum_{j=1}^{K} Y_{ij} A_{ij}$ from reconstruction-weighted ground-truth labels of the sample's K nearest same-person neighbors in the encoder's feature space, and places it alongside the ground-truth label $Y_i$ and the pseudo label $\\hat{Y}^p_i$. Two angular statistics, TupleMD and TripleMD, are fed to a two-component Gaussian Mixture Model, producing a label confidence and an image confidence. The label confidence drives label correction (a blend of ground truth, pseudo, neighboring, and horizontally flipped labels), the image confidence drives sample weighting, and two networks co-train with exchanged confidences. Concretely, the method lowers mean angular error on EyeDiap from 5.46° to 5.05° with the Gaze360 backbone and from 5.34° to 5.04° with the GazeTR backbone, and on Gaze360 from 10.78° to 10.52° with the Gaze360 backbone.","pith_inferences":["The neighboring-label premise could be stress-tested by running SUGE without reducing the feature dimension to 16; if gains shrink when nearest neighbors in high-dimensional space no longer align with label similarity, the local-smoothness assumption is the active ingredient.","The same triplet-label recipe should transfer to other continuous regression tasks with locally smooth labels, such as head-pose or body-orientation estimation, where noisy annotations and low-quality frames are also common.","The hard confidence truncation at $\\tau = 0.5$ discards partial evidence from moderately uncertain samples; a soft, temperature-scaled weighting might extract more signal from them.","Because corrected labels are partly built from the model's own pseudo labels, systematic model errors could reinforce annotation errors; co-training reduces but does not remove this risk, so testing with a third independent network is a natural safeguard."],"forward_implications":["With the Gaze360 backbone, SUGE lowers EyeDiap mean angular error from 5.46° to 5.05° and Gaze360 from 10.78° to 10.52°.","The same scheme improves the GazeTR transformer backbone on EyeDiap (5.34° to 5.04°) and Gaze360 (10.61° to 10.51°), showing the uncertainty module transfers across architecture families.","SUGE beats two noisy-label learning methods adapted to gaze regression, CoTeaching and DivideMix, on both EyeDiap and Gaze360 with the Gaze360 backbone.","Ablations attribute the gain to all designed parts: removing label correction costs 0.12°, removing sample weighting costs 0.05°, removing neighboring labels costs 0.11°, and switching from co-training to self-training costs 0.09°."],"supporting_citations":[{"why":"Provides the Gaze360 dataset and the CNN gaze-estimation backbone that SUGE wraps and improves.","marker":"Kellnhofer et al. 2019"},{"why":"Provides the GazeTR transformer backbone used as the second architecture for SUGE.","marker":"Cheng and Lu 2022"},{"why":"Supplies the data partitioning and preprocessing protocol that all comparisons follow.","marker":"Cheng et al. 2021"},{"why":"Defines the EyeDiap dataset, the main benchmark for ablations and parameter analysis.","marker":"Funes Mora, Monay, and Odobez 2014"},{"why":"Defines the MPIIFaceGaze dataset used to evaluate cross-dataset behavior.","marker":"Zhang et al. 2017b"},{"why":"Provides ETH-XGaze, the pretraining set for the GazeTR backbone.","marker":"Zhang et al. 2020b"},{"why":"Supplies the co-training strategy and the DivideMix noisy-label baseline compared in Table 2.","marker":"Li, Socher, and Hoi 2020"},{"why":"Supplies the Gaussian Mixture Model that turns the two uncertainty metrics into label and image confidences.","marker":"Permuter, Francos, and Jermyn 2006"},{"why":"Provides the CoTeaching noisy-label learning baseline adapted for gaze regression comparison.","marker":"Han et al. 2018"}],"fun_headline_variants":["Triplet-label consistency tames gaze uncertainty, hits 5.05° on EyeDiap","SUGE's triple-label check slashes gaze error to 5.05° on EyeDiap","Uncertainty in gaze? SUGE suppresses it with triple-label consistency","Triplet-label weighting corrects noisy gaze labels, down to 5.05°","Gaze error from 5.46° to 5.05°: SUGE suppresses uncertainty"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that images that look similar to the model should have similar gaze directions, so a weighted average of nearby images' labels is a trustworthy reference; if this local smoothness fails, the uncertainty scores, the sample weights, and the corrected labels are all biased.","fun_headline_variants_meta":{"raw":{"variants":["Triplet-label consistency tames gaze uncertainty, hits 5.05° on EyeDiap","SUGE's triple-label check slashes gaze error to 5.05° on EyeDiap","Uncertainty in gaze? SUGE suppresses it with triple-label consistency","Triplet-label weighting corrects noisy gaze labels, down to 5.05°","Gaze error from 5.46° to 5.05°: SUGE suppresses uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000899,"raw_usage":{"total_tokens":3901,"prompt_tokens":1004,"completion_tokens":2897,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":2781}},"tokens_in":620,"tokens_out":2897,"duration_ms":20480,"temperature":1.0,"reasoning_tokens":2781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:37:11.971134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random subset of EyeDiap or Gaze360 training images independently and check whether samples flagged with low label confidence are precisely those whose original labels disagree with the re-annotation, and whether the corrected labels land closer to the re-annotation; if they do not, the triplet-label uncertainty is not measuring real label noise.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Gaze360 dataset and the CNN gaze-estimation backbone that SUGE wraps and improves."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the GazeTR transformer backbone used as the second architecture for SUGE."},{"cited_title":"Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark","cited_arxiv_id":"2104.12668","evidence_quote":"Supplies the data partitioning and preprocessing protocol that all comparisons follow."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian Mixture Model that turns the two uncertainty metrics into label and image confidences."}],"review_version":1}