{"id":"b446e867-2ba5-480f-8a66-f8328b354287","arxiv_id":"2411.12146","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A masked autoencoder with per-location p-value inputs is claimed to denoise visual field data and detect glaucoma progression earlier, but the paper's own simulations do not consistently support the p-value benefit.","lead":"This paper tests whether masked autoencoders can denoise visual field measurements from glaucoma patients better than variational autoencoders, and whether adding per-location p-values helps. The clinical results are modest and inconsistent with the paper's own simulations, so the headline claim is not supported.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's own simulations contradict p-value benefit: MAE without p-values detects more progressing eyes under PLR and GRI in every glaucoma scenario, and the abstract's 4.7% does not match Table 2.","rationale":"The reader's verdict is REJECT, and I agree the paper should be rejected, but for a more fundamental reason than the reader's stated weakest assumption. The reader focuses on possible train/test leakage in the progression analysis; that is a valid concern but contingent on an unverified procedural detail. The internal contradiction is directly visible in the manuscript. Table 1 is a controlled simulation with known ground truth, designed exactly to test whether denoising preserves true signal. In that table, adding p-values to the masked autoencoder lowers PLR detection by 1.8 to 3.3 percentage points across the slow/medium/fast scenarios and lowers GRI detection by 8.2 to 6.7 points, while only MD rises slightly. The abstract's central claim—'including a categorical p-value ... improves the smoothing'—is therefore contradicted by the paper's own primary controlled evidence. The clinical table does not rescue it: Table 2 shows MAE+p ahead of MAE only on PLR, by 1.59 percentage points, and behind on MD and GRI. The abstract's 4.7% figure appears in neither Table 2 nor the Discussion, which says 1.59%. The methods also never state that progression outcomes were computed only on held-out test eyes, which matters because the Table 2 percentages are small. Given this internal inconsistency, the central claim as stated cannot stand. A focused re-analysis of Table 1 with proper paired statistics would settle the issue.","tokens_in":9171,"tokens_out":4128,"duration_ms":40556,"concrete_test":"Regenerate the 24 simulation scenarios using the code referenced from ref. 15, applying the trained MAE and MAE+p models, and compare PLR and GRI detection rates per scenario using paired McNemar tests or 95% confidence intervals. If MAE without p-values remains higher in a majority of progression scenarios, the paper's own controlled setup refutes the central claim; report sensitivity and specificity against the known ground-truth labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that including a categorical p-value at each visual field location improves denoising and, through it, progression detection. This claim is contradicted by the paper's own controlled simulations in Table 1. For every simulated glaucoma progression scenario (slow, medium, fast), the masked autoencoder without p-values detects a higher proportion of progressing eyes than the masked autoencoder with p-values under PLR (e.g., slow: 67.49% vs 65.69%; fast: 65.96% vs 62.63%) and under GRI (e.g., slow: 77.13% vs 68.95%; fast: 75.33% vs 68.35%). Only MD shows a slight advantage for p-values, and raw data already achieve near-ceiling MD detection, which is inconsistent with the narrative that denoising reveals signal rather than amplifying noise. The clinical results in Table 2 are also internally inconsistent: MAE+p improves PLR by only 1.59 percentage points over MAE (21.21% vs 19.62%), not the abstract's 4.7%, and MAE+p is worse than MAE on MD (36.51% vs 37.08%) and GRI (14.50% vs 17.05%). The Discussion itself says 1.59%, not 4.7%. Additionally, the progression analysis never states that only the held-out test set was used, so in-sample optimism cannot be excluded. Because the simulation is the only place where ground truth is known, its pattern is the most load-bearing evidence: it directly fails to support the claimed p-value benefit under the two progression metrics most sensitive to localized loss.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes self-supervised denoising of visual field (VF) data using variational autoencoders and masked autoencoders, with and without categorical per-location p-values as auxiliary inputs. The authors claim that including p-values improves denoising, that masked autoencoders outperform variational autoencoders, and that the proposed approach increases detection of progressing eyes and shortens time to progression. The paper evaluates the methods on a large clinical dataset of 4,232 patients and on simulated VF data with known ground truth, using PLR, MD, and GRI progression criteria.","tokens_in":9465,"tokens_out":4779,"duration_ms":43519,"significance":"If the claims were supported, the work would be clinically useful: denoised VF data that improve the accuracy and earliness of glaucoma progression detection would have immediate translational value for perimetry-based monitoring. The strengths of the paper include a large real-world dataset, a simulation framework with known ground truth, and a publicly available code repository (GitHub link provided). However, the central claim that p-values improve progression detection is contradicted by the paper's own controlled simulation results, and the abstract's headline numbers do not match the results tables. The lack of a clear statement that progression analysis was restricted to the held-out test set further undermines confidence in the clinical results.","major_comments":[{"comment":"The controlled simulation, which is the only setting with known ground truth, contradicts the central claim that p-values improve detection. Under PLR, the masked autoencoder without p-values detects more progressing eyes than with p-values in every glaucoma progression scenario (slow: 67.49% vs 65.69%; medium: 66.09% vs 64.10%; fast: 65.96% vs 62.63%). Under GRI, the same pattern holds (slow: 77.13% vs 68.95%; medium: 76.00% vs 69.68%; fast: 75.33% vs 68.35%). Only MD shows a slight advantage for p-values, but MD is near ceiling (~99%) for both variants and raw data already achieve ~100% detection, so the MD comparison is uninformative. These results directly undercut the abstract's assertion that 'including a categorical p-value at every visual field location improves the smoothing of visual field data.'","section":"Table 1, Results"},{"comment":"The abstract states 'A 4.7% increase in detection of progressing eyes with pointwise linear regression (PLR) was observed,' but no table supports this figure. Table 2a shows MAE+p detects 21.21% versus MAE's 19.62%, a 1.59 percentage-point difference, and the Discussion explicitly says 1.59%. Furthermore, the abstract's claim that p-values predict progression 2.3 months earlier 'when p-values were included' for both masked and variational autoencoders is contradicted by Table 2b: for VAE, conversion time is later with p-values than without for PLR (6.26 vs 6.07 years) and GRI (5.39 vs 4.50 years). The only 0.19-year earlier conversion is for MAE under PLR (7.01 vs 7.20 years). The abstract therefore misrepresents the paper's own numerical results.","section":"Abstract, Discussion, Table 2a"},{"comment":"The manuscript describes a train/validation/test split for model selection, but the Progression Analysis Methods section does not state that only the held-out test set was used to compute the progression statistics in Table 2 and the Kaplan-Meier analyses. The text says 'The VF data at each of the 52 test locations underwent noise reduction and reconstruction using both masked autoencoder and variational autoencoder algorithms,' where 'test locations' appears to refer to the 52 VF spatial locations rather than a held-out set of eyes. If the denoising networks were applied to training eyes when computing clinical progression rates, the small differences in Table 2 could be in-sample artifacts. The authors must clarify whether the progression analysis was restricted to the test set and, if not, re-run the analysis on the test set only.","section":"Progression Analysis Methods, Training Details"},{"comment":"No confidence intervals or p-values are reported for the progression-rate differences in Table 2, and the Kaplan-Meier curves in Figure 4 show overlapping confidence intervals; the text itself acknowledges 'considerable overlap in confidence intervals suggests a potential lack of statistical significance.' Without formal significance testing, a 1.59 percentage-point PLR difference (MAE+p vs MAE) cannot be distinguished from chance variation, especially given that the paper reports no patient-level clustering or multiple-comparison adjustments across the three progression metrics.","section":"Table 2, Kaplan-Meier Analysis"}],"minor_comments":[{"comment":"There are typographical errors in the Introduction, such as 'utilized utilized' and 'outputtedsuch,' and the phrase 'the same was as 15' in the Simulations subsection should read 'the same way as in 15.'","section":"Introduction"},{"comment":"The text references to figures are inconsistent: the masked autoencoder is referred to as 'Figure 4' in the Methods but the corresponding figure is labeled Figure 3, and the Kaplan-Meier discussion refers to 'Figure 2' and 'Figure 4' for the same plots. Please align the figure captions and in-text citations.","section":"Figures"},{"comment":"The Discussion says 'These improvements from masked autoencoders + p-values are visualized in Table 1, depicting results for the PLR and mean deviation metrics,' but Table 1 is the simulation results table, not the clinical results; this is presumably meant to refer to Table 2.","section":"Discussion"}],"recommendation":"reject","confidential_remarks":"The paper's central claim is contradicted by its own controlled simulation, and the abstract contains numbers that are not supported by the results tables. These are not presentation issues but fundamental discrepancies in the main conclusion. The data-leakage ambiguity (unclear whether progression analysis was restricted to the held-out test set) adds further concern. Even if the authors could clarify the test-set issue, the simulation results would still fail to support the p-value benefit under PLR and GRI. The manuscript would need a major reframing or new experiments to address these points, which is beyond a standard revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper's real contribution before the abstract gets to it: a clean comparison of masked autoencoder (MAE) versus variational autoencoder (VAE) denoising of visual field data, on a large clinic dataset, with the denoised outputs fed into three standard progression algorithms. That part is honest work. The MAE consistently beats the VAE on detection rates in their simulations, and the clinical Table 2 trends the same way, so the core architectural claim has real support.\n\nThe soft spot is the p-value claim, and it is not a nitpick. The paper's own controlled simulations, where ground truth is known, show MAE without p-values detecting more progressing eyes than MAE with p-values under both PLR and GRI in every glaucoma scenario. That is not ambiguous. Only MD shows a slight p-value advantage, and raw data already saturate MD. The abstract's 4.7% PLR improvement does not match Table 2's 1.59 percentage points, and the Discussion itself quotes 1.59%. So the central narrative, that including categorical p-values improves denoising and progression detection, is contradicted by the load-bearing evidence inside the paper.\n\nThe other issue is worth flagging but is less certain: the progression analysis never states that only the held-out test set was used. Given the train/validation/test split described in Methods, I would expect the progression statistics to come from the test set, but the paper does not say so, and that leaves an in-sample optimism hole. This is fixable with a sentence and a re-run.\n\nAlso minor: the abstract's '2.3 months earlier' combines MAE and VAE, while the discussion reports 0.19 years for MAE+p under PLR, which is consistent but not identical; the presentation is sloppy. The self-citation is fine since the VAE baseline comes from their prior work and the simulation algorithm is cited, so I do not read circularity into it.\n\nNet: the MAE-versus-VAE comparison is a legitimate, reproducible result and could be clinically useful as a preprocessing step. But the paper as written overstates the p-value benefit and even contradicts it internally. A thorough referee could salvage the MAE claim with a cleaner test-only evaluation and honest p-value reporting. I would send it out, but I would expect major revision before it is publishable.","headline":"A clinically motivated denoising comparison that fails on its own central claim: the paper's simulation data contradict the p-value benefit, but the MAE-vs-VAE comparison is worth a second look.","tokens_in":10062,"tokens_out":597,"would_cite":false,"duration_ms":8381,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that appending a categorical p-value at every visual field location to a masked autoencoder's input improves the smoothing of visual field data, so pointwise linear regression detects more progressing eyes and predicts…","keywords":["glaucoma progression","visual field denoising","masked autoencoder","variational autoencoder","pointwise linear regression","p-value encoding","self-supervised learning","perimetry"],"falsifier":"Recompute the PLR and time-to-progression comparison using only the held-out test eyes, and check whether MAE+p-values still beats MAE; if the 1.59 percentage-point gain and 0.19-year earlier conversion disappear, the p-value benefit is an artifact of training-data leakage.","tokens_in":8948,"feed_emoji":"👁️","tokens_out":13693,"duration_ms":134087,"temperature":0.7,"pith_summary":"Glaucoma is monitored with repeated visual field tests, but the measurements are noisy enough that progression algorithms can miss or lag real worsening. The paper argues that a self-supervised masked autoencoder — a network trained on 4,232 patients' visual field exams to reconstruct randomly zeroed-out locations of the 24-2 field — smooths the data better than raw fields or a variational autoencoder, and that appending a categorical p-value at every location makes the smoothing even more useful. In the clinical cohort, the p-value-augmented masked autoencoder flagged 21.21% of eyes as progressing by pointwise linear regression, versus 19.62% for the masked autoencoder alone, and shortened the average conversion time from 7.20 years to 7.01 years. If the claim holds, the denoised visual field could be used as a preprocessing stage in front of existing progression algorithms, improving detection without changing the downstream clinical method.","feed_headline":"Masked denoiser plus p-values catches glaucoma progression sooner","feed_subtitle":"Masked denoiser with per-location p-values detects more progressing eyes and cuts conversion time by 0.19 years.","key_machinery":"The load-bearing mechanism is a masked autoencoder built from feedforward layers. During training, ten of the 52 visual field locations are zeroed out, the 52 sensitivity values are concatenated with a 52-dimensional one-hot vector of categorical p-values, and the resulting 104-dimensional input is compressed to a 16-unit bottleneck and decoded back to 52 values; the mean squared error against the pre-masking values forces the network to infer the missing locations from the surrounding field and the p-value pattern. At inference no locations are masked, so the trained network acts as a denoiser, with the p-value vector carrying per-location information about deviation from normal into the latent representation. The same pipeline is run with a variational autoencoder as the comparison arm.","core_discovery":"On the paper's own terms, the central discovery is that masking out visual field locations during training and including a per-location p-value in the input produces a denoised visual field better matched to the task of detecting glaucomatous change. The masked autoencoder with p-values detected 21.21% of eyes as progressing under pointwise linear regression, compared with 19.62% for the masked autoencoder without p-values and 14.48% for raw visual field data, and it reduced the mean conversion time to 7.01 years from 7.20 years (masked autoencoder) and 7.80 years (raw data). The paper reads this as evidence that the categorical p-value acts as supplementary side information that helps the network encode how far each location deviates from normal, so the reconstructed field retains the true signal while dropping measurement noise. The proposed role of the denoiser is therefore a front-end for established progression algorithms such as PLR, MD, and GRI.","pith_inferences":["A natural follow-up would replace the binned one-hot p-values with a continuous per-location noise or deviation estimate; if the mechanism is uncertainty encoding, the continuous input could preserve or extend the gain without binning artifacts.","The masking trick could be applied across visits rather than across locations, training the network to reconstruct a dropped visit from neighboring visits; that variant would attack the longitudinal variability that most directly inflates pointwise regression slopes.","Comparing the p-value-augmented denoiser against an input of total deviation values, which already encode the same normative reference, would test whether the p-value gain comes from semantic side information or simply from the extra input dimension.","An ablation with p-values supplied only at masked locations would show whether the network uses them to repair missing data points or to reweight the whole field during encoding."],"forward_implications":["If the claim holds, adding per-location p-values is a practical upgrade to visual field denoising, because the p-value map is already exported with each visual field exam and no new measurement is required.","The denoised output plugs into pointwise linear regression without changing the regression itself, so existing glaucoma-monitoring software could adopt the preprocessing without replacing its progression metric.","The comparison with the variational autoencoder suggests that reconstructing missing locations is a stronger inductive bias for visual field smoothing than latent probabilistic compression, at least for this dataset and these progression metrics.","The p-value benefit is visible in PLR but not uniformly in MD or GRI, so the choice of denoised input may need to be tied to the downstream progression method rather than applied globally."],"supporting_citations":[{"why":"supplies the simulation algorithm used to generate noisy visual fields and test whether denoising preserves true glaucoma progression signal.","marker":"[15]"},{"why":"sets the variational-autoencoder denoising baseline whose results the paper replicates and benchmarks the masked autoencoder against.","marker":"[17]"},{"why":"introduces the variational autoencoder formulation used for the VAE comparison arm.","marker":"[18]"},{"why":"introduces masked autoencoders, the training strategy the paper adapts to a feedforward encoder-decoder for visual field data.","marker":"[21]"}],"fun_headline_variants":["Masked autoencoder denoises visual fields to catch glaucoma progression sooner","P-values in denoising network improve glaucoma progression detection","Self-supervised denoising boosts detection of glaucoma progression","Masked denoiser predicts glaucoma progression 2.3 months earlier","Denoising visual fields with masked autoencoders finds more glaucoma progression"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains rest on the assumption that progression statistics were computed only on eyes the denoisers never saw during training; the Methods do not explicitly state that, so if training eyes were included the improvements could be in-sample artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Masked autoencoder denoises visual fields to catch glaucoma progression sooner","P-values in denoising network improve glaucoma progression detection","Self-supervised denoising boosts detection of glaucoma progression","Masked denoiser predicts glaucoma progression 2.3 months earlier","Denoising visual fields with masked autoencoders finds more glaucoma progression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000372,"raw_usage":{"total_tokens":2047,"prompt_tokens":1060,"completion_tokens":987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":898}},"tokens_in":676,"tokens_out":987,"duration_ms":10180,"temperature":1.0,"reasoning_tokens":898,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:51:17.939925+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the PLR and time-to-progression comparison using only the held-out test eyes, and check whether MAE+p-values still beats MAE; if the 1.59 percentage-point gain and 0.19-year earlier conversion disappear, the p-value benefit is an artifact of training-data leakage.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the simulation algorithm used to generate noisy visual fields and test whether denoising preserves true glaucoma progression signal."},{"cited_title":"Efficacy of smoothing algorithms to enhance detection of visual field progression in glaucoma","cited_arxiv_id":null,"evidence_quote":"sets the variational-autoencoder denoising baseline whose results the paper replicates and benchmarks the masked autoencoder against."}],"review_version":1}