{"id":"f7f75f10-b974-470c-9e0b-bf2a7ecf0b43","arxiv_id":"2412.01203","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A saliency-supervised VAE that converts fundus images into unadversarial examples is claimed to improve frozen-model DR grading under a new model-agnostic online adaptation setting.","lead":"Diabetic retinopathy screening models often fail when hospital images differ from the training data. This paper proposes a setting where the model is hidden and patient data streams in, then uses a VAE to turn images into saliency-like \"unadversarial examples\" to help the frozen model, reporting modest accuracy gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical foundation is unsound: ∂δ0/∂x is undefined or zero for independent noise, and Theorem 2's ∂s/∂x ∝ s is not established; with no model-agnostic baseline, the central claim is unsupported.","rationale":"The reader's weakest assumption identifies the same load-bearing defect: the paper's theory requires ∂δ0/∂x to be a well-defined, image-dependent quantity bounded by the saliency map. In the canonical iteration, δ0 is random noise independent of x, so the derivative is zero or undefined. The proof of Theorem 2 also contains a clear non sequitur: Eq. (27) does not imply ∂s/∂x ∝ s. Because the method's name, architecture, and supervision choice are all justified through these theorems, the theoretical contribution collapses. The empirical tables show moderate gains and a plausible small-batch robustness effect, but they are not decisive because no model-agnostic baseline with error bars is given; without such a baseline, the gains could be due to generic input smoothing or contrast change rather than to the proposed saliency-bounded generative perturbation. The reader's REJECT verdict is therefore appropriate, and no adjustment is needed.","tokens_in":20844,"tokens_out":5804,"duration_ms":51024,"concrete_test":"Perform one analytical check: write out the generative process actually implemented in Eq. (4), where the latent is z = Eτ(x) and the perturbation is δ = D(z). Then evaluate ∂δ0/∂x under the initialization used in Eq. (10). If δ0 is sampled independently of x, then ∂δ0/∂x = 0, making the latent argument of FΦ in Theorem 1 constant across inputs and contradicting the encoder's dependence on x; if δ0 is instead redefined as z(x, τ), then Eq. (10) is no longer the iterative process being reformulated, so Theorem 1 is not a reformulation of conventional unadversarial learning. Separately, numerically estimate ∂s/∂x from Eq. (21) on a sample of APTOS images and regress it on s; if the proportionality ∂s/∂x ∝ s fails, Theorem 2 is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that GUES is a theoretically grounded, model-agnostic adaptation method—rests on Theorem 1 and Theorem 2. Theorem 1 (Supp. 8.1, Eq. 11–20) is effectively a tautology: after deriving an upper bound δ_{k+1} ≤ δ0 + Um[k + ∂δ0/∂x + Σ h_i(∂δ0/∂x)], the proof defines FΦ(⋅) as that entire bracket and then says 'let V be a value that makes the equality relationship hold.' Since V and FΦ can be chosen after the fact, any iterative sequence can be represented this way; no generative reformulation is established. More seriously, Theorem 2 (Supp. 8.2, Eq. 22–30) differentiates the initial random noise δ0 with respect to the input x. In the conventional iteration defined in Eq. (10), δ0 is an independently sampled initialization, not a function of x; hence ∂δ0/∂x is zero (or undefined). The proof then invokes the chain rule ∂δ0/∂x = (∂δ0/∂s)(∂s/∂x), which presupposes δ0 is a function of the saliency map s, and concludes ∂s/∂x ∝ s from Eq. (27). But Eq. (27) yields (1/2 − Σ IΔi/IΔ), a value depending on the perturbation Δx, not on s; the proportionality is asserted, not derived. Consequently, the identification of the VAE latent z with ∂δ0/∂x and the use of the saliency map as an upper bound (Sec. 4.2) have no valid basis. The experimental comparison is also not controlled for the model-absence assumption: SFDA/TTA baselines such as SHOT and TENT access source-model parameters, while GUES does not, and no model-agnostic input-transformation baseline (e.g., histogram equalization, contrast normalization, or random noise) is provided. The reported +4.5% ACC / +2.8% QWK therefore cannot be attributed to the proposed unadversarial mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new setting, OMG-DA (Online Model-aGnostic Domain Adaptation), in which a frozen, unseen source model is adapted to streaming unlabeled target data without access to the model's parameters or source data. To operate in this setting, the authors introduce GUES, a VAE-based method that generates per-image perturbations by regressing the augmented image onto its fine-grained saliency map. The paper claims two theorems: Theorem 1 reformulates iterative unadversarial perturbation optimization as a generative function of a latent variable (the derivative of initial noise with respect to the input), and Theorem 2 asserts that the saliency map provides an upper bound on this latent variable. Experiments on four diabetic retinopathy datasets across 12 transfer tasks report average improvements of 4.5% ACC and 2.8% QWK over the unaugmented source model, as well as gains when GUES is combined with TTA methods such as SHOT-IM and TENT.","tokens_in":21251,"tokens_out":3361,"duration_ms":30223,"significance":"The OMG-DA setting is clinically motivated and genuinely interesting: model absence, source-data privacy, and streaming target data are all realistic constraints in hospital deployment. The 12-task evaluation across four DR datasets and the batch-size robustness study are useful empirical contributions, and the paper is commendably transparent about its limitations for natural images. However, the claimed theoretical grounding is the central novelty, and it is not sound. If the theory were removed, the method would reduce to an empirical image-to-saliency regression with no demonstrated advantage over simpler model-agnostic input transforms. The empirical evaluation also lacks a model-agnostic control baseline, so the central claims are currently unsupported.","major_comments":[{"comment":"Theorem 1 is vacuous as proven. The proof bounds δ_{k+1} by δ0 + U_m[k + ∂δ0/∂x + h_1(∂δ0/∂x) + ... + h_k(∂δ0/∂x)] and then defines F_Φ(∂δ0/∂x) to be exactly the bracketed expression and chooses V 'to make the equality relationship hold.' Because F_Φ and V are constructed after the fact to absorb the upper bound, any iterative sequence can be represented in the form δ_k = δ0 + V·F_Φ(∂δ0/∂x); no substantive generative reformulation is established. This is load-bearing because Section 4.2 uses Theorem 1 to justify replacing iterative optimization with a VAE that predicts perturbations.","section":"Supplementary Sec. 8.1, Eqs. (19)-(20)"},{"comment":"Theorem 2 is not established. In Eq. (10), δ0 is an independently sampled random initialization, not a function of the input x, so ∂δ0/∂x is either zero or undefined. The proof invokes the chain rule ∂δ0/∂x = (∂δ0/∂s)(∂s/∂x), which presupposes that δ0 is a differentiable function of the saliency map s; no such dependency is given. Moreover, Eq. (27) yields ∂s/∂x = 1/2 − Σ_i I_{Δi}/I_Δ in the treated case, a quantity that depends on the perturbation Δx and is not shown to be proportional to the saliency map s. The conclusion ∂s/∂x ∝ s and hence ∂δ0/∂x ≤ U·s is asserted rather than derived. This invalidates the theoretical basis for choosing saliency maps as pseudo-perturbation labels and for the bound used in Section 4.2.","section":"Supplementary Sec. 8.2, Eqs. (23)-(30)"},{"comment":"The identification of the VAE latent variable z with ∂δ0/∂x is unsupported once Theorems 1 and 2 fail. The claim that the KL term L_KL 'ensures the learning of ∂δ0/∂x' and that the saliency map 'provides an upper bound on the function input' rests entirely on those theorems. As it stands, the method is an empirical regression of x_t + δ_t onto the saliency map g_t; the 'generative unadversarial examples' interpretation and the theoretical motivation for the specific loss are not justified.","section":"Sec. 4.2, Eqs. (4)-(8)"},{"comment":"The experimental comparison does not control for the model-absence assumption. The OMG-DA setting forbids access to the source model, yet the main baselines in Table 2 (SHOT, NRC, CoWA, PLUE, TPDS, SHOT-IM, TENT, SAR) access model parameters or outputs. No model-agnostic input-only baseline (e.g., histogram equalization, contrast enhancement, or other image transforms) is included. The reported 4.5% ACC / 2.8% QWK improvement over the unaugmented source model therefore cannot be attributed to GUES's specific mechanism, since any input transformation that shifts the target distribution could produce similar gains.","section":"Sec. 5.3 and Table 2"}],"minor_comments":[{"comment":"There is a typo in the subsection title: 'Souce model pre-training' should read 'Source model pre-training.'","section":"Sec. 5.2"},{"comment":"The word 'demostrate' should be 'demonstrate'.","section":"Conclusion"},{"comment":"The subsection title 'Visualization of unadversaisal examples' contains a typo; it should be 'unadversarial examples.'","section":"Sec. 5.5"},{"comment":"The algorithm input lists the VAE parameters but not the hyperparameters α and β, and step 6 refers to g_i before defining how the saliency map is computed; adding these details would improve reproducibility.","section":"Algorithm 1"},{"comment":"The batch-size comparison reports point estimates without error bars or significance tests, so the claimed robustness of GUES at small batch sizes is not statistically supported.","section":"Sec. 5.4 and Fig. 4"},{"comment":"The accuracy formula ACC = (TP+TN)/(TP+TN+FP+FN) is written for binary classification, but the DR grading task is multiclass; the macro- or micro-averaging scheme should be specified.","section":"Eq. (31)"}],"recommendation":"reject","confidential_remarks":"The OMG-DA setting is a meaningful contribution and the empirical setup is extensive, but the theoretical core of the paper—Theorems 1 and 2—is unsound in ways that cannot be repaired by local edits, and the experimental section lacks the model-agnostic baseline needed to support the central claim. I would recommend rejection, though the authors may wish to resubmit a revised version that either removes the theoretical pretensions and frames GUES as a purely empirical method with appropriate baselines, or supplies a genuinely valid derivation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nQuick take on arXiv:2412.01203. The OMG-DA setting is a real practical need, and the saliency-supervised VAE is a simple empirical idea that might have legs. But the paper's central intellectual claim doesn't hold: the two theorems that are supposed to ground the method are not valid, and the experiments don't isolate the setting the paper claims to address.\n\nWhat's genuinely new: the problem statement—adapting a frozen, fully hidden model to a stream of unlabeled target images—is clearly motivated by privacy and data-flow constraints in hospitals. The paper is readable, the method is simple, and there's a real experimental effort: 12 transfer tasks across four DR datasets, ablations, and batch-size robustness. The finding that GUES improves small-batch stability when combined with TENT and SHOT-IM is a useful empirical signal, if it survives scrutiny.\n\nWhere it falls apart:\n\n(1) Theorem 1 is a tautology. The proof explicitly defines F_Phi and V after the fact to reproduce the sum, so any iterative sequence could be 'reformulated' that way. It doesn't establish a generative reformulation.\n\n(2) Theorem 2 differentiates the initial random noise delta0 with respect to the input x. Since delta0 is sampled independently of x, this derivative is zero or undefined. The paper never supplies a parameterization where delta0 depends on x, so the claimed upper bound by the saliency map is not a theorem. The subsequent identification of the VAE latent with d delta0/dx is unsupported.\n\n(3) The theory is load-bearing: the paper sells GUES as theoretically grounded, and the saliency-map supervision is justified by that bound. Remove the theory and the method becomes a heuristic that needs empirical validation on its own.\n\n(4) The experimental comparison is not controlled for the model-absence assumption. SHOT, NRC, TENT, etc. access source-model parameters and are therefore out-of-setting baselines. There is no model-agnostic baseline—no histogram equalization, contrast normalization, or simple input noise. So the +4.5% ACC / +2.8% QWK over the frozen source model cannot be attributed to the proposed mechanism. Per-task numbers also vary widely; several tasks get worse, so average improvements may be driven by a few favorable tasks. No error bars or code make that hard to judge.\n\nThat said, I'm not convinced the core idea is worthless. The Table 3 ablations suggest both losses and the saliency target do something, and small-batch robustness is a practically valuable property. A revision that drops the invalid theory, adds fair black-box baselines, and reports variance might produce a useful paper for the clinical transfer community.\n\nMy call: this deserves a serious referee rather than a desk reject—the setting is novel and the empirical question is testable—but as it stands it should be rejected. If you send it out, ask referees to benchmark GUES against simple input transforms and to check whether the saliency regression is actually doing the work.\n\nBest","headline":"OMG-DA is a useful new problem statement and the saliency-supervised VAE shows some empirical promise, but the theoretical framing is vacuous and the comparisons don't isolate the model-absence setting.","tokens_in":21871,"tokens_out":6177,"would_cite":false,"duration_ms":52899,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A VAE trained on saliency maps can adapt a frozen, unseen diabetic-retinopathy model to new hospital data.","keywords":["diabetic retinopathy grading","online model-agnostic domain adaptation","unadversarial examples","test-time adaptation","source-free domain adaptation","variational autoencoder","saliency map","fundus image domain shift"],"falsifier":"Compute the Jacobian $\\partial \\delta_0 / \\partial x$ for the actual random noise $\\delta_0$ used in GUES and check numerically whether its magnitude is bounded by $U \\cdot s$ across all pixels and all 12 transfer tasks; a single violation, or a case where the derivative is undefined, would falsify Theorem 2 and remove the theoretical reason for using saliency maps.","tokens_in":20575,"feed_emoji":"👁️","tokens_out":6428,"duration_ms":52487,"temperature":0.7,"pith_summary":"This paper proposes a way to adapt a diabetic-retinopathy grading model to a new hospital's images when the model itself is hidden and the new images arrive as a stream—a setting it calls Online Model-aGnostic Domain Adaptation (OMG-DA). Instead of retraining or even touching the model, it rewrites each incoming image into a generative unadversarial example: a small, individualized perturbation that makes the image easier for the hidden model to grade. The perturbation generator is a Variational AutoEncoder trained to regress target images onto their own saliency maps, with the saliency map doing double duty as pseudo-label and theoretical upper bound. Across 12 source-target tasks built from four fundus datasets, the approach raises average accuracy by 4.5% and quadratic weighted kappa by 2.8% over the untouched source model, and it stabilizes existing test-time adaptation methods at small batch sizes. The paper's point is that domain adaptation can be done from the data side alone, with no model access, no source data, and no labels.","feed_headline":"Saliency-trained VAE adapts hidden DR model, +4.5% ACC","feed_subtitle":"A VAE rewrites each incoming fundus image so a hidden grading model performs better across 12 hospital-to-hospital tasks.","key_machinery":"The load-bearing object is the generative unadversarial perturbation function $F_\\Phi(\\partial \\delta_0 / \\partial x)$, instantiated as a Variational AutoEncoder. The encoder, with the reparameterization trick, samples the latent variable $z$ that stands for the derivative of the initial random noise with respect to the image, while the decoder generates the individual perturbation $\\delta_t$. A by-pass connection adds $\\delta_t$ to the input image to form the unadversarial example $\\hat{x}_t$, and the training loss is $L_{\\mathrm{GUES}} = \\alpha D_{\\mathrm{KL}}(q(z|x_t) \\| \\mathcal{N}(0,I)) + \\beta \\|\\hat{x}_t - g_t\\|^2$, where $g_t$ is the fine-grained saliency map of $x_t$. The saliency map thus serves simultaneously as the regression target, the carrier of lesion-related signal, and the theoretical upper bound that the paper uses to justify identifying the latent input.","core_discovery":"The central claim is that GUES—a VAE that turns an input fundus image into an individual perturbation by regressing toward the image's fine-grained saliency map—can adapt a completely unseen frozen source model to a streaming target domain. The method reformulates the iterative perturbation optimization of unadversarial learning into a generative form, $\\delta_k = \\delta_0 + V F_\\Phi(\\partial \\delta_0 / \\partial x)$, in which the encoder with reparameterization supplies the latent input and the decoder acts as the perturbation generation function. The saliency map is chosen as the pseudo-perturbation label because it highlights hemorrhages, exudates, and other DR-relevant lesions, and because the paper proves (Theorem 2) that the saliency map bounds the latent input. In experiments on APTOS, DDR, DeepDR, and Messidor-2, GUES outperforms the unaugmented source model by 4.5% average ACC and 2.8% average QWK over 12 transfer tasks, and when prepended to SHOT-IM or TENT it improves those trainable test-time methods as well.","pith_inferences":["Editorial extension: if the saliency-bound argument is the only reason for choosing saliency maps, then an ablation with a generic edge map or a random fixed target would separate the bound's role from the VAE's denoising effect; the paper's ablations replace saliency with the input image or a mixup, but not with a non-lesion structural map.","Editorial extension: the method's stated limitation implies a natural boundary—on natural images with rich backgrounds the saliency bound constant $U$ becomes too loose, so the same recipe would likely fail outside medical fundus photography; a transfer to chest X-rays or dermoscopy, where backgrounds are more contained, would test how far the data-centric trick generalizes.","Editorial extension: the fact that perturbations are individualized rather than class-specific suggests GUES is learning a style or domain translation rather than a label-conditioned shift; comparing GUES perturbations with oracle-trained Grad-CAM maps could reveal whether the hidden model's actual failure modes are being corrected or merely generic contrast enhancement."],"forward_implications":["A hospital can deploy a proprietary grading model without revealing its parameters or architecture and still improve its accuracy on the hospital's own incoming images.","GUES can be composed with existing trainable test-time adaptation methods such as SHOT-IM and TENT, improving their accuracy and reducing their sensitivity to small batch sizes.","Because the generator is trained only on unlabeled target images and their saliency maps, it does not require labels, source data, or model access, which addresses data-privacy and model-privacy constraints in clinical deployment.","The visualization analyses suggest the generated perturbations direct the hidden model's attention to hemorrhages, soft exudates, and hard exudates, the features clinicians use for grading."],"supporting_citations":[{"why":"defines unadversarial examples and the iterative perturbation optimization that GUES reformulates generatively","marker":"[24]"},{"why":"supplies the VAE used to instantiate the perturbation generation function with a latent input variable","marker":"[10]"},{"why":"provides the fine-grained saliency map computation used both as pseudo-perturbation label and theoretical upper bound","marker":"[19]"},{"why":"provides the SHOT-IM baseline and the target method that GUES is combined with in the trainable case","marker":"[15]"},{"why":"provides the TENT baseline and the target method that GUES is combined with in the trainable case","marker":"[35]"}],"fun_headline_variants":["Model-blind VAE adapts DR grading on streaming data","Hidden model? VAE adapts DR grading with saliency maps","GUES: VAE perturbations adapt unseen DR model, +4.5% ACC","Streaming DR images adapt frozen model via generative tweaks","Data-centric DR adaptation: VAE mimics perturbations, no model needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole saliency-map choice rests on Theorem 2's claim that the derivative of the initial random noise with respect to the image exists and is bounded by the saliency map; since that noise is drawn independently of the image, the derivative may be zero, undefined, or only boundable under a specific parameterization the paper does not give, and if it fails, the saliency pseudo-labels lose their theoretical justification.","fun_headline_variants_meta":{"raw":{"variants":["Model-blind VAE adapts DR grading on streaming data","Hidden model? VAE adapts DR grading with saliency maps","GUES: VAE perturbations adapt unseen DR model, +4.5% ACC","Streaming DR images adapt frozen model via generative tweaks","Data-centric DR adaptation: VAE mimics perturbations, no model needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2499,"prompt_tokens":1041,"completion_tokens":1458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":1367}},"tokens_in":657,"tokens_out":1458,"duration_ms":9778,"temperature":1.0,"reasoning_tokens":1367,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:36:20.685320+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Jacobian $\\partial \\delta_0 / \\partial x$ for the actual random noise $\\delta_0$ used in GUES and check numerically whether its magnitude is bounded by $U \\cdot s$ across all pixels and all 12 transfer tasks; a single violation, or a case where the derivative is undefined, would falsify Theorem 2 and remove the theoretical reason for using saliency maps.","supporting_citations":[{"cited_title":"Unadversarial exam- ples: Designing objects for robust vision","cited_arxiv_id":null,"evidence_quote":"defines unadversarial examples and the iterative perturbation optimization that GUES reformulates generatively"},{"cited_title":"Human detection using a mobile platform and novel features derived from a visual saliency mechanism","cited_arxiv_id":null,"evidence_quote":"provides the fine-grained saliency map computation used both as pseudo-perturbation label and theoretical upper bound"},{"cited_title":"Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation","cited_arxiv_id":null,"evidence_quote":"provides the SHOT-IM baseline and the target method that GUES is combined with in the trainable case"},{"cited_title":"Tent: Fully test-time adaptation by entropy minimization","cited_arxiv_id":null,"evidence_quote":"provides the TENT baseline and the target method that GUES is combined with in the trainable case"}],"review_version":1}