{"id":"14d1d489-f4b9-42a0-aea1-0fd4f7150df8","arxiv_id":"2411.16175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A test-time self-supervised super-resolution method that rescales degradation embeddings by an LPIPS-based quality score and regularizes SR features toward CLIP features improves LPIPS and NRQM on real-world benchmarks.","lead":"This paper combines a quality-based degradation controller with a CLIP-style feature alignment loss to make self-supervised super-resolution produce more natural high-resolution images during test-time fine-tuning. It reports improved perceptual quality over prior self-supervised methods on real-world SR benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains are not self-supervised: HR-guided early stopping on a validation split of the test set, plus finetuning on the full test set, makes Table 1 an artifact-prone transductive comparison; the headline perceptual improvements are not yet evidenced.","rationale":"The reader's weakest assumption about the scalar controller is plausible but secondary: the ablations suggest the controller contributes less than FAR, and even if the scalar-rescaling assumption is imperfect, a correct evaluation could still show gains. The protocol issue, by contrast, is the foundation of every number in Table 1. The paper is transparent about using a validation split of the testing data for early stopping; because the finetuning is on the whole testing set, this is effectively model selection by HR-based quality on training data. That is not self-supervised and can manufacture comparative wins. A fixed-budget, held-out evaluation is the one check that would settle whether the central claim lands. I keep a conditional posture rather than outright rejection because the method could survive the corrected protocol, but the condition is non-negotiable: redo the evaluation without HR-guided early stopping and without transductive test-set fitting.","tokens_in":12954,"tokens_out":12588,"duration_ms":118481,"concrete_test":"Re-run Table 1 under a genuinely LR-only protocol: finetune each method on a held-out LR subset of the target domain (or on the same domain's training split) with fixed iteration budgets of 300, 450, and 600, and evaluate on LR images never used for finetuning; do not use HR-based LPIPS on a validation split from the test set for early stopping. Report PSNR/SSIM/LPIPS/NRQM for Ours, LWay, SRTTA, and the off-the-shelf models at each budget. If Ours fails to beat the off-the-shelf model and LWay on held-out images across budgets, the reported perceptual gains are an artifact of the protocol.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Sec. 5.2 states that finetuning runs on the whole testing set and that for LWay and Ours 'we split a validation set from the testing data and use the early stopping strategy'; Fig. 9 then selects stopping iterations by LPIPS on those validation images. This means the final models in Table 1 are selected using ground-truth HR information on images that are also in the finetuning set. The reported LPIPS and NRQM numbers are therefore not achievable from LR-only self-supervision, and the comparison to SRTTA and to the off-the-shelf models is not controlled because those baselines are not given the same HR-guided iteration selection. The paper claims in Sec. 5.3 that the method is robust when early stopping is infeasible, but no table provides numbers for that setting; all headline comparisons use HR-guided early stopping. The controller and FAR contributions may be real, but their load-bearing evidence is the transductive, HR-supervised protocol. If the protocol is corrected, the central claim of state-of-the-art perceptual quality must be re-established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes two additions to the LR-reconstruction self-supervised SR framework of LWay: a controller that rescales the degradation embedding by an LPIPS-based quality indicator (HQI), and FAR, a Gram-matrix feature-alignment regularizer that encourages super-resolved outputs to match CLIP feature statistics of natural HR images. The method is applied by finetuning pretrained RealESRGAN+ and SwinIRGAN on target real-world LR test sets. Reported results on NTIRE20, AIM2019, and RealSR show improved LPIPS and NRQM over SRTTA, LWay, and the off-the-shelf models, with ablations attributing the gains to both components.","tokens_in":13173,"tokens_out":6941,"duration_ms":65571,"significance":"The two components are simple, intuitive, and clearly described, and the ablation structure is a sensible way to isolate their contributions. A notable strength is that the authors explicitly disclose the finetuning and early-stopping protocol, which allows the reader to assess the evidence. If the reported gains survive a clean evaluation protocol, the controller and FAR would be useful additions to test-time self-supervised SR. However, the central evidence is currently protocol-dependent, the LWay baseline is an unofficial reproduction, no code is released, and no error bars are reported, so the state-of-the-art claim is not yet established.","major_comments":[{"comment":"The headline evaluations select models using ground-truth HR information in a transductive protocol. Sec. 5.2 states that finetuning runs on the whole testing set and that for LWay and Ours 'we split a validation set from the testing data and use the early stopping strategy'; Fig. 9 then selects stopping iterations by LPIPS on those validation images. Thus the final models in Table 1 are chosen with access to HR references on images drawn from the same test sets, while the SRTTA and off-the-shelf baselines are not given the same HR-guided selection. This makes the reported improvements unattainable from LR-only self-supervision and breaks the comparability of Table 1. Please provide results under a clean protocol (fixed iteration count chosen without HR, or a validation split from an independent domain) and report both early-stopped and non-early-stopped numbers.","section":"Sec. 5.2, Sec. 5.3, Table 1, Fig. 9"},{"comment":"The only comparison to LWay uses an unofficial reproduction because LWay does not release source code or training data. Since Table 1's state-of-the-art claim depends on outperforming LWay, an uncontrolled reproduction can bias the comparison. Please release the reproduction details (training iterations, reconstruction-loss weights, and the exact validation schedule) and ideally use the official implementation or provide a verification check, such as reproducing LWay's published numbers on a common benchmark.","section":"Sec. 5.2"},{"comment":"No repeated runs, error bars, or significance tests are reported. Several headline margins are small (e.g., NTIRE20 with RealESRGAN+: LPIPS 0.2369 vs. 0.2498 for LWay; NRQM 6.137 vs. 6.121 for the off-the-shelf model). Without seed variance or a paired test, the improvements cannot be distinguished from noise. Please report mean plus/minus standard deviation over at least three seeds and apply a paired significance test for the main comparisons.","section":"Tables 1-4"},{"comment":"The controller's sign and the HQI ordering are assumed rather than derived. In Eq. (8), the pretraining and finetuning definitions use opposite signs of HQI, and the finetuning relation is justified only by the observation that a smaller LR/SR distance indicates more degradation in the SR output. Because LPIPS is also the primary evaluation metric, the objective and the metric are entangled; improvements in LPIPS may reflect optimization of the same distance rather than genuine perceptual quality. The paper should include a sensitivity analysis of the controller sign and evaluate with perceptual metrics not used in the loss (e.g., DISTS, NIQE, FID, or a human study). The NRQM gains mitigate this concern but do not eliminate it.","section":"Eqs. (7)-(8), Sec. 4.3"},{"comment":"The ablation table does not specify the evaluation dataset, and its baseline numbers (PSNR 26.2, SSIM 0.7889, LPIPS 0.2525, NRQM 4.511) do not match any off-the-shelf model row in Table 1 for NTIRE20, AIM2019, RealSR-Canon, or RealSR-Nikon. Please state the dataset and settings used for Table 2 and explain the discrepancy.","section":"Table 2"}],"minor_comments":[{"comment":"There are several typos: 'cosince' should be 'cosine' in Eq. (7); 'KernalGAN' should be 'KernelGAN' in Table 1; the Fig. 2 caption contains 'inX' instead of 'in X'; and the conclusion uses 'fintunes' instead of 'finetunes'.","section":"Eq. (7), Table 1, Fig. 2, Conclusion"},{"comment":"The interpolation experiment in Eq. (12) and Fig. 7 does not state which dataset is used, nor whether the controller s is applied during pretraining, finetuning, or both. Please add these details so the experiment can be reproduced.","section":"Sec. 5.3, LR reconstruction vs. SR performance"},{"comment":"No code availability statement is provided. Releasing the code and pretrained reconstruction network would substantially improve reproducibility, especially since the LWay baseline must be reimplemented.","section":"General"},{"comment":"The motivation for FAR relies on reference [30]'s observation that self-supervised tasks degrade under distribution shift, but the connection between that observation and Gram-matrix alignment with CLIP is informal. A short explanation of why this particular alignment should reveal distribution shift would help readers evaluate the design.","section":"Sec. 4.4, reference [30]"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is real and, on reading the paper, it lands: Sec. 5.2's early stopping on a validation split of the test set is disclosed in the text and is load-bearing for the Table 1 results. I am not rejecting the paper because the methodological ideas are plausible and the ablations are informative; however, the central claim of state-of-the-art self-supervised perceptual quality must be re-established under a protocol that does not use HR information for model selection, with variance reported and an official or carefully verified LWay baseline. If those changes are made, a positive outcome is likely."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this paper. The two proposed mechanisms—the quality-controlled degradation embedding (controller s) and the CLIP-based feature-alignment regularizer (FAR)—are sensible, and the ablations suggest each does something on the metrics tested. But the headline comparison in Table 1 is not a self-supervised evaluation. The authors finetune on the full test set and select the stopping iteration using LPIPS on a validation split of that same test set. Baselines SRTTA and the off-the-shelf models are not given the same HR-guided selection. So the reported LPIPS and NRQM gains are largely explained by selecting models with ground-truth HR information from the test set, not by the method's own self-supervision.\n\nWhat is actually new: grafting a scalar quality-scaled degradation controller and a CLIP feature-alignment regularizer onto the LWay reconstruction paradigm is a legitimate extension, though not a new framework. The controller design is heuristic—one scalar multiplier on a 512-dimensional embedding—but the paper gives a nice diagnostic (Fig. 2b) showing why plain LR reconstruction can favor degraded outputs. FAR is also plausible, and Table 2 supports that it helps perceptual metrics.\n\nSoft spots, in proportion: the evaluation protocol is the load-bearing flaw. Section 5.2 states they split a validation set from the testing data and use early stopping for LWay and Ours; Fig. 9 shows they pick the iteration with best LPIPS on those validation images. Since the test set is also the finetuning set, the model is selected by peeking at HR labels. The paper's robustness claim in Sec. 5.3 (that the method is appropriate when early stopping is infeasible) is not supported by any table—all Table 1 numbers use the HR-guided selection. There are also minor issues: no error bars, no significance tests, no code release, and LWay is an unofficial reproduction. The scalar controller assumption is heuristic but not fatal; the ablation confirms it helps.\n\nBottom line: the method is worth a look, but the claimed state-of-the-art perceptual quality is not evidenced. A corrected evaluation with a truly held-out validation set or no early stopping is needed. This paper is for people working on test-time adaptation for SR, and as a cautionary example of transductive evaluation pitfalls. I would send it to peer review, but with the requirement that the authors rerun without HR-guided early stopping and report both settings.","headline":"Plausible components undermined by a transductive evaluation that uses HR labels from the test set to pick the model; the method may work, but Table 1 doesn't show it.","tokens_in":13693,"tokens_out":2064,"would_cite":false,"duration_ms":21196,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A quality gauge and a feature-alignment regularizer let self-supervised super-resolution adapt to real-world domains without paired data.","keywords":["self-supervised super-resolution","real-world super-resolution","degradation embedding controller","feature-alignment regularizer","LPIPS","CLIP","test-time adaptation","perceptual quality"],"falsifier":"Construct a synthetic dataset where low-resolution images are degraded by independent and unequal amounts of blur and noise, then finetune an off-the-shelf SR model with the controller and FAR. If two images with identical HQI but different blur-noise mixes cause the controller to produce higher LPIPS after finetuning than the off-the-shelf model, or if sweeping the scalar $s$ while holding all else fixed changes reconstruction error along an unintended degradation axis, the scalar-control assumption fails.","tokens_in":12741,"feed_emoji":"🖼️","tokens_out":6560,"duration_ms":58262,"temperature":0.7,"pith_summary":"The paper argues that self-supervised super-resolution underperforms because it relies on degradation models built from low-resolution inputs alone, missing what high-resolution natural images look like. It proposes two additions: a controller that scales the degradation embedding according to how close the super-resolved output is to a bicubic upsampling of the input, and a feature-alignment regularizer that pulls the statistics of super-resolved images toward those of natural high-resolution images as judged by a fixed CLIP encoder. Finetuning off-the-shelf SR models on real-world test sets with these components lowers perceptual distance and raises no-reference quality scores while keeping fidelity roughly stable. If correct, this gives a practical recipe to adapt any pretrained SR model to a specific camera or domain without collecting paired high-resolution data.","feed_headline":"Quality gauge steers super-resolution to real-world domains","feed_subtitle":"A controller and feature-alignment regularizer adapt off-the-shelf SR models to real images without paired high-res data.","key_machinery":"The central mechanism is the low-resolution reconstruction network from the LWay paradigm—a degradation encoder, an image encoder, and a reconstructor—augmented by two new parts. The first is a scalar controller $s$ that multiplies the 512-dimensional degradation embedding $\\mathbf{e}_d$, defined as $s = n + (1 - \\mathrm{HQI})$ in pretraining and $s = n + \\mathrm{HQI}$ in finetuning, where $n$ is Gaussian noise and $\\mathrm{HQI} = 1 - \\mathrm{LPIPS}(f_\\uparrow(X^r), M(X^r))$ measures how close the super-resolved output is to a bicubic upsampling of the low-resolution input. The second is the Feature-Alignment Regularizer (FAR), which computes average- and max-pooled Gram-matrix statistics of features from a trainable image encoder and a frozen CLIP encoder, aligns their dimensions with linear layers, and penalizes the Frobenius-norm difference. FAR is trained on natural high-resolution images during pretraining, then applied to super-resolved outputs during finetuning, pulling the output distribution toward natural-image statistics.","core_discovery":"The paper establishes that the low-resolution reconstruction objective can be realigned with super-resolution quality by (i) multiplying the degradation embedding with a scalar controller that tracks the LPIPS distance between the super-resolved output and the bicubic-upsampled low-resolution input, and (ii) adding a feature-alignment regularizer that enforces the Gram-matrix statistics of the super-resolved output to match those of natural high-resolution images as encoded by a frozen CLIP image encoder. The controller prevents the SR model from degrading its own output to satisfy the reconstruction loss, while the regularizer constrains the output distribution toward natural-image statistics that are invisible in the low-resolution input alone. On NTIRE20, AIM2019, and RealSR benchmarks, finetuning RealESRGAN+ and SwinIRGAN with these components yields lower LPIPS and higher NRQM than prior self-supervised finetuning baselines, with comparable or better fidelity.","pith_inferences":["The HQI based on LPIPS ties the quality measure to the bicubic upsampling of the low-resolution input; a no-reference quality metric that does not depend on that reference could generalize to settings where bicubic upsampling is a poor baseline, such as heavily compressed low-resolution images.","FAR uses CLIP as the fixed encoder, which biases the distribution constraint toward semantic features; ablations show that an SR-trained encoder (EDSR) gives higher NRQM but worse fidelity, suggesting a multi-encoder ensemble might better balance perceptual quality and fidelity.","The controller scales the entire degradation embedding by a single scalar; a per-degradation-type adjustment (blur vs noise vs compression) could be more effective on real-world data where these degradations are independent, as the paper itself lists finer degradation adjustments as a future direction."],"forward_implications":["Any off-the-shelf SR model, GAN-based or transformer-based, can be adapted to a target real-world domain by finetuning with the reconstruction loss plus the quality-scaled controller and FAR, requiring no paired high-resolution data.","FAR constrains the distribution of super-resolved outputs toward natural-image statistics, which should reduce over-smoothing and unnatural textures that pure reconstruction objectives produce, as reflected in higher NRQM and lower LPIPS.","The controller's dependence on HQI makes the finetuning self-regulating: if the SR model degenerates toward bicubic interpolation, the controller increases the degradation load, preventing the model from trivializing the reconstruction objective.","Finetuning becomes more stable over many iterations: the paper shows LPIPS does not degrade after 600 iterations with FAR, whereas the LWay baseline overfits the reconstruction objective and suffers performance drops.","The method operates on a whole test set with consistent degradation, making it suitable for camera-specific adaptation rather than per-image optimization."],"supporting_citations":[{"why":"Supplies the low-resolution reconstruction network (degradation encoder, image encoder, reconstructor) and the pretraining/finetuning paradigm that the method extends.","marker":"[4]"},{"why":"Provides the observation that a well-trained self-supervised task shows a significant performance drop on distribution-shifted test data, motivating the design of FAR.","marker":"[30]"},{"why":"Provides the fixed CLIP image encoder whose features FAR aligns with as a prior for natural high-resolution images.","marker":"[23]"},{"why":"Defines LPIPS, used both to build the High-resolution Quality Indicator in the controller and as a perceptual evaluation metric.","marker":"[37]"},{"why":"Supplies the off-the-shelf RealESRGAN+ model and the second-order degradation pipeline used to synthesize pretraining data.","marker":"[32]"}],"fun_headline_variants":["Controller tunes degradation for realistic super-resolution","Feature alignment brings natural look to self-supervised SR","Self-supervised SR gains high-res awareness via quality gauge","Real-world super-resolution refined with CLIP-based regularizer","No ground truth needed: SR tuned with quality-aware controller"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a single scalar—how similar the super-resolved output is to the bicubic upsampling of the low-resolution input—correctly orders images by degradation extent, and that multiplying the whole degradation embedding by this scalar changes only the amount of degradation without altering its type.","fun_headline_variants_meta":{"raw":{"variants":["Controller tunes degradation for realistic super-resolution","Feature alignment brings natural look to self-supervised SR","Self-supervised SR gains high-res awareness via quality gauge","Real-world super-resolution refined with CLIP-based regularizer","No ground truth needed: SR tuned with quality-aware controller"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1374,"prompt_tokens":878,"completion_tokens":496,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":420}},"tokens_in":494,"tokens_out":496,"duration_ms":4686,"temperature":1.0,"reasoning_tokens":420,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:27:39.736994+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a synthetic dataset where low-resolution images are degraded by independent and unequal amounts of blur and noise, then finetune an off-the-shelf SR model with the controller and FAR. If two images with identical HQI but different blur-noise mixes cause the controller to produce higher LPIPS after finetuning than the off-the-shelf model, or if sweeping the scalar $s$ while holding all else fixed changes reconstruction error along an unintended degradation axis, the scalar-control assumption fails.","supporting_citations":[{"cited_title":"Low-res leads the way: Improving generalization for super- resolution by self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Supplies the low-resolution reconstruction network (degradation encoder, image encoder, reconstructor) and the pretraining/finetuning paradigm that the method extends."},{"cited_title":"Convolu- tional visual prompt for robust visual perception","cited_arxiv_id":null,"evidence_quote":"Provides the observation that a well-trained self-supervised task shows a significant performance drop on distribution-shifted test data, motivating the design of FAR."},{"cited_title":"The unreasonable effectiveness of deep features as a perceptual metric","cited_arxiv_id":null,"evidence_quote":"Defines LPIPS, used both to build the High-resolution Quality Indicator in the controller and as a perceptual evaluation metric."}],"review_version":1}