{"id":"8421f055-2ac5-4d00-85ff-83656b88d42d","arxiv_id":"2412.07149","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"By fine-tuning Stable Diffusion on a curated aesthetic dataset with learned quality-specific prompt tokens, RAP-SR improves the perceptual quality of several diffusion-based super-resolution methods, as measured by no-reference metrics.","lead":"This paper fine-tunes a pretrained image diffusion model on 5,000 carefully selected high-quality images, adding learned prompt tokens for 'high quality' and 'low quality', then drops the tuned model into existing super-resolution pipelines to make outputs look sharper and more realistic. It reports consistent improvements on no-reference quality scores, but ships no code or data and uses the same metrics for dataset curation and evaluation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No-reference metric gains may reflect curation bias; without human evaluation, the realism claim is unsupported.","rationale":"The reader's identified weakest assumption concerns the negative identifier being trained only on synthetic RealESRGAN degradations and potentially failing to transfer to real-world degradations. That is a legitimate mechanism-level concern. However, the most load-bearing issue for the central claim is more fundamental: the evidence used to establish 'improved realism' is constituted by no-reference metrics that overlap with the metrics used to curate the training data. The HFAID dataset was selected to maximize MANIQA and CLIPIQA, and the fine-tuned model is trained on that dataset. Evaluating on those same metrics is therefore circular to an important degree. Even if the negative-prompt mechanism transfers perfectly to real degradations, the reported metric gains could be largely an artifact of the model's prior shifting toward HFAID-like images, which were chosen specifically to score high on those metrics. The paper even shows LPIPS consistently worsens with RAP-SR, indicating a loss of fidelity that is compensated by higher no-reference scores. Without a human preference study or evaluation on a metric not used in curation, we cannot determine whether RAP-SR truly improves restoration realism or merely biases outputs toward a metric-pleasing aesthetic. This does not make the method implausible, but it raises the bar for evidence. The reader's CONDITIONAL verdict is appropriate; the missing human study is an additional condition that should be explicitly required. Hence I do not change the verdict, but I partially disagree with the reader's focus on the degradation-distribution mismatch rather than the metric-circularity concern, which I regard as more load-bearing.","tokens_in":98,"tokens_out":3615,"duration_ms":55194,"concrete_test":"Run a forced-choice human perceptual study on 100 images from RealSR and DrealSR, comparing each baseline (StableSR, DiffBIR, SeeSR) against its RAP-SR version. Each pair is shown side-by-side in random order to at least 20 naive raters; if RAP-SR is preferred at a statistically significant level (e.g., p<0.05, two-tailed binomial test), the realism claim is provisionally supported. Otherwise, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that RAP-SR enhances restoration prior and improves realism when plugged into diffusion-based SR methods. The only quantitative evidence is gains on no-reference quality metrics (MANIQA, MUSIQ, CLIPIQA, BRISQUE) in Table 1. However, the HFAID training set was explicitly curated using MANIQA, CLIPIQA, and NIQE as selection criteria (Section 3.1, Appendix B). The fine-tuned base model is trained to reproduce HFAID images, shifting its output distribution toward images that score highly on those same metrics. Consequently, the reported metric improvements are not independent evidence of improved realism; they may simply reflect that the model now generates images that resemble the metric-selected training set, which could be stylistically biased rather than more faithful to the input LR image. This concern is reinforced by the consistent LPIPS degradation in Table 1, indicating lost perceptual fidelity to the original image. The paper offers no human perceptual study and no comparison with current state-of-the-art SR methods, despite claiming 'state-of-the-art results.' Thus the load-bearing assumption that no-reference metric gains equate to enhanced restoration realism is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RAP-SR, a plug-and-play enhancement for diffusion-based real-world image super-resolution. The authors curate a 5,000-image High-Fidelity Aesthetic Image Dataset (HFAID) using a four-stage Quality-Driven Aesthetic Image Selection Pipeline (QDAISP) that filters on no-reference quality metrics and a multi-modal aesthetic model. They then fine-tune Stable Diffusion 2.1 on HFAID with a restoration-oriented prompt optimization scheme: positive and negative restoration identifier tokens are appended to semantic captions, with negative samples generated by the RealESRGAN degradation pipeline (Algorithm 1). At inference, classifier-free guidance combines positive and negative identifier prompts (Eq. 3). The fine-tuned model is used to replace the base diffusion model in StableSR, DiffBIR, and SeeSR, and is evaluated on DIV2K, RealSR, and DrealSR using both full-reference and no-reference metrics. The central claim is that this prior enhancement improves the realism of diffusion-based super-resolution, with consistent no-reference metric gains and state-of-the-art results.","tokens_in":17211,"tokens_out":6634,"duration_ms":68662,"significance":"If validated, RAP-SR would be a useful orthogonal contribution: it is a lightweight, method-agnostic way to inject restoration-aware priors into existing diffusion-based SR pipelines, and HFAID could be reused by the community. The paper's strengths include the simple plug-and-play design, the consistent CLIPIQA and BRISQUE improvements across three base methods in most rows of Table 1, the ablations on dataset size and prompt design, and the commitment to releasing code and data. However, the evidence supporting the central realism claim is currently incomplete: the no-reference metrics used for evaluation overlap with the metrics used to curate the training set, no human perceptual study is provided, LPIPS degrades in most rows, and there is no comparison with current state-of-the-art Real-SR methods. These issues are addressable in revision, but they are load-bearing for the paper's conclusions.","major_comments":[{"comment":"The evaluation is partly circular. HFAID is curated by selecting images with high CLIPIQA, MANIQA, and NIQE scores, and the main results in Table 1 are reported on the same no-reference metric family, including MANIQA, CLIPIQA, and BRISQUE. Fine-tuning on metric-selected images can inflate those metrics without improving perceived realism, because the model is trained to reproduce images that already score well on those metrics. The authors should add a human perceptual study (e.g., pairwise preference) and/or evaluate on held-out no-reference metrics that were not part of QDAISP, and report whether the gains persist.","section":"Section 3.1, Appendix B, Table 1"},{"comment":"The full-reference fidelity results undermine the realism claim as currently presented. LPIPS worsens in 7 of 9 rows (e.g., SeeSR on DIV2K: 0.3168 to 0.3501; SeeSR on RealSR: 0.3096 to 0.3388; SeeSR on DrealSR: 0.3243 to 0.3542; StableSR on DIV2K: 0.3118 to 0.3542), while PSNR/SSIM improve only in some rows. Since the paper claims more realistic output and provides no human evaluation, the no-reference gains alone do not establish realism; they may indicate a shift toward a preferred style rather than improved restoration. A user study or a detailed fidelity-realism trade-off analysis is needed.","section":"Table 1"},{"comment":"Table 1 reports single values with no error bars, no number of runs, and no significance tests. Several improvements are very small relative to expected run-to-run variation (e.g., SeeSR MUSIQ on DIV2K: 68.66 to 68.86), and some no-reference metrics actually decrease (e.g., StableSR MANIQA on DIV2K: 0.6193 to 0.6017). The statement that the method achieves 'great improvements in almost all no-reference metrics' is not statistically supported without repeated runs and variability measures.","section":"Section 4.1, Table 1"},{"comment":"The dataset-size ablation does not support the claims made in the text. The text says that at 8,000 images 'both reference and no-reference metrics show a decline,' but Table 2 reports only MANIQA and CLIPIQA; no full-reference metrics are shown. In addition, the choice of the default 5,000-image size, the positive ratio r=0.8, and the guidance scale lambda_s is not accompanied by a described selection protocol, so it is unclear whether these values were tuned on the DrealSR test set. The authors should report the full metric table and clarify the hyperparameter selection procedure.","section":"Section 4.2, Table 2"},{"comment":"The negative restoration identifier [V] is trained only on degradations synthesized by the RealESRGAN pipeline, while the method is evaluated on real-world LR images with unknown and mixed degradations. If the synthetic degradation model does not cover the real test distribution, the learned association between [V] and 'low quality' may not transfer, and the classifier-free guidance in Eq. (3) would not steer correctly on real inputs. The authors should analyze the degradation gap, for example by testing on additional synthetic degradation types or including real LR/HR pairs, to demonstrate that the negative identifier generalizes.","section":"Section 3.2, Algorithm 1, Eq. (3)"},{"comment":"The paper claims 'state-of-the-art results,' but Section 4.1 compares only each base method with and without RAP-SR. No comparison is made with other published Real-SR methods such as SUPIR, PASD, CoSeR, or recent GAN-based baselines. The authors should either add such comparisons or restrict the claim to 'improves the base methods,' which is all the current evidence supports.","section":"Abstract, Section 4.1, Conclusion"}],"minor_comments":[{"comment":"The text contains a typo: 'RealESRAGN' should be 'RealESRGAN.'","section":"Section 4"},{"comment":"The paragraph under Quantitative Comparison refers to 'PAR-SR' instead of 'RAP-SR.'","section":"Section 4.1"},{"comment":"The dataset name is spelled 'HAFID' in the qualitative comparison section; it should be 'HFAID' for consistency.","section":"Appendix A.2"},{"comment":"The row header 'Filck2K' is a typo for 'Flickr2K.'","section":"Table 3"},{"comment":"The DrealSR MUSIQ value for SeeSR + RAP-SR is reported as '70.0644,' which appears to have an extra digit; please verify the formatting.","section":"Table 1"},{"comment":"The text says 'As illustrated in figure 2, our dataset excels in both image quality and detail richness,' but the dataset-quality comparison is shown in Figure 4, not Figure 2; the cross-reference should be corrected.","section":"Section 3.1, Figure 4"},{"comment":"The notation z_t_lr is used without definition; please define the latent variable and its relationship to the low-resolution image x_lr.","section":"Eq. (1)-(3)"},{"comment":"There is an inconsistency: Appendix B reports that MUSIQ and BRISQUE correlate less with human aesthetic preferences, while Appendix D states that no-reference metrics 'such as MANIQA, MUSIQ, and CLIPIQA' align more closely with human perception. This should be reconciled.","section":"Appendix B and D"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: this is a plausible and useful paper, but the evidence for its central claim is softer than the prose suggests. The idea is simple: take Stable Diffusion 2.1, fine-tune it briefly on 5,000 carefully filtered aesthetic images, and bind positive and negative identifier tokens to \"high quality\" and \"degraded\" during training. Then plug the fine-tuned UNet into StableSR, DiffBIR, or SeeSR. That's a reasonable application of Emu-style quality tuning plus DreamBooth-style binding to restoration prompts. The authors are honest that most components are known recipes; what's new is the application and the identifier scheme.\n\nThe dataset HFAID is likely the most useful contribution. The curation pipeline (CLIP-IQA, MANIQA, NIQE filtering, aesthetic assessment, human verification) does produce a dataset that scores higher on every no-reference metric than LSDIR, DIV2K, etc. I'd believe these images are genuinely prettier and cleaner.\n\nNow the soft spots. The biggest issue is the evaluation: the training set was curated with CLIP-IQA and MANIQA, and the main results are reported on those same metrics. That creates a partial circularity. However, the paper also reports consistent gains on MUSIQ and BRISQUE, which were explicitly held out from curation, so the effect is not purely metric-fitting. The deeper problem is that \"realism\" is never validated by humans. The paper shows a few crops and reports no-reference scores, but there is no perceptual study. Given the known unreliability of no-reference metrics, the claim that RAP-SR produces more realistic outputs is not established.\n\nThere are smaller issues: no error bars or significance tests anywhere; the \"state-of-the-art\" claim is unsupported because the authors don't compare with SUPIR, CoSeR, or PASD; LPIPS consistently gets worse, which they wave away with a single appendix example; and the negative identifier is trained only on RealESRGAN degradations, so transfer to real-world mixed degradation is untested. Code and data are promised but not released. Also, there are typos (\"PAR-SR\", \"HAFID\") and a confusing sentence about the 8,000-image ablation.\n\nOverall: I think the recipe works, and the dataset is probably a genuine resource. But the paper currently overclaims. A serious referee should ask for a human evaluation, a proper SOTA comparison, error bars, and the artifacts before acceptance. I'd take a look at this at a reading group, and I'd cite the dataset if it ships.","headline":"RAP-SR is a sensible plug-and-play recipe that likely improves diffusion SR realism, but the evidence is weakened by metric curation overlap and the absence of a human study.","tokens_in":17708,"tokens_out":3196,"would_cite":true,"duration_ms":32384,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By fine-tuning a pretrained diffusion model on a curated 5,000-image aesthetic dataset with learned positive and negative restoration identifiers, RAP-SR produces a restoration prior that plugs into existing diffusion super-resolution…","keywords":["real-world image super-resolution","diffusion model fine-tuning","restoration prior","restoration prompt optimization","dataset curation","classifier-free guidance","no-reference image quality assessment","plug-and-play enhancement"],"falsifier":"Train RAP-SR exactly as described, then test the plugged-in models on real low-resolution photos whose degradations are known to be unlike RealESRGAN's synthetic pipeline, such as long-kernel motion blur or heavy sensor noise; if the no-reference metric gains over the unmodified base methods disappear or reverse, the learned negative token does not transfer to unseen degradations.","tokens_in":16768,"feed_emoji":"🖼️","tokens_out":9582,"duration_ms":77971,"temperature":0.7,"pith_summary":"RAP-SR claims that the restoration prior inside a general-purpose text-to-image diffusion model can be strengthened with a small, carefully chosen dataset, and that this stronger prior transfers to existing diffusion-based super-resolution methods. The paper builds a curated set of 5,000 high-quality, aesthetic images with rich captions, then fine-tunes Stable Diffusion 2.1 on those images paired with two learned tokens: one for high quality and one for degraded quality. At inference, the fine-tuned model is swapped into StableSR, DiffBIR, or SeeSR without retraining, and classifier-free guidance with the learned tokens steers generation toward realistic details. On three benchmarks, the plug-in boosts no-reference quality scores such as CLIPIQA, MANIQA, MUSIQ, and BRISQUE, at the cost of some full-reference scores. A sympathetic reader would care because it offers a cheap, method-agnostic way to improve the realism of diffusion-based super-resolution.","feed_headline":"A 5,000-image tune sharpens three super-resolution methods","feed_subtitle":"Curated images and learned quality tokens make StableSR, DiffBIR, and SeeSR outputs more lifelike.","key_machinery":"The load-bearing mechanism is a pair of learned restoration identifiers, [X] (positive) and [V] (negative), combined with classifier-free guidance. These tokens are optimized during the Restoration-Oriented Prompt Optimization (ROPO) phase, which appends the positive token to high-quality images and the negative token to RealESRGAN-degraded images together with the semantic caption, then fine-tunes the diffusion model (Restoration Priors Refinement, RPR) on the curated HFAID dataset. At inference the negative token's prediction is subtracted from the positive token's prediction (Equation 3), so the model learns a direction in output space that leads away from degradation; the dataset's role is to make this direction correspond to genuinely high-fidelity, aesthetic detail.","core_discovery":"The central discovery is that the quality-tuning stage of pretrained text-to-image diffusion models, normally used to improve aesthetics, can be repurposed to build a restoration prior: fine-tuning on a tiny dataset of 5,000 ultra-high-quality images, with captions augmented by learned restoration identifiers, teaches the model to associate those identifiers with image quality. During training the model sees both high-quality images paired with a positive identifier and, with probability 0.2, degraded versions of the same images paired with a negative identifier, where the degradations come from the RealESRGAN pipeline. At inference, classifier-free guidance with the positive and negative identifiers produces a restoration direction that pushes the generated image away from degraded features. Swapping the fine-tuned backbone into existing SR methods is enough to improve their no-reference perceptual metrics across synthetic and real-world datasets.","pith_inferences":["A testable extension is to train the negative identifier on multiple degradation pipelines rather than only RealESRGAN; if the plug-in gains then transfer to benchmarks with unknown degradations, the restoration direction is not tied to one synthetic degradation family.","Because the paper's evidence relies on no-reference metrics, a direct human-perception study on paired real-world images would clarify whether the improved scores correspond to subjectively better restoration, since the appendix shows these metrics can disagree with full-reference scores.","Since RAP-SR only replaces the T2I backbone, it may combine with future SR adapters unchanged, making it a reusable component rather than a one-off method."],"forward_implications":["StableSR, DiffBIR, and SeeSR all improve on no-reference quality metrics when their Stable Diffusion backbone is replaced with the RAP-SR fine-tuned model, with no additional fine-tuning of the SR methods.","Training converges in about 3,000 steps with batch size 40, so the restoration-prior enhancement is cheap relative to training a full SR diffusion model.","The gains come with a tradeoff: full-reference metrics such as PSNR, SSIM, and LPIPS may worsen on some datasets, so RAP-SR is tuned for perceptual realism rather than pixel fidelity.","Ablations show that both the positive and negative identifiers matter, with the negative prompt contributing more to the no-reference score gains."],"supporting_citations":[{"why":"Supplies the quality-tuning phase concept and the observation that a small high-quality dataset can enhance a pretrained diffusion model.","marker":"(Dai et al. 2023)"},{"why":"DreamBooth's unique-identifier binding is the technique ROPO extends to restoration prompts.","marker":"(Ruiz et al. 2023)"},{"why":"Classifier-free guidance is the inference mechanism that combines positive and negative restoration identifiers.","marker":"(Ho and Salimans 2022)"},{"why":"RealESRGAN's degradation pipeline generates the low-quality negative samples used to train the negative identifier.","marker":"(Wang et al. 2021b)"},{"why":"StableSR is one of the three diffusion SR methods whose backbone RAP-SR replaces for plug-in testing.","marker":"(Wang et al. 2024)"},{"why":"DiffBIR is another base method evaluated with the RAP-SR backbone.","marker":"(Lin et al. 2024)"},{"why":"SeeSR is the third base method and the model used for all ablation studies.","marker":"(Wu et al. 2024)"},{"why":"CLIPIQA acts as both a dataset-selection criterion and an evaluation metric for the improved outputs.","marker":"(Wang, Chan, and Loy 2023)"},{"why":"MANIQA is used to select HFAID images and to measure restoration quality.","marker":"(Yang et al. 2022)"},{"why":"NIQE is one of the no-reference metrics selected for the HFAID curation pipeline.","marker":"(Zhang, Zhang, and Bovik 2015)"}],"fun_headline_variants":["5,000 images teach diffusion models to restore","Tiny fine-tune, sharper super-resolution","Plug-and-play prior boost for diffusion SR","Quality tokens sharpen three SR methods","Aesthetic fine-tune turns diffusion into SR enhancer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The negative restoration identifier is trained only on degradations synthesized by the RealESRGAN pipeline, so if real-world image degradation lies outside that synthetic distribution, the classifier-free guidance may not steer generation away from the actual defects.","fun_headline_variants_meta":{"raw":{"variants":["5,000 images teach diffusion models to restore","Tiny fine-tune, sharper super-resolution","Plug-and-play prior boost for diffusion SR","Quality tokens sharpen three SR methods","Aesthetic fine-tune turns diffusion into SR enhancer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001041,"raw_usage":{"total_tokens":4405,"prompt_tokens":1000,"completion_tokens":3405,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":3336}},"tokens_in":616,"tokens_out":3405,"duration_ms":23666,"temperature":1.0,"reasoning_tokens":3336,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:04:20.385056+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RAP-SR exactly as described, then test the plugged-in models on real low-resolution photos whose degradations are known to be unlike RealESRGAN's synthetic pipeline, such as long-kernel motion blur or heavy sensor noise; if the no-reference metric gains over the unmodified base methods disappear or reverse, the learned negative token does not transfer to unseen degradations.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DreamBooth's unique-identifier binding is the technique ROPO extends to restoration prompts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"StableSR is one of the three diffusion SR methods whose backbone RAP-SR replaces for plug-in testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MANIQA is used to select HFAID images and to measure restoration quality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"NIQE is one of the no-reference metrics selected for the HFAID curation pipeline."}],"review_version":1}