REVIEW 4 major objections 5 minor 46 references
Learning Multi-scale Spatial-frequency Features for Image Denoising
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Image denoising improves when a network is fed a pyramid of downscaled noisy images and learns to split its Fourier spectrum into high- and low-frequency bands with an adaptive mask; the proposed MADNet reports 39.78 dB on SIDD and 39.98…
desk verdict Plausible incremental denoising paper whose global fusion block is undefined as written; worth a referee's time if the resizing omission is fixed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the adaptive spatial-frequency learning unit (ASFU), which contains an adaptive frequency enhancement block (AFEB) and an adaptive spatial enhancement block (ASEB). AFEB applies a fast Fourier transform, multiplies the spectrum by a learnable rectangular mask whose dimensions scale with the input size, and uses the mask to split the signal into high- and low-frequency components; each branch is transformed back, refined by an MLP and by transposed self-attention, then added together. ASEB is that transposed self-attention, which models long-range channel dependencies. The global feature fusion block (GFFB) concatenates the four pyramid-scale features along the channel dimension and applies ASEB to produce a unified global feature. Together these units let the network adaptively choose how much of each frequency band and each scale to trust, which is the mechanism behind the claimed denoising gains.
What would settle it
Run the released implementation on a 256x256 dummy input and print the spatial shapes of the four features entering the global feature fusion block; if they are H, H/2, H/4, and H/8 and are concatenated without interpolation, the operation fails, showing the architecture is not as described. Independently, re-training with a fixed random mask instead of the learned adaptive mask would show whether the learned separation drives the reported PSNR gain.
Extended reading notes
Core claim
MADNet reaches state-of-the-art denoising performance on both synthetic and real noisy images by combining three mechanisms: multi-scale pyramid inputs that supply less-noisy low-resolution views of the same scene; an adaptive spatial-frequency learning unit (ASFU) whose learnable mask separates the Fourier spectrum into high- and low-frequency components, processes each band with MLPs and transposed self-attention, and recombines them; and a global feature fusion block (GFFB) that concatenates deep features from all four scales so coarse semantic structure can enhance fine detail. The paper reports PSNR 39.78 dB on SIDD and 39.98 dB on DND, and its ablation study attributes the gain to all three components rather than to any single one.
Load-bearing premise
The central claim rests on a concatenation of feature maps that have different resolutions in the global feature fusion block, and the paper does not specify the resizing that would make this concatenation possible.
Editorial extensions
If this is right
- A single trained MADNet model should restore clean images at all four pyramid scales simultaneously, so the multi-scale outputs could be ensembled or used as extra supervision without extra inference cost.
- The learnable frequency-separation mask means the network can adapt its high/low split to the noise level, so the same architecture should work across the tested sigma=15 to sigma=50 range and on real sensor noise.
- If the reported benchmarks hold, MADNet becomes the reference point for real-image denoising on SIDD and DND, so future methods will need to beat 39.78 dB and 39.98 dB.
- The ablation results imply that neither scale information nor frequency separation alone accounts for the gain; both are needed, so simpler single-domain or single-scale variants should underperform the full model.
Reading between the lines
- Because the pyramid input works by presenting the network with progressively less noisy versions of the same scene, the same trick could transfer to other restoration tasks such as deblurring or super-resolution, where low-resolution views also carry cleaner global information.
- The learned frequency mask is a potential interpretability probe: extracting the trained mask and plotting its shape across noise levels would show whether the network settles on a fixed high/low split or adapts per image, a test the paper does not run.
- The reported gaps over the nearest competitors are small (about 0.2 to 0.3 dB on SIDD and DND), and the paper does not report variance across training runs; a multi-seed rerun would indicate whether the improvement is systematic or within noise.
- If the global feature fusion concatenation requires an interpolation step that the paper omits, the fix is straightforward, but the description in Eq. (9) needs that step spelled out before the architecture can be reproduced as written.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MADNet, a multi-scale adaptive dual-domain network for image denoising. The architecture uses image pyramid inputs, a dual-domain modulation layer with an adaptive spatial-frequency learning unit (ASFU) that separates high- and low-frequency components via a learnable mask, and a global feature fusion block (GFFB) in skip connections. The authors report experiments on synthetic (CBSD68, Set12) and real (SIDD, DND) datasets, claiming state-of-the-art PSNR/SSIM results. The central claim is that the combination of multi-scale inputs, frequency separation, and global feature fusion improves denoising performance.
Significance. If the architecture is implementable as described, the paper makes a plausible empirical contribution by combining multi-scale pyramid inputs, adaptive frequency separation, and cross-scale feature fusion, and the reported gains on SIDD and DND are nonzero. The ablation study attempts to attribute improvements to individual components, and the paper includes both real and synthetic noise benchmarks. However, the method description contains a fundamental tensor-dimension inconsistency in the GFFB, and the ablation table has internal inconsistencies; until these are resolved, the central architectural innovation and its claimed benefit cannot be verified. The SOTA claim is also weakened by the absence of several widely used recent baselines and the lack of error bars. The work is not circular: metrics come from held-out test sets and the learnable mask is a trained parameter.
major comments (4)
- [Section 3.4, Eq. (9)] The global feature fusion block concatenates features from different pyramid scales along the channel dimension, but Section 3.1 defines these features as F_i ∈ R^{H/n × W/n × nC} with n = 1, 2, 4, 8. Concatenation along channels requires identical spatial dimensions; no upsampling, interpolation, or pooling operation is described anywhere in the paper. Since Table 5 attributes a 0.063 dB improvement (39.777 vs 39.714 dB) to GFFB, this is a load-bearing definitional gap. Please specify how the resolution mismatch is resolved (e.g., bilinear upsampling to the finest scale) and update Figure 3 and the ablation description accordingly.
- [Section 3.3.1, Eq. (4) and Section 3.4, Eq. (10)] The mask operation in Eq. (4), F_high, F_low = Mask(F_freq), is not defined: the paper does not state how the binary mask is constructed, how the learnable coefficient controls the mask dimensions, or whether the separation is an element-wise multiplication in the Fourier domain. In addition, Eq. (10) uses ASEB(F_g, F_i) as a two-argument operation, while Eq. (7) defines transposed self-attention for a single input feature. The intended mechanism by which the global feature F_g modulates each scale-specific feature is unspecified. These details are needed to reproduce the core ASFU and GFFB components.
- [Table 5] The ablation table contains internally inconsistent checkmark patterns. For example, the rows labeled 'AFEB w/o Sep' and 'AFEB w/o Enh' have ASEB=✗ and AFEB=✗, meaning both the spatial and frequency enhancement blocks are removed, which does not match the row labels. The reported values 39.625 and 39.621 dB therefore cannot be interpreted as isolating the effect of 'Sep' or 'Enh' alone. The text in Section 4.6.2 also says 'Removal of frequency separation (39.670 dB)' but that row (ASEB+AFEB w/o Sep) has a different configuration. Please correct the table so the ablation conditions match the components being removed, or clarify the intended experimental design.
- [Tables 2-4] The claim of state-of-the-art performance is not adequately supported by the chosen baselines. The comparisons omit several widely used recent denoisers, including Restormer, Uformer, and other top-performing methods on SIDD and DND, and no error bars or statistical significance tests are provided. The reported margins over the strongest included baseline (DRANet) are small (e.g., 0.24 dB on SIDD, 0.35 dB on DND), so the SOTA claim should be either backed by comparisons against those methods or rephrased as competitive performance against the included baselines.
minor comments (5)
- [Section 4.6.1, Table 5] The text 'removing both results in a more noticeable 0.04 dB reduction (39.695 dB vs 39.735 dB)' does not match the table: the value 39.735 dB does not appear in Table 5, and the actual drop from the full model (39.777 dB) to 'w/o MSI & GFFB' (39.695 dB) is 0.082 dB. Please correct the numbers.
- [Section 4.4, Table 3] The text states that on Set12 the method 'maintains leading PSNR values of 33.05 dB at σ=15', but Table 3 lists MADNet's Set12 PSNR at σ=15 as 33.00 dB. Please align the text with the table.
- [Section 4.2 and Table 2] Section 4.2 says evaluation is performed at noise levels 15, 25, 30, and 50, but the caption and text in Section 4.3 refer to 'three noise levels (σ=15, 25, 50)' while the table actually includes four columns (σ=15, 25, 30, 50). Please correct the inconsistent wording.
- [Table 2] The DudeNet row is duplicated in the CBSD68 block.
- [Section 4.7] Section 4.7 is titled 'Limitation' but it reports FLOPs and parameter counts; the actual limitation (the large parameter count) is discussed in Section 5. Please consider retitling or reorganizing this section.
Circularity Check
No significant circularity: the paper's claims are empirical benchmark results with independently evaluated held-out test sets, and the cited self-works are related-work background rather than load-bearing evidence.
full rationale
MADNet is an empirical architecture paper. The central claims are PSNR/SSIM numbers on CBSD68, Set12, SIDD, and DND, which are external benchmarks evaluated on held-out test sets. No parameter is fitted to the test data and then reported as a prediction; the learnable frequency mask and network weights are optimized on training splits (Flickr2K, SIDD Medium) only. The multi-scale pyramid inputs, ASFU, and GFFB are architectural components whose contributions are assessed by ablations, not by construction that forces the reported improvements. The loss functions in Eqs. (11)-(13) supervise training with ground-truth targets, and the final metrics are computed independently on test benchmarks. The paper does cite works by overlapping authors (e.g., [19], [30], [33], [34]), but these appear in related-work and frequency-learning context, not as load-bearing justification of MADNet's design or performance; none is invoked as a uniqueness theorem or as a substitute for experimental evidence. The strongest concern in the manuscript is that Eq. (9) concatenates features of different spatial resolutions along the channel dimension without describing a resizing operation; this is an implementability and reproducibility defect, not a circularity defect, because it does not make the claimed result equivalent to an input by construction. Accordingly, no circular step meeting the required evidence standard is present.
Assumptions & free parameters
free parameters (3)
- Number of pyramid scales =
4
- Loss scale weights alpha_s =
not reported
- Learnable frequency mask coefficient =
not reported
assumptions (3)
- standard math FFT and IFFT are defined as in Eq. (2) and can be used inside a differentiable network.
- domain assumption Low-resolution versions of a noisy image contain less noise and provide cleaner contextual information.
- ad hoc to paper The adaptive binary frequency mask is trainable by gradient descent.
Cite this review
Pith. "Pith review of Learning Multi-scale Spatial-frequency Features for Image Denoising." pith.science (2026). https://pith.science/paper/HV4KN7J5
@misc{pith2026250616307,
author = {Pith},
title = {Pith review of: Learning Multi-scale Spatial-frequency Features for Image Denoising},
year = {2026},
howpublished = {\url{https://pith.science/paper/HV4KN7J5}},
note = {Machine review of arXiv:2506.16307}
}
read the original abstract
Recent advancements in multi-scale architectures have demonstrated exceptional performance in image denoising tasks. However, existing architectures mainly depends on a fixed single-input single-output Unet architecture, ignoring the multi-scale representations of pixel level. In addition, previous methods treat the frequency domain uniformly, ignoring the different characteristics of high-frequency and low-frequency noise. In this paper, we propose a novel multi-scale adaptive dual-domain network (MADNet) for image denoising. We use image pyramid inputs to restore noise-free results from low-resolution images. In order to realize the interaction of high-frequency and low-frequency information, we design an adaptive spatial-frequency learning unit (ASFU), where a learnable mask is used to separate the information into high-frequency and low-frequency components. In the skip connections, we design a global feature fusion block to enhance the features at different scales. Extensive experiments on both synthetic and real noisy image datasets verify the effectiveness of MADNet compared with current state-of-the-art denoising approaches.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
J. Bao, X. Tao, Y . Zhou, An emotion recognition method based on eye movement and audiovisual features in mooc learning environment, IEEE Transactions on Computational Social Systems 11 (1) (2022) 171–183
work page 2022
- [3]
-
[4]
X. Xie, Y . Wu, H. Ni, C. He, Node-imgnet: a pde-informed effective and robust model for image denoising, Pattern Recognition 148 (2024) 110176. 19
work page 2024
-
[5]
E. Ning, Y . Wang, C. Wang, H. Zhang, X. Ning, Enhancement, integration, ex- pansion: Activating representation of detailed features for occluded person re- identification, Neural Networks 169 (2024) 532–541
work page 2024
-
[6]
X. Ji, Y . Cao, Y . Tai, C. Wang, J. Li, F. Huang, Real-world super-resolution via kernel estimation and noise injection, in: proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition workshops, 2020, pp. 466–467
work page 2020
- [7]
-
[8]
S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, Restormer: Efficient transformer for high-resolution image restoration, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5728–5739
2022
Show all 46 references
-
[9]
Fan, T.-J
C.-M. Fan, T.-J. Liu, K.-H. Liu, Sunet: Swin transformer unet for image denois- ing, in: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2022, pp. 2333–2337
2022
-
[10]
E. Ning, W. Li, J. Fang, J. Yuan, Q. Duan, G. Wang, 3d-guided multi-feature semantic enhancement network for person re-id, Information Fusion 117 (2025) 102863
2025
-
[11]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10012–10022
2021
-
[12]
Zhang, M
K. Zhang, M. Long, J. Chen, M. Liu, J. Li, Cfpnet: A denoising network for complex frequency band signal processing, IEEE Transactions on Multimedia 25 (2023) 8212–8224
2023
-
[13]
M. Yan, J. Qian, R. Wang, S. Gao, J. Yang, Dynamic group difference coding based on thermal infrared face image for fever screening, IEEE Transactions on Instrumentation and Measurement 72 (2023) 1–13. 20
2023
-
[14]
R. Ma, S. Li, B. Zhang, Z. Li, Generative adaptive convolutions for real-world noisy image denoising, in: Proceedings of the AAAI conference on artificial in- telligence, V ol. 36, 2022, pp. 1935–1943
2022
-
[15]
M. Zhou, J. Huang, C.-L. Guo, C. Li, Fourmer: An efficient global modeling paradigm for image restoration, in: International conference on machine learning, PMLR, 2023, pp. 42589–42601
2023
-
[16]
X. Hu, B. Zhong, Q. Liang, S. Zhang, N. Li, X. Li, Towards modalities corre- lation for rgb-t tracking, IEEE Transactions on Circuits and Systems for Video Technology (2024)
2024
-
[17]
E. Ning, C. Wang, H. Zhang, X. Ning, P. Tiwari, Occluded person re- identification with deep learning: a survey and perspectives, Expert systems with applications 239 (2024) 122419
2024
-
[18]
S. Chen, Y . Chen, Y . Zheng, Z.-X. Yang, E. Wu, A transformer-based adaptive prototype matching network for few-shot semantic segmentation, in: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 2024, pp. 659–667
2024
-
[19]
X. Hu, Y . Tai, X. Zhao, C. Zhao, Z. Zhang, J. Li, B. Zhong, J. Yang, Exploiting multimodal spatial-temporal patterns for video object tracking, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39, 2025, pp. 3581–3589
2025
-
[20]
L. Wang, S. Li, F. Yang, J. Wang, Z. Zhang, Y . Liu, Y . Wang, J. Yang, Not all parameters matter: Masking diffusion models for enhancing generation ability, in: Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 12880–12890
2025
-
[21]
Z. Wang, Z. Yan, J. Pan, G. Gao, K. Zhang, J. Yang, Dornet: A degradation ori- ented and regularized network for blind depth super-resolution, in: Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 15813– 15822. 21
2025
-
[22]
Y . Wu, Z. Yan, Z. Wang, X. Li, L. Hui, J. Yang, Deep height decoupling for precise vision-based 3d occupancy prediction, arXiv preprint arXiv:2409.07972 (2024)
2024 arXiv
-
[23]
Zhang, S
H. Zhang, S. Chen, L. Luo, J. Yang, Few-shot learning with long-tailed labels, Pattern Recognition 156 (2024) 110806
2024
-
[24]
C. Ren, X. He, C. Wang, Z. Zhao, Adaptive consistency prior based deep network for image denoising, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8596–8606
2021
-
[25]
R. K. Thakur, S. K. Maji, Multi scale pixel attention and feature extraction based neural network for image denoising, Pattern recognition 141 (2023) 109603
2023
-
[26]
W. Wu, S. Liu, Y . Xia, Y . Zhang, Dual residual attention network for image de- noising, Pattern Recognition 149 (2024) 110291
2024
-
[27]
X. An, L. Zhao, C. Gong, N. Wang, D. Wang, J. Yang, SHaRPose: Sparse High- Resolution Representation for Human Pose Estimation, Proceedings of the AAAI Conference on Artificial Intelligence 38 (2) (2024) 691–699
2024
-
[28]
E. Ning, C. Zhang, C. Wang, X. Ning, H. Chen, X. Bai, Pedestrian re-id based on feature consistency and contrast enhancement, Displays 79 (2023) 102467
2023
-
[29]
Z. Wang, X. Cun, J. Bao, W. Zhou, J. Liu, H. Li, Uformer: A general u-shaped transformer for image restoration, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17683–17693
2022
-
[30]
C. Zhao, W. Cai, C. Hu, Z. Yuan, Cycle contrastive adversarial learning with structural consistency for unsupervised high-quality image deraining transformer, Neural Networks (2024) 106428
2024
-
[31]
X. Hu, B. Zhong, Q. Liang, S. Zhang, N. Li, X. Li, R. Ji, Transformer track- ing via frequency fusion, IEEE Transactions on Circuits and Systems for Video Technology 34 (2) (2023) 1020–1031. 22
2023
-
[32]
Fuoli, L
D. Fuoli, L. Van Gool, R. Timofte, Fourier space losses for efficient perceptual image super-resolution, in: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, 2021, pp. 2360–2369
2021
-
[33]
C. Zhao, W. Cai, C. Dong, Z. Zeng, Toward sufficient spatial-frequency inter- action for gradient-aware underwater image enhancement, in: ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 3220–3224
2024
-
[34]
C. Zhao, W. Cai, C. Dong, C. Hu, Wavelet-based fourier information interaction with frequency diffusion adjustment for underwater image restoration, in: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2024, pp. 8281–8291
2024
-
[35]
Lai, J.-B
W.-S. Lai, J.-B. Huang, N. Ahuja, M.-H. Yang, Fast and accurate image super- resolution with deep laplacian pyramid networks, IEEE transactions on pattern analysis and machine intelligence 41 (11) (2018) 2599–2613
2018
-
[36]
B. Lim, S. Son, H. Kim, S. Nah, K. Mu Lee, Enhanced deep residual networks for single image super-resolution, in: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136–144
2017
-
[37]
S. Roth, M. J. Black, Fields of experts: A framework for learning image priors, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), V ol. 2, IEEE, 2005, pp. 860–867
2005
-
[38]
Abdelhamed, S
A. Abdelhamed, S. Lin, M. S. Brown, A high-quality denoising dataset for smart- phone cameras, in: IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2018
2018
-
[39]
Plotz, S
T. Plotz, S. Roth, Benchmarking denoising algorithms with real photographs, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1586–1595
2017
-
[40]
C. Tian, Y . Xu, W. Zuo, B. Du, C.-W. Lin, D. Zhang, Designing and training of a dual cnn for image denoising, Knowledge-Based Systems 226 (2021) 106949. 23
2021
-
[41]
Liang, J
J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, R. Timofte, Swinir: Image restoration using swin transformer, in: Proceedings of the IEEE/CVF interna- tional conference on computer vision, 2021, pp. 1833–1844
2021
-
[42]
Y . Gou, P. Hu, J. Lv, J. T. Zhou, X. Peng, Multi-scale adaptive network for single image denoising, Advances in Neural Information Processing Systems 35 (2022) 14099–14112
2022
-
[43]
B. Li, X. Liu, P. Hu, Z. Wu, J. Lv, X. Peng, All-in-one image restoration for unknown corruption, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17452–17462
2022
-
[44]
C. Tian, M. Zheng, W. Zuo, B. Zhang, Y . Zhang, D. Zhang, Multi-stage image denoising with the wavelet transform, Pattern Recognition 134 (2023) 109050
2023
-
[45]
J. W. Soh, N. I. Cho, Variational deep image restoration, IEEE Transactions on Image Processing 31 (2022) 4363–4376
2022
-
[46]
J. Du, X. Qiao, Z. Yan, H. Zhang, W. Zuo, Flexible image denoising model with multi-layer conditional feature modulation, Pattern Recognition 152 (2024) 110372. 24
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.