REVIEW 3 major objections 5 minor 29 references
Enhancing Frequency for Single Image Super-Resolution with Learnable Separable Kernels
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Replacing square kernels with learnable rank-one ones cuts super-resolution cost and sharpens high-frequency detail.
desk verdict The separable-kernel trick is a known low-rank factorization with clean complexity math, but the frequency story is unproven and the abstract overstates the gains; the scale-dependent trend is fragile but worth one careful look. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Learnable Separable Kernel (LSK), an n×n matrix of rank one that factors as a product of an n×1 and a 1×n vector. The paper replaces a square convolution layer with two consecutive 1D convolution layers without an activation between them, so the two operations can be merged back into a single separable kernel at inference. Because the intermediate feature map is computed without nonlinearity, the whole two-layer operation is algebraically equivalent to a sum of LSKs, and this decomposition is what produces the parameter and FLOP reductions while preserving the representational capacity of a full-rank kernel. The frequency-enhancement argument rests on the observation that rank-one high-pass operators such as Sobel are separable, so the learned LSKs are expected to behave as frequency enhancers during super-resolution.
What would settle it
Train S-SRCNN on a standard benchmark, extract the learned 1D kernels from an LSK layer, form their outer products, and compute the 2D Fourier transform of each kernel; if the average response at high spatial frequencies is not greater than the response at low frequencies, the claimed frequency-enhancement mechanism is contradicted.
Extended reading notes
Core claim
The central claim is that a square convolution kernel in a super-resolution network can be replaced by a sum of rank-one separable kernels, called LSKs, and that this replacement does two things at once: it cuts the parameter count and FLOPs to about 2/n_k of the original for an n_k×n_k kernel, and it gives the network an inherent tendency to enhance image frequency components. The paper reasons that edge-detecting kernels such as Sobel are separable and high-pass, so learning rank-one kernels should let the network acquire frequency-enhancing filters. Empirically, S-SRCNN, S-ESPCN, and S-VDSR (the separable versions) reduce parameters and FLOPs by over 60% in the SRCNN case and 32–43% in the VDSR/ESPCN cases, while staying within about 0.1 dB of the original models in PSNR. The paper also reports that on ×4 upscaling, the separable versions win more often than the normal versions on both PSNR and SSIM, consistent with the idea that larger scale factors lose more high-frequency information and therefore benefit more from frequency-enhancing kernels.
Load-bearing premise
The whole frequency-enhancement argument rests on the premise that a learned rank-one kernel will naturally behave as a high-frequency enhancer, but rank-one kernels can just as easily learn low-pass or smoothing filters, and the paper provides no spectral measurement of the learned weights to rule that out.
Editorial extensions
If this is right
- Models using LSKs cut parameters and FLOPs by roughly 2/n_k, so a 5×5 kernel replacement halves the cost and a 3×3 replacement reduces it to two-thirds, yielding the reported >60% reduction in the SRCNN case.
- LSK-based super-resolution is compatible with pre-upsampling, post-upsampling, and residual-based architectures, so the module is genuinely plug-and-play across the main SISR framework families.
- The relative performance of LSK models improves as the scale factor goes from ×2 to ×4, implying that the benefit grows precisely when the low-resolution input has lost more high-frequency content.
- Because the two 1D kernels are merged during inference, the computational savings are realized at deployment time, not only during training.
- LSK models produce feature maps that show less separation between frequency-related and detail-lacking channels, which the paper interprets as the kernels integrating frequency enhancement into each feature rather than isolating it.
Reading between the lines
- The paper does not measure the frequency response of the learned LSK weights, so the claimed mechanism — that learned rank-one kernels will be high-pass — remains unverified; a test that computes the FFT of trained LSKs would directly probe this assumption.
- Because a sum of n_fe rank-one matrices has rank at most n_fe, the expressiveness of the replaced square layer is bounded by the number of intermediate feature maps, so the method implicitly trades a controlled amount of capacity for efficiency.
- The observed improvement at larger scale factors may partly reflect a regularization effect of the lower-parameter separable architecture, which could help on harder tasks where the baseline overfits; this alternative explanation is not ruled out by the experiments.
- The authors suggest combining LSKs with auxiliary structures; a natural extension is pairing LSKs with a frequency-aware loss, such as gradient variance loss, to test whether the two frequency-enhancement mechanisms are additive.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Learnable Separable Kernels (LSKs), rank-one convolutional kernels implemented as two 1D convolutions, as a plug-and-play replacement for square kernels in single image super-resolution. The authors argue from a frequency perspective that LSKs enhance high-frequency content, derive parameter and FLOP reductions, and evaluate the replacement on SRCNN, ESPCN, and VDSR variants across five benchmark datasets. They report large parameter/computation savings and comparable or improved PSNR/SSIM, with larger relative gains at higher upscaling factors.
Significance. If the frequency-enhancement claim were established, the paper would offer a simple and broadly applicable low-rank reparameterization for SISR, with exact complexity reductions that are verified in Table 3. The complexity analysis in Sec. 3.3 is arithmetically correct, and the experimental comparison across three architectures and five datasets is a useful empirical study of kernel separability. The main weakness is that the mechanistic explanation in terms of frequency enhancement is not supported by the evidence provided, and the abstract overstates the parameter savings. The contribution is therefore currently a sound efficiency technique with an unsubstantiated frequency-based motivation.
major comments (3)
- [Sec. 3.2 (and Sec. 3.1, final paragraph)] The central frequency claim is not established. The paper argues that LSKs enhance frequency because Sobel, a separable rank-one kernel, is high-pass. However, rank-one kernels can equally be low-pass: the 3x3 box filter is the outer product of two [1,1,1] vectors, and the learned 1D kernels are unconstrained. The manuscript provides no Fourier-domain analysis of learned kernels or feature maps, and Sec. 4.4's two-example visualization is qualitative. A direct spectral measurement of trained LSKs or of output feature maps, or a constrained formulation that enforces high-pass behavior, is needed before the frequency mechanism can be accepted.
- [Abstract and Table 3] The abstract's claim of 'over 60% reduction in both the number of parameters and computational requirements' as a general property of the baseline methods is contradicted by Table 3: only S-SRCNN exceeds 60% (62.45%), while S-ESPCN is 43.14% and the S-VDSR variants are 32–35%. The sentence should be restricted to the SRCNN case or rephrased to report per-method reductions.
- [Sec. 4.3 and Table 4] The scale-dependent benefit is not convincingly established. Table 4 shows that at x2 the separable variants win only 33% of PSNR comparisons and 10% of SSIM comparisons, and many differences in Table 2 are below 0.1 dB with no error bars or significance tests. The monotone trend in Fig. 3 is consistent with the frequency hypothesis, but it could also reflect optimization or parameterization effects; the post hoc explanation does not constitute a predictive test.
minor comments (5)
- [Sec. 3.1, Eq. (3)] The product of an n x 1 and a 1 x n kernel is an outer product, not a convolution, and the '*' notation is used ambiguously; this should be clarified.
- [Abstract and Sec. 5] The word 'orthogonal' in describing the 1D kernels is not defined, and no orthogonality constraint is imposed on the learned kernels.
- [Sec. 4.3, Table 4] The proportions in Table 4 are not accompanied by the number of comparisons or confidence intervals, which makes values such as 0.33 versus 0.67 difficult to interpret.
- [Table 3 caption] The caption states that FLOPs are calculated when upscaling to 512x512 on x2, x3, and x4 tasks, but the text does not clarify whether the same input resolution is used for all scales; this should be stated explicitly.
- [Fig. 3] The axis labels in Fig. 3 are not legible in the provided version, making the trend difficult to verify.
Circularity Check
No circularity: the rank-one frequency claim is an invalid generalization, not a definitional equivalence or self-citation chain.
full rationale
Walking the derivation chain: LSKs are defined as rank-one matrices (Sec. 3.1), and the replacement of square kernels by sums of such matrices is given by Eqs. (3)-(4). The load-bearing frequency claim appears in Sec. 3.2: "A rank-1 matrix can enhance the frequency," supported only by the Sobel example. This is an overgeneralization, since a rank-one outer product can also be a box blur or other low-pass filter, and the paper offers no spectral analysis of the learned kernels. However, this is not circular: the definition of LSK does not build frequency enhancement into the term, the assertion is not derived by substituting equations into one another, and no fitted parameter is renamed as a prediction. The scale-dependent performance trend in Sec. 4.3 and Table 4 is an empirical observation made after the experiments; it is consistent with the hypothesis but was not an output forced by construction. The parameter and FLOP reductions in Sec. 3.3 are direct analytical calculations, independently tabulated in Table 3; the abstract's "over 60%" phrasing overstates the reductions for ESPCN and VDSR, but that is a reporting overclaim rather than a circular step. There are no load-bearing self-citations or imported uniqueness theorems. The central weakness, namely that rank-one does not by itself imply high-frequency enhancement, is a correctness/evidence gap that would require direct spectral measurement, not a circularity.
Assumptions & free parameters
free parameters (2)
- n_fe (number of intermediate feature maps in the separable layer) =
n_fe = n_fm (set equal to the output feature map count of the next layer)
- Layers selected for replacement =
Only layers where n_fm-1 > 1 and n_fm > 1 (e.g., layer 2 of SRCNN)
assumptions (3)
- standard math Convolution is linear, allowing two 1D convolutions without an intermediate activation to be merged into a single convolution.
- ad hoc to paper Rank-one kernels enhance image frequency content, as exemplified by Sobel.
- domain assumption The complexity analysis assumes n_fm-1 = n_fe = n_fm to derive simple ratios.
Cite this review
Pith. "Pith review of Enhancing Frequency for Single Image Super-Resolution with Learnable Separable Kernels." pith.science (2026). https://pith.science/paper/ISMDHVCR
@misc{pith2026250604555,
author = {Pith},
title = {Pith review of: Enhancing Frequency for Single Image Super-Resolution with Learnable Separable Kernels},
year = {2026},
howpublished = {\url{https://pith.science/paper/ISMDHVCR}},
note = {Machine review of arXiv:2506.04555}
}
read the original abstract
Existing approaches often enhance the performance of single-image super-resolution (SISR) methods by incorporating auxiliary structures, such as specialized loss functions, to indirectly boost the quality of low-resolution images. In this paper, we propose a plug-and-play module called Learnable Separable Kernels (LSKs), which are formally rank-one matrices designed to directly enhance image frequency components. We begin by explaining why LSKs are particularly suitable for SISR tasks from a frequency perspective. Baseline methods incorporating LSKs demonstrate a significant reduction of over 60\% in both the number of parameters and computational requirements. This reduction is achieved through the decomposition of LSKs into orthogonal and mergeable one-dimensional kernels. Additionally, we perform an interpretable analysis of the feature maps generated by LSKs. Visualization results reveal the capability of LSKs to enhance image frequency components effectively. Extensive experiments show that incorporating LSKs not only reduces the number of parameters and computational load but also improves overall model performance. Moreover, these experiments demonstrate that models utilizing LSKs exhibit superior performance, particularly as the upscaling factor increases.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Abrahamyan, L., Truong, A.M., Philips, W., Deligiannis, N.: Gradient variance loss for structure-enhanced image super-resolution. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 3219–3223. IEEE (2022)
work page 2022
-
[2]
ACM Journal on Emerging Technologies in Computing Systems (JETC) 13(3), 1–18 (2017)
Anwar, S., Hwang, K., Sung, W.: Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC) 13(3), 1–18 (2017)
work page 2017
-
[3]
In: Pro- ceedings of the British Machine Vision Conference
Bevilacqua, M., Roumy, A., Guillemot, C., line Alberi Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Pro- ceedings of the British Machine Vision Conference. pp. 135.1–135.10. BMVA Press (2012)
work page 2012
-
[4]
In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Ding, X., Guo, Y., Ding, G., Han, J.: Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 1911–1920 (2019)
work page 2019
-
[5]
In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Ding, X., Zhang, X., Han, J., Ding, G.: Diverse branch block: Building a convolu- tion as an inception-like unit. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10881–10890 (2021)
work page 2021
-
[6]
In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., Sun, J.: Repvgg: Making vgg-style convnets great again. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 13728–13737 (2021)
work page 2021
-
[7]
IEEE Transactions on Pattern Analysis and Machine Intelligence 38(2), 295–307 (2016)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolu- tional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(2), 295–307 (2016)
work page 2016
-
[8]
http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html
2012
Show all 29 references
-
[9]
He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
2016
-
[10]
In: 2015 IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR)
Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR). pp. 5197–5206 (2015)
2015
-
[11]
In: Proceedings of the British Machine Vision Conference (2014)
Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural net- works with low rank expansions. In: Proceedings of the British Machine Vision Conference (2014)
2014
-
[12]
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutionalnetworks.In:2016IEEEConferenceonComputerVisionandPattern Recognition (CVPR). pp. 1646–1654 (2016)
2016
-
[13]
Irbm42(2), 120–133 (2021)
Li, Y., Sixou, B., Peyrin, F.: A review of the deep learning methods for medical images super resolution problems. Irbm42(2), 120–133 (2021)
2021
-
[14]
In: Proceedings Eighth IEEE International Conference on Computer Vision
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natu- ral images and its application to evaluating segmentation algorithms and measur- ing ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. vol....
2001
-
[15]
Multimedia Tools and Applications76, 21811–21838 (2017) Title Suppressed Due to Excessive Length 15
Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications76, 21811–21838 (2017) Title Suppressed Due to Excessive Length 15
2017
-
[16]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVP...
2016
-
[17]
Neurocomputing487, 46–65 (2022)
Su, J., Xu, B., Yin, H.: A survey of deep learning approaches to image restoration. Neurocomputing487, 46–65 (2022)
2022
-
[18]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the incep- tion architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2818–2826 (2016)
2016
-
[19]
In: 2013 IEEE International Conference on Com- puter Vision
Timofte, R., De, V., Gool, L.V.: Anchored neighborhood regression for fast example-based super-resolution. In: 2013 IEEE International Conference on Com- puter Vision. pp. 1920–1927 (2013)
2013
-
[20]
In: Computer Vision–ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Re- vised Selected Papers, Part IV 12
Timofte, R., De Smet, V., Van Gool, L.: A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Computer Vision–ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Re- vised Selected Papers, Part IV 12. pp. 111–12...
2015
-
[21]
In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Wang, Y.: Edge-enhanced feature distillation network for efficient super-resolution. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp. 776–784 (2022)
2022
-
[22]
IEEE Transactions on Pattern Analysis and Machine Intelligence43(10), 3365– 3387 (2021)
Wang,Z.,Chen,J.,Hoi,S.C.H.:Deeplearningforimagesuper-resolution:Asurvey. IEEE Transactions on Pattern Analysis and Machine Intelligence43(10), 3365– 3387 (2021)
2021
-
[23]
IEEE Transactions on Image Processing 13(4), 600–612 (2004)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)
2004
-
[24]
In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Xie, W., Song, D., Xu, C., Xu, C., Zhang, H., Wang, Y.: Learning frequency-aware dynamic network for efficient super-resolution. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4288–4297 (2021)
2021
-
[25]
IEEE Transactions on Image Processing19(11), 2861–2873 (2010)
Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse rep- resentation. IEEE Transactions on Image Processing19(11), 2861–2873 (2010)
2010
-
[26]
In: Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse- representations. In: Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7. pp. 711–730. Springer (2012)
2012
-
[27]
In: 2021 IEEE/CVF Inter- national Conference on Computer Vision (ICCV)
Zhan, Z., Gong, Y., Zhao, P., Yuan, G., Niu, W., Wu, Y., Zhang, T., Jayaweera, M., Kaeli, D., Ren, B., Lin, X., Wang, Y.: Achieving on-mobile real-time super- resolution with neural architecture and pruning search. In: 2021 IEEE/CVF Inter- national Conference on Computer Visio...
2021
-
[28]
IEEE Transactions on Pattern Analysis and Ma- chine Intelligence38(10), 1943–1955 (2016)
Zhang, X., Zou, J., He, K., Sun, J.: Accelerating very deep convolutional networks for classification and detection. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence38(10), 1943–1955 (2016)
2016
-
[29]
In: Proceedings of the 29th ACM International Conference on Multimedia
Zhang, X., Zeng, H., Zhang, L.: Edge-oriented convolution block for real-time su- per resolution on mobile devices. In: Proceedings of the 29th ACM International Conference on Multimedia. pp. 4034–4043 (2021)
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.