Pith. sign in

REVIEW 3 major objections 5 minor 36 references

SESF-Fuse: An Unsupervised Deep Model for Multi-Focus Image Fusion

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that measuring spatial frequency on deep features, rather than feature intensity, gives state-of-the-art multi-focus image fusion from an unsupervised encoder-decoder, with the highest reported mean Qg (0.7105) and Qcb…

desk verdict A plausible unsupervised twist on multi-focus fusion whose 'state-of-the-art' claim is not backed by the evidence; worth a serious referee, but not a reliable SOTA result as written. read the letter →

arxiv 1908.01703 v2 pith:DV4IGXXW submitted 2019-08-05 cs.CV

Boyuan Ma , Xiaojuan Ban , Haiyou Huang , Yu Zhu This is my paper · ORCID
classification cs.CV
keywords multi-focusimagefusionunsuperviseddeeplearningencoder-decodernetworkspatialfrequencyfeaturegradientdecisionmapconsistencyverificationmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that, for multi-focus image fusion, the right way to use an unsupervised deep network is to measure gradient sharpness (spatial frequency) on the learned features rather than feature intensity. If true, it means a network trained only to reconstruct ordinary images can supply a focus detector, removing the need for synthetic blurred training pairs. On 38 multi-focus test pairs, the method reports the highest average Qg (0.7105) and Qcb (0.7848) among 16 comparison methods, and the greatest number of per-image first places. A sympathetic reader would take the contribution as evidence that deep-feature gradient activity, followed by consistency verification, is a strong and general fusion rule.

What carries the argument

The carrying object is a pixel-wise spatial frequency computed on deep feature vectors, not on gray levels. For each position the row and column frequencies are summed over a radius-r neighborhood of the feature map and combined as $SF(x,y)=\sqrt{CF(x,y)^2+RF(x,y)^2}/(2r+1)^2$; the decision map is simply which source has the larger $SF$ at each pixel. This expresses the paper's assumption that objects within the depth of field are sharp and therefore have larger feature gradients. The map is then cleaned by alternating opening and closing with a disk structuring element, small-region reversal below $0.01\times H\times W$, and an edge-preserving guided filter, before the weighted-average fusion step.

What would settle it

Use a multi-focus pair with a known ground-truth focus mask and compare the decision maps from deep-feature spatial frequency versus raw grayscale spatial frequency under identical post-processing; if raw features match or beat the deep version, the deep encoder is not the source of the advantage. Likewise, if randomly reinitializing the encoder leaves the three metrics essentially unchanged, the learned reconstruction features are not what the method relies on.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that activity level in multi-focus fusion should be measured by feature gradient rather than feature intensity. The paper builds an encoder-decoder trained only to reconstruct grayscale MS-COCO images, extracts deep feature maps, computes a pixel-wise spatial frequency on those maps, and thresholds the comparison to form a focus decision map. Consistency verification, through morphological opening and closing, removal of regions below 0.01 by image height and width, and guided filtering, refines the map, and the fused image is a weighted average of the two sources. The reported result is state-of-the-art fusion performance versus 16 methods, with the highest mean Qg and Qcb and the most first-place finishes across 38 image pairs.

Load-bearing premise

The load-bearing premise is that spatial frequency computed on deep features from a network trained on unrelated grayscale photos reliably measures focus on new multi-focus pairs, and that the hand-set post-processing parameters do not quietly account for the reported gains.

Editorial extensions

If this is right

  • Multi-focus fusion can be done without synthetic blurred training pairs: a network trained only to reconstruct ordinary grayscale images supplies the focus cue.
  • Gradient-based activity on deep features is a better fusion rule than intensity-based L1-norm, max, average, or abs-max rules, according to the paper's ablation study.
  • The consistency-verification post-processing converts noisy per-pixel focus decisions into clean region boundaries, which is why the decision-map variant outperforms direct deep-feature fusion.
  • The scheme naturally extends to any number of source images by serial pairwise fusion.
  • The same deep-feature spatial-frequency recipe is suggested as applicable to multi-exposure, infrared-visible, and medical image fusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the deep-feature spatial-frequency measure generalizes, it could serve as a focus cue for focal-stack depth estimation in microscopy and computational photography, where no paired blurred images exist.
  • Because the network was trained on generic photo content, the method's performance on non-photographic imagery such as medical scans, text, or synthetic graphics is untested; a cross-domain evaluation would bound its generality.
  • The comparison is summarized by three metric averages, and since SESF-Fuse is below LP on mean Qm, a per-image paired significance test over the 38 pairs would settle whether the overall state-of-the-art claim holds under a different aggregation.
  • A controlled sensitivity analysis of the hand-set parameters (disk radius, area threshold, guided filter radius and regularization) is a natural next experiment; stable rankings across a wide parameter range would make the method genuinely parameter-free in practice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SESF-Fuse, an unsupervised encoder-decoder model for multi-focus image fusion. The network is trained on MS-COCO grayscale images to reconstruct inputs, and at inference time spatial frequency computed on deep features is used to build an initial decision map, followed by morphological consistency verification and guided-filter refinement. The fused image is obtained by pixelwise weighted averaging of the two source images according to the refined decision map. The authors compare against 16 existing fusion methods on 38 multi-focus image pairs using three objective metrics (Qg, Qm, Qcb) and qualitative visualization, and claim state-of-the-art performance based on the highest mean Qg and Qcb values and the largest number of first-place results.

Significance. If the claimed state-of-the-art result holds, the paper makes a useful contribution by demonstrating that feature-gradient activity measures computed on unsupervised deep features can outperform supervised and hand-crafted fusion rules for multi-focus fusion. Strengths of the work include the unsupervised training protocol on independent MS-COCO data, the clear ablation study separating the contributions of spatial-frequency fusion and decision-map refinement, the availability of code, and the use of standard objective metrics and a large comparison set. However, the statistical and generalization evidence for the central claim is currently insufficient, and several fusion hyperparameters appear to be chosen on the test set used for the headline comparisons.

major comments (3)
  1. [Experiments, Table 2] The state-of-the-art claim rests on mean differences of Qg=0.7105 versus 0.7102 for CNN-Fuse and Qcb=0.7848 versus 0.7839 for CNN-Fuse, i.e., margins of 0.0003 and 0.0009 over 38 test pairs. No per-image variance, confidence intervals, or paired significance tests are reported, so these margins may be within noise. Please provide per-image scores and a paired significance test (e.g., Wilcoxon signed-rank test) for Qg, Qcb, and Qm, and report effect sizes, before claiming superiority over CNN-Fuse.
  2. [Method, Fusion subsection; Experiments] Several fusion hyperparameters are explicitly described as empirically set: the guided filter parameters r=4 and ε=0.1, the small-region area threshold 0.01×H×W, and the structuring-element radius tied to the spatial-frequency kernel radius. If these values were selected using the same 38 test pairs that appear in Table 2, the reported metrics are optimistically biased. Please report how these parameters were chosen, use a separate validation set or cross-validation, and include a sensitivity analysis to demonstrate that the results are not artifacts of this tuning.
  3. [Experiments, Table 2 and Conclusion] The proposed method's mean Qm is 2.8886, which is lower than CNN-Fuse (2.9654) and LP (2.9642). The paper argues that the largest first-place count (14 versus 7 and 8) makes the method more robust, but first-place counts do not account for the magnitude of differences or near-ties and are not a substitute for a statistical comparison of the Qm distributions. The concluding claim that the method 'generally outperform[s] other methods, leading to state-of-the-art performance' is therefore not supported on all three metrics.
minor comments (5)
  1. [Experiments, Objective Image Fusion Quality Metrics] The phrase 'In order to access the fusion performance' should be 'assess the fusion performance'.
  2. [Abstract and throughout] The text inconsistently uses 'state-of-art' and 'state-of-the-art'; please standardize to 'state-of-the-art'.
  3. [Abstract] 'The key point behind of proposed method' should be 'behind the proposed method'.
  4. [Experiments, Experimental Settings] The sentence 'we use 38 pairs of multi-focus images as testing set for evaluation, which are publicly available online' does not name the specific datasets or sources; please list them precisely for reproducibility.
  5. [References] The author name 'Savi´c' appears with a malformed accent; please use proper Unicode or LaTeX encoding.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; mild evaluation-tuning risk from post-processing hyperparameters set on the same 38-image test set.

  1. fitted input called prediction [Section III-B 'Consistency Verification' and Section IV-A 'Experimental Settings']
    "In this work, we experimentally set local window radius r to 4 and the regularization parameter ε to 0.1 in guided filter algorithm."

    The 38 multi-focus pairs are introduced as the testing set for evaluation ('we use 38 pairs of multi-focus images as testing set for evaluation'), and the guided-filter parameters that shape the final decision map are described as 'experimentally set' with no separate validation split. Because the headline SOTA metrics in Table 2 are computed on those same 38 pairs, any parameter choice informed by those metrics makes the comparison partly a fit to the evaluation set. The margins over CNN-Fuse are minute (Qg 0.7105 vs 0.7102; Qcb 0.7848 vs 0.7839), so even mild tuning can flip the ranking.

full rationale

The paper's core derivation is not circular. The encoder-decoder is trained unsupervised on MS-COCO, a dataset independent of the multi-focus test pairs, and the spatial-frequency activity measure is defined from the deep features via Eqs. (4)-(6) without using the fusion target. The decision map and fusion rule (Eqs. (7)-(8)) are explicit and do not reintroduce the output metric as an input. There are no self-citations by the present authors: every cited prior method (DenseFuse, CNN-Fuse, guided filtering, etc.) is external. The only circularity-adjacent concern is that the post-processing parameters (guided-filter r and ε, and similarly the disk radius and area threshold) are reported as 'experimentally set' while the same 38-image set is used for the headline evaluation; if those parameters were tuned on the test metrics, the reported advantage is partly fitted. However, the paper does not explicitly state that the parameters were optimized on the test set, and the central unsupervised-deep-feature-plus-spatial-frequency scheme retains independent content. I therefore assign a low score of 2, reflecting a mild evaluation-tuning risk rather than a definitional or self-citation-driven circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the standard focus-sharpness assumption, the untested hypothesis that deep-feature gradients are the right focus measure, and a set of fusion-specific parameters whose values appear to have been chosen on the evaluation set. The autoencoder itself is trained on independent MS-COCO data, which is the paper's strongest independent grounding.

free parameters (4)
  • Spatial frequency kernel radius r = not stated in paper (implicitly tuned)
    Radius of the window used to compute RF and CF in Eqs. (4)-(6); also tied to the disk structuring element radius. The value is not specified, suggesting it was chosen empirically.
  • Small region area threshold = 0.01*H*W
    Regions smaller than this fraction of the image are 'reversed'; set 'usually' in the paper and likely tuned on the test set.
  • Guided filter parameters = r=4, epsilon=0.1
    Explicitly stated to be set experimentally; these smooth the decision map and affect boundary quality.
  • Reconstruction loss weight lambda = 3
    Taken from DenseFuse; not tuned here, but a free parameter of network training that influences the learned features.
assumptions (3)
  • domain assumption In multi-focus photography, regions within the depth of field are sharp and regions outside are blurred; hence a sharpness/activity measure can separate focused and defocused pixels.
    This is the standard premise of multi-focus fusion, cited in the introduction and used throughout.
  • domain assumption Spatial frequency (gradient energy) on deep feature maps is a valid proxy for focus/sharpness.
    The central hypothesis of the paper, asserted in the introduction ('what really matter is feature gradient, not feature intensity') and supported only by internal ablation, not by an independent theoretical argument.
  • domain assumption A reconstruction-trained autoencoder on MS-COCO grayscale images transfers to multi-focus fusion images well enough to extract focus-discriminative features.
    The encoder is trained on natural images for reconstruction, and then frozen; the paper assumes these features generalize to the test fusion pairs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SESF-Fuse: An Unsupervised Deep Model for Multi-Focus Image Fusion." pith.science (2026). https://pith.science/paper/DV4IGXXW

@misc{pith2026190801703,
  author       = {Pith},
  title        = {Pith review of: SESF-Fuse: An Unsupervised Deep Model for Multi-Focus Image Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DV4IGXXW}},
  note         = {Machine review of arXiv:1908.01703}
}
read the original abstract

In this work, we propose a novel unsupervised deep learning model to address multi-focus image fusion problem. First, we train an encoder-decoder network in unsupervised manner to acquire deep feature of input images. And then we utilize these features and spatial frequency to measure activity level and decision map. Finally, we apply some consistency verification methods to adjust the decision map and draw out fused result. The key point behind of proposed method is that only the objects within the depth-of-field (DOF) have sharp appearance in the photograph while other objects are likely to be blurred. In contrast to previous works, our method analyzes sharp appearance in deep feature instead of original image. Experimental results demonstrate that the proposed method achieves the state-of-art fusion performance compared to existing 16 fusion methods in objective and subjective assessment.

Figures

Figures reproduced from arXiv: 1908.01703 by the authors.

Figure 1
Figure 1. The schematic diagram of proposed algorithm. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The detailed fusion strategy. In this paper, the encoder provides high dimensional deep feature for each pixel in an image. However, the original spatial frequency is calculated on gray image with single channel. Thus, for deep features, we modify the spatial fre￾quency calculation method. Let F represents the deep fea￾tures driven from encoder block. F(x,y) represents one fea￾ture vector, (x, y) refers to the coord… view at source ↗
Figure 3
Figure 3. Visualization of fused results. The first row is near focused source image and the second row is far focused source [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of different ’leaf’ and ’Sydney Opera House’ fused results. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The difference images for each ’beer’ fused results [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The difference images for each ’golf’ fused results [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 35 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    Aslantas, V., and Kurban, R. 2010. Fusion of multi-focus images using differential evolution algorithm. Expert Systems with Applications 37(12):8861 -- 8870

    work page 2010

  3. [3]

    Burt , P., and Adelson , E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications 31(4):532--540

    work page 1983

  4. [4]

    Chen, Y., and Blum, R. S. 2009. A new automated quality assessment algorithm for image fusion. Image and Vision Computing 27(10):1421 -- 1432. Special Section: Computer Vision Methods for Ambient Intelligence

    work page 2009

  5. [5]

    De, I.; Chanda, B.; and Chattopadhyay, B. 2006. Enhancing effective depth-of-field by image fusion using mathematical morphology. Image and Vision Computing 24(12):1278 -- 1287

    work page 2006

  6. [6]

    Facebook. 2019. Pytorch. https://pytorch.org

    work page 2019

  7. [7]

    He , K.; Sun , J.; and Tang , X. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6):1397--1409

    work page 2013

  8. [8]

    Hu, J.; Shen, L.; and Sun, G. 2018. Squeeze-and-excitation networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2018

Show all 36 references
  1. [9]

    P., and Ba, J

    Kingma, D. P., and Ba, J. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations

    2015

  2. [10]

    J.; O’Callaghan, R

    Lewis, J. J.; O’Callaghan, R. J.; Nikolov, S. G.; Bull, D. R.; and Canagarajah, N. 2007. Pixel- and region-based image fusion with complex wavelets. Information Fusion 8(2):119 -- 130. Special Issue on Image Fusion: Advances in the State of the Art

    2007

  3. [11]

    Li , H., and Wu , X. 2019. Densefuse: A fusion approach to infrared and visible images. IEEE Transactions on Image Processing 28(5):2614--2623

    2019

  4. [12]

    Li, S.; Kang, X.; Hu, J.; and Yang, B. 2013. Image matting for fusion of multi-focus images in dynamic scenes. Information Fusion 14(2):147 -- 162

    2013

  5. [13]

    Li, S.; Kang, X.; Fang, L.; Hu, J.; and Yin, H. 2017. Pixel-level image fusion: A survey of the state of the art. Information Fusion 33:100 -- 112

    2017

  6. [14]

    Li , S.; Kang , X.; and Hu , J. 2013. Image fusion with guided filtering. IEEE Transactions on Image Processing 22(7):2864--2875

    2013

  7. [15]

    T.; and Wang, Y

    Li, S.; Kwok, J. T.; and Wang, Y. 2001. Combination of images with diverse focuses using the spatial frequency. Information Fusion 2(3):169 -- 176

    2001

  8. [16]

    Li, H.; Manjunath, B.; and Mitra, S. 1995. Multisensor image fusion using the wavelet transform. Graphical Models and Image Processing 57(3):235 -- 245

    1995

  9. [17]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Fleet, D.; Pajdla, T.; Schiele, B.; and Tuytelaars, T., eds., Computer Vision -- ECCV 2014 , 740--755. Cham: Springer ...

    2014

  10. [18]

    Liu , Z.; Blasch , E.; Xue , Z.; Zhao , J.; Laganiere , R.; and Wu , W. 2012. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(1):94--109

    2012

  11. [19]

    Liu, Y.; Chen, X.; Peng, H.; and Wang, Z. 2017. Multi-focus image fusion with a deep convolutional neural network. Information Fusion 36:191 -- 207

    2017

  12. [20]

    Liu, Y.; Liu, S.; and Wang, Z. 2015. Multi-focus image fusion with dense sift. Information Fusion 23:139 -- 155

    2015

  13. [21]

    Liu , Z. 2012. Image fusion metrics. https://github.com/zhengliu6699/imageFusionMetrics

    2012

  14. [22]

    Liu, Y. 2019. Image fusion. http://www.escience.cn/people/liuyu1/Codes.html

    2019

  15. [23]

    Nejati, M.; Samavi, S.; and Shirani, S. 2015. Multi-focus image fusion using dictionary-based sparse representation. Information Fusion 25:72 -- 84

    2015

  16. [24]

    Nencini, F.; Garzelli, A.; Baronti, S.; and Alparone, L. 2007. Remote sensing image fusion using the curvelet transform. Information Fusion 8(2):143 -- 156. Special Issue on Image Fusion: Advances in the State of the Art

    2007

  17. [25]

    Peng-wei Wang , and Bo Liu . 2008. A novel image fusion metric based on multi-scale analysis. In 2008 9th International Conference on Signal Processing , 965--968

    2008

  18. [26]

    Prabhakar., R. 2017. Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. In The IEEE International Conference on Computer Vision (ICCV)

    2017

  19. [27]

    Savi \'c , S., and Babi \'c , Z. 2012. Multifocus image fusion based on empirical mode decomposition. In 19th IEEE International Conference on Systems, Signals and Image Processing (IWSSIP)

    2012

  20. [28]

    Stathaki, T. 2011. Image fusion: algorithms and applications . Elsevier

    2011

  21. [29]

    Toet, A. 1989. Image fusion by a ratio of low-pass pyramid. Pattern Recognition Letters 9(4):245 -- 253

    1989

  22. [30]

    C.; Sheikh, H

    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; and Simoncelli, E. P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4):600--612

    2004

  23. [31]

    Wikipedia. 2019. Focus stacking. https://github.com/cmcguinness/focusstack

    2019

  24. [32]

    Xu, K. 2019. Image fusion. http://xudongkang.weebly.com/index.html

    2019

  25. [33]

    S., and Petrovic , V

    Xydeas , C. S., and Petrovic , V. 2000. Objective image fusion performance measure. Electronics Letters 36(4):308--309

    2000

  26. [34]

    Yang , B., and Li , S. 2010. Multifocus image fusion and restoration with sparse representation. IEEE Transactions on Instrumentation and Measurement 59(4):884--892

    2010

  27. [35]

    Zhang, Q., and long Guo, B. 2009. Multifocus image fusion using the nonsubsampled contourlet transform. Signal Processing 89(7):1334 -- 1346

    2009

  28. [36]

    Zhou, Z.; Li, S.; and Wang, B. 2014. Multi-scale weighted gradient-based fusion for multi-focus images. Information Fusion 20:60 -- 72

    2014

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.