pith. sign in

arxiv: 1907.03217 · v1 · pith:43NEQLJYnew · submitted 2019-07-07 · 📡 eess.IV · cs.CV

Adaptive Weighting Depth-variant Deconvolution of Fluorescence Microscopy Images with Convolutional Neural Network

Da He , De Cai , Jiasheng Zhou , Jiajia Luo , Sung-Liang Chen This is my paper

Pith reviewed 2026-05-25 01:45 UTC · model grok-4.3

classification 📡 eess.IV cs.CV
keywords fluorescence microscopydepth-variant PSFdeconvolutionconvolutional neural networkdefocus predictionimage restorationout-of-focus imagesadaptive weighting
0
0 comments X

The pith

A convolutional neural network predicts defocus levels of image patches to enable depth-variant deconvolution that restores out-of-focus fluorescence microscopy images.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that training a convolutional neural network to classify image patches into one of eleven defocus levels supplies the correct depth-variant point spread function for each patch. This prediction step is followed by patch-wise deconvolution and an adaptive weighting step that stitches the results into a full image while removing boundary artifacts. Traditional deconvolution assumes a single blur function across all depths and therefore leaves residual blur in thick specimens. The reported accuracy of 98.2 percent on the training dataset produces measured gains of up to 6.6 dB in peak signal-to-noise ratio and 11 percent in structural similarity index after deconvolution. The work therefore supplies a concrete route from learned blur estimation to improved image recovery in fluorescence microscopy.

Core claim

The central claim is that DelpNet, a convolutional neural network trained to predict defocus level, achieves 98.2 percent accuracy on the authors' microscopy dataset and thereby supplies accurate depth-variant point spread functions. These functions are inserted into adaptive weighting depth-variant deconvolution, which processes image patches separately and then combines them with weighting to suppress boundary artifacts. Validation on patches at eleven defocus levels shows maximum improvements of 6.6 dB in peak signal-to-noise ratio and 11 percent in structural similarity index. The method is presented as the first to handle out-of-focus fluorescence images with a learning-based depth-

What carries the argument

DelpNet, the convolutional neural network that classifies image patches by defocus level to select the matching depth-variant point spread function for deconvolution.

If this is right

  • Patch-wise deconvolution guided by predicted defocus levels recovers detail in out-of-focus regions that depth-invariant methods leave blurred.
  • Adaptive weighting of the patch results removes visible seams at patch boundaries in the final image.
  • The approach yields higher peak signal-to-noise ratio and structural similarity index than prior methods on the same dataset.
  • The technique supplies a practical route to higher-quality images of thick specimens without changing the microscope hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the eleven discrete levels prove too coarse for specimens thicker than those in the training set, the residual blur after deconvolution may increase.
  • Retraining DelpNet on images from a different microscope would be required before the method can be used on data collected under altered optical conditions.
  • Combining the depth prediction step with other restoration algorithms could further reduce artifacts in live imaging sequences.

Load-bearing premise

The network trained on the authors' dataset will correctly classify defocus level on new fluorescence images and the eleven discrete levels will adequately represent the continuous depth variation present in thick specimens.

What would settle it

Apply the trained DelpNet to a fresh collection of real fluorescence images acquired from different specimens or microscopes and check whether defocus prediction accuracy falls below 98 percent or whether the reported PSNR and SSIM gains after deconvolution disappear.

Figures

Figures reproduced from arXiv: 1907.03217 by Da He, De Cai, Jiajia Luo, Jiasheng Zhou, Sung-Liang Chen.

Figure 1
Figure 1. Figure 1: Two examples of fluorescence microscopy images. (a) An in-focus image. (b) An out-of-focus image. two kinds of blurs in fluorescence microscopy: one is caused by the depth-variant microscopic point spread function (PSF) and another by Poisson noise. The former is also associated with the limited depth of field in a fluorescence micro￾scope, which inevitably causes low-quality images in out-of-focus regions… view at source ↗
Figure 2
Figure 2. Figure 2: The overview of the method. (a) The process of generating synthetic datasets. (b) The architecture of DelpNet. (c) An example of the patch-wise defocus level pre￾diction process. The prediction results are expressed in different colors, and the “bg” here indicates a background patch. (d) The AWDVD for two cases: (i) the stride is the same as the patch size and it is just the non-overlapping patch-wise deco… view at source ↗
Figure 3
Figure 3. Figure 3: Evaluation result of DelpNet. The confusion matrix shows the distribution of prediction results of DelpNet on the held-out test set. 3.2 Analysis of Special Settings in DelpNet We manually set an extra label “bg” to split patches with almost full background from normal defocused patches. To analyze the value of this strategy, we trained the almost same DelpNet with two settings: with “bg” label and without… view at source ↗
Figure 4
Figure 4. Figure 4: The validation loss curves of three Batch Normalization momentum parameters. The blue curve showing momentum of 0.99 is disordered. Different from most works that apply Batch Normalization with the momen￾tum parameter of a relatively large value (e.g., default 0.99 in [1]), we assigned this parameter 0.60. This is because we found large Batch Normalization mo￾mentum parameters easily lead to oscillations o… view at source ↗
Figure 5
Figure 5. Figure 5: Cumulative match curves of different settings. Each node represents the corre￾sponding top-k accuracy of a specified training strategy (e.g., the most left blue node indicates that if we train DelpNet model without “bg” label, the top-1 accuracy on the test set is about 96.8%.). The variable k is in the range of 1–11 for the blue curve and the range of 1–12 for other curves. 3.3 Comparison among Architectu… view at source ↗
Figure 6
Figure 6. Figure 6: Cumulative match curves of trained DelpNet and other 4 representative CNNs. Each node represents the corresponding top-k accuracy of a CNN architecture (e.g., the most left blue node indicates that using the trained ResNet18 model, the top-1 accuracy on the test set is about 88.7%.). All the architectures were retrained by us with slight modifications for input and output formats [PITH_FULL_IMAGE:figures/… view at source ↗
Figure 7
Figure 7. Figure 7: The PSF with a random defocus level blurred the in-focus image, result [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 7
Figure 7. Figure 7: An image patch for (a) original in-focus image, (b) defocused image, (c) noisy defocused image, (d) depth-variant deconvolved image, and (e) depth-invariant decon￾volved image. (f) The 1D profile of the above five images along the red arrow direction. (g) The SSIM and PSNR values of the above five images, taking (a) as the comparison reference. red arrows (the whole 84 pixels) in Figs. 7(a)–7(e). As shown … view at source ↗
Figure 8
Figure 8. Figure 8: (a) PSNR improvement for depth-variant and depth-invariant deconvolved im￾ages. (b) SSIM of the noisy defocused, depth-variant deconvolved, and depth-invariant deconvolved images; the SSIM ratio (the orange curve and text), defined as the SSIM of the depth-variant deconvolved image over that of the noisy defocused one. culties for recovery. The SSIM of the depth-variant deconvolution ranged from 0.66 to 1.… view at source ↗
Figure 9
Figure 9. Figure 9: Whole image results for (a) realistic image, (b) AWDVD of case (i) in non￾overlapping patch-wise deconvolution, and (c) AWDVD of case (ii) in a bilinear inter￾polation way. (d)–(f) are the zoomed-in image blocks in the red dashed boxes in (a)–(c) respectively. the number of defocus levels as those in [30], our model achieved tremendous accuracy improvement from 95% in [30] to 98.2%. The improvement could b… view at source ↗
read the original abstract

Fluorescence microscopy plays an important role in biomedical research. The depth-variant point spread function (PSF) of a fluorescence microscope produces low-quality images especially in the out-of-focus regions of thick specimens. Traditional deconvolution to restore the out-of-focus images is usually insufficient since a depth-invariant PSF is assumed. This article aims at handling fluorescence microscopy images by learning-based depth-variant PSF and reducing artifacts. We propose adaptive weighting depth-variant deconvolution (AWDVD) with defocus level prediction convolutional neural network (DelpNet) to restore the out-of-focus images. Depth-variant PSFs of image patches can be obtained by DelpNet and applied in the afterward deconvolution. AWDVD is adopted for a whole image which is patch-wise deconvolved and appropriately cropped before deconvolution. DelpNet achieves the accuracy of 98.2%, which outperforms the best-ever one using the same microscopy dataset. Image patches of 11 defocus levels after deconvolution are validated with maximum improvement in the peak signal-to-noise ratio and structural similarity index of 6.6 dB and 11%, respectively. The adaptive weighting of the patch-wise deconvolved image can eliminate patch boundary artifacts and improve deconvolved image quality. The proposed method can accurately estimate depth-variant PSF and effectively recover out-of-focus microscopy images. To our acknowledge, this is the first study of handling out-of-focus microscopy images using learning-based depth-variant PSF. Facing one of the most common blurs in fluorescence microscopy, the novel method provides a practical technology to improve the image quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes DelpNet, a CNN that classifies image patches into one of 11 discrete defocus levels to estimate depth-variant PSFs, which are then used within an adaptive weighting depth-variant deconvolution (AWDVD) pipeline to restore out-of-focus fluorescence microscopy images. It reports 98.2% classification accuracy on the authors' dataset (outperforming prior work) together with maximum post-deconvolution gains of 6.6 dB PSNR and 11% SSIM on patches from the same 11 levels, and claims the method eliminates patch-boundary artifacts while providing the first learning-based treatment of depth-variant blur.

Significance. If the reported accuracy and improvement figures prove reproducible and the method generalizes beyond the training distribution, the work would supply a concrete, patch-wise engineering solution to a routine practical problem in thick-specimen fluorescence imaging where depth-invariant PSF assumptions produce artifacts. The adaptive-weighting step to suppress boundary seams is a modest but useful implementation detail.

major comments (2)
  1. [Abstract] Abstract: the central performance claims (98.2% accuracy, 6.6 dB PSNR, 11% SSIM) are presented with no information on training-set size, loss function, optimizer, regularization, or cross-validation procedure, so it is impossible to judge whether the numbers support the claim that DelpNet enables effective depth-variant recovery.
  2. [Abstract] Abstract (and the paragraph describing DelpNet and validation): no experiments are reported that test generalization to images acquired on different microscopes, with different noise statistics, or from specimens whose thickness produces continuous rather than discrete depth variation; without such tests the claim that the 11-level discrete sampling plus learned PSF yields artifact-free recovery on practical data remains unsupported.
minor comments (1)
  1. [Abstract] Abstract: 'To our acknowledge' is a typographical error and should read 'To our knowledge'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed review and constructive suggestions. We address each major comment below and indicate where revisions have been made to the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central performance claims (98.2% accuracy, 6.6 dB PSNR, 11% SSIM) are presented with no information on training-set size, loss function, optimizer, regularization, or cross-validation procedure, so it is impossible to judge whether the numbers support the claim that DelpNet enables effective depth-variant recovery.

    Authors: We agree that the abstract would be strengthened by including key experimental details. The body of the manuscript (Section 3) specifies the training set size, cross-entropy loss, Adam optimizer, and 5-fold cross-validation procedure used to obtain the 98.2% accuracy. We will revise the abstract to briefly summarize these elements so that the performance numbers can be evaluated in context. revision: yes

  2. Referee: [Abstract] Abstract (and the paragraph describing DelpNet and validation): no experiments are reported that test generalization to images acquired on different microscopes, with different noise statistics, or from specimens whose thickness produces continuous rather than discrete depth variation; without such tests the claim that the 11-level discrete sampling plus learned PSF yields artifact-free recovery on practical data remains unsupported.

    Authors: The presented experiments focus on the authors' dataset with its 11 discrete defocus levels, which is the setting in which the 98.2% accuracy and PSNR/SSIM gains are demonstrated. We acknowledge that additional tests on other microscopes, noise conditions, or continuous depth variation would further support broader claims. In the revised manuscript we have expanded the discussion to explicitly state the discrete-level assumption and its implications for practical thick-specimen imaging, while noting generalization as future work. revision: partial

Circularity Check

0 steps flagged

No significant circularity; standard supervised CNN pipeline with empirical validation

full rationale

The paper describes DelpNet as a CNN classifier trained to predict one of 11 discrete defocus levels from image patches, followed by patch-wise depth-variant deconvolution and adaptive weighting. Reported metrics (98.2% accuracy, 6.6 dB PSNR, 11% SSIM gains) are direct empirical measurements on held-out patches from the training distribution. No equations, fitted parameters, or self-citations are presented that reduce these outputs to the inputs by construction. The derivation chain consists of standard supervised learning steps with no self-definitional, fitted-input, or uniqueness-imported circularity. The method remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 2 invented entities

The central claim rests on the existence of a discrete set of 11 defocus levels that capture depth variation and on the ability of a CNN to map image appearance to those levels; both are domain assumptions rather than derived quantities.

free parameters (1)
  • number of defocus levels
    Set to 11 for the microscopy dataset; used to discretize continuous depth variation.
axioms (1)
  • domain assumption Fluorescence microscopy images can be modeled as convolution with a depth-dependent PSF plus noise
    Standard optical model invoked to justify deconvolution step
invented entities (2)
  • DelpNet no independent evidence
    purpose: Classify image patches into one of 11 defocus levels
    New CNN architecture introduced for the prediction task
  • AWDVD no independent evidence
    purpose: Perform patch-wise depth-variant deconvolution with boundary artifact suppression
    New adaptive weighting procedure proposed

pith-pipeline@v0.9.0 · 5825 in / 1371 out tokens · 21193 ms · 2026-05-25T01:45:06.649326+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · 5 internal anchors

  1. [1]

    In: 12th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI)

    Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghe- mawat, S., Irving, G., Isard, M., et al.: Tensorflow: A system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI). pp. 265–283 (2016)

    work page 2016

  2. [2]

    Optics Communications 284(6), 1548–1552 (2011)

    Arines, J.: Partially compensated deconvolution from wavefront sensing images of the eye fundus. Optics Communications 284(6), 1548–1552 (2011)

    work page 2011

  3. [3]

    Elements of the theory of diffraction, pp

    Born, M., Wolf, E.: Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, chap. Elements of the theory of diffraction, pp. 412–512. Cambridge University Press (1999)

    work page 1999

  4. [4]

    Biomedical optics express 8(12), 5675–5687 (2017)

    Fei, X., Zhao, J., Zhao, H., Yun, D., Zhang, Y.: Deblurring adaptive optics retinal images using deep convolutional neural networks. Biomedical optics express 8(12), 5675–5687 (2017)

    work page 2017

  5. [5]

    In: Proceedings of the IEEE international conference on computer vision

    He, K., Gkioxari, G., Doll´ ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision. pp. 2961–2969 (2017)

    work page 2017

  6. [6]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016) 15

    work page 2016

  7. [7]

    Improving neural networks by preventing co-adaptation of feature detectors

    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  8. [8]

    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  9. [9]

    Scientific reports 5, 9894 (2015)

    Kim, B., Naemura, T.: Blind depth-variant deconvolution of 3d data in wide-field fluorescence microscopy. Scientific reports 5, 9894 (2015)

    work page 2015

  10. [10]

    Adam: A Method for Stochastic Optimization

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  11. [11]

    Network In Network

    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  12. [12]

    Molecular cell 58(4), 644–659 (2015)

    Liu, Z., Lavis, L.D., Betzig, E.: Imaging live-cell dynamics and structure at the single-molecule level. Molecular cell 58(4), 644–659 (2015)

    work page 2015

  13. [13]

    Nature methods 9(7), 637–637 (2012)

    Ljosa, V., Sokolnicki, K.L., Carpenter, A.E.: Annotated high-throughput mi- croscopy image sets for validation. Nature methods 9(7), 637–637 (2012)

    work page 2012

  14. [14]

    The astronomical journal 79, 745 (1974)

    Lucy, L.B.: An iterative technique for the rectification of observed distributions. The astronomical journal 79, 745 (1974)

    work page 1974

  15. [15]

    In: 2013 IEEE International Conference on Image Processing

    Morin, R., Bidon, S., Basarab, A., Kouam´ e, D.: Semi-blind deconvolution for reso- lution enhancement in ultrasound imaging. In: 2013 IEEE International Conference on Image Processing. pp. 1413–1417 (2013)

    work page 2013

  16. [16]

    Cold Spring Harbor Protocols 2011(12), pdb–top066936 (2011)

    Murray, J.M.: Methods for imaging thick specimens: confocal microscopy, decon- volution, and structured illumination. Cold Spring Harbor Protocols 2011(12), pdb–top066936 (2011)

    work page 2011

  17. [17]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 779–788 (2016)

    work page 2016

  18. [18]

    JOSA 62(1), 55–59 (1972)

    Richardson, W.H.: Bayesian-based iterative method of image restoration. JOSA 62(1), 55–59 (1972)

    work page 1972

  19. [19]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: In- verted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4510–4520 (2018)

    work page 2018

  20. [20]

    JOSA A 24(9), 2622–2627 (2007)

    Shaevitz, J.W., Fletcher, D.A.: Enhanced three-dimensional deconvolution mi- croscopy using a measured depth-varying point-spread function. JOSA A 24(9), 2622–2627 (2007)

    work page 2007

  21. [21]

    In: 2018 25th IEEE International Conference on Image Processing (ICIP)

    Shajkofci, A., Liebling, M.: Semi-blind spatially-variant deconvolution in optical microscopy with local point spread function estimation by use of convolutional neural networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP). pp. 3818–3822 (2018)

    work page 2018

  22. [22]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  23. [23]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the incep- tion architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2818–2826 (2016)

    work page 2016

  24. [24]

    In: Adaptive Optics Systems V

    Thi´ ebaut,´E., Denis, L., Soulez, F., Mourya, R.: Spatially variant psf modeling and image deblurring. In: Adaptive Optics Systems V. vol. 9909, p. 99097N (2016)

    work page 2016

  25. [25]

    Mathematical Problems in Engineering 2014 (2014)

    Tsai, H.C., Wu, J.L.: An improved adaptive deconvolution algorithm for single image deblurring. Mathematical Problems in Engineering 2014 (2014)

    work page 2014

  26. [26]

    IEEE transactions on image processing 13(4), 600–612 (2004) 16 He et al

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., et al.: Image quality as- sessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4), 600–612 (2004) 16 He et al

    work page 2004

  27. [27]

    In: Advances in Neural Information Processing Systems

    Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Advances in Neural Information Processing Systems. pp. 1790– 1798 (2014)

    work page 2014

  28. [28]

    Optics letters 37(14), 2778–2780 (2012)

    Yan, L., Fang, H., Zhong, S.: Blind image deconvolution with spatially adaptive total variation regularization. Optics letters 37(14), 2778–2780 (2012)

    work page 2012

  29. [29]

    Applied spectroscopy 66(11), 1334–1346 (2012)

    Yan, L., Liu, H., Zhong, S., Fang, H.: Semi-blind spectral deconvolution with adap- tive tikhonov regularization. Applied spectroscopy 66(11), 1334–1346 (2012)

    work page 2012

  30. [30]

    BMC bioinformatics 19(1), 77 (2018)

    Yang, S.J., Berndl, M., Ando, D.M., Barch, M., Narayanaswamy, A., Christiansen, E., Hoyer, S., Roat, C., Hung, J., Rueden, C.T., et al.: Assessing microscope image focus quality with deep learning. BMC bioinformatics 19(1), 77 (2018)

    work page 2018