REVIEW 3 major objections 4 minor 29 references
Learned reconstructions for practical mask-based lensless imaging
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Unrolled ADMM networks make mask-based lensless cameras practical for interactive use, reconstructing in 75 ms with better perceptual quality than a converged solver.
desk verdict A solid, reproducible demonstration that unrolled ADMM makes lensless imaging 20x faster with better-looking outputs, though the headline 3x LPIPS gain is inflated because LPIPS is also the training loss. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the unrolled ADMM network, in which each iteration of the alternating direction method of multipliers is treated as one layer of a neural network. The update equations—soft-thresholding for sparsity, least-squares for data fidelity, non-negativity projection, and dual-variable updates—stay intact, but the per-layer penalty parameters $\mu_1,\mu_2,\mu_3,\tau$ are learned by backpropagation. Optional learned pieces are a small U-Net used as a regularizer inside the iterations (Le-ADMM*) and a larger U-Net applied as a final denoiser (Le-ADMM-U). The learned parameters are what absorb forward-model error, while the fixed update structure is what keeps the network bounded-compute and sample-efficient.
What would settle it
Measure a point source with the same DiffuserCam at several distances, such as 10 cm, 50 cm, and 2 m, and compare the recorded PSFs: if the pattern changes shape with depth instead of simply translating, the shift-invariant forward model in Eq. (1) is violated. A complementary test is to collect aligned lensed/lensless pairs of natural 3D scenes and compute LPIPS and MSE for Le-ADMM-U versus converged ADMM; if the learned advantage disappears outside the monitor plane, the generalization claim is not supported.
Extended reading notes
Core claim
The central claim is that unrolling ADMM with the lensless-specific variable splitting $b=CHx$ into five layers produces reconstructions comparable to 100-iteration ADMM at one twentieth of the runtime, and that adding learned components lets the network absorb model mismatch rather than being penalized by it. Concretely, Le-ADMM-U averages LPIPS $0.1904$ and MSE $0.0074$ on a 1,000-image held-out set, versus $0.5711$ and $0.0622$ for converged ADMM, in 75 ms rather than 1.5 s. The paper also claims that this training transfers beyond the flat-screen training setup: its networks reconstruct natural objects in the wild with image quality similar to or better than converged ADMM.
Load-bearing premise
The load-bearing premise is that a single measured point-spread function, taken at the training distance, describes how the camera blurs any scene point, including natural scenes at other depths and with different lighting; if that shift-invariance breaks, the wild-scene reconstructions do not establish generalization.
Editorial extensions
If this is right
- A DiffuserCam-style lensless imager using Le-ADMM or Le-ADMM-U can show the user a reconstruction in about 75 ms, making interactive preview of the scene feasible rather than waiting roughly 1.5 s for converged ADMM.
- On the 1,000-image test set, Le-ADMM-U improves average LPIPS from 0.5711 to 0.1904 and MSE from 0.0622 to 0.0074, so perceptual quality is about three times closer to the lensed-camera ground truth.
- With only 20 learned parameters, Le-ADMM matches the perceptual quality of converged ADMM while using far fewer training pairs, which suggests the physical model does most of the work.
- Because the learned networks trade data fidelity for perceptual quality, users can choose where to sit on the model-to-deep spectrum depending on whether measurement consistency or visual appeal matters more.
- The same networks, trained on images of a flat screen, reconstruct natural 3D objects under different lighting in the paper's qualitative wild tests, indicating the approach is not confined to the training display.
Reading between the lines
- The reported 20× speedup is for GPU execution at a fixed five-layer budget; extending interactive preview to low-power or mobile hardware would require quantization or distillation, which the paper does not address.
- A natural testable extension is to train the same unrolled architecture on multiple depth-dependent PSFs, which could turn the current flat-screen-trained network into a depth-aware lensless imager without changing the unrolling principle.
- Because the lensed-camera labels define what 'ground truth' means, the achievable quality is bounded by the training camera's aberrations and field of view; applications without a lensed reference would need a self-supervised or physics-consistency loss.
- The observed data-fidelity-versus-quality trade-off suggests that a practical product could expose a 'fidelity slider' by varying the number of unrolled layers or the denoiser strength.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes unrolled ADMM networks for DiffuserCam lensless imaging, with three variants: Le-ADMM (learned penalty and sparsity parameters), Le-ADMM* (adds a learned CNN regularizer), and Le-ADMM-U (adds a jointly trained U-Net denoiser). The networks are trained on 25,000 experimentally captured lensed/lensless image pairs displayed on a monitor, then compared with classical ADMM and a purely deep U-Net on a 1,000-image held-out test set. The paper reports a 20x reconstruction speedup for Le-ADMM and Le-ADMM-U, and a 3x improvement in LPIPS for Le-ADMM-U over converged ADMM, and shows qualitative reconstructions of natural objects in the wild.
Significance. If the central claims hold, this is a practically valuable contribution: it demonstrates a physically informed, bounded-compute reconstruction framework for a real lensless camera, provides a sizable experimental paired dataset, and makes the source code available. The explicit comparison along a model-based-to-deep spectrum is useful, and the result that a small number of learned ADMM parameters already matches classical ADMM at 20x lower cost is credible and worth publishing. The main caveat is that the headline perceptual-quality claim rests on a metric that is also used as the training loss, and the wild generalization claim is only qualitative; both need additional evidence before the claims can stand as stated.
major comments (3)
- [Sec. 4.5, Sec. 6.1, Table 2] The headline 3x LPIPS improvement of Le-ADMM-U over converged ADMM is partly by construction. The training loss in Sec. 4.5 is a combination of MSE and LPIPS, and the evaluation in Table 2 uses the same LPIPS metric. Le-ADMM-U also has markedly worse data fidelity (22.14 vs. 13.62), so it is free to generate plausible texture that lowers LPIPS without being faithful to the measurement. An independent perceptual evaluation is needed: for example, a human observer study, a perceptual metric that was not used in training, or an evaluation of reconstructions trained with an MSE-only objective. Without such evidence, the phrase 'better perceptual image quality' should be tempered.
- [Sec. 6.2, Fig. 6] The claim that the networks generalize to images in the wild is supported only by qualitative visual comparisons, with no ground-truth images or quantitative metrics for the natural scenes. Because the training data are flat monitor images at a fixed focal distance, the transfer to 3D objects with variable lighting and depth is a load-bearing premise. I would like to see a quantitative evaluation on natural scenes, for example by capturing simultaneous lensed ground truth for a small set of wild scenes or by imaging known targets with registered reference images.
- [Sec. 6.1, Table 2] The ADMM baseline is labeled 'converged' at 100 iterations, but no convergence criterion or residual values are reported. If the baseline is not fully converged, the reported LPIPS improvement factor could be inflated. Please report the ADMM stopping criterion, primal/dual residuals, or rename the baseline '100-iteration ADMM' to avoid overstating the comparison.
minor comments (4)
- [Sec. 4.1] The heading 'Learned AMMM (Le-ADMM)' appears to contain a typo; it should be 'Learned ADMM'.
- [Sec. 5] The MSE/LPIPS loss weighting schedule is described only qualitatively ('weight MSE more heavily during earlier epochs'); for reproducibility, the exact schedule or a pointer to the released code settings should be provided.
- [Table 2] The U-Net data fidelity entry is listed as '19', which is ambiguous and likely missing a decimal point; please format all numbers consistently.
- [Fig. 5] The subfigure labels in Fig. 5(b) appear corrupted or misformatted ('10/zero.sups', '1 2 3 4', etc.); these should be fixed for legibility.
Circularity Check
The headline 3x LPIPS improvement is partially circular because LPIPS is a training loss, not an independent perceptual evaluation.
-
fitted input called prediction
[Sec. 4.5 (Loss functions), Sec. 5 (Implementation), Sec. 6.1 (Test set results, Table 2)]
"Here, we will use the Learned Perceptual Image Patch Similarity metric (LPIPS) that uses deep features and aims to quantify a perceptual distance between two images, as introduced in [25]. During training, we use a combination of both MSE and LPIPS, as outlined in Section 5. ... Overall, Le-ADMM-U has 3× better image quality than converged ADMM as measured by the LPIPS metric."
The headline quality claim is evaluated with LPIPS, but the same LPIPS function is a component of the training loss (with MSE) and is explicitly weighted more heavily in later epochs. The network parameters are therefore fitted to minimize exactly the metric that is later reported as independent evidence of perceptual quality. The test set is held out, so the comparison is not tautological, but the 3x LPIPS improvement largely reflects successful optimization of the training objective rather than an external perceptual judgment. This is reinforced by Table 2: Le-ADMM-U's data fidelity is much worse (22.14) than ADMM's (13.62), indicating the LPIPS gain is achieved by departing from the physical model, i.e., by optimizing the loss rather than by better reconstruction.
full rationale
The paper's forward model (Eq. 1) and the unrolled ADMM updates (Eqs. 4-5) are presented in full and are not defined in terms of the outputs they are used to predict. The DiffuserCam and ADMM citations to the authors' prior work reproduce the actual update equations, so they are not load-bearing as self-citations in a circular way; a measured PSF is used for calibration. The central circularity is confined to the perceptual-quality claim: LPIPS appears both as a training loss (Sec. 4.5: 'we use a combination of both MSE and LPIPS') and as the evaluation metric for the headline 3x improvement (Sec. 6.1 and Table 2). Because the network is trained to minimize LPIPS, reporting test LPIPS as independent evidence of better perceptual quality is partly a fitted-input-called-prediction pattern. However, the architecture, the experimental dataset, the speedup, the held-out MSE numbers, and the qualitative wild reconstructions are independent contributions, so the paper is not wholly circular. Score 5 reflects moderate, partial circularity in the central perceptual metric rather than a fully forced derivation.
Assumptions & free parameters
free parameters (4)
- Per-layer ADMM penalty and sparsity parameters {mu1^k, mu2^k, mu3^k, tau^k} =
Not reported individually; learned from training data
- CNN regularizer weights N in Le-ADMM* =
32,135 learned parameters
- U-Net denoiser weights U in Le-ADMM-U =
10,605,927 learned parameters
- MSE/LPIPS loss weighting schedule =
Not precisely specified
assumptions (4)
- domain assumption The forward model b = C H x uses a shift-invariant point spread function h, measured once with an LED, and this model stays valid for all scenes.
- domain assumption Lensed camera images, after OpenCV distortion correction, are valid ground-truth labels for the scene radiance.
- domain assumption Training on images of a flat computer screen at a fixed distance transfers to natural 3D objects and different lighting conditions.
- standard math A fixed number of five unrolled ADMM iterations is sufficient for the learned network to approximate the desired reconstruction.
Cite this review
Pith. "Pith review of Learned reconstructions for practical mask-based lensless imaging." pith.science (2026). https://pith.science/paper/QOVEWLJ2
@misc{pith2026190811502,
author = {Pith},
title = {Pith review of: Learned reconstructions for practical mask-based lensless imaging},
year = {2026},
howpublished = {\url{https://pith.science/paper/QOVEWLJ2}},
note = {Machine review of arXiv:1908.11502}
}
read the original abstract
Mask-based lensless imagers are smaller and lighter than traditional lensed cameras. In these imagers, the sensor does not directly record an image of the scene; rather, a computational algorithm reconstructs it. Typically, mask-based lensless imagers use a model-based reconstruction approach that suffers from long compute times and a heavy reliance on both system calibration and heuristically chosen denoisers. In this work, we address these limitations using a bounded-compute, trainable neural network to reconstruct the image. We leverage our knowledge of the physical system by unrolling a traditional model-based optimization algorithm, whose parameters we optimize using experimentally gathered ground-truth data. Optionally, images produced by the unrolled network are then fed into a jointly-trained denoiser. As compared to traditional methods, our architecture achieves better perceptual image quality and runs 20x faster, enabling interactive previewing of the scene. We explore a spectrum between model-based and deep learning methods, showing the benefits of using an intermediate approach. Finally, we test our network on images taken in the wild with a prototype mask-based camera, demonstrating that our network generalizes to natural images.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
FlatCam: Replacing lenses with masks and computation,
M. S. Asif, A. Ayremlou, A. Veeraraghavan, R. Baraniuk, and A. Sankaranarayanan, “FlatCam: Replacing lenses with masks and computation,” inComputer Vision Workshop (ICCVW), 2015 IEEE International Conference on, (IEEE, 2015), pp. 663–666
work page 2015
-
[2]
Thinobservationmodulebyboundoptics: conceptandexperimentalverification,
J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thinobservationmodulebyboundoptics: conceptandexperimentalverification,”Appl.Opt. 40,1806–1813(2001)
work page 2001
-
[3]
DiffuserCam: lenslesssingle-exposure 3D imaging,
N.Antipa,G.Kuo,R.Heckel,B.Mildenhall,E.Bostan,R.Ng,andL.Waller,“DiffuserCam: lenslesssingle-exposure 3D imaging,” Optica5, 1–9 (2018)
work page 2018
-
[4]
3D Fluorescence Microscopy with DiffuserCam,
G. Kuo, N. Antipa, R. Ng, and L. Waller, “3D Fluorescence Microscopy with DiffuserCam,” inComputational Optical Sensing and Imaging, (Optical Society of America, 2018), pp. CM3E–3
work page 2018
-
[5]
Miniature 3D Fluorescence Microscope Using Random Microlenses,
K. Yanny, N. Antipa, R. Ng, and L. Waller, “Miniature 3D Fluorescence Microscope Using Random Microlenses,” in Optics and the Brain, (Optical Society of America, 2019), pp. BT3A–4
work page 2019
-
[6]
Single-shot 3D fluorescence microscopy with Fourier DiffuserCam,
F. L. Liu, V. Madhavan, N. Antipa, G. Kuo, S. Kato, and L. Waller, “Single-shot 3D fluorescence microscopy with Fourier DiffuserCam,” inNovel Techniques in Microscopy,(Optical Society of America, 2019), pp. NS2B–3
work page 2019
-
[7]
Three-Dimensional Information Acquisition Using a Compound Imaging System,
R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-Dimensional Information Acquisition Using a Compound Imaging System,” Opt. Rev.14, 347–350 (2007)
work page 2007
-
[8]
Video from Stills: Lensless Imaging with Rolling Shutter
N. Antipa, P. Oare, E. Bostan, R. Ng, and L. Waller, “Video from Stills: Lensless Imaging with Rolling Shutter,” arXiv preprint arXiv:1905.13221 (2019)
work page Pith review arXiv 2019
Show all 29 references
-
[9]
Single-frame3DfluorescencemicroscopywithultraminiaturelenslessFlatScope,
J.K.Adams,V.Boominathan,B.W.Avants,D.G.Vercosa,F.Ye,R.G.Baraniuk,J.T.Robinson,andA.Veeraraghavan, “Single-frame3DfluorescencemicroscopywithultraminiaturelenslessFlatScope,”Sci.advances 3,e1701548(2017)
2017
-
[10]
Thermal Escher Sensors: Pixel-efficient Lensless Imagers Based on Tiled Optics,
P. R. Gill, J. Tringali, A. Schneider, S. Kabir, D. G. Stork, E. Erickson, and M. Kellam, “Thermal Escher Sensors: Pixel-efficient Lensless Imagers Based on Tiled Optics,” inComputational Optical Sensing and Imaging, (Optical Society of America, 2017), pp. CTu3B–3
2017
-
[11]
Lensless light-field imaging with multi-phased fresnel zone aperture,
K.Tajima, T. Shimano, Y. Nakamura, M. Sao, and T. Hoshizawa, “Lensless light-field imaging with multi-phased fresnel zone aperture,” in2017 IEEE International Conference on Computational Photography (ICCP), (2017), pp. 76–82
2017
-
[12]
DiffuserCam: diffuser-based lensless cameras,
G. Kuo, N. Antipa, R. Ng, and L. Waller, “DiffuserCam: diffuser-based lensless cameras,” inComputational Optical Sensing and Imaging, (Optical Society of America, 2017), pp. CTu3B–2
2017
-
[13]
Optical, mathematical, and computational foundations of lensless ultra-miniature diffractive imagers and sensors,
D. G. Stork and P. R. Gill, “Optical, mathematical, and computational foundations of lensless ultra-miniature diffractive imagers and sensors,” Int. J. on Adv. Syst. Meas.7, 4 (2014)
2014
-
[14]
A fast iterative shrinkage-thresholding algorithm for linear inverse problems,
A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM journal on imaging sciences2, 183–202 (2009)
2009
-
[15]
Distributed optimization and statistical learning via the alternating direction method of multipliers,
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations Trends Mach. learning3, 1–122 (2011)
2011
-
[16]
Imaging through glass diffusers using densely connected convolutional networks,
S. Li, M. Deng, J. Lee, A. Sinha, and G. Barbastathis, “Imaging through glass diffusers using densely connected convolutional networks,” Optica5, 803–813 (2018)
2018
-
[17]
Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media,
Y. Li, Y. Xue, and L. Tian, “Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media,” Optica5, 1181–1190 (2018)
2018
-
[18]
DeeplearningapproachforFourierptychographymicroscopy,
T.Nguyen,Y.Xue,Y.Li,L.Tian,andG.Nehmetallah,“DeeplearningapproachforFourierptychographymicroscopy,” Opt. express26, 26470–26484 (2018)
2018
-
[19]
Learning fast approximations of sparse coding,
K. Gregor and Y. LeCun, “Learning fast approximations of sparse coding,” inProceedings of the 27th International Conference on International Conference on Machine Learning, (Omnipress, 2010), pp. 399–406
2010
-
[20]
Shrinkage fields for effective image restoration,
U. Schmidt and S. Roth, “Shrinkage fields for effective image restoration,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2014), pp. 2774–2781
2014
-
[21]
Dirty pixels: Optimizing image classification architectures for raw sensor data,
S. Diamond, V. Sitzmann, S. Boyd, G. Wetzstein, and F. Heide, “Dirty pixels: Optimizing image classification architectures for raw sensor data,” arXiv preprint arXiv:1701.06487 (2017)
2017 arXiv
-
[22]
Unrolled optimization with deep priors,
S. Diamond, V. Sitzmann, F. Heide, and G. Wetzstein, “Unrolled optimization with deep priors,” arXiv preprint arXiv:1705.08041 (2017)
2017 arXiv
-
[23]
Deep ADMM-Net for compressive sensing MRI,
J. Sun, H. Li, and Z. Xu, “Deep ADMM-Net for compressive sensing MRI,” inAdvances in neural information processing systems,(2016), pp. 10–18
2016
-
[24]
U-net: Convolutional networks for biomedical image segmentation,
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, (Springer, 2015), pp. 234–241
2015
-
[25]
The unreasonable effectiveness of deep features as a perceptual metric,
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2018), pp. 586–595
2018
-
[26]
The OpenCV Library,
G. Bradski, “The OpenCV Library,” Dr. Dobb’s J. Softw. Tools (2000)
2000
-
[27]
The MIR Flickr Retrieval Evaluation,
M. J. Huiskes and M. S. Lew, “The MIR Flickr Retrieval Evaluation,” inMIR ’08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, (ACM, New York, NY, USA, 2008)
2008
-
[28]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014)
2014 arXiv
-
[29]
Lensless learning repository,
K. Monakhova, J. Yurtsever, G. Kuo, N. Antipa, K. Yanny, and L. Waller, “Lensless learning repository,”https: //github.com/Waller-Lab/LenslessLearning/ (2019). Accessed: 2019-08-22
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.