Integrated Forward-Inverse Network for Lensless Image Reconstruction

Donggeon Bae; Jaewoo Jung; Jongho Kim; Joonsik Park; Kyung Chul Lee; Sangjun Byun; Seung Ah Lee; Taeyoung Kim; Yong Guk Kang

REVIEW 3 major objections 7 minor 17 references

Integrated Forward-Inverse Network for Lensless Image Reconstruction

T0 review · 3 major / 7 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Interleaving forward projections with inverse updates at every scale recovers sharp images from highly multiplexed lensless measurements and adapts the point-spread field under mismatch.

desk verdict Solid hybrid architecture for lensless reconstruction: multi-scale bidirectional FSO/ISO plus a learned PSF field, with real SOTA gains and a new dataset. read the letter →

arxiv 2607.04608 v1 pith:UODNYKRG submitted 2026-07-06 cs.CV

Donggeon Bae , Jaewoo Jung , Yong Guk Kang , Kyung Chul Lee , Taeyoung Kim , Jongho Kim , Sangjun Byun , Joonsik Park

show 1 more author

Seung Ah Lee

This is my paper · ORCID

classification cs.CV

keywords lenslessimagingcomputationalinverseproblemsmodel-baseddeeplearningpoint-spreadfunctionshift-variantdeconvolutionencoder-decoder

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Lensless cameras replace bulky lenses with thin coded elements, but the resulting measurements mix light over large spatial footprints, so inversion is severely ill-conditioned and fragile to calibration error. Existing hybrid networks often apply physics only once, after which measurement cues are lost. This paper claims that repeatedly coupling a differentiable forward operator with a learnable inverse operator inside every level of an encoder–decoder hierarchy keeps measurement-domain and image-domain information available to each other throughout refinement. The same shared learnable point-spread field drives both operators, so the kernels themselves adapt under model uncertainty. On three lensless benchmarks, including a new wide-field dataset, the method reports clear gains in reconstruction quality; the same interleaving pattern also remains competitive when the operators are swapped for Gaussian deblurring or for angular-spectrum holography. The result matters because it turns physics consistency into a continuous architectural principle rather than a single pre- or post-processing step, and because it reduces the need for dense per-system calibration.

What carries the argument

The Integrated Forward–Inverse Block (IFIB): at each scale it runs a Forward System Operator that projects the current image features into the measurement domain and an Inverse System Operator that restores image features from the measurement, then fuses both streams with learnable gates before the next scale, all driven by a shared multi-scale point-spread field.

What would settle it

Train and evaluate the identical architecture with the forward and inverse operators replaced by identity maps (or with a deliberately mismatched fixed PSF) on DiffuserCam or WiderCam; if the reported PSNR/LPIPS gains over strong hybrid baselines disappear, the claim that bidirectional physics coupling is the source of the improvement is falsified.

Watch

Extended reading notes

Core claim

The authors establish that an encoder–decoder whose every scale interleaves a differentiable forward projection with a learnable inverse update, both conditioned on a jointly learned shift-variant point-spread field, yields state-of-the-art reconstruction quality on DiffuserCam, WiderCam and MultiWienerNet and remains competitive when the same interleaving is applied to Gaussian deblurring and simulated inline holography.

Load-bearing premise

A sufficiently faithful, differentiable linear forward model (and a compatible inverse or adjoint) must still be available at test time; severe saturation or large geometry changes erase recoverable information that no reconstructor can restore.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a clean, usable advance in hybrid physics–ML reconstruction for lensless cameras. The new piece is not “add a Wiener layer,” but the systematic dual-stream interleaving of a differentiable forward operator and a learnable inverse at every encoder–decoder scale, plus a jointly learned multi-kernel PSF field with ROI blending. That design is carried through carefully and produces clear gains.

What they do well: three real benchmarks (DiffuserCam, their new WiderCam with strong field dependence, and MultiWienerNet), strong classical/CNN/hybrid baselines, and ablations that actually isolate the claim—identity operators, FSO-only, ISO-only, regularizer, padding, k-scaling, learned vs frozen PSFs. The holography transfer (swap angular-spectrum operators, keep the interleaving backbone) is a useful check that the idea is not locked to convolution. Code, supplement, and WiderCam are promised; the modeling assumptions (linear differentiable forward model, paired target-domain data) are standard for the field and are stated honestly in the limitations.

Soft spots are real but secondary. FFT-based FSO/ISO cost scales with k and padding; k=1 already beats the baselines, so denser fields are an accuracy–compute trade-off rather than a free lunch. No error bars or multi-seed stats. Gains on Gaussian deblurring are modest until the blur is large. None of that undercuts the main numbers (+1.63 / +0.65 / +2.58 dB PSNR on the three lensless sets).

This is for people building or deploying thin computational cameras and for anyone designing multi-scale physics–NN invertors. The math is standard Fourier deconvolution with learnable regularizers; the data and citation pattern look solid. I would send it to referees and would cite the architecture and WiderCam if I work in this space.

Referee Report

3 major / 7 minor

Summary. The paper proposes IFIN, an encoder–decoder architecture for lensless image reconstruction that interleaves differentiable forward system operators (FSO) with learnable inverse system operators (ISO) inside Integrated Forward–Inverse Blocks at every scale, while jointly learning a multi-kernel shift-variant PSF field with ROI blending. Motivated by locality mismatch, one-sided inversion, and representation bottlenecks under large-footprint PSFs, the design keeps coupled measurement- and image-domain streams so that physics consistency cues are reintroduced throughout decoding. Empirically, IFIN reports state-of-the-art PSNR/LPIPS/SSIM on DiffuserCam, a newly introduced wide-FoV WiderCam dataset, and MultiWienerNet, with component ablations isolating bidirectional coupling and learned vs. calibrated PSFs, plus transfer experiments on Gaussian deblurring and simulated inline holography (with modality-matched operators).

Significance. If the reported gains hold under fair comparison, the work is a solid contribution to hybrid physics–NN reconstruction for computational imaging. The bidirectional multi-scale interleaving is a clear architectural idea relative to one-sided unrolled or single-inversion hybrids, the learnable PSF field reduces dense calibration burden, and the new WiderCam benchmark (strong shift variance, >100° FoV) is a useful community resource. Code, supplementary material, and the WiderCam dataset are promised, which strengthens reproducibility. The holography transfer, while simulated, supports the claim that the interleaving template is not tied to a single convolutional PSF model.

major comments (3)

Table 1 SOTA margins, especially WiderCam (+0.65 dB PSNR over MoDL) and several LPIPS/SSIM gaps, are reported from single runs with no error bars, multi-seed statistics, or significance tests. Given free parameters (k, loss weights, gates, ϵ, ROI maps) and the claim of state-of-the-art quality, please report mean±std over at least 3 seeds for IFIN and the strongest baselines on all three lensless benchmarks, or otherwise justify that the ranking is stable.
§5 and S1–S2: fairness of the hybrid baseline comparison is not fully transparent. The text states a “unified reconstruction objective” and fixed (λ_img, λ_perc) for retrained learning-based methods, but does not tabulate whether each baseline (UPDN, MWDNs, LensNet, MoDL, MWNet, etc.) was retrained end-to-end on the same splits/resolutions with matched epochs, PSF normalization, and data augmentation, or evaluated from published checkpoints. Please add a short protocol table so the +1.63/+0.65/+2.58 dB claims can be audited.
§4.2 and S9: the multi-kernel design is asymmetric—FSO uses a single averaged PSF while ISO applies region-wise Wiener-like inversion with ROI blending. The efficiency argument is reasonable, but the paper does not quantify how much accuracy is left on the table versus a shift-variant FSO (even on a subset). A controlled experiment or bound on the measurement-consistency residual under the averaged FSO would strengthen the claim that bidirectional coupling, not only multi-kernel ISO, drives the gains under strong shift variance (WiderCam/MWNet).

minor comments (7)

Table 5 / S11: inference latency for IFIN (k=1) is ~142 ms vs. 12–60 ms for strong baselines; the conclusion correctly flags FFT overhead, but a brief note on whether mixed-precision, reduced padding, or fewer IFIB stages can close the gap would help practitioners choose the k=1 operating point.
Fig. 1 notation: n indexes both the per-scale PSF embedding h_n and stage-(n) representations; a one-line clarification that h_n is shared across encoder/decoder IFIBs at the same scale would reduce ambiguity.
Eq. (4): ϵ(u,v) is described as a learnable non-negative 2D regularizer; state its spatial resolution relative to the FFT grid and whether it is shared across scales or per IFIB.
WiderCam (S4): affine alignment is estimated offline from a deconvolution baseline and LoFTR correspondences. Please state whether any test images were excluded for failed registration and whether metrics are computed after inverse-warping only (as claimed) without additional cropping that could favor methods with less geometric distortion.
Table 2: for σ=5, IFIN trails NAFNet/RCAN slightly; the text already notes graceful degradation for larger blur, but a short clause that the physics prior is not free in the mild-blur regime would set expectations.
Inline holography (Tab. 3, S5) is fully simulated; the abstract’s “suggesting … can extend beyond lensless cameras” is fair, but please mark the holography result as simulated in the abstract or introduction to avoid over-reading experimental transfer.
Typos/style: abstract and title use both “Forward-Inverse” and “Forward–Inverse”; unify en-dash usage. Occasional missing spaces after periods in the compiled text (e.g., “reconstructionfromtheresulting”) appear to be PDF extraction artifacts—verify the camera-ready source.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: IFIN is an empirical hybrid architecture whose SOTA claims rest on held-out benchmarks and ablations, not on definitions or self-citation chains.

full rationale

The paper’s load-bearing claims are architectural (bidirectional FSO/ISO interleaving at every encoder–decoder scale plus a jointly learned shift-variant PSF field) and empirical (PSNR/LPIPS/SSIM gains on DiffuserCam, WiderCam, and MultiWienerNet; competitive transfer to Gaussian deblurring and inline holography). These are not algebraic identities of the training loss or of fitted constants: models are trained on train splits and scored on held-out test splits against classical, data-driven, and hybrid baselines, with component ablations (identity FSO/ISO, FSO-only, ISO-only, learned vs. frozen calibrated PSFs) isolating the claimed mechanism. Self-citations (e.g., prior lensless hardware/fabrication work by overlapping authors) are ordinary background references and do not define the reconstruction metrics or force the reported gains. There is no uniqueness theorem imported from the authors, no fitted parameter renamed as a prediction, and no ansatz smuggled in as a first-principles result. The acknowledged operating assumptions (faithful differentiable linear forward model; target-domain paired supervision) are standard for hybrid physics–NN reconstruction and do not create circularity in the reported results. Score 0 is therefore the correct outcome.

Assumptions & free parameters 4 free parameters · 3 assumptions · 2 invented entities

The central empirical claim rests on standard supervised deep-learning practice plus the modeling assumption that a linear, differentiable forward operator (and a compatible inverse) adequately describes the imaging system. Free parameters are ordinary training and architecture choices; no new physical entities are postulated.

free parameters (4)

loss weights (λ_img, λ_perc, λ_psf) = (1.0, 0.05, 0.1)
Fixed to (1.0, 0.05, 0.1) by validation; they directly trade off pixel, perceptual and PSF non-negativity terms that shape the final reconstruction.
number of PSF kernels k = s² = 1–16 (dataset-dependent)
Chosen per dataset (k∈{1,4,9,16}); larger k improves shift-variant modeling at linear cost in ISO compute and is a free design knob that affects reported metrics.
learnable frequency regularizer ϵ(u,v) and ROI maps
Optimized end-to-end; they control noise amplification and spatial blending of region-wise deconvolutions and are not fixed by physics alone.
gate initializations α=0.8, β=0.2 and learning rates
Hand-chosen starting points and separate AdamW rates for network vs PSF parameters that influence convergence and final quality.

assumptions (3)

domain assumption The imaging process is adequately modeled by a linear operator (shift-invariant or shift-variant convolution, or angular-spectrum propagation) plus additive noise.
Stated in Sec. 2–3 and used to define FSO/ISO; severe nonlinearities or saturation are acknowledged as out of scope.
domain assumption Paired scene–measurement training data (or synthetic data generated from a known forward model) are available for supervised end-to-end training.
All reported results use supervised losses on paired datasets; the paper does not claim unsupervised or zero-shot transfer.
ad hoc to paper Wiener-style frequency-domain inversion with a learnable non-negative regularizer is a sufficiently expressive inverse operator for the ISO.
Chosen as the concrete ISO implementation (Eq. 4); alternatives are not exhaustively compared.

invented entities (2)

Integrated Forward–Inverse Block (IFIB) with dual FSO/ISO streams
purpose: Architectural unit that exchanges measurement- and image-domain features at every scale.
Defined by the authors; no independent physical existence outside the network.
Learnable multi-scale PSF field with ROI blending
purpose: Captures shift-variant blur without dense calibration and is shared by FSO and ISO.
A trainable parameterization introduced for robustness; validated only inside the same supervised pipeline.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Integrated Forward-Inverse Network for Lensless Image Reconstruction." pith.science (2026). https://pith.science/paper/UODNYKRG

@misc{pith2026260704608,
  author       = {Pith},
  title        = {Pith review of: Integrated Forward-Inverse Network for Lensless Image Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UODNYKRG}},
  note         = {Machine review of arXiv:2607.04608}
}

read the original abstract

Lensless imaging enables compact and versatile computational cameras by replacing bulky optics with thin coded elements. However, reconstruction from the resulting measurements is challenging: large-footprint point-spread functions (PSFs) produce highly multiplexed observations, making inversion severely ill-conditioned and sensitive to calibration errors and model mismatch. While deep learning approaches, including hybrid models that incorporate physics priors, have shown promise, explicitly maintaining data fidelity throughout the network hierarchy remains difficult. Here, we propose the Integrated Forward-Inverse Network (IFIN), a physics-guided architecture that interleaves differentiable forward projections with learnable inverse updates at every scale, enabling complementary cues to be exploited jointly in the measurement and image domains. This bidirectional coupling supports progressive, physics-consistent refinement and permits system-constrained PSF kernel adaptation under model uncertainty. On challenging lensless benchmarks, including a newly introduced dataset, IFIN achieves state-of-the-art reconstruction quality. We further observe competitive performance on Gaussian deblurring and simulated inline holography reconstruction, suggesting that the same interleaving principle can extend beyond lensless cameras.

Figures

Figures reproduced from arXiv: 2607.04608 by the authors.

**Figure 1.** Overall architecture of IFIN. The network follows an encoder–decoder structure, where Integrated Forward–Inverse Blocks (IFIBs) are inserted at each scale to jointly apply the Forward System Operator (FSO) and Inverse System Operator (ISO). A shared learnable PSF field guides both operators, ensuring forward–inverse consistency across scales. Notation: n indexes the per-scale PSF embedding hn (one per scale, share… view at source ↗

**Figure 2.** (a) Schematic of the forward–inverse pairing in the IFIB. (b,c) Single-PSF setting: FSO uses 2D convolution; ISO uses Wiener-like deconvolution. (d,e) PSFfield setting: FSO uses a single representative (averaged) PSF from the PSF field; ISO applies region-wise deconvolution blended by learnable region-of-interest (ROI) maps. features to improve their consistency with the forward model and to better support the sub… view at source ↗

**Figure 3.** Visual comparison on DiffuserCam display–capture data. IFIN preserves color fidelity and high-frequency textures while suppressing artifacts. Insets mark zoomed regions and structures. WiderCam. WiderCam (ours) consists of 25,000 wide field-of-view (FoV, > 100◦ ) lensless measurements (24,000 train / 1,000 test), resized to 480 × 270 from 4608 × 2592 sensor frames, with affine-aligned supervision. The dataset exhibi… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Comparison on WiderCam dataset. Compared to prior methods, IFIN mitigates field-dependent peripheral blur and geometric distortion, while preserving fine textures and edges across the entire image [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

**Figure 5.** Figure 5: In-the-wild WiderCam measurements. IFIN generalizes to diverse scenes and lighting, reducing ringing and color shifts while preserving edges and textures. Without ground truth, we additionally report no-reference image quality metrics: MANIQA↑ [47] and NIQE↓ [31]. Gaus…

**Figure 6.** Figure 6: Comparison on MWNet dataset. The first row shows simulated spatially varying measurements and reconstructions; the second row shows experimental miniscope captures of a USAF target [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 1 linked inside Pith

[1]

Optica5(1), 1–9 (2017)

Antipa, N., Kuo, G., Heckel, R., Mildenhall, B., Bostan, E., Ng, R., Waller, L.: Diffusercam: lensless single-exposure 3d imaging. Optica5(1), 1–9 (2017)

2017
[2]

arXiv preprint arXiv:2505.01755 (2025)

Bai, J., Yin, Y., Dong, Y., Zhang, X., Pun, C.M., Chen, X.: Lensnet: An end-to- end learning framework for empirical point spread function modeling and lensless imaging reconstruction. arXiv preprint arXiv:2505.01755 (2025)

arXiv 2025
[3]

IEEE Transactions on Computa- tional Imaging11, 213–227 (2025)

Bezzam, E., Perron, Y., Vetterli, M.: Towards robust and generalizable lensless imaging with modular learned reconstruction. IEEE Transactions on Computa- tional Imaging11, 213–227 (2025)

2025
[4]

Foundations and Trends®in Machine learning3(1), 1–122 (2011)

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed opti- mizationandstatisticallearningviathealternatingdirectionmethodofmultipliers. Foundations and Trends®in Machine learning3(1), 1–122 (2011)

2011
[5]

In: European conference on computer vision

Chen, L., Chu, X., Zhang, X., Sun, J.: Simple baselines for image restoration. In: European conference on computer vision. pp. 17–33. Springer (2022)

2022
[6]

In: Proceedings of the 1st ACM international conference on Multimedia information retrieval

Huiskes, M.J., Lew, M.S.: The mir flickr retrieval evaluation. In: Proceedings of the 1st ACM international conference on Multimedia information retrieval. pp. 39–43 (2008)

2008
[7]

Optics Express30(26), 46324–46335 (2022)

Kingshott, O., Antipa, N., Bostan, E., Akşit, K.: Unrolled primal-dual networks for lensless cameras. Optics Express30(26), 46324–46335 (2022)

2022
[8]

Optica10(1), 72–80 (2023)

Lee, K.C., Bae, J., Baek, N., Jung, J., Park, W., Lee, S.A.: Design and single-shot fabrication of lensless cameras with arbitrary point spread functions. Optica10(1), 72–80 (2023)

2023

Show all 17 references

[9]

Optics Express31(23), 39088–39101 (2023) 22 D

Li, Y., Li, Z., Chen, K., Guo, Y., Rao, C.: Mwdns: reconstruction in multi-scale feature spaces for lensless imaging. Optics Express31(23), 39088–39101 (2023) 22 D. Bae et al

2023
[10]

Optics express27(20), 28075–28090 (2019)

Monakhova, K., Yurtsever, J., Kuo, G., Antipa, N., Yanny, K., Waller, L.: Learned reconstructions for practical mask-based lensless imaging. Optics express27(20), 28075–28090 (2019)

2019
[11]

In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

Poudel,A.,Nakarmi,U.:Deeplir:Attention-basedapproachformask-basedlensless image reconstruction. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 431–439 (2024)

2024
[12]

In: International Conference on Medical image computing and computer-assisted intervention

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)

2015
[13]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: Loftr: Detector-free local fea- ture matching with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8922–8931 (2021)

2021
[14]

The MIT press (1964)

Wiener, N.: Extrapolation, interpolation, and smoothing of stationary time series. The MIT press (1964)

1964
[15]

Light: Science & Applications9(1), 171 (2020)

Yanny, K., Antipa, N., Liberti, W., Dehaeck, S., Monakhova, K., Liu, F.L., Shen, K., Ng, R., Waller, L.: Miniscope3d: optimized single-shot miniature 3d fluores- cence microscopy. Light: Science & Applications9(1), 171 (2020)

2020
[16]

Optica9(1), 96–99 (2022)

Yanny, K., Monakhova, K., Shuai, R.W., Waller, L.: Deep learning for fast spatially varying deconvolution. Optica9(1), 96–99 (2022)

2022
[17]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018) Integrated Forward–Inverse Network 23 Fig.S...

2018

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.

[1] [1]

Optica5(1), 1–9 (2017)

Antipa, N., Kuo, G., Heckel, R., Mildenhall, B., Bostan, E., Ng, R., Waller, L.: Diffusercam: lensless single-exposure 3d imaging. Optica5(1), 1–9 (2017)

2017

[2] [2]

arXiv preprint arXiv:2505.01755 (2025)

Bai, J., Yin, Y., Dong, Y., Zhang, X., Pun, C.M., Chen, X.: Lensnet: An end-to- end learning framework for empirical point spread function modeling and lensless imaging reconstruction. arXiv preprint arXiv:2505.01755 (2025)

arXiv 2025

[3] [3]

IEEE Transactions on Computa- tional Imaging11, 213–227 (2025)

Bezzam, E., Perron, Y., Vetterli, M.: Towards robust and generalizable lensless imaging with modular learned reconstruction. IEEE Transactions on Computa- tional Imaging11, 213–227 (2025)

2025

[4] [4]

Foundations and Trends®in Machine learning3(1), 1–122 (2011)

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed opti- mizationandstatisticallearningviathealternatingdirectionmethodofmultipliers. Foundations and Trends®in Machine learning3(1), 1–122 (2011)

2011

[5] [5]

In: European conference on computer vision

Chen, L., Chu, X., Zhang, X., Sun, J.: Simple baselines for image restoration. In: European conference on computer vision. pp. 17–33. Springer (2022)

2022

[6] [6]

In: Proceedings of the 1st ACM international conference on Multimedia information retrieval

Huiskes, M.J., Lew, M.S.: The mir flickr retrieval evaluation. In: Proceedings of the 1st ACM international conference on Multimedia information retrieval. pp. 39–43 (2008)

2008

[7] [7]

Optics Express30(26), 46324–46335 (2022)

Kingshott, O., Antipa, N., Bostan, E., Akşit, K.: Unrolled primal-dual networks for lensless cameras. Optics Express30(26), 46324–46335 (2022)

2022

[8] [8]

Optica10(1), 72–80 (2023)

Lee, K.C., Bae, J., Baek, N., Jung, J., Park, W., Lee, S.A.: Design and single-shot fabrication of lensless cameras with arbitrary point spread functions. Optica10(1), 72–80 (2023)

2023