REVIEW 4 major objections 4 minor 27 references
Contrast-invariant deep ptychography neural networks
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A ptychography network with real/imaginary decoder outputs reconstructs datasets across order-of-magnitude photon-flux differences by fitting two scale parameters at inference, without retraining.
desk verdict Solid scaling factorization, overstated beamline generalization claim; worth a serious referee with revisions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the real/imaginary decoder representation together with the quadratic intensity factorization in Eq. (5). Because rectangular coordinates are linear through the Fourier transform, the exit wave separates into two terms, each linear in one scaling parameter, so the measured intensity becomes a quadratic form in $(s_1, s_2)$ whose coefficients are determined entirely by the network's normalized predictions and the known probe. This makes the scaling parameters directly optimizable at inference time and removes the nonlinear coupling that arises with amplitude/phase outputs. The probe-weighted patch stitching in Eq. (8), which uses the probe intensity as a confidence weight, and the synthetic object sampling from empirical real-imaginary distributions are supporting components that address signal-to-noise in patch assembly and phase-distribution mismatch in training data.
What would settle it
Take a trained PtychoPINN-CI network and apply it to a dataset acquired with a deliberately different probe, such as a different defocus setting or a changed focusing optic, while keeping the network's stored probe fixed; if the inference-time scaling fit cannot bring the predicted diffraction intensities into agreement with the measured ones, the probe-constancy assumption is violated and the central claim fails for that case.
Extended reading notes
Core claim
The central claim is that representing the decoder output in rectangular coordinates, as normalized real and imaginary components $\tilde{a}_o(\theta)$ and $\tilde{b}_o(\theta)$, makes the far-field intensity a quadratic function of two real scaling parameters $s_1$ and $s_2$: $I = s_1^2|\Psi_a|^2 + 2s_1 s_2 \mathrm{Re}[\bar{\Psi}_a \Psi_b] + s_2^2|\Psi_b|^2$, where $\Psi_a$ and $\Psi_b$ are Fourier transforms of probe-weighted basis fields that depend only on the network's normalized predictions and the known probe. Because this intensity is quadratic in the scale parameters, the optimal $(s_1, s_2)$ for any new dataset can be found by solving a linear least-squares problem after freezing the network weights. This decouples the learned object texture from the dataset-dependent measurement scale, so a single trained network can produce photon-consistent reconstructions across datasets with very different photon fluxes. The paper further claims that probe-intensity-weighted stitching and synthetic training data sampled from empirical real-imaginary correlations are needed to recover accurate amplitude and phase contrast, and that together these components reduce Fourier error by up to a factor of five compared with the previous baseline across five experimental datasets from multiple beamlines.
Load-bearing premise
The probe function is assumed known and identical between training and inference; if the probe changes (for example, from optics drift or a different focusing condition), the scaling factorization and the reported cross-dataset generalization break down.
Editorial extensions
If this is right
- A network trained once on data from a given instrument can be reused for subsequent measurements with fluctuating photon fluxes, since the two scale parameters absorb the per-dataset intensity scale at inference with negligible computational cost.
- The correctly scaled output makes neural-network reconstructions usable as initialization for iterative phase retrieval, potentially skipping the coarse refinement stage.
- The scaling factorization transfers to more expressive network backbones, such as vision transformers, because it is a property of the output representation and forward model, not of the specific architecture.
- The probe-weighted stitching principle, borrowed from iterative ptychography, can be applied generally to any patch-based reconstruction network that assembles overlapping object predictions.
- Synthetic-only training can work for a new instrument as long as a probe function is available, removing the need for large experimental training sets at each new beamline.
Reading between the lines
- The same quadratic-scaling trick should apply to any coherent imaging problem where the forward model is a product of the object with a known illumination and a linear transform, such as Fourier ptychography or in-line holography, provided the object is represented in rectangular coordinates.
- One could extend the per-dataset scaling fit to a per-region or per-patch fit to handle spatially varying illumination intensity within a single dataset, though this would trade global consistency for local flexibility.
- The observed phase compression, which the paper attributes to limited network capacity or remaining synthetic-data mismatch, suggests that replacing the empirical sampling distribution with a learned generative model of complex-valued object statistics could be a direct next step.
- Because the factorization requires a known, fixed probe, a natural stress test is to deliberately defocus or drift the optics between training and inference; if the scaling fit still compensates, the framework may tolerate more probe variation than the stated assumption allows.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes PtychoPINN-CI, a feed-forward ptychography reconstruction network with a real/imaginary decoder output, test-time fitting of two global scaling parameters (s1, s2), probe-intensity-weighted stitching, and synthetic training data sampled from an empirical real/imaginary distribution. The central claim is that, with a known and fixed probe, the far-field intensity depends quadratically on (s1, s2) when the object is represented in rectangular components, so fitting two scalars at inference makes a single trained network measurement-consistent under changes in illumination intensity. The authors report qualitative improvements and up to a 5x Fourier-error reduction over PtychoPINN-torch on five experimental datasets from multiple facilities.
Significance. The scaling factorization in Eqs. (1)-(5) is a clean, parameter-free consequence of the forward model and is correct as stated; if the fixed-probe assumption holds, it gives a principled and architecture-agnostic way to remove the global scale ambiguity at inference. The probe-weighted stitching and the empirical-correlation synthetic sampling are sensible engineering contributions that appear to improve reconstruction quality. The main value is in enabling zero-retraining deployment under changing photon flux. However, the empirical support is currently incomplete: quantitative metrics are not tabulated, and the cross-beamline generalization claim is not demonstrated under a single trained network and probe model.
major comments (4)
- [Section 3.1, Fig. 3(c)] The headline claim of 'up to a 5x reduction in Fourier error' is not supported by a quantitative table or error bars. Figure 3(c) shows only a schematic comparison, and no per-dataset R-factor values (Eq. S13) or run-to-run variability are reported. Please provide a table with per-dataset R-factors for all configurations, including the baseline, and repeat training runs with multiple seeds.
- [Abstract, Section 5, and Section 4] The abstract and conclusion state that 'a single trained network' produces consistent reconstructions 'across 5 experimental datasets spanning multiple beamlines and facilities,' but the reported protocol trains separate models with per-instrument measured probes, and Section 4 explicitly limits robustness to cases where 'the probe geometry does not substantially change.' This is a load-bearing overstatement: with a different probe, the quadratic coefficients in Eq. (5) change, and the two scalars (s1, s2) cannot compensate. Either report an experiment with one frozen network and one probe model applied to datasets from different beamlines, or revise the claim to fixed-probe, variable-flux/contrast generalization.
- [Section 3.3, Fig. 4] The correlated synthetic distributions are derived from the TP2 reference reconstruction and then evaluated on TP2 (Fig. 4), so the evaluation target's complex-value statistics are present in the training-data generation. This does not invalidate the other Velociprobe datasets, but the TP2 result should be reported as a leakage-affected or semi-supervised case, and the generalization claim should rely on the held-out datasets.
- [Section 2.4, Eq. (9)] The training loss as printed is not the Poisson negative log-likelihood: LPoiss = sum [I_pred - I_meas log(I_pred)]^2 is a squared residual of a likelihood expression, not the likelihood itself. Please correct the equation or justify the squared form; as written it is ambiguous and affects reproducibility.
minor comments (4)
- [Section 2.4, Eq. (10)] The typesetting of the RMS scaling formula contains malformed radicals; please re-typeset the equation.
- [References] Reference [22] is an incomplete citation containing 'TODO: Complete citation once published'; this must be completed before publication.
- [Section 2.4 and Eq. (9)] The use of the index n in Eq. (9) is inconsistent with the surrounding notation ('n' appears both as a summation index and inside the parentheses); please clean up the notation.
- [Section 4] The claim of '30% fewer parameters' than PtychoPINN-torch is not accompanied by a table of architecture sizes; please report the parameter counts explicitly.
Circularity Check
Scaling factorization is self-contained, but the TP2 synthetic-data evaluation is partly circular because the training distribution is built from the TP2 reference itself.
-
fitted input called prediction
[Section 2.5 and Section 3.3 / Figure 4]
"we construct an empirical distribution by normalizing a 2D histogram of complex values from a conventionally reconstructed object. We then sample synthetic object textures using this empirical distribution... All correlated sampling distributions are derived from the TP2 dataset... Figure 4: ... (a) ... reference reconstruction of the TP2 Dataset ... (b) Predicted objects from models trained on the corresponding per-row synthetic datasets."
The DLC and WNC synthetic training sets draw real and imaginary values from the empirical joint distribution of the TP2 reference reconstruction, and then the same TP2 reference is used as the evaluation target. Thus the claimed minimization of phase distribution mismatch on TP2 holds by construction: the training marginal distribution is exactly the test object's marginal distribution. The TP2 result is therefore a fitted-input evaluation, not an independent prediction. The paper partially acknowledges this concern and provides out-of-distribution results on other Velociprobe datasets, which limits the severity, but the specific TP2 phase-distribution improvement remains self-fulfilling.
full rationale
The core scaling contribution is not circular. Equation (5) is a parameter-free consequence of linearity through the Fourier transform: with real/imaginary decoder outputs and a known probe, the predicted intensity is exactly a quadratic function of (s1, s2), with coefficients built from the network's normalized predictions and the probe. The two scaling parameters are explicitly fitted at test time against the measured intensities (Eq. 7), and the paper presents this as calibration rather than as an unseen prediction; this is a legitimate test-time fit, not a hidden input. The fixed-probe assumption stated in Section 2.1 is a real scope limitation for the cross-beamline generalization claim, but it is an assumption, not circularity. Self-citations to the authors' prior PtychoPINN-torch work [22] are used for baselines, dataset provenance, and the previously observed phase-distribution mismatch; they are not load-bearing for the new factorization derivation, and the new empirical-sampling experiments provide independent evidence on non-TP2 datasets. The only concrete circular loop is the TP2 synthetic-data evaluation, where the training distribution is derived from the TP2 reference and the TP2 reference is then used as the test target; for that dataset the phase-distribution match is forced by construction. Because the central scaling claim and the generalization to other Velociprobe datasets remain independently supported, the overall circularity is partial rather than pervasive.
Assumptions & free parameters
free parameters (2)
- s1, s2 scaling parameters =
Per-dataset, fit via least squares at inference
- Empirical real/imaginary distribution from TP2 reference =
2D histogram of complex values from the TP2 reconstruction
assumptions (4)
- domain assumption Known and fixed probe function P between training and inference
- domain assumption The forward model of ptychography (Fourier transform of probe times object, intensity detection) is exact
- domain assumption Poisson negative log-likelihood is the correct loss
- ad hoc to paper The empirical real/imaginary distribution from TP2 is representative of other experimental objects
Cite this review
Pith. "Pith review of Contrast-invariant deep ptychography neural networks." pith.science (2026). https://pith.science/paper/DYBU7KVJ
@misc{pith2026260802869,
author = {Pith},
title = {Pith review of: Contrast-invariant deep ptychography neural networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/DYBU7KVJ}},
note = {Machine review of arXiv:2608.02869}
}
read the original abstract
Ptychography neural networks suffer from scaling inconsistencies when generalizing out of distribution, limiting their real world viability. We address this scaling mismatch using a factorization strategy which decouples the learned object texture from measurement scaling, enabling a single trained network to produce measurement-consistent reconstructions across varying illumination conditions. This requires predicting the learned object in real and imaginary units instead of the canonical amplitude and phase representation. We additionally introduce a synthetic object sampling strategy that minimizes phase distribution mismatch between synthetic training data and experimental targets. These improvements yield up to a 5x reduction in Fourier error over the previous PtychoPINN-torch baseline across 5 experimental datasets spanning multiple beamlines and facilities.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Helen Mary Louise Faulkner and John M Rodenburg. Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm.Physical Review Letters, 93(2):023903, 2004
work page 2004
-
[2]
High-resolution scanning x-ray diffraction microscopy.Science, 321(5887):379–382, 2008
Pierre Thibault, Martin Dierolf, Andreas Menzel, Oliver Bunk, Christian David, and Franz Pfeiffer. High-resolution scanning x-ray diffraction microscopy.Science, 321(5887):379–382, 2008
2008
-
[3]
Huaidong Jiang, Changyong Song, Chien-Chun Chen, Rui Xu, Kevin S Raines, Benjamin P Fahimian, Chien-Hung Lu, Ting-Kuo Lee, Akio Nakashima, Jun Urano, et al. Quantitative 3d imaging of whole, unstained cells by using x-ray diffraction microscopy.Proceedings of the National Academy of Sciences, 107(25):11234–11239, 2010
work page 2010
-
[4]
Mirko Holler, Manuel Guizar-Sicairos, Esther HR Tsai, Roberto Dinapoli, Elisabeth Müller, Oliver Bunk, Jörg Raabe, and Gabriel Aeppli. High-resolution non-destructive three- dimensional imaging of integrated circuits.Nature, 543(7645):402–406, 2017
work page 2017
-
[5]
Aaron Michelson, Brian Minevich, Hamed Emamy, Xiaojing Huang, Yong S Chu, Hanfei Yan, and Oleg Gang. Three-dimensional visualization of nanoparticle lattices and multimaterial frameworks.Science, 376(6589):203–207, 2022
work page 2022
-
[6]
Computational microscopy with coherent diffractive imaging and ptychography
Jianwei Miao. Computational microscopy with coherent diffractive imaging and ptychography. Nature, 637(8045):281–295, 2025
2025
-
[7]
An improved ptychographical phase retrieval algorithm for diffractive imaging.Ultramicroscopy, 109(10):1256–1262, 2009
Andrew M Maiden and John M Rodenburg. An improved ptychographical phase retrieval algorithm for diffractive imaging.Ultramicroscopy, 109(10):1256–1262, 2009
2009
-
[8]
Anakha V Babu, Tao Zhou, Saugat Kandel, Tekin Bicer, Zhengchun Liu, William Judge, 27 Daniel J Ching, Yi Jiang, Sinisa Veseli, Steven Henke, et al. Deep learning at the edge enables real-time streaming ptychographic imaging.Nature Communications, 14(1):7059, 2023
work page 2023
Show all 27 references
-
[9]
APS scientific computing strategy
Nicholas Schwarz. APS scientific computing strategy. Technical report, Argonne National Laboratory, 2024. URLhttps://www.aps.anl.gov/files/APS-Uploads/XSD/ XSD-Strategic-Plans/APSScientificComputingStrategy-2024-09-11-FINAL.pdf
2024
-
[10]
En- abling scientific discovery at next-generation light sources with advanced ai and hpc
NicholasSchwarz, StuartCampbell, AlexanderHexemer, ApurvaMehta, andJanaThayer. En- abling scientific discovery at next-generation light sources with advanced ai and hpc. InSmoky Mountains Computational Sciences and Engineering Conference, pages 145–156. Springer, 2020
2020
-
[11]
Deep-learning real-time phase retrieval of imperfect diffraction patterns from x-ray free-electron lasers.npj Computational Materials, 11(1):68, 2025
Sung Yun Lee, Do Hyung Cho, Chulho Jung, Daeho Sung, Daewoong Nam, Sangsoo Kim, and Changyong Song. Deep-learning real-time phase retrieval of imperfect diffraction patterns from x-ray free-electron lasers.npj Computational Materials, 11(1):68, 2025
2025
-
[12]
PID3Net: adeep learning approach for single-shot coherent x-ray diffraction imaging of dynamic phenomena
Tien-Sinh Vu, Minh-Quyet Ha, Adam Mukharil Bachtiar, Duc-Anh Dao, Truyen Tran, Hiori Kino, ShuntaroTakazawa, NozomuIshiguro, YuheiSasaki, MasakiAbe, etal. PID3Net: adeep learning approach for single-shot coherent x-ray diffraction imaging of dynamic phenomena. npj Computationa...
2025
-
[13]
Deep-learning electron diffractive imaging.Physical Review Letters, 130(1):016101, 2023
Dillan J Chang, Colum M O’Leary, Cong Su, Daniel A Jacobs, Salman Kahn, Alex Zettl, Jim Ciston, Peter Ercius, and Jianwei Miao. Deep-learning electron diffractive imaging.Physical Review Letters, 130(1):016101, 2023
2023
-
[14]
Compleximagingofphasedomains by deep neural networks.IUCrJ, 8(1):12–21, 2021
LonglongWu, PavolJuhas, ShinjaeYoo, andIanRobinson. Compleximagingofphasedomains by deep neural networks.IUCrJ, 8(1):12–21, 2021
2021
-
[15]
Ptychodv: Vision transformer-based deep unrolling network for pty- chographic image reconstruction.IEEE Open Journal of Signal Processing, 5:539–547, 2024
Weijie Gan, Qiuchen Zhai, Michael T McCann, Cristina Garcia Cardona, Ulugbek S Kamilov, and Brendt Wohlberg. Ptychodv: Vision transformer-based deep unrolling network for pty- chographic image reconstruction.IEEE Open Journal of Signal Processing, 5:539–547, 2024
2024
-
[16]
AI-enabled high-resolution scanning coherent diffraction imaging
Mathew J Cherukara, Tao Zhou, Youssef Nashed, Pablo Enfedaque, Alex Hexemer, Ross J Harder, and Martin V Holt. AI-enabled high-resolution scanning coherent diffraction imaging. Applied Physics Letters, 117(4), 2020. 28
2020
-
[17]
Pty- choFormer: A transformer-based model for ptychographic phase retrieval.arXiv preprint arXiv:2410.17377, 2024
Ryuma Nakahata, Shehtab Zaman, Mingyuan Zhang, Fake Lu, and Kenneth Chiu. Pty- choFormer: A transformer-based model for ptychographic phase retrieval.arXiv preprint arXiv:2410.17377, 2024
2024 arXiv
-
[18]
PtychoNet: Fast and high quality phase retrieval for ptychography
Ziqiao Guan, Esther H Tsai, Xiaojing Huang, Kevin G Yager, and Hong Qin. PtychoNet: Fast and high quality phase retrieval for ptychography. Technical report, Brookhaven National Lab.(BNL), Upton, NY (United States), 2019
2019
-
[19]
Ptychographic image reconstruction from limited data via score-based diffusion models with physics-guidance.arXiv preprint arXiv:2502.18767, 2025
Refik Mert Cam, Junjing Deng, Rajkumar Kettimuthu, Mathew J Cherukara, and Tekin Bicer. Ptychographic image reconstruction from limited data via score-based diffusion models with physics-guidance.arXiv preprint arXiv:2502.18767, 2025
2025 arXiv
-
[20]
Fourier ptychographic reconstruction with denoising diffusion probabilistic models.Optics & Laser Technology, 176:111016, 2024
Kai Wu, An Pan, and Wei Gao. Fourier ptychographic reconstruction with denoising diffusion probabilistic models.Optics & Laser Technology, 176:111016, 2024
2024
-
[21]
On the use of deep learning for phase recovery.Light: Science & Applications, 13(1):4, 2024
Kaiqiang Wang, Li Song, Chutian Wang, Zhenbo Ren, Guangyuan Zhao, Jiazhen Dou, Jianglei Di, George Barbastathis, Renjie Zhou, Jianlin Zhao, et al. On the use of deep learning for phase recovery.Light: Science & Applications, 13(1):4, 2024
2024
-
[22]
To- wards generalizable deep ptychography neural networks
Albert Vong, Steven Henke, Oliver Hoidn, Hanna Ruth, Junjing Deng, Alexander Hexemer, David Shapiro, Apurva Mehta, Arianna Gleason, Levi Hancock, and Nicholas Schwarz. To- wards generalizable deep ptychography neural networks. 2025. TODO: Complete citation once published
2025
-
[23]
Ptychographic phase retrieval via a deep-learning-assisted iterative algorithm.Applied Crystallography, 57(5), 2024
Koki Yamada, Natsuki Akaishi, Kohei Yatabe, and Yuki Takayama. Ptychographic phase retrieval via a deep-learning-assisted iterative algorithm.Applied Crystallography, 57(5), 2024
2024
-
[24]
Physics constrained unsupervised deep learning for rapid, high resolution scanning coherent diffraction reconstruction.Scientific Reports, 13(1):22789, 2023
Oliver Hoidn, Aashwin Ananda Mishra, and Apurva Mehta. Physics constrained unsupervised deep learning for rapid, high resolution scanning coherent diffraction reconstruction.Scientific Reports, 13(1):22789, 2023
2023
-
[25]
Pty-chi: a pytorch-based modern ptychographic data analysis package.arXiv preprint arXiv:2510.20929, 2025
Ming Du, Hanna Ruth, Steven Henke, Yi Jiang, Viktor Nikitin, Ashish Tripathi, Junjing Deng, Jeffrey Klug, Peco Myint, Tao Zhou, et al. Pty-chi: a pytorch-based modern ptychographic data analysis package.arXiv preprint arXiv:2510.20929, 2025. 29
-
[26]
Ptychography reduces spectral distortions intrinsic to conventional zone-plate-based x-ray spectromicroscopy.Microscopy and Microanalysis, 27(6):1448–1453, 2021
Matthew A Marcus, David A Shapiro, and Young-Sang Yu. Ptychography reduces spectral distortions intrinsic to conventional zone-plate-based x-ray spectromicroscopy.Microscopy and Microanalysis, 27(6):1448–1453, 2021
2021
-
[27]
Learning to see by looking at noise.Advances in Neural Information Processing Systems, 34: 2556–2569, 2021
Manel Baradad Jurjo, Jonas Wulff, Tongzhou Wang, Phillip Isola, and Antonio Torralba. Learning to see by looking at noise.Advances in Neural Information Processing Systems, 34: 2556–2569, 2021. 30
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.