Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

TorchOptics: An open-source Python library for differentiable Fourier optics simulations

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read TorchOptics claims to make Fourier optics simulation differentiable end-to-end, enabling gradient-based inverse design of optical systems and joint optimization with neural networks.

desk verdict A solid, useful PyTorch library paper for differentiable Fourier optics; the forward simulations are well validated, but the paper omits two easy checks—gradient correctness and quantitative coherence propagation—that would fully back its flagship claims. read the letter →

arxiv 2411.18591 v1 pith:EPD7RPNZ submitted 2024-11-27 physics.optics physics.class-phphysics.comp-ph

classification physics.opticsphysics.class-phphysics.comp-ph
keywords FourieropticsautomaticdifferentiationinversedesignopticalsimulationPyTorchspatialcoherencepolarizationcomputationalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TorchOptics is a Python library that treats Fourier optics simulations as differentiable tensor computations, so that light propagation through lenses, modulators, and other elements can be optimized by gradient descent. The paper's central claim is that this makes optical hardware design, including inverse design and end-to-end training of hybrid optics-plus-neural-network models, practical within a single framework. It further claims support for polarized light via Jones calculus and for arbitrary spatial coherence via the mutual coherence function, going beyond the usual scalar coherent simulations. A sympathetic reader would care because this lowers the barrier to jointly optimizing physical optics and machine learning post-processing.

What carries the argument

The load-bearing objects are the Field, Element, and System classes, all inheriting from PyTorch's Module, together with two FFT-based propagation algorithms: the Direct Integration method for the Rayleigh-Sommerfeld integral and the Angular Spectrum method, selected automatically by a critical-distance criterion. Modulation is a pointwise product with a (possibly trainable) complex profile; propagation is an FFT convolution or spectral filter; both are differentiable. For coherence, the library stores the mutual coherence function as a four-dimensional tensor and propagates it by the four-fold diffraction integral, at quadratic memory cost relative to coherent fields.

What would settle it

Compare TorchOptics gradients against finite differences for a two-lens system with trainable phase profiles, and compare its simulated partially coherent diffraction pattern against the analytic Gaussian Schell-model propagation or an independent numerical integration; disagreement beyond numerical tolerance would falsify the central claims.

Watch

Extended reading notes

Core claim

The central discovery is that the operations of computational Fourier optics—pointwise modulation, Rayleigh-Sommerfeld propagation by direct integration, angular spectrum propagation, and their compositions—can be implemented as PyTorch module operations whose parameters are ordinary trainable tensors. On top of this, the Field, Element, and System classes give the user a small object model: fields carry sampled wavefronts, elements apply differentiable transformations, and systems sequence them along the optical axis. The same automatic differentiation that trains neural networks therefore computes gradients of a simulation loss with respect to element positions, phase profiles, and other continuous properties. The paper demonstrates this by training three phase modulators to split a Gaussian beam, and it extends the framework to polarized fields (Jones vectors and matrices) and partially coherent fields (mutual coherence tensors).

Load-bearing premise

The framework's usefulness rests on the assumption that automatic differentiation through every propagation and modulation operation yields correct gradients and that the mutual coherence propagation is physically accurate; the paper provides demonstrations but not a reference validation of either.

Editorial extensions

If this is right

  • Optical system parameters such as phase profiles, element positions, and modulation responses can be optimized with standard gradient-based optimizers such as Adam.
  • Hybrid models that cascade physical optics into neural networks can be trained end to end in a single backward pass.
  • Inverse design is available for systems using polarized light and spatially partially coherent illumination, not just coherent scalar fields.
  • GPU acceleration and batching over multiple fields make large-scale optimization of optical systems practical.
  • A built-in critical-distance rule of thumb tells users which propagation algorithm to choose for better sampling accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not include a finite-difference or analytic verification of the gradients; a direct check of backpropagated versus numerical gradients for a small lens system would settle whether the advertised differentiability is exact.
  • The mutual coherence propagation is shown only qualitatively; comparing a simulated Gaussian Schell-model field against the known analytic propagation would test the coherence machinery.
  • Because coherence simulation scales quadratically in memory, its practical reach is limited; the library could be extended with approximations such as mode decomposition for large partially coherent fields.
  • The same field-element-system design could be reused for time-domain or non-paraxial propagators, though the paper does not claim those capabilities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces TorchOptics, an open-source Python library for differentiable Fourier optics simulations built on PyTorch. It describes three core classes (Field, Element, System), the underlying scalar diffraction theory (Rayleigh-Sommerfeld direct integration and angular spectrum propagation, Eqs. 2-5), and the use of PyTorch automatic differentiation for gradient-based optimization of optical elements and hybrid optical-neural models. Additional features include polarized fields through Jones calculus (Sec. 5.1), fields with arbitrary spatial coherence represented by the mutual coherence function (Sec. 5.2), polychromatic fields (Sec. 5.3), and standard spatial profiles (Sec. 5.4). The paper validates forward propagation accuracy against a SciPy quadrature reference (Fig. 3), shows computation-time scaling (Fig. 2), demonstrates a phase-modulator training example (Listing 2, Fig. 4), and gives a polarizer power-consistency example (Listing 3). It presents the coherence propagation of Eq. (13) only through qualitative intensity plots (Fig. 5).

Significance. If the results are trustworthy, TorchOptics would be a useful community resource: it is open-source under MIT, installable from PyPI, GPU-accelerated, and follows PyTorch conventions, which lowers the barrier for applying automatic differentiation to Fourier-optics inverse design and end-to-end hybrid optimization. The mathematical formulation (Eqs. 1-14) is standard and, as far as it goes, correctly presented. The paper gives reproducible code listings, a quantitative forward-propagation accuracy check against an independent quadrature solver, and a concrete demonstration of parameter optimization. These are real strengths. However, the two flagship claims for which the library would be adopted—reliable gradients for inverse design and simulation of arbitrary spatial coherence—are exactly the parts that lack quantitative validation. The central value proposition is therefore plausible but not yet established.

major comments (3)
  1. [Sec. 5.2] No numerical verification of the gradients is provided. The paper's central claim is that TorchOptics enables gradient-based optimization and end-to-end differentiable modeling, and this claim rests entirely on PyTorch autodiff through FFT-based convolution, pointwise complex multiplication, grid interpolation, and parametrized constraints. The loss curve in Fig. 4(c) is weak evidence: an incorrect gradient can still decrease the loss in a favorable case, and it says nothing about the correctness of gradients for the coherence or polarization paths. I recommend adding a finite-difference (or analytic) gradient check for representative operations, including the complex FFT scaling in Eqs. (2)-(5), modulation of Eq. (1), and a simple propagation through a lens. This is a load-bearing validation for the primary purpose of the library.
  2. [Sec. 3] The mutual coherence propagation is not quantitatively validated. Eq. (13) is a standard result, but the manuscript only demonstrates it through the qualitative diffraction patterns of Fig. 5. Since the abstract explicitly claims support for 'fields with arbitrary spatial coherence,' the implementation should be checked against a reference solution—for example, the analytic propagation of a Gaussian-Schell-model beam, or a numerical evaluation by independent quadrature—including a comparison of both intensity and coherence (e.g., visibility or the cross-spectral density at selected point pairs). The paper should also specify how the four-dimensional tensor in CoherenceField is discretized and how boundary conditions and zero-padding are handled in the double convolution of Eq. (13). Without this, the coherence feature is an unverified flagship capability.
  3. [Sec. 3 / Fig. 3] The forward-propagation validation covers only the DI method, not the AS method. The text claims both methods are implemented and that the library automatically selects between them based on Eq. (6), but Fig. 3 reports errors only for the DI method and only for z-values satisfying the sampling condition. Since the AS method is the default for short propagation distances, an analogous error analysis for AS (or at least a statement that AS has been checked against the same quadrature reference) would make the accuracy claim for the library's core operation complete.
minor comments (4)
  1. [Abstract and throughout] There are several typographical issues, including 'di fferent' in the abstract and introduction, 'theAngular' in Sec. 3, and 'evaulation' in the caption of Fig. 2. These should be corrected in a final revision.
  2. [Fig. 3 caption] The caption calls this a 'one-dimensional aperture simulation,' but panel (a) shows a two-dimensional yz-plane intensity map. Please clarify whether the aperture is one-dimensional in y with uniformity along x, or a two-dimensional slit of finite x extent, since this affects how the reader interprets the error metric.
  3. [Listing 5] In the polychromatic example, `torch.zeros_like(input_data)` may not match the dtype or shape of `output_field.intensity()`. Initializing with `torch.zeros_like(output_field.intensity())` after the first iteration, or explicitly specifying dtype and device, would be safer and clearer.
  4. [Sec. 5.4] The list of profiles is useful, but the paper does not mention whether the generated profiles are differentiable with respect to their parameters (e.g., waist radius, grating period). A brief statement about this would help users know whether these profiles can be used directly in inverse design.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TorchOptics is an implementation of standard, external Fourier-optics results, with no fitted parameters or self-referential predictions.

full rationale

TorchOptics reports a software implementation, not a derived scientific prediction. The forward model uses standard scalar diffraction theory: Eq. (1) is pointwise modulation, Eqs. (2)-(5) are the Rayleigh-Sommerfeld integral and angular-spectrum transfer function, and Eqs. (11)-(13) are the standard mutual-coherence propagation relations. These equations are taken from external textbooks and prior numerical-methods literature (Goodman; Voelz; Mandel and Wolf), not defined in terms of the library's outputs. There are no fitted parameters, no trained surrogate, and no quantity is predicted from data; the inverse-design demonstrations are optimizations with a specified loss (Eq. 7) and do not claim to validate a theory. The only self-citation is [31] (the authors' prior Optics Express paper), used to motivate spatial-coherence simulation and to state that mutual-coherence storage scales quadratically; that claim is a complexity fact about a 4D tensor and is not load-bearing for any derived result. The absence of a finite-difference gradient check or of an experimental validation of Eq. (13) is a correctness/validation gap, not circularity: the library's claims do not reduce to their inputs by construction. Accordingly, no circular step is identified.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The library introduces new software abstractions but no new physical entities, free parameters, or fitted constants. The central numerical methods are established in the optics literature. The main load-bearing assumptions are about the numerical accuracy and differentiability of the implementations, which are not exhaustively validated in the paper.

assumptions (5)
  • domain assumption Rayleigh-Sommerfeld diffraction integral (Eq. 2) with impulse response h_z (Eq. 3) is a valid scalar model for the simulated systems.
    The propagation engine is built on this integral; it is standard scalar diffraction theory and is invoked without proof in Sec. 3.
  • domain assumption The FFT-based numerical evaluation of the diffraction integral (DI method) and angular spectrum method accurately approximate the continuous solution under the sampling conditions described.
    Sec. 3 relies on the sampling criterion from Voelz [28] but provides no formal convergence guarantee.
  • domain assumption The angular spectrum transfer function H_z in Eq. (5) is a valid propagation model for the distances at which the method is used.
    Sec. 3; assumes evanescent field components are negligible or handled correctly.
  • domain assumption The mutual coherence function propagation rule in Eq. (13) is a correct extension of scalar propagation to partially coherent fields.
    Sec. 5.2; based on Mandel and Wolf [32], but its numerical implementation is not validated against a reference.
  • standard math PyTorch automatic differentiation returns mathematically correct gradients through FFT, modulation, and intensity operations.
    Sec. 4; this underpins the inverse design claim, but no gradient check is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TorchOptics: An open-source Python library for differentiable Fourier optics simulations." pith.science (2026). https://pith.science/paper/EPD7RPNZ

@misc{pith2026241118591,
  author       = {Pith},
  title        = {Pith review of: TorchOptics: An open-source Python library for differentiable Fourier optics simulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EPD7RPNZ}},
  note         = {Machine review of arXiv:2411.18591}
}
read the original abstract

TorchOptics is an open-source Python library for differentiable Fourier optics simulations, developed using PyTorch to enable GPU-accelerated tensor computations and automatic differentiation. It provides a comprehensive framework for modeling, analyzing, and designing optical systems using Fourier optics, with applications in imaging, diffraction, holography, and signal processing. The library leverages PyTorch's automatic differentiation engine for gradient-based optimization, enabling the inverse design of complex optical systems. TorchOptics supports end-to-end optimization of hybrid models that integrate optical systems with machine learning architectures for digital post-processing. The library includes a wide range of optical elements and spatial profiles, and supports simulations with polarized light and fields with arbitrary spatial coherence.

Figures

Figures reproduced from arXiv: 2411.18591 by the authors.

Figure 1
Figure 1. Intensity distributions of the optical field at each focal plane along the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Computation time of field propagation in TorchOptics as a function [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Field propagation and error analysis for a one-dimensional aperture [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training results of an optical system consisting of three phase mod [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Diffraction patterns from three circular apertures with varying spa￾tial coherence. The apertures are separated by 2 mm and each has a radius of 200 µm. The intensity distributions along the yz-plane at x = 0 mm (left) and corresponding intensity profiles at z = 2000 m…
Figure 6
Figure 6. Figure 6: Imaging fields with Gaussian-shaped optical spectra using a 4 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TurPy: a physics-based and differentiable optical turbulence simulator for algorithmic development and system optimization

    physics.optics 2026-04 unverdicted novelty 6.0 of 10

    TurPy delivers a differentiable wave-optics turbulence simulator that reproduces theoretical beam broadening and scintillation to 98% accuracy and enables end-to-end optimization of diffractive neural networks.

  2. Diffractive neural networks for mode-sorting with flexible detection regions

    physics.optics 2025-08 conditional novelty 6.0 of 10

    A diffractive neural network mode-sorter with trainable output detection regions achieves higher efficiency at equal crosstalk than fixed-region sorters.

  3. Locality-aware Surrogates for Gradient-based Black-box Optimization

    cs.LG 2025-01 conditional novelty 5.0 of 10

    A locality-aware surrogate loss derived from the gradient theorem improves gradient estimation and query efficiency in black-box optimization.

Reference graph

Works this paper leans on

32 extracted references · 16 canonical work pages · cited by 3 Pith papers

  1. [1]

    LeCun, Y

    Y . LeCun, Y . Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444. doi:10.1038/nature14539

  2. [3]

    Wetzstein, A

    G. Wetzstein, A. Ozcan, S. Gigan, S. Fan, D. Englund, M. Solja ˇci´c, C. Denz, D. A. B. Miller, D. Psaltis, Inference in artificial intelligence with deep optics and photonics, Nature 588 (7836) (2020) 39–47. doi: 10.1038/s41586-020-2973-6

  3. [4]

    B. J. Shastri, A. N. Tait, T. Ferreira de Lima, W. H. P. Pernice, H. Bhaskaran, C. D. Wright, P. R. Prucnal, Photonics for artificial in- telligence and neuromorphic computing, Nature Photonics 15 (2) (2021) 102–114. doi:10.1038/s41566-020-00754-y

  4. [5]

    Genty, L

    G. Genty, L. Salmela, J. M. Dudley, D. Brunner, A. Kokhanovskiy, S. Kobtsev, S. K. Turitsyn, Machine learning and applications in ultra- fast photonics, Nature Photonics 15 (2) (2021) 91–101. doi:10.1038/ s41566-020-00716-4

  5. [6]

    Mengu, M

    D. Mengu, M. S. Sakib Rahman, Y . Luo, J. Li, O. Kulce, A. Ozcan, At the intersection of optics and deep learning: Statistical inference, computing, and inverse design, Advances in Optics and Photonics 14 (2) (2022) 209. doi:10.1364/AOP.450345

  6. [7]

    Sinha, J

    A. Sinha, J. Lee, S. Li, G. Barbastathis, Lensless computational imaging through deep learning, Optica 4 (9) (2017) 1117–1125. doi:10.1364/ OPTICA.4.001117

  7. [8]

    Rivenson, Z

    Y . Rivenson, Z. G ¨or¨ocs, H. G ¨unaydin, Y . Zhang, H. Wang, A. Ozcan, Deep learning microscopy, Optica 4 (11) (2017) 1437. doi:10.1364/ OPTICA.4.001437

  8. [9]

    Doster, A

    T. Doster, A. T. Watnik, Machine learning approach to OAM beam de- multiplexing via convolutional neural networks, Applied Optics 56 (12) (2017) 3386–3396. doi:10.1364/AO.56.003386

Show all 32 references
  1. [10]

    Barbastathis, A

    G. Barbastathis, A. Ozcan, G. Situ, On the use of deep learning for com- putational imaging, Optica 6 (8) (2019) 921. doi:10.1364/OPTICA.6. 000921

  2. [11]

    W. J. Smith, Modern Optical Engineering, 4th Ed.: The Design of Optical Systems, 4th Edition, McGraw Hill, New York, 2008

  3. [12]

    T. W. Hughes, I. A. D. Williamson, M. Minkov, S. Fan, Wave physics as an analog recurrent neural network, Science Advances 5 (12) (2019) eaay6946. doi:10.1126/sciadv.aay6946

  4. [13]

    T. Zhou, X. Lin, J. Wu, Y . Chen, H. Xie, Y . Li, J. Fan, H. Wu, L. Fang, Q. Dai, Large-scale neuromorphic optoelectronic computing with a re- configurable diffractive processing unit, Nature Photonics 15 (5) (2021) 367–373. doi:10.1038/s41566-021-00796-w

  5. [14]

    A. H. Dorrah, F. Capasso, Tunable structured light with flat optics, Sci- ence 376 (6591) (2022) eabi6860. doi:10.1126/science.abi6860

  6. [15]

    Goodman, Introduction to Fourier Optics, fourth edition Edition, W

    J. Goodman, Introduction to Fourier Optics, fourth edition Edition, W. H. Freeman, New York, 2017

  7. [16]

    Mengu, Y

    D. Mengu, Y . Luo, Y . Rivenson, A. Ozcan, Analysis of Diffractive Optical Neural Networks and Their Integration With Electronic Neural Networks, IEEE Journal of Selected Topics in Quantum Electronics 26 (1) (2020) 1–

  8. [17]

    doi:10.1109/JSTQE.2019.2921376

  9. [18]

    Rodr ´ıguez, S

    C. Rodr ´ıguez, S. Arlt, L. M ¨ockl, M. Krenn, XLuminA: An Auto- differentiating Discovery Framework for Super-Resolution Microscopy (May 2024). arXiv:2310.08408, doi:10.48550/arXiv.2310. 08408

  10. [19]

    D. S. Hazineh, S. W. D. Lim, Z. Shi, F. Capasso, T. Zickler, Q. Guo, D-Flat: A Di fferentiable Flat-Optics Framework for End-to-End Meta- surface Visual Sensor Design (Aug. 2022). arXiv:2207.14780. 8

  11. [20]

    D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning representations by back-propagating errors, Nature 323 (6088) (1986) 533–536. doi: 10.1038/323533a0

  12. [21]

    Sitzmann, S

    V . Sitzmann, S. Diamond, Y . Peng, X. Dun, S. Boyd, W. Heidrich, F. Heide, G. Wetzstein, End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging, ACM Transactions on Graphics 37 (4) (2018) 1–13. doi: 10.1145/3...

  13. [22]

    S.-H. Baek, H. Ikoma, D. S. Jeon, Y . Li, W. Heidrich, G. Wetzstein, M. H. Kim, Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics, in: 2021 IEEE /CVF International Conference on Computer Vi- sion (ICCV), 2021, pp. 2631–2640. doi:10.1109/ICCV48922.2021. 00265

  14. [23]

    Z. Lin, C. Roques-Carmes, R. Pestourie, M. Solja ˇci´c, A. Majumdar, S. G. Johnson, End-to-end nanophotonic inverse design for imaging and po- larimetry, Nanophotonics 10 (3) (2021) 1177–1187. doi:10.1515/ nanoph-2020-0579

  15. [24]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, PyTorch: An Imperative Style, High- Perfor...

  16. [25]

    D. P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization (Jan. 2017). arXiv:1412.6980

  17. [26]

    N. A. Rubin, G. D’Aversa, P. Chevalier, Z. Shi, W. T. Chen, F. Capasso, Matrix Fourier optics enables a compact full-Stokes polarization cam- era, Science 365 (6448) (2019) eaax1839. doi:10.1126/science. aax1839

  18. [27]

    J. W. Goodman, Statistical Optics, John Wiley & Sons, 2015

  19. [28]

    F. Shen, A. Wang, Fast-Fourier-transform based numerical integration method for the Rayleigh-Sommerfeld di ffraction formula, Applied Op- tics 45 (6) (2006) 1102. doi:10.1364/AO.45.001102

  20. [29]

    V oelz, Computational Fourier Optics: A MATLAB Tutorial, no

    D. V oelz, Computational Fourier Optics: A MATLAB Tutorial, no. 89 in Tutorial Texts in Optical Engineering, SPIE Press, Bellingham, Wash, 2011

  21. [30]

    Virtanen, R

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y . Fen...

  22. [31]

    Hecht, Optics, 4th Edition, Addison-Wesley, Reading, Mass, 2002

    E. Hecht, Optics, 4th Edition, Addison-Wesley, Reading, Mass, 2002

  23. [32]

    M. J. Filipovich, A. Malyshev, A. I. Lvovsky, Role of spatial coherence in diffractive optical neural networks, Optics Express 32 (13) (2024) 22986. doi:10.1364/OE.523619

  24. [33]

    Mandel, E

    L. Mandel, E. Wolf, Optical Coherence and Quantum Optics, Cambridge University Press, Cambridge, 1995. doi:10.1017/ CBO9781139644105. 9

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.