Pith. sign in

REVIEW 4 major objections 6 minor 22 references

Deep Learning for Optical Misalignment Diagnostics in Multi-Lens Imaging Systems

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Trained only on simulated spot diagrams and camera images, a neural network recovers per-element translation and tilt errors in a six-lens camera accurately enough for automated alignment.

desk verdict A re-implementable simulation study showing that neural networks can regress per-element misalignments from spot diagrams and images, but the practical claims outrun the evidence. read the letter →

arxiv 2506.23173 v1 pith:ROHNQ4A3 submitted 2025-06-29 physics.optics cs.AIcs.LG

Tomer Slor , Dean Oren , Shira Baneth , Tom Coen , Haim Suchowski This is my paper · ORCID
classification physics.opticscs.AIcs.LG
keywords opticalmisalignmentdiagnosticsdeeplearningspotdiagramthrough-focusimaginglensalignmentsynthetictrainingdatasim-to-realtransfermulti-lenssystem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a deep network can reverse-engineer element-by-element alignment errors in a multi-lens camera from the light it images, without opening the lens or using an interferometer. Two complementary pipelines are tested: one feeds ray-traced spot diagrams into a fully connected network and recovers five degrees of freedom (two lateral shifts, axial shift, two tilts) for each of four optical groups in a six-lens photographic prime, reaching 0.0317 mm mean absolute error in translation and 0.011 degrees in tilt. The other feeds grayscale through-focus images into a ResNet-based model and estimates four degrees of freedom (decenter and tilt) for two- and six-lens systems, with mean absolute errors of 0.044 mm and 0.121 degrees, and 0.089 mm and 0.505 degrees, respectively. If the synthetic-to-real transfer holds, such a system would turn an ordinary camera and a few defocused snapshots into an alignment diagnostic for manufacturing and field calibration.

What carries the argument

The load-bearing object for Method 1 is the spot diagram: the measured $(x_i, y_i)$ coordinates where rays from a 2x2 grid of field points at two wavelengths hit two detector planes, flattened into one 3200-dimensional input vector. Misalignments deform this pattern, and a five-layer fully connected network with residual connections learns the mapping from the deformation to the 20-dimensional misalignment vector. Method 2's machinery is the grayscale through-focus image stack: irradiance planes simulated at several detector positions, each encoded by a ResNet18, then fused with crop-coordinate metadata that records where and how large the bright image region is. The crop coordinates are passed through a dedicated MLP, so the network sees both the gross position and scale of the image patch and the fine structure inside it.

What would settle it

Assemble the six-lens prime from the cited patent, introduce known decenter and tilt offsets to each element, capture grayscale images at the five focus positions (or spot diagrams at the two screen distances), and compare the network's per-element predictions with the applied perturbations; if the errors exceed the reported test-set MAEs, or if predictions do not track the offsets, the central claim is not transferable to hardware. A minimal version can be done purely in simulation: train with rayopt spot diagrams and test on 3DOptix images of the same system, since a cross-simulator gap would already bound the sim-to-real gap.

Watch

Extended reading notes

Core claim

The central claim is that the inverse map from optical measurements to misalignment states is learnable in high-dimensional, multi-element systems. Using only simulated data—spot diagrams generated by ray tracing and grayscale irradiance images from a physical simulation platform—the networks predict per-element decenter and tilt errors across the full assembly, with no symmetry assumptions, no internal access, and no interferometric inputs. The spot-diagram model treats the collection of ray hit coordinates as a 3200-dimensional feature vector and regresses a 20-dimensional output encoding five degrees of freedom for each of the four optical elements. The image model encodes each through-focus irradiance plane with a ResNet18 per image, concatenates crop-coordinate metadata, and regresses four degrees of freedom per element. On test samples, re-simulating the system with the predicted misalignment vector reproduces the misaligned image closely, and for the image-based method the corrected irradiance approaches the ideal aligned reference.

Load-bearing premise

The scheme stands on the assumption that spot diagrams and images generated in the two simulation environments match real optical behavior closely enough that models trained entirely on synthetic data will diagnose misalignments on physical hardware; the paper reports no experiment that tests this transfer.

Editorial extensions

If this is right

  • Method 1 reaches a mean absolute error of 0.0317 mm in translation and 0.011 degrees in tilt on the six-lens test set, predicting all five degrees of freedom for each optical element.
  • Method 2 reaches 0.044 mm and 0.121 degrees on the two-lens system and 0.089 mm and 0.505 degrees on the six-lens system using only grayscale through-focus images and crop metadata.
  • Both pipelines avoid symmetry assumptions, internal lens access, wavefront sensors, and interferometry, relying only on spot diagrams or incoherent images.
  • Re-simulating with the predicted misalignment vector yields irradiance images close to those of the ground-truth alignment, and for the image-based method the corrected image approaches the ideal aligned reference.
  • The paper identifies acquisition geometry—screen placement, ray range, and source location—as decisive: poor choices degrade performance or make the inverse problem ill-posed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the synthetic-to-real transfer is confirmed, the same simulation pipeline could be reused for other lens prescriptions, turning each new design's optical prescription into a one-time training set and making the method a drop-in quality-control step.
  • Editorial inference: the two methods read different physical information—spot diagrams encode ray angles directly, while through-focus images encode intensity—so a hybrid that fuses both could outperform either, especially for axial shift, which Method 2 omits.
  • Editorial inference: a natural testable extension is to train on simulated data and fine-tune on a small set of physically measured misalignments; if even a handful of real examples closes the gap, the sim-to-real concern becomes a calibration problem rather than a barrier.
  • Editorial inference: the crop-metadata channel suggests the network partly solves alignment by tracking the location and size of the illuminated patch, so diagnostic accuracy may depend on having a high-contrast target such as the USAF resolution mask used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes two deep learning methods for diagnosing lens misalignments from externally observable optical data. Method 1 uses ray-traced spot diagrams (from the rayopt library) and a fully connected network to regress 5-DOF errors (decenter, axial shift, tilt) for each of four optical elements in a six-lens photographic prime, reporting a mean absolute error of 0.0317 mm in translation and 0.011 degrees in tilt on a held-out synthetic test set. Method 2 uses synthetic grayscale irradiance images (from the 3DOptix platform) processed by ResNet18 encoders with crop metadata, estimating 4-DOF errors in a two-lens system (MAE 0.044 mm, 0.121 degrees) and a six-lens system (MAE 0.089 mm, 0.505 degrees). The authors claim that because the simulations incorporate realistic physical effects, the trained models 'generalize well to experimental conditions' and can be readily applied in manufacturing and quality control.

Significance. If the claimed sim-to-real transfer were established, the framework would be a practical contribution to automated optical alignment, replacing specialized metrology with standard spot-diagram or camera-image measurements. The paper contributes large synthetic datasets (500,000 spot-diagram samples and 124,000 image samples), a reproducible pipeline with open-source code, and a clean demonstration that a neural network can regress high-dimensional misalignment states from simulated data. However, the significance is currently limited by the absence of any experimental validation and by the clear overfitting in the six-lens image model. The central premise, that synthetic training transfers to physical hardware, is asserted rather than demonstrated, and the paper provides no baseline comparisons or uncertainty measures.

major comments (4)
  1. [Section 4 (Summary), final paragraph] The claim that 'the trained model generalizes well to experimental conditions' is not supported by any experiment in the manuscript. All reported evaluation is on held-out synthetic samples drawn from the same simulators used for training (rayopt for Method 1, 3DOptix for Method 2). No noise, sensor pixelation, stray light, calibration error, or domain randomization is modeled, and no hardware experiment is reported. Since the stated motivation is manufacturing and quality control, this load-bearing claim must either be verified experimentally or removed and replaced with a clearly limited claim about simulation performance. As written, the abstract and summary overstate the applicability of the method.
  2. [Section 3, 'Image-based model (6-lens system)' and Fig. 4(a)] The six-lens image model shows substantial overfitting: the validation loss plateaus near 0.387 after about 160 epochs while the training loss continues to decrease, yet the text characterizes this as 'mild overfitting'. The reported MAEs (0.089 mm translation, 0.505 degrees tilt) correspond to roughly 18% and 8% of the full perturbation ranges (0.5 mm and 6 degrees), and the representative sample in Fig. 4(c) contains per-element errors of up to 0.128 mm in Δx (Element 3) and 0.79 degrees in Δθx (Element 2). These numbers do not support the sentence that the model 'generalizes well to multi-element alignment regression directly from sensor data'; at minimum, the overfitting should be addressed with regularization or early stopping, and the claims tempered accordingly.
  3. [Section 2.2 (Image-based method) and Supplementary S.2/S.3] The network for Method 2 receives not only the cropped image but also the crop coordinates (x,y) and pre-crop dimensions (width, height) as metadata. These coordinates are computed directly from the simulated irradiance distribution via intensity thresholding, so they are strongly correlated with the global image shift induced by the misalignments. The paper does not analyze whether the model exploits this metadata as a shortcut rather than learning from image content, nor does it discuss whether such metadata would be available with comparable accuracy from a physical camera (where thresholding and coordinate calibration introduce noise). An ablation study without the metadata, or an analysis of the model's reliance on it, is needed to support the claim that the method works 'from raw irradiance data.'
  4. [Section 3, 'Spot diagram model'] The headline accuracy of 0.0317 mm / 0.011 degrees is a same-simulator test on rayopt-generated spot diagrams, which are idealized geometric ray-hit coordinates without noise, diffraction, or detector effects. Because the network is trained on the exact same simulation code it is tested on, it may exploit fine synthetic details (such as the precise ray-ID-based masking or exact screen positions) that would not be observable in a real experiment. The paper provides no cross-simulator validation (e.g., using a different ray tracer), no injected noise, and no sensitivity analysis of the results to simulation parameters such as ray count, field grid, or screen distance. Consequently, the claim that this accuracy 'would be challenging to achieve with analytical or manual optimization approaches' is not established for physical systems.
minor comments (6)
  1. [Section 2.2 and Supplementary S.2] Method 2 is described as using 'through-focus intensity images,' but the two-lens system uses only a single detector plane; only the six-lens system uses multiple detector positions. Please adjust the terminology to avoid ambiguity.
  2. [Throughout] The term 'inverse-design' is not standard for this task; the paper solves an inverse problem of estimating misalignments from measurements. Consider using 'inverse problem' or 'misalignment regression' instead.
  3. [Section 3] No error bars or standard deviations are reported for the MAE values. All results appear to come from a single training run; reporting the mean and variance over multiple seeds or test subsets would strengthen the reliability of the numbers.
  4. [Fig. 2(c) and Fig. 4(c)] The tables show only one representative test sample. Please provide per-element error statistics (e.g., mean and percentiles) across the full test set, as a single sample can be unrepresentative.
  5. [Section 2.1 and Supplementary S.1] The description of the ray generation is ambiguous: the text says 'Each sample consists of 400 rays traced from a 2×2 grid of field positions,' while S.1 says 'Rays (400 per field point ...) are emitted from a 2×2 grid.' Please clarify the total number of rays per spot diagram and how the four 400-ray inputs are constructed.
  6. [Supplementary S.1] The 'Surface sequence' string ('cgtctcgtcgtctcgtcgtctcgtct') is unexplained; a reference to the notation or a table of surface types would help the reader interpret the prescription.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the network maps simulated measurements to independently sampled misalignment labels and is evaluated on held-out simulated samples; sim-to-real generalization is an untested validity assumption, not a circular derivation.

full rationale

The paper's derivation chain is: (1) sample random misalignment vectors from specified uniform distributions; (2) run deterministic ray-tracing (rayopt) or irradiance simulation (3DOptix) to generate spot diagrams or images; (3) train a network to regress the misalignment vector from those measurements; (4) evaluate on a held-out split of the same simulated dataset. The ground-truth misalignments are output labels, not input features, and they are not fitted from the measurements; they are independently drawn before simulation. The reported MAE of 0.0317 mm translation and 0.011 deg tilt (Results, 'Spot diagram model') is a same-simulator held-out evaluation, which is ordinary generalization testing rather than circular reasoning. No parameter is fitted to a subset of data and then renamed a prediction; the labels are never fed into the network as inputs. There are no self-citations used as load-bearing support: the cited simulators (rayopt, 3DOptix) are external tools, and the patent-based lens prescriptions are independent external specifications. No uniqueness theorem or ansatz is imported from the authors' own previous work. The Summary's statement that 'because our simulations incorporate realistic physical effects, including surface scattering, polarization, and reflection, the trained model generalizes well to experimental conditions' is an unsupported assertion about sim-to-real transfer; it is a validity and correctness concern, not a circularity concern, because no experimental data is involved either in defining the method or in producing the reported numbers. The reader's weakest_assumption identifies this gap correctly, and the reader's circularity score of 0 is appropriate.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. It relies on the fidelity of commercial and open-source simulators and on the standard supervised-learning assumption that held-out simulated samples represent the deployment distribution. Free parameters include hand-chosen simulation ranges, screen positions, cropping thresholds, and training hyperparameters that affect the reported errors.

free parameters (5)
  • Perturbation ranges for spot diagram data = ±1 mm, ±1 degree
    Chosen by hand; results depend on fault-space extent.
  • Screen positions for spot diagram = 16 mm and 26 mm behind last surface
    Selected to maximize observability; stated as important in Summary.
  • Cropping intensity thresholds for image method = 10, 10, 10, 9, 5 W/cm2
    Hand-chosen thresholds for each of five detectors.
  • Training hyperparameters = Spot: lr=1e-5, wd=0.01; Image: lr=0.01, wd=0.001
    Hyperparameters chosen by hand; no sensitivity analysis.
  • Tilt range for image method = ±3 degrees
    Hand-chosen; MAE of 0.505 degrees for six-lens system should be interpreted relative to this range.
assumptions (3)
  • domain assumption Ray tracing in rayopt and 3DOptix accurately models the real optical behavior of the lens systems, including scattering, polarization, and reflection.
    Invoked throughout Methods (S.1, S.2, S.3) and Summary; no experimental verification is provided.
  • domain assumption A neural network trained solely on synthetic data will transfer to real measurements without significant domain shift.
    Central to the claimed practical value, stated explicitly in Summary.
  • domain assumption The inverse mapping from spot diagrams and images to element misalignments is learnable and unique over the sampled range.
    Assumed by training regressors; the paper notes observability depends on configuration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Learning for Optical Misalignment Diagnostics in Multi-Lens Imaging Systems." pith.science (2026). https://pith.science/paper/ROHNQ4A3

@misc{pith2026250623173,
  author       = {Pith},
  title        = {Pith review of: Deep Learning for Optical Misalignment Diagnostics in Multi-Lens Imaging Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ROHNQ4A3}},
  note         = {Machine review of arXiv:2506.23173}
}
abstract

In the rapidly evolving field of optical engineering, precise alignment of multi-lens imaging systems is critical yet challenging, as even minor misalignments can significantly degrade performance. Traditional alignment methods rely on specialized equipment and are time-consuming processes, highlighting the need for automated and scalable solutions. We present two complementary deep learning-based inverse-design methods for diagnosing misalignments in multi-element lens systems using only optical measurements. First, we use ray-traced spot diagrams to predict five-degree-of-freedom (5-DOF) errors in a 6-lens photographic prime, achieving a mean absolute error of 0.031mm in lateral translation and 0.011$^\circ$ in tilt. We also introduce a physics-based simulation pipeline that utilizes grayscale synthetic camera images, enabling a deep learning model to estimate 4-DOF, decenter and tilt errors in both two- and six-lens multi-lens systems. These results show the potential to reshape manufacturing and quality control in precision imaging.

Figures

Figures reproduced from arXiv: 2506.23173 by the authors.

Figure 1
Figure 1. Overview of the misalignment diagnosis framework. (a) Comparison between spot diagrams from a nominally aligned system and a misaligned system, where small perturbations in lens positions or orientations produce measurable shifts in the ray intersection pattern at the image plane. (b) Schematic of the learning pipeline for Method 1: the spot diagram (collection of (𝑥𝑖 , 𝑦𝑖) ray hit coordinates) is flattened into a f… view at source ↗
Figure 2
Figure 2. Results for the 6-lens system using Method 1. (a) Training and validation MSE [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results for the 2-lens system. (a) Training and validation MSE loss curves. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Results for the 6-lens system. (a) Training and validation MSE loss curves. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 21 canonical work pages

  1. [1]

    Malacara,Optical Shop Testing (John Wiley & Sons, Hoboken, NJ, 2007), 3rd ed

    D. Malacara,Optical Shop Testing (John Wiley & Sons, Hoboken, NJ, 2007), 3rd ed. Wiley Series in Pure and Applied Optics

    work page 2007

  2. [2]

    W. J. Smith,Modern Optical Engineering: The Design of Optical Systems (McGraw-Hill, New York, 2008), 4th ed

    work page 2008

  3. [3]

    Extended hartmann test based on the pseudoguiding property of a hartmann maskcompleted by a phase chessboard,

    J. Primot, V. Daru, and B. Guénoche, “Extended hartmann test based on the pseudoguiding property of a hartmann maskcompleted by a phase chessboard,” Appl. Opt.39, 571–580 (2000)

    work page 2000

  4. [4]

    The use of primary mirrors as hartmann masks for in situ alignment of segmented mirror telescopes,

    S. M. Crawford, M. Wells, and H. Gajjar, “The use of primary mirrors as hartmann masks for in situ alignment of segmented mirror telescopes,” inProc. SPIE, vol. 7012 (2008), p. 70123P

    work page 2008

  5. [5]

    Developments in optical testing technology during the last decade,

    J. C. Wyant, “Developments in optical testing technology during the last decade,” inLatin America Optics and Photonics Conf. (LAOP), (Optical Society of America, 2010), p. TuH1

    work page 2010

  6. [6]

    Telescope alignment by out-of-focus stellar image analysis,

    E. Luna, A. Cordero, J. Valdez,et al., “Telescope alignment by out-of-focus stellar image analysis,” Publ. Astron. Soc. Pac.111, 104–110 (1999)

    work page 1999

  7. [7]

    Neural-network-directed alignment of optical systems using the laser-beam spatial filter as an example,

    A. J. Decker, M. J. Krasowski, and K. E. Weiland, “Neural-network-directed alignment of optical systems using the laser-beam spatial filter as an example,” Tech. Rep. NASA TP-3372, NASA Lewis Research Center (1993)

    work page 1993

  8. [8]

    Deep learning in optical metrology: a review,

    C. Zuo, J. Qian, S. Feng,et al., “Deep learning in optical metrology: a review,” Light. Sci. & Appl.11, 39 (2022)

    work page 2022

Show all 22 references
  1. [9]

    Deep learning-enabled framework for automatic lens-design starting-point generation,

    G. Côté, J. Lalonde, and S. Thibault, “Deep learning-enabled framework for automatic lens-design starting-point generation,” Opt. Express29, 3841–3854 (2021). Published: January 25 2021

    2021

  2. [10]

    Accelerating optics-design optimizations with deep learning,

    R. S. Hegde, “Accelerating optics-design optimizations with deep learning,” Opt. Eng.58, 065103 (2019)

    2019

  3. [11]

    Approach for self-optimising assembly of optical systems,

    M. Holters, A. Gatej, S. Haag,et al., “Approach for self-optimising assembly of optical systems,” Int. J. Comput. Integr. Manuf.29, 1227–1237 (2016)

    2016

  4. [12]

    Determining optimal assembly condition for lens module production by combining genetic algorithm and c-blstm,

    H. Min, Y. Son, and Y. Choi, “Determining optimal assembly condition for lens module production by combining genetic algorithm and c-blstm,” Processes12, 452 (2024)

    2024

  5. [13]

    Opticalsystemoptimizationmethodforas-builtperformancebasedonnodalaberration theory,

    Z.Gu,Y.Wang,andC.Yan,“Opticalsystemoptimizationmethodforas-builtperformancebasedonnodalaberration theory,” Opt. Express28, 7928–7942 (2020)

    2020

  6. [14]

    Application of deep learning in active alignment leads to high-efficiency and accurate camera lens assembly,

    H. Liu, W. Li, S. Gao,et al., “Application of deep learning in active alignment leads to high-efficiency and accurate camera lens assembly,” Opt. Express32, 43834–43849 (2024)

    2024

  7. [15]

    Pointspreadfunctionestimationforwide-fieldsmall-aperturetelescopeswithdeepneural networks and calibration data,

    P.Jia, X.Wu, Z.Li, et al., “Pointspreadfunctionestimationforwide-fieldsmall-aperturetelescopeswithdeepneural networks and calibration data,” Mon. Notices Royal Astron. Soc.505, 4717–4725 (2021)

    2021

  8. [16]

    Improvingneural-network-basedpredictionmodelsformisalignmentinoff-axisthree-mirror anastigmat telescopes,

    I.BaşlarandM.Dursun,“Improvingneural-network-basedpredictionmodelsformisalignmentinoff-axisthree-mirror anastigmat telescopes,” Appl. Opt.63, 7747–7755 (2024)

    2024

  9. [17]

    Machine learning for improving stellar image-based alignment in wide-field telescopes,

    Z. Wu, Y. Zhang, R. Tang,et al., “Machine learning for improving stellar image-based alignment in wide-field telescopes,” Res. Astron. Astrophys.22, 015008 (2022)

    2022

  10. [18]

    Perception of misalignment states for sky survey telescopes with digital twin and deep neural networks,

    M. Zhang, P. Jia, and Z. Li, “Perception of misalignment states for sky survey telescopes with digital twin and deep neural networks,” Opt. Eng.62, 44054 (2023)

    2023

  11. [19]

    Tolerance-aware deep optics,

    J. Dai, L. Chen, and T. Xue, “Tolerance-aware deep optics,” arXiv preprintarXiv:2502.04719(2025)

    2025 arXiv

  12. [20]

    Method for optical adjustment with deep learning to quantitatively predict misalignment in optics,

    R. Hashimoto, S. Matsuura, and Y. Iida, “Method for optical adjustment with deep learning to quantitatively predict misalignment in optics,” Appl. Opt.63, 6794–6805 (2024)

    2024

  13. [21]

    GitHub repository

    RayOpt, “Rayopt,”https://github.com/quartiq/rayopt (2020). GitHub repository

    2020

  14. [22]

    3doptix: Cloud-based optical design and simulation platform,

    3DOptix Ltd., “3doptix: Cloud-based optical design and simulation platform,” (2025)

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.