Hardware-In-The-Loop Training of a 4f Optical Correlator with Logarithmic Complexity Reduction for CNNs

Federico Corradi; Lorenzo Pes; Maryam Dehbashizadeh Chehreghan; Rick Luiken; Ripalta Stabile; Sander Stuijk

REVIEW 2 major objections 4 minor 12 references

Hardware-In-The-Loop Training of a 4f Optical Correlator with Logarithmic Complexity Reduction for CNNs

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A forward-only learning rule trains a physical 4f optical correlator on MNIST to 87.6 percent accuracy, close to backpropagation, with a logarithmic complexity reduction.

desk verdict A credible first demonstration of PEPITA hardware-in-the-loop training on a 4f correlator, but the logarithmic complexity claim does not survive contact with their own FFT-based kernel update. read the letter →

arxiv 2501.04141 v1 pith:PQZ7ILSA submitted 2025-01-07 cs.NE

Lorenzo Pes , Maryam Dehbashizadeh Chehreghan , Rick Luiken , Sander Stuijk , Ripalta Stabile , Federico Corradi This is my paper · ORCID

classification cs.NE

keywords 4fopticalcorrelatorhardware-in-the-looptrainingforward-onlylearningPEPITAneuralnetworkMNISTconvolutionalcomplexityreduction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a physical 4f optical correlator can be trained directly, without a differentiable software model of the device, using a forward-only learning rule called PEPITA. On a 600-sample MNIST subset with hardware-in-the-loop training, PEPITA reaches 87.6% test accuracy, essentially matching backpropagation's 88.8% within noise. The authors also claim that PEPITA's parameter update costs $O(n^2)$ arithmetic while backpropagation's costs $O(n^2 \log n)$, a logarithmic reduction. If correct, this means analog optical hardware that is hard to simulate can still learn on-device from its own forward passes.

What carries the argument

The key object is the PEPITA update rule, a forward-only learning rule that modulates the input with the final-layer error, performs a second forward pass, and uses the difference between the two activation maps ($h$ and $h_{\mathrm{mod}}$) to update the weights locally. In the 4f correlator, convolution is done optically: a lens Fourier-transforms the input, a spatial light modulator multiplies by the kernel, and a second lens inverse-transforms, so the PEPITA update can be formed directly from optical outputs without a differentiable device model. This combination is what removes backpropagation's gradient path and yields the claimed $O(n^2)$ complexity.

What would settle it

Measure total wall-clock time and energy for both algorithms on the same optical setup using the full MNIST training set; if PEPITA's per-epoch time is not lower than backpropagation's, the claimed logarithmic complexity reduction does not translate to practice.

Watch

Extended reading notes

Core claim

The central discovery is that a forward-only learning algorithm, PEPITA, can train a 4f optical correlator in the loop, using two forward passes through the physical device to compute parameter updates and never computing a gradient through a device model. The paper reports 87.6% test accuracy with PEPITA versus 88.8% with backpropagation on the same hardware-in-the-loop setup, and attributes the gap to limited hyperparameter tuning. It also reports that the PEPITA update has $O(n^2)$ computational complexity compared to $O(n^2 \log n)$ for backpropagation, because backpropagation needs FFT-based derivatives and error propagation through layers, while PEPITA only combines activations from the two forward passes.

Load-bearing premise

The claim of a logarithmic complexity reduction holds only if the comparison is restricted to update-rule FLOPs; counting the 25 ms optical forward passes and the backpropagation gradient terms the paper omits could shrink or reverse the advantage.

Editorial extensions

If this is right

PEPITA eliminates the need for a software gradient model, so hardware imperfections are incorporated into the learned weights during training.
The per-update cost drops from $O(n^2 \log n)$ to $O(n^2)$, an advantage that grows with image size in Fourier-based convolutional layers.
Forward-only training requires only two forward passes during training and one at inference, making it practical on slow optical hardware.
The near-backpropagation accuracy on a 600-sample subset suggests the approach tolerates the device's optical imperfections, which had an average SSIM of 0.8 against ideal convolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension beyond the paper's claims is to measure wall-clock time end-to-end, since the 25 ms SLM setup time per optical pass could dominate and potentially erase the logarithmic complexity advantage.
The authors do not state this, but PEPITA's model-agnostic two-forward-pass update should transfer to other analog accelerators with drift or noise without re-deriving the learning rule.
The reported accuracy is on only 600 training samples; extending to the full MNIST or higher-resolution inputs would reveal whether the 1.2-point gap and the complexity reduction hold outside that regime.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper is that it does genuinely show PEPITA can train a 4f optical correlator in the loop to near-BP accuracy on a small MNIST subset, and that is a useful data point for the backprop-free photonic training crowd. But the advertised logarithmic complexity reduction over BP does not hold up when you count the full training step, including the FFT the paper itself says it runs every iteration.

What's new and good: To my knowledge this is the first HWL training of a 4f correlator specifically with PEPITA. The experimental setup is standard but the result is credible: five seeds, overlapping error bars (87.6±3 vs 88.8±4), and they are upfront that the dataset is tiny (600/100), the SLM setup is slow (25 ms), and no hyperparameter tuning was done. The SSIM check between software and optical convolutions is a nice sanity check.

Where it goes soft: The headline complexity claim is the problem. The abstract and Sec. 2 say PEPITA is O(n^2) vs BP O(n^2 log n), but Sec. 3 describes updating the kernel as W' = W - ηΔW and then transforming it with an FFT to load onto SLM2. An FFT of an n×n kernel is O(n^2 log n), so that step alone eats the claimed logarithmic advantage. On top of that, the PEPITA update for a convolutional layer is a cross-correlation between the input and the activation difference; doing that exactly is not free O(n^2), it is O(n^2 log n) via FFT or worse directly. So the O(n^2) figure counts part of the loop and ignores the rest. The paper's own footnote that BP FLOPS exclude dL/dzhw actually makes BP look better, not worse, in the comparison. The accuracy result stands, but the complexity contribution as stated does not.

There is also the usual reproducibility gap: no code, no data, no exact sampling details, which makes the empirical claim hard to verify. That is addressable but still missing.

Verdict: The paper is worth a serious referee because the empirical setup is new and the flaw is fixable. But the authors should be asked to either provide a full cost accounting for both algorithms or drop the logarithmic reduction claim entirely. As written, the central selling point is not supported.

Referee Report

2 major / 4 minor

Summary. The paper reports hardware-in-the-loop training of a 4f optical correlator on a subsampled MNIST dataset using the PEPITA forward-only learning algorithm. The central claim is that PEPITA achieves 87.6% accuracy versus 88.8% for backpropagation while reducing per-iteration computational complexity from O(n^2 log n) to O(n^2). The experiments use a single optical convolution layer with 8 kernels of size 28×28, a 2×2 max-pooling layer, and a fully connected classifier, with the convolution physically executed on the optical device during training and testing.

Significance. If the complexity reduction were justified, this would be an important step toward scalable optical training without a differentiable device model. The empirical demonstration that a forward-only algorithm can train a physical 4f correlator to near-backpropagation accuracy with overlapping error bars is a useful proof of concept. However, as argued in the major comments, the complexity claim is not supported by the paper's own description of the training loop, and this undermines the paper's headline contribution.

major comments (2)

[Section 3, abstract, conclusion] The claimed O(n^2) complexity for PEPITA omits the FFT that the paper explicitly states is performed on the updated kernel each iteration: "the updated kernel W′ = W − η∆W ... is transformed to the Fourier domain using the Fast Fourier Transform (FFT)." This FFT is O(n^2 log n) and is mandatory because the kernel is represented in the frequency domain inside the 4f correlator. Including it makes the per-iteration cost at least O(n^2 log n), equal to the quoted BP cost, so the logarithmic reduction claimed in the abstract and conclusion is not an end-to-end property of the training loop.
[Section 2] The statement that PEPITA has "a total computational complexity of O(n^2)" for the update rule is not justified for a convolution layer. The update ∆W is a function of the input and the activation difference (h − h_mod); computing this cross-correlation costs O(n^2 k^2) directly or O(n^2 log n) with FFTs, not O(n^2). The paper provides no derivation or algorithmic detail showing how an O(n^2) computation of ∆W is achieved. Thus the complexity comparison in Fig. 2d and the surrounding text is incomplete.

minor comments (4)

[Fig. 2d caption] The caption acknowledges that BP FLOPS exclude dL/dzhw from downstream layers; this caveat should also appear in the main text and the abstract, since it affects the interpretation of the comparison.
[Notation throughout] The complexity expressions are rendered inconsistently (e.g., "O(n2)" and "O(n2 log n)" in the abstract). Please use superscript notation O(n^2) and O(n^2 log n).
[Section 2] The definition of n is ambiguous: it is introduced as the input image size, but the introduction's convolution complexity formula uses O(n^2 k^2) with a kernel size k. Clarify whether the experimental 28×28 kernels make k = n, and define n and k explicitly in the complexity analysis.
[Section 4] The SSIM value of approximately 0.8 between software and optical convolutions is reported without a standard deviation or the number of measurements; please provide the distribution or a confidence interval.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical HWL accuracy result is externally benchmarked, and the complexity-accounting concern is a correctness/incompleteness issue, not a circular derivation.

full rationale

The paper's central empirical claim—that PEPITA can train a physical 4f optical correlator on MNIST to 87.6% accuracy versus 88.8% for backpropagation—is an externally measured result against a held-out test set, not a quantity fitted to the model and then relabeled as a prediction. PEPITA itself is imported from prior work by Dellaferrera and Kreiman, with no load-bearing dependence on the present authors' own citations. The only candidate for a circular step is the claimed O(n^2) complexity for PEPITA in Section 2. That claim is not derived from an equation that reduces to its own input; rather, it is an incomplete cost accounting. Section 3 states that the updated kernel W' is transformed to the Fourier domain using an FFT, which adds an O(n^2 log n) step per training iteration, and the Fig. 2d caption itself acknowledges that the BP FLOPS are undercounted. These are correctness weaknesses in the complexity comparison, not a case where the conclusion is equivalent by construction to the premise. The accuracy result remains self-contained and externally falsifiable, so the appropriate circularity finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The paper relies on standard Fourier optics, the PEPITA algorithm, and a small MNIST benchmark. The free parameters are experimental and architectural choices, not fitted constants, but they influence the reported accuracy and complexity.

free parameters (5)

learning rate = 0.001
Fixed for all experiments; chosen by hand without tuning and affects convergence.
camera exposure = 20 ms
Set for the optical readout; affects signal quality but is not varied or fitted.
number of kernels = 8
Architecture choice, not fitted; central to model capacity.
kernel size = 28 x 28
Matches input size, making the convolution a filter operation; chosen by design.
MNIST training subset size = 600
Chosen due to SLM sequential processing speed; affects variance and accuracy.

assumptions (4)

standard math Convolution theorem: convolution in the spatial domain equals pointwise multiplication in the Fourier domain.
Used throughout to justify 4f correlator operation (Section 1, Fig. 1).
domain assumption The 4f correlator implements forward Fourier and inverse Fourier transforms and pointwise kernel multiplication with fidelity sufficient for learning.
Required for HWL training; SSIM ≈ 0.8 shows fidelity is imperfect but sufficient (Section 4).
domain assumption PEPITA updates using two forward passes approximate error gradients without a backward pass.
Imported from [6]; the paper relies on its correctness without re-deriving it (Section 2).
domain assumption MNIST subset accuracy generalizes from 600 training samples to the intended use case.
The authors infer PEPITA's effectiveness from this small benchmark (Section 3).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Hardware-In-The-Loop Training of a 4f Optical Correlator with Logarithmic Complexity Reduction for CNNs." pith.science (2026). https://pith.science/paper/PQZ7ILSA

@misc{pith2026250104141,
  author       = {Pith},
  title        = {Pith review of: Hardware-In-The-Loop Training of a 4f Optical Correlator with Logarithmic Complexity Reduction for CNNs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQZ7ILSA}},
  note         = {Machine review of arXiv:2501.04141}
}

read the original abstract

This work evaluates a forward-only learning algorithm on the MNIST dataset with hardware-in-the-loop training of a 4f optical correlator, achieving 87.6% accuracy with O(n2) complexity, compared to backpropagation, which achieves 88.8% accuracy with O(n2 log n) complexity.

Figures

Figures reproduced from arXiv: 2501.04141 by the authors.

**Figure 1.** HWL with 4f Optical Correlator. a Physical 4f correlator. b Full system view with input and output processing. c HWL training with BP. d HWL training with PEPITA. In both c and d, the blue path represents the physical device. In red are the computational complexity of the update rules represented by the orange equations. Notably, PEPITA does not require a differentiable model of the device, but instead, it uses acti… view at source ↗

**Figure 2.** HWL vs. software training results. a Mean training loss and train accuracy per epoch for each algorithm with HWL with 300 MNIST samples. b Software versus (experimental) optical convolution with an edge detection kernel. The average SSIM is reported inside. c End of training mean test accuracy in optical device. d Total FLOPS for BP and PEPITA. Approximate FLOPS for BP are computed without accounting for dL/dzhw fro… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

[1]

Photonics for artificial intelligence and neuromorphic computing

Bhavin J Shastri, Alexander N Tait, Thomas Ferreira de Lima, Wolfram HP Pernice, Harish Bhaskaran, C David Wright, and Paul R Prucnal. Photonics for artificial intelligence and neuromorphic computing. Nature Photonics , 15(2), 2021

work page 2021
[2]

Massively parallel amplitude-only fourier neural network

Mario Miscuglio, Zibo Hu, Shurui Li, Jonathan K George, Roberto Capanna, Hamed Dalir, Philippe M Bardet, Puneet Gupta, and Volker J Sorger. Massively parallel amplitude-only fourier neural network. Optica , 7(12):1812--1819, 2020

work page 2020
[3]

Hybrid training of optical neural networks

James Spall, Xianxin Guo, and Alexander I Lvovsky. Hybrid training of optical neural networks. Optica , 9(7), 2022

work page 2022
[4]

Signal propagation: The framework for learning and inference in a forward pass

Adam Kohan, Edward A Rietman, and Hava T Siegelmann. Signal propagation: The framework for learning and inference in a forward pass. IEEE Transactions on Neural Networks and Learning Systems , 2023

work page 2023
[5]

The forward-forward algorithm: Some preliminary investigations

Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations. arXiv:2212.13345 , 2022

arXiv 2022
[6]

Error-driven input modulation: Solving the credit assignment problem without a backward pass

Giorgia Dellaferrera and Gabriel Kreiman. Error-driven input modulation: Solving the credit assignment problem without a backward pass. In International Conference on Machine Learning , pages 4937--4955. PMLR, 2022

work page 2022
[7]

Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks

Danilo Pietro Pau and Fabrizio Maria Aymone. Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks. In 2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS) . IEEE, 2023

work page 2023
[8]

Fully forward mode training for optical neural networks

Zhiwei Xue, Tiankuang Zhou, Zhihao Xu, Shaoliang Yu, Qionghai Dai, and Lu Fang. Fully forward mode training for optical neural networks. Nature , 632(8024):280--286, 2024

work page 2024

Show all 12 references

[9]

Forward--forward training of an optical neural network

Ilker Oguz, Junjie Ke, Qifei Weng, Feng Yang, Mustafa Yildirim, Niyazi Ulas Dinc, Jih-Liang Hsieh, Christophe Moser, and Demetri Psaltis. Forward--forward training of an optical neural network. Optics Letters , 48(20):5249--5252, 2023

2023
[10]

Backpropagation-free training of deep physical neural networks

Ali Momeni, Babak Rahmani, Matthieu Mall \'e jac, Philipp Del Hougne, and Romain Fleury. Backpropagation-free training of deep physical neural networks. Science , 382(6676):1297--1303, 2023

2023
[11]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter edition editor eid howpublished institution journal key month note number organization pages publisher school series title type volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state...
[12]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Photonics for artificial intelligence and neuromorphic computing

Bhavin J Shastri, Alexander N Tait, Thomas Ferreira de Lima, Wolfram HP Pernice, Harish Bhaskaran, C David Wright, and Paul R Prucnal. Photonics for artificial intelligence and neuromorphic computing. Nature Photonics , 15(2), 2021

work page 2021

[2] [2]

Massively parallel amplitude-only fourier neural network

Mario Miscuglio, Zibo Hu, Shurui Li, Jonathan K George, Roberto Capanna, Hamed Dalir, Philippe M Bardet, Puneet Gupta, and Volker J Sorger. Massively parallel amplitude-only fourier neural network. Optica , 7(12):1812--1819, 2020

work page 2020

[3] [3]

Hybrid training of optical neural networks

James Spall, Xianxin Guo, and Alexander I Lvovsky. Hybrid training of optical neural networks. Optica , 9(7), 2022

work page 2022

[4] [4]

Signal propagation: The framework for learning and inference in a forward pass

Adam Kohan, Edward A Rietman, and Hava T Siegelmann. Signal propagation: The framework for learning and inference in a forward pass. IEEE Transactions on Neural Networks and Learning Systems , 2023

work page 2023

[5] [5]

The forward-forward algorithm: Some preliminary investigations

Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations. arXiv:2212.13345 , 2022

arXiv 2022

[6] [6]

Error-driven input modulation: Solving the credit assignment problem without a backward pass

Giorgia Dellaferrera and Gabriel Kreiman. Error-driven input modulation: Solving the credit assignment problem without a backward pass. In International Conference on Machine Learning , pages 4937--4955. PMLR, 2022

work page 2022

[7] [7]

Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks

Danilo Pietro Pau and Fabrizio Maria Aymone. Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks. In 2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS) . IEEE, 2023

work page 2023

[8] [8]

Fully forward mode training for optical neural networks

Zhiwei Xue, Tiankuang Zhou, Zhihao Xu, Shaoliang Yu, Qionghai Dai, and Lu Fang. Fully forward mode training for optical neural networks. Nature , 632(8024):280--286, 2024

work page 2024