{"id":"382266de-0185-4123-9155-f826dbf4b3c2","arxiv_id":"2501.04141","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Hardware-in-the-loop training of a 4f optical correlator with the PEPITA forward-only algorithm reaches MNIST accuracy comparable to backpropagation while removing the need for software gradients and claiming a log-factor reduction in training complexity.","lead":"The authors trained a physical 4f optical correlator on a small MNIST task using a forward-only algorithm called PEPITA, reaching 87.6% test accuracy versus 88.8% for backpropagation. The result suggests optical neural networks can be trained in the loop without a differentiable software model, at a lower stated computational cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The O(n^2) complexity claim for PEPITA omits the FFT required to reload the updated kernel onto SLM2, so the logarithmic reduction over BP is unsupported.","rationale":"The reader correctly flags the complexity comparison as the weakest assumption, but the specific concern differs. The reader focuses on latency and omitted BP FLOPS; the sharper issue is that PEPITA's own O(n^2) count omits the FFT used to update the frequency-domain kernel on SLM2, as acknowledged in the paper's setup. This is not a question of wall-clock time but of asymptotic algorithmic complexity, which is the paper's central quantitative claim. Since the title and abstract advertise a logarithmic reduction, and the reduction does not hold when the full training loop is counted, the central claim is unsupported. The empirical comparison on a small MNIST subset may still be valid, but the paper's primary contribution is the complexity reduction; without it, the contribution reduces to an incremental hardware demonstration. The error bars overlap, so the accuracy parity is plausible, but the complexity argument is internally inconsistent with the described pipeline. Therefore the verdict should move from CONDITIONAL to REJECT, or at least require substantial revision of the complexity claim. The proposed concrete test would settle whether the omitted FFT is indeed load-bearing; based on the manuscript text, it is.","tokens_in":3495,"tokens_out":9668,"duration_ms":95404,"concrete_test":"Recompute the total per-training-step FLOPs for the described architecture, explicitly including (a) the FFT of the updated kernel W′ to load onto SLM2 (acknowledged in Sec. 3, O(n^2 log n)) and (b) the cross-correlation in the PEPITA ΔW computation for a full-size 28×28 kernel. If both are included, PEPITA's asymptotic cost becomes O(n^2 log n), matching BP, so the logarithmic reduction disappears. For a direct check, instrument a software simulation of the PEPITA update rule and count FFT and correlation operations per epoch.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is the logarithmic complexity reduction, but the paper's own text undermines it. Section 3 states: 'the updated kernel W′ = W − η∆W ... is transformed to the Fourier domain using the Fast Fourier Transform (FFT).' This FFT is O(n^2 log n) and must be executed every training iteration to update SLM2. The O(n^2) complexity assigned to PEPITA in Sec. 2 and the abstract excludes this mandatory step. Even if the PEPITA update rule ΔW were computed in O(n^2), the additional FFT makes the per-step cost O(n^2 log n), equal to BP. Moreover, for a convolution layer with an n×n kernel, the PEPITA update ΔW = (h − h_mod) correlated with the input is itself a cross-correlation, which requires O(n^2 log n) via FFT (or O(n^4) direct), not O(n^2). Thus the claimed logarithmic reduction is not an end-to-end property of the training loop; it is an artifact of counting only a subset of operations. The accuracy result (87.6% vs 88.8%, overlapping error bars) is unaffected, but the paper's headline complexity contribution fails.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports hardware-in-the-loop training of a 4f optical correlator on a subsampled MNIST dataset using the PEPITA forward-only learning algorithm. The central claim is that PEPITA achieves 87.6% accuracy versus 88.8% for backpropagation while reducing per-iteration computational complexity from O(n^2 log n) to O(n^2). The experiments use a single optical convolution layer with 8 kernels of size 28×28, a 2×2 max-pooling layer, and a fully connected classifier, with the convolution physically executed on the optical device during training and testing.","tokens_in":3730,"tokens_out":5091,"duration_ms":45707,"significance":"If the complexity reduction were justified, this would be an important step toward scalable optical training without a differentiable device model. The empirical demonstration that a forward-only algorithm can train a physical 4f correlator to near-backpropagation accuracy with overlapping error bars is a useful proof of concept. However, as argued in the major comments, the complexity claim is not supported by the paper's own description of the training loop, and this undermines the paper's headline contribution.","major_comments":[{"comment":"The claimed O(n^2) complexity for PEPITA omits the FFT that the paper explicitly states is performed on the updated kernel each iteration: \"the updated kernel W′ = W − η∆W ... is transformed to the Fourier domain using the Fast Fourier Transform (FFT).\" This FFT is O(n^2 log n) and is mandatory because the kernel is represented in the frequency domain inside the 4f correlator. Including it makes the per-iteration cost at least O(n^2 log n), equal to the quoted BP cost, so the logarithmic reduction claimed in the abstract and conclusion is not an end-to-end property of the training loop.","section":"Section 3, abstract, conclusion"},{"comment":"The statement that PEPITA has \"a total computational complexity of O(n^2)\" for the update rule is not justified for a convolution layer. The update ∆W is a function of the input and the activation difference (h − h_mod); computing this cross-correlation costs O(n^2 k^2) directly or O(n^2 log n) with FFTs, not O(n^2). The paper provides no derivation or algorithmic detail showing how an O(n^2) computation of ∆W is achieved. Thus the complexity comparison in Fig. 2d and the surrounding text is incomplete.","section":"Section 2"}],"minor_comments":[{"comment":"The caption acknowledges that BP FLOPS exclude dL/dzhw from downstream layers; this caveat should also appear in the main text and the abstract, since it affects the interpretation of the comparison.","section":"Fig. 2d caption"},{"comment":"The complexity expressions are rendered inconsistently (e.g., \"O(n2)\" and \"O(n2 log n)\" in the abstract). Please use superscript notation O(n^2) and O(n^2 log n).","section":"Notation throughout"},{"comment":"The definition of n is ambiguous: it is introduced as the input image size, but the introduction's convolution complexity formula uses O(n^2 k^2) with a kernel size k. Clarify whether the experimental 28×28 kernels make k = n, and define n and k explicitly in the complexity analysis.","section":"Section 2"},{"comment":"The SSIM value of approximately 0.8 between software and optical convolutions is reported without a standard deviation or the number of measurements; please provide the distribution or a confidence interval.","section":"Section 4"}],"recommendation":"reject","confidential_remarks":"The paper's main selling point is the logarithmic complexity reduction, which is not supported by the authors' own algorithmic description: the mandatory kernel FFT in Section 3 and the unaccounted cross-correlation in the PEPITA update rule invalidate the O(n^2) claim. The experimental result is a modest proof of concept on a small MNIST subset, and without the complexity claim the contribution is limited. A resubmission that substantially revises or removes the complexity claim might be considered, but the current central claim is untenable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that it does genuinely show PEPITA can train a 4f optical correlator in the loop to near-BP accuracy on a small MNIST subset, and that is a useful data point for the backprop-free photonic training crowd. But the advertised logarithmic complexity reduction over BP does not hold up when you count the full training step, including the FFT the paper itself says it runs every iteration.\n\nWhat's new and good: To my knowledge this is the first HWL training of a 4f correlator specifically with PEPITA. The experimental setup is standard but the result is credible: five seeds, overlapping error bars (87.6±3 vs 88.8±4), and they are upfront that the dataset is tiny (600/100), the SLM setup is slow (25 ms), and no hyperparameter tuning was done. The SSIM check between software and optical convolutions is a nice sanity check.\n\nWhere it goes soft: The headline complexity claim is the problem. The abstract and Sec. 2 say PEPITA is O(n^2) vs BP O(n^2 log n), but Sec. 3 describes updating the kernel as W' = W - ηΔW and then transforming it with an FFT to load onto SLM2. An FFT of an n×n kernel is O(n^2 log n), so that step alone eats the claimed logarithmic advantage. On top of that, the PEPITA update for a convolutional layer is a cross-correlation between the input and the activation difference; doing that exactly is not free O(n^2), it is O(n^2 log n) via FFT or worse directly. So the O(n^2) figure counts part of the loop and ignores the rest. The paper's own footnote that BP FLOPS exclude dL/dzhw actually makes BP look better, not worse, in the comparison. The accuracy result stands, but the complexity contribution as stated does not.\n\nThere is also the usual reproducibility gap: no code, no data, no exact sampling details, which makes the empirical claim hard to verify. That is addressable but still missing.\n\nVerdict: The paper is worth a serious referee because the empirical setup is new and the flaw is fixable. But the authors should be asked to either provide a full cost accounting for both algorithms or drop the logarithmic reduction claim entirely. As written, the central selling point is not supported.","headline":"A credible first demonstration of PEPITA hardware-in-the-loop training on a 4f correlator, but the logarithmic complexity claim does not survive contact with their own FFT-based kernel update.","tokens_in":4282,"tokens_out":2759,"would_cite":false,"duration_ms":26786,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A forward-only learning rule trains a physical 4f optical correlator on MNIST to 87.6 percent accuracy, close to backpropagation, with a logarithmic complexity reduction.","keywords":["4f optical correlator","hardware-in-the-loop training","forward-only learning","PEPITA","optical neural network","MNIST","convolutional neural network","complexity reduction"],"falsifier":"Measure total wall-clock time and energy for both algorithms on the same optical setup using the full MNIST training set; if PEPITA's per-epoch time is not lower than backpropagation's, the claimed logarithmic complexity reduction does not translate to practice.","tokens_in":3305,"feed_emoji":"🔬","tokens_out":7535,"duration_ms":64470,"temperature":0.7,"pith_summary":"This paper sets out to show that a physical 4f optical correlator can be trained directly, without a differentiable software model of the device, using a forward-only learning rule called PEPITA. On a 600-sample MNIST subset with hardware-in-the-loop training, PEPITA reaches 87.6% test accuracy, essentially matching backpropagation's 88.8% within noise. The authors also claim that PEPITA's parameter update costs $O(n^2)$ arithmetic while backpropagation's costs $O(n^2 \\log n)$, a logarithmic reduction. If correct, this means analog optical hardware that is hard to simulate can still learn on-device from its own forward passes.","feed_headline":"Optical correlator trains at 87.6% accuracy without backprop","feed_subtitle":"Forward-only PEPITA cuts training compute to quadratic scaling and removes the need for a device model.","key_machinery":"The key object is the PEPITA update rule, a forward-only learning rule that modulates the input with the final-layer error, performs a second forward pass, and uses the difference between the two activation maps ($h$ and $h_{\\mathrm{mod}}$) to update the weights locally. In the 4f correlator, convolution is done optically: a lens Fourier-transforms the input, a spatial light modulator multiplies by the kernel, and a second lens inverse-transforms, so the PEPITA update can be formed directly from optical outputs without a differentiable device model. This combination is what removes backpropagation's gradient path and yields the claimed $O(n^2)$ complexity.","core_discovery":"The central discovery is that a forward-only learning algorithm, PEPITA, can train a 4f optical correlator in the loop, using two forward passes through the physical device to compute parameter updates and never computing a gradient through a device model. The paper reports 87.6% test accuracy with PEPITA versus 88.8% with backpropagation on the same hardware-in-the-loop setup, and attributes the gap to limited hyperparameter tuning. It also reports that the PEPITA update has $O(n^2)$ computational complexity compared to $O(n^2 \\log n)$ for backpropagation, because backpropagation needs FFT-based derivatives and error propagation through layers, while PEPITA only combines activations from the two forward passes.","pith_inferences":["A testable extension beyond the paper's claims is to measure wall-clock time end-to-end, since the 25 ms SLM setup time per optical pass could dominate and potentially erase the logarithmic complexity advantage.","The authors do not state this, but PEPITA's model-agnostic two-forward-pass update should transfer to other analog accelerators with drift or noise without re-deriving the learning rule.","The reported accuracy is on only 600 training samples; extending to the full MNIST or higher-resolution inputs would reveal whether the 1.2-point gap and the complexity reduction hold outside that regime."],"forward_implications":["PEPITA eliminates the need for a software gradient model, so hardware imperfections are incorporated into the learned weights during training.","The per-update cost drops from $O(n^2 \\log n)$ to $O(n^2)$, an advantage that grows with image size in Fourier-based convolutional layers.","Forward-only training requires only two forward passes during training and one at inference, making it practical on slow optical hardware.","The near-backpropagation accuracy on a 600-sample subset suggests the approach tolerates the device's optical imperfections, which had an average SSIM of 0.8 against ideal convolution."],"supporting_citations":[{"why":"Defines the PEPITA forward-only learning rule that the paper trains on the optical correlator.","marker":"[6]"},{"why":"Introduces hardware-in-the-loop training of optical neural networks, the method this paper applies to a 4f correlator.","marker":"[3]"},{"why":"Demonstrates massively parallel amplitude-only Fourier neural networks, the optical architecture used here.","marker":"[2]"},{"why":"Reports fully forward-mode training for optical neural networks, a prior forward-only approach this work extends.","marker":"[8]"},{"why":"Provides a memory-efficient PEPITA variant and benchmark that motivates choosing PEPITA over other forward-only algorithms.","marker":"[7]"},{"why":"Shows backpropagation-free training of deep physical neural networks, the broader setting of this result.","marker":"[10]"}],"fun_headline_variants":["Forward-only training matches backprop on optical correlator","PEPITA cuts CNN training complexity on optical correlator","Optical correlator trains without backprop, near backprop accuracy","Hardware-in-the-loop forward-only training achieves 87.6% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim of a logarithmic complexity reduction holds only if the comparison is restricted to update-rule FLOPs; counting the 25 ms optical forward passes and the backpropagation gradient terms the paper omits could shrink or reverse the advantage.","fun_headline_variants_meta":{"raw":{"variants":["Forward-only training matches backprop on optical correlator","PEPITA cuts CNN training complexity on optical correlator","Optical correlator trains without backprop, near backprop accuracy","Hardware-in-the-loop forward-only training achieves 87.6% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001047,"raw_usage":{"total_tokens":4296,"prompt_tokens":739,"completion_tokens":3557,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":355,"completion_tokens_details":{"reasoning_tokens":3485}},"tokens_in":355,"tokens_out":3557,"duration_ms":23547,"temperature":1.0,"reasoning_tokens":3485,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:40:35.492259+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure total wall-clock time and energy for both algorithms on the same optical setup using the full MNIST training set; if PEPITA's per-epoch time is not lower than backpropagation's, the claimed logarithmic complexity reduction does not translate to practice.","supporting_citations":[{"cited_title":"Error-driven input modulation: Solving the credit assignment problem without a backward pass","cited_arxiv_id":null,"evidence_quote":"Defines the PEPITA forward-only learning rule that the paper trains on the optical correlator."},{"cited_title":"Hybrid training of optical neural networks","cited_arxiv_id":null,"evidence_quote":"Introduces hardware-in-the-loop training of optical neural networks, the method this paper applies to a 4f correlator."},{"cited_title":"Massively parallel amplitude-only fourier neural network","cited_arxiv_id":null,"evidence_quote":"Demonstrates massively parallel amplitude-only Fourier neural networks, the optical architecture used here."},{"cited_title":"Fully forward mode training for optical neural networks","cited_arxiv_id":null,"evidence_quote":"Reports fully forward-mode training for optical neural networks, a prior forward-only approach this work extends."},{"cited_title":"Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks","cited_arxiv_id":null,"evidence_quote":"Provides a memory-efficient PEPITA variant and benchmark that motivates choosing PEPITA over other forward-only algorithms."},{"cited_title":"Backpropagation-free training of deep physical neural networks","cited_arxiv_id":null,"evidence_quote":"Shows backpropagation-free training of deep physical neural networks, the broader setting of this result."}],"review_version":1}