{"id":"56b39130-5f39-4088-9cdf-1548b55ff4f6","arxiv_id":"2501.07991","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A digital twin of a multimode fiber, updated online during training, lets gradients pass through a nonlinear optical layer and improves hybrid network accuracy and drift resilience.","lead":"This paper trains a hybrid neural network in which laser pulses propagating through a multimode fiber act as a fixed optical layer, and a separate neural network called the Optical Layer Twin predicts the fiber's output so gradients can flow through the physical layer. The authors show that continuously updating this twin during training improves classification accuracy and resilience to fiber drift, pointing toward low-power optical hardware for AI inference.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Output-level fidelity does not validate the Jacobian approximation that the training algorithm depends on; the paper lacks a direct gradient-alignment check.","rationale":"The reader's weakest assumption correctly identifies the Jacobian approximation as the load-bearing premise. The central claim is not merely that a neural network can mimic the optical layer's input-output map; it is that backpropagating through that network trains the preceding layers, which requires accurate gradient information. The paper supplies strong output-fidelity evidence and a plausible online-refinement argument, and the experimental comparison (online greater than offline greater than no preprocessor) is consistent with the method working. However, all of that evidence is indirect. The one missing measurement is the Jacobian itself. This is the most load-bearing gap because every upstream weight update depends on it. Output-space fidelity is necessary but not sufficient for faithful gradients, and training the OLT by output MSE does not guarantee derivative fidelity. The proposed check is feasible because the experimental setup already allows controlled preprocessor perturbations, and it would directly quantify J_OLT versus J_OS. I agree with the reader's conditional verdict: the method is plausible and internally consistent, but the paper should not be accepted as fully demonstrating the mechanism until this check is provided. The lack of repeated trials and error bars is a secondary concern: even with a correct Jacobian, the small 3-5 point accuracy differences need statistical support, but that is not the central logical vulnerability.","tokens_in":10349,"tokens_out":4391,"duration_ms":47313,"concrete_test":"Perform a direct Jacobian-alignment experiment after training. Select 20 held-out images and 10 random unit-norm directions δθ in preprocessor-weight space. For a small step ε, measure the experimental output change Δy_exp = f_OS(f_p(θ+εδθ)) - f_OS(f_p(θ)) and compare it with the OLT-based prediction Δy_pred = J_OLT(f_p(θ)) · J_fp(θ) · εδθ. Report the cosine similarity and relative L2 error averaged over directions. If mean cosine similarity is below roughly 0.7 while output MAE remains near 1e-2, the Jacobian proxy is not faithful and the central claim needs revision; if it is above roughly 0.9, the concern is settled in the authors' favor. The same check can be repeated after a fiber perturbation to test gradient fidelity under drift.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive step in the algorithm is the substitution J_OLT ≈ J_OS during backpropagation, as stated in Appendix Note 2. The paper's fidelity evidence for the OLT is entirely in output space: normalized MAE 1.03e-2 and visually similar speckle patterns (Figs. 2c, 2d). For a high-dimensional, nonlinear multimode mapping, small output error does not imply small Jacobian error; a U-Net trained by intensity MSE can match speckle statistics while producing gradients that are misaligned with the true physical response. The online refinement procedure also trains the OLT only against output MSE, L_refine = |y_OS - y_OLT|^2, so it does not directly constrain derivatives either. If J_OLT and J_OS disagree substantially, the stated explanation for the accuracy gain, that the preprocessor receives faithful physical gradients, is unsupported. The observed 80% versus 77% versus 75% could then arise from other causes, such as the preprocessing layer adding trainable capacity or acting as a regularizer, rather than from correct gradient flow through the optical system. No gradient-alignment measurement appears anywhere in the manuscript; without one, the central mechanism is asserted rather than demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes training hybrid neural networks that include a physical nonlinear multimode-fiber layer by replacing the optical system with a learned differentiable surrogate, called the Optical Layer Twin (OLT), during backpropagation. The OLT is a U-Net trained on experimental input-output pairs and is either kept fixed (offline learning) or continuously refined with fresh experimental data during training (online learning). On a 1500-sample Fashion-MNIST subset, the authors report 80% test accuracy with online learning, 77% with offline learning, and 75% without a preprocessing layer; they also report OLT fidelity of normalized MAE 1.03e-2 (SNR 96.8). In a separate set of experiments, a mechanical actuator introduces controlled fiber perturbations, and the online learning scheme is reported to improve final accuracy by up to 39% relative to offline learning. The paper also studies how OLT architecture size affects fidelity and computational cost.","tokens_in":10586,"tokens_out":6004,"duration_ms":64623,"significance":"If the central claims hold, the work is a valuable step toward training physical neural networks with complex nonlinear analog layers: the online-refinement idea addresses input-distribution shift and system drift, and the reported OLT inference latency (30 ms) versus an analytical nonlinear-Schrödinger-equation simulation (~500 s for 15 of 240 modes) is a concrete practical advantage. The experimental demonstration is nontrivial, and the OLT output fidelity is high. However, the paper's central mechanism, that the OLT's Jacobian faithfully approximates the optical system's Jacobian, is asserted rather than demonstrated, and the headline accuracy differences are small and appear to come from single runs without error bars. These issues are load-bearing for the main claim and require additional evidence or experiments.","major_comments":[{"comment":"The training algorithm depends on the approximation J_OLT ≈ J_OS, but the manuscript only validates the OLT in output space: normalized MAE 1.03e-2 and visually similar speckle patterns in Figs. 2c and 2d. For a high-dimensional nonlinear multimode mapping, small output error does not imply small Jacobian error, and the refinement loss L_refine = |y_OS - y_OLT|^2 also constrains only outputs. As a result, the explanation that the preprocessor receives faithful physical gradients is not directly supported; the 80% versus 77% versus 75% differences could in principle arise from the added preprocessing capacity or from a regularizing effect. I recommend adding a direct gradient-alignment check, for example comparing u^T J_OLT v with finite-difference or experimentally measured directional derivatives, or an ablation using a deliberately wrong or frozen OLT during backward passes.","section":"Appendix Note 2"},{"comment":"The headline results rest on small margins (80% vs 77% vs 75% test accuracy) and on single training runs. No error bars, repeated seeds, or statistical significance tests are reported anywhere in the manuscript, and Fig. 3c reports the perturbation-rate experiments without indicating the number of independent runs. Given that a 2–3 percentage point gap can arise from initialization or experimental variability, please provide repeated trials with means and standard deviations, and state the number of independent experimental repetitions for the drift experiments.","section":"Fig. 2b and Fig. 3c"},{"comment":"The experimental and training protocols are underspecified for reproducibility. The manuscript does not state how many experimental input-output pairs were used to pretrain the offline OLT, how many refinement steps are taken per batch or per epoch in online learning, the OLT's learning rate and optimizer, or the exact train/test split. The description of the preprocessing block as '6 linear convolutional layers of 1 kernel with 6 × 6 parameters' is ambiguous: it could mean six separate convolution kernels of size 6×6, or six sequential layers each containing one 6×6 kernel. Please clarify these details, since they directly affect the claimed parameter efficiency and the interpretation of the results.","section":"Appendix Note 2, last paragraph"},{"comment":"The claim of 'state-of-the-art image classification accuracies' is not supported by the presented baselines. An accuracy of 80% on a 1500-sample Fashion-MNIST subset is not state-of-the-art in a general sense, and the only comparisons are the offline and no-preprocessor variants of the same hybrid system. Please include an equivalent fully digital network trained on the same data and, if possible, a random-features baseline, so that the benefit of the physical layer and of the OLT gradient mechanism can be isolated from the effect of simply adding trainable capacity.","section":"Abstract and Fig. 2"}],"minor_comments":[{"comment":"Define 'normalized MAE' explicitly and state how SNR is computed from it; as written, the reader cannot verify the reported SNR of 96.8.","section":"Fig. 2c"},{"comment":"The sentence 'the slight increase in error stems directly from slow drifts in the experimental system over time' is confusing for the no-preprocessor condition, since no drift is intentionally induced in Fig. 2; clarify what drift means in that context.","section":"Guiding Model Training with the Optical Layer Twin"},{"comment":"The definition of the nonlinear mode coupling tensor η_p,l,m,n contains a denominator that appears typeset incorrectly (a product of four separate integrals with unclear brackets); please correct the equation.","section":"Appendix Note 1"},{"comment":"The manuscript does not include a data or code availability statement. Given the complexity of the experimental setup and the OLT architecture, a reproducibility statement is necessary.","section":"Methods and Results"},{"comment":"Reference [13] (Wright et al., Nature 2022) is closely related and should be discussed more explicitly in the introduction to clarify the novelty of the present approach relative to that work.","section":"References"},{"comment":"Specify whether the MAE values in Fig. 3b are averaged over all test examples or computed for a single representative input, and state the units.","section":"Fig. 3b"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a good fit for the journal, and the central idea is promising, but the single-run, small-margin results and the lack of any direct Jacobian validation make the main claim conditional. The stress-test concern about output-level fidelity not validating the Jacobian approximation is justified: the approximation J_OLT ≈ J_OS is the load-bearing step, yet no gradient-alignment measurement or control experiment appears. I do not see grounds for rejection, because the deficiency is fixable within the manuscript's scope: a direct gradient-alignment check or an ablation with a deliberately wrong OLT would substantially strengthen the paper. The authors should also provide repeated runs with error bars before the accuracy comparisons can be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper takes the Wright et al. digital-twin idea and adds a genuinely practical twist—continuously updating the surrogate during training so it tracks both input distribution shifts and physical drift. The drift-perturbation experiment with the actuator is the most interesting part, and the OLT architecture scaling study is thorough. The paper is clearly written and properly cites the prior work it builds on.\n\nWhat is new is the online refinement protocol, not the concept of backpropagating through a neural surrogate. The experimental demonstration on a 1500-sample Fashion-MNIST subset is internally consistent: online learning reaches 80% test accuracy vs 77% offline and 75% without preprocessing, and the OLT fidelity numbers are reported (MAE 1.03e-2, SNR 96.8). The comparison between online and offline updating is a fair test of the claimed benefit.\n\nThe soft spots are real but not fatal. First, there are no error bars or repeated runs. The accuracy differences are 3 and 2 percentage points, which could easily be noise on a single trial. Second, no code or data is released, which limits reproducibility. Third, there is no baseline of a purely digital network with comparable capacity, so we don't know if the physical layer actually helps or if the gain is just from the extra trainable parameters in the preprocessor. Fourth, the central mechanism—that the OLT's Jacobian is a faithful proxy for the optical system's Jacobian—is asserted in Appendix Note 2 but never directly tested. The stress-test note is correct: output-level MAE does not guarantee gradient alignment, and the refinement loss is also only in output space. This is a missing measurement rather than a demonstrated contradiction, but it is load-bearing for the explanation of why online learning works. If the gradients are misaligned, the accuracy gain could come from the preprocessor acting as a regularizer or adding capacity.\n\nThe 'state-of-the-art' and 'exceptional resilience' wording in the abstract is also overblown for a single small-dataset experiment.\n\nWho is this for? Researchers in optical computing and anyone using physical layers in neural networks. It deserves a serious referee. The authors should add a gradient-alignment check (e.g., correlation between OLT and experimental VJPs on a few inputs), run multiple trials with error bars, add a digital baseline, and release code/data. With those additions, this could be a solid contribution. As is, it is a promising but incomplete demonstration.\n\nMy recommendation: send to peer review, request those revisions.\n\nBest,","headline":"Online surrogate refinement is a useful twist on the digital-twin idea, but the paper never measures the Jacobian alignment that its training scheme depends on, and the accuracy gains are single-run small numbers.","tokens_in":11153,"tokens_out":3001,"would_cite":true,"duration_ms":28618,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A continuously updated neural twin makes an opaque nonlinear multimode fiber trainable by backpropagation, lifting a 1500-sample Fashion-MNIST task to 80% accuracy and resisting drift.","keywords":["optical neural networks","digital twin","multimode fiber","nonlinear optics","backpropagation","online learning","U-Net","hybrid neural networks"],"falsifier":"On the setup of Fig. 2a, perturb each phase pixel on the spatial light modulator one at a time during online learning, record the camera intensity change, and assemble the resulting sensitivity matrix of the physical system; compare it with the twin's sensitivity matrix on the same inputs. If the two diverge under the input distribution shifts or fiber perturbations where online learning is claimed to help, then the gradient updates to the preprocessor are not the true gradients and the reported accuracy and drift-resilience results would not follow.","tokens_in":10162,"feed_emoji":"⚡","tokens_out":11855,"duration_ms":104766,"temperature":0.7,"pith_summary":"This paper shows that a physical optical system—ultrashort pulse propagation through a multimode fiber—can be embedded in a neural network as a fixed, untrainable layer while the digital layers around it are still trained with ordinary error backpropagation. The enabler is a separate neural network, called the Optical Layer Twin (OLT), that learns the fiber's input–output map and then stands in for the fiber during the backward pass, supplying the gradients the physics does not provide in closed form. Because the fiber's response drifts and because updating the preceding layer changes what the fiber sees, the authors continuously refresh the twin with live experimental data during training, a procedure they call online learning. In their Fashion-MNIST experiment on 1500 training samples, online learning reaches 80% test accuracy, versus 77% for a fixed twin and 75% when the optical layer receives raw images. If the scheme scales, it points toward energy-efficient AI in which large nonlinear physical devices perform the heavy computation and only a small digital portion of the network is trained.","feed_headline":"Reach 80% on Fashion-MNIST with a twin-trained optical layer","feed_subtitle":"A live neural surrogate supplies the fiber's missing gradient, so hybrid optical AI can train end-to-end and resist drift.","key_machinery":"The load-bearing object is the Optical Layer Twin (OLT), a convolutional U-Net that maps the two-dimensional phase pattern written on the spatial light modulator to the two-dimensional intensity pattern measured on the camera. It carries the argument because the backward pass replaces the optical system's unknown Jacobian with the twin's Jacobian, the approximation being $J_{\\mathrm{OLT}} \\approx J_{\\mathrm{OS}}$. Gradients are computed as vector–Jacobian products rather than full Jacobian matrices, since a full matrix for a batch of ten $128\\times128$ images would contain roughly $1.7\\times10^{10}$ elements and exceed 100 GB of memory. The second essential mechanism is online refinement: at each training step the twin is updated on the experimental input–output pairs collected in the forward pass, so it tracks both the evolving input distribution caused by preprocessor weight updates and slow physical drift, which is what keeps the gradient approximation valid.","core_discovery":"The central claim is that a data-driven neural network can approximate both the forward map and the Jacobian of a nonlinear multimode-fiber optical system closely enough that backpropagating through this twin trains the layers on either side of the physical layer, and that continuously refining the twin during training is what keeps it accurate as the input distribution shifts and the hardware drifts. The authors demonstrate this on a three-layer hybrid network whose middle layer is the fiber: a single convolutional preprocessor, the optical system, and a fully connected classifier. Online learning reaches 80% test accuracy on a 1500-sample Fashion-MNIST subset, compared with 77% for an offline, fixed twin and 75% with no preprocessing, while the twin's output predictions reach a normalized mean absolute error of $1.03\\times10^{-2}$ (SNR 96.8). Under deliberately accelerated mechanical drift of the fiber, online learning maintains its advantage, improving final classification accuracy by up to 39% over a fixed twin. The paper also reports that the trained twin predicts optical outputs in about 30 ms on a consumer GPU, whereas a truncated 15-mode numerical simulation of the fiber takes roughly 500 s.","pith_inferences":["If the Jacobian approximation holds beyond this setup, the same twin scheme could in principle stack several physical layers in width and depth, or be applied to other non-differentiable physical systems such as scattering media, waveguides, or mechanical reservoirs; the paper hints at this but does not demonstrate it.","The online refinement loop is essentially an adaptive model of a drifting plant; an untested extension is whether it can also compensate drift during deployed inference, not just during training, by continuing to collect pairs of inputs and outputs.","The twin itself is a 70M-parameter U-Net trained on live data, so the total training compute is not obviously reduced; whether the approach saves energy overall depends on how often and how long the twin must be updated, which the paper does not quantify.","The accuracy comparisons are on a 1500-sample Fashion-MNIST subset; whether the online advantage persists on full-size datasets or more complex tasks is an open empirical question."],"forward_implications":["Adding a differentiable digital twin of the physical layer lets the standard error-backpropagation algorithm train layers that precede and follow an optical system with no analytic gradient; the demonstrated network trains a convolutional preprocessor before the fiber and a classifier after it.","Continuously updating the twin with live experimental data (online learning) is what maintains fidelity as the preprocessor changes what the fiber sees; this is the difference between 80% test accuracy and 77% with a fixed twin.","The same online updating confers resilience to slow physical drift: under mechanically induced fiber perturbation, online learning improves final accuracy by up to 39% relative to a fixed twin.","The neural twin is fast enough to make training practical: 30 ms per prediction on a consumer GPU versus roughly 500 s for a truncated 15-mode numerical simulation of the same fiber.","Because the optical layer contributes large nonlinear transformations without any digitally trainable weights, the trained network can run inference with only the small preprocessor and classifier computed digitally; the physical layer itself consumes no digital operations during inference."],"supporting_citations":[{"why":"It establishes the baseline approach of training physical neural networks with backpropagation through a differentiable model of the physical layer, which the OLT adapts for multimode fibers.","marker":"[13]"},{"why":"It provides the optimum input power level for Kerr-dominated nonlinear propagation in the multimode fiber, which the experimental Methods adopt.","marker":"[18]"},{"why":"It is the Fashion-MNIST dataset used for the 1500-sample classification benchmark and the reported accuracy numbers.","marker":"[26]"},{"why":"It supplies the generalized multimode nonlinear Schrödinger equation solver whose roughly 500-second runtime is compared with the OLT's 30-millisecond inference.","marker":"[27]"},{"why":"It explains the bending-induced loss of higher-order modes that the authors cite to account for reduced task accuracy when the mechanical actuator is added.","marker":"[28]"},{"why":"It is the U-Net architecture on which the Optical Layer Twin is built.","marker":"[29]"},{"why":"It is the source of the multimode pulse-propagation equation used in Appendix Note 1 to describe the fiber's linear and nonlinear mode coupling.","marker":"[32]"}],"fun_headline_variants":["Digital twin backprop trains optical fiber layer to 80% accuracy","Neural twin lets multimode fiber join end-to-end training","Online twin beats offline sim for hybrid optical AI","Train through multimode fiber with a live digital twin"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim stands on the twin's gradients staying close to the real optical system's gradients throughout training, even as the preprocessor changes what the fiber sees and as the fiber is mechanically perturbed; the paper shows the twin's output images match well, but it does not measure the gradient match directly.","fun_headline_variants_meta":{"raw":{"variants":["Digital twin backprop trains optical fiber layer to 80% accuracy","Neural twin lets multimode fiber join end-to-end training","Online twin beats offline sim for hybrid optical AI","Train through multimode fiber with a live digital twin"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2648,"prompt_tokens":950,"completion_tokens":1698,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":1631}},"tokens_in":566,"tokens_out":1698,"duration_ms":13563,"temperature":1.0,"reasoning_tokens":1631,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:29:34.074943+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the setup of Fig. 2a, perturb each phase pixel on the spatial light modulator one at a time during online learning, record the camera intensity change, and assemble the resulting sensitivity matrix of the physical system; compare it with the twin's sensitivity matrix on the same inputs. If the two diverge under the input distribution shifts or fiber perturbations where online learning is claimed to help, then the gradient updates to the preprocessor are not the true gradients and the reported accuracy and drift-resilience results would not follow.","supporting_citations":[{"cited_title":"Kazakov, J","cited_arxiv_id":null,"evidence_quote":"It supplies the generalized multimode nonlinear Schrödinger equation solver whose roughly 500-second runtime is compared with the OLT's 30-millisecond inference."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It explains the bending-induced loss of higher-order modes that the authors cite to account for reduced task accuracy when the mechanical actuator is added."},{"cited_title":"Ronneberger, P","cited_arxiv_id":null,"evidence_quote":"It is the U-Net architecture on which the Optical Layer Twin is built."},{"cited_title":"Mafi, ‘Pulse Propagation in a Short Nonlinear Graded-Index Multimode Optical Fiber’, J","cited_arxiv_id":null,"evidence_quote":"It is the source of the multimode pulse-propagation equation used in Appendix Note 1 to describe the fiber's linear and nonlinear mode coupling."}],"review_version":1}