{"id":"d7945247-591a-4679-b600-99b33908af04","arxiv_id":"2510.24347","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A physics-gated neural network predicts MARFE worsening 40 ms ahead on HL-3 with claimed AUC 0.969, but the target labels are constructed from the same cleaned visual features the model uses.","lead":"A new system predicts a plasma instability called a MARFE about 40 milliseconds before it worsens on the HL-3 tokamak, using camera images plus plasma measurements. The authors report high accuracy and a 1-millisecond control-loop service, but the evaluation relies on labels the system built from its own inputs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported AUC/F1 may measure self-consistency: the worsening label in Eq. (8) is a thresholded future jump of the same cleaned features the model consumes, so independent MARFE ground truth is needed before the 40 ms warning claim is established.","rationale":"Agree with the reader's weakest assumption: the worsening label is self-referential. The model could be a high-quality predictor of its own cleaned feature trajectory, which is useful for control only if that feature is a reliable MARFE proxy, but no evidence establishes that proxy independently of the physics variables that also drive the gate. The intrinsic validation in §3.1 cannot break the circle because it uses the same EM/physics assumptions. The abstract/body inconsistencies are secondary but make protocol uncertainty concrete. Since the concern is about external validity rather than internal contradiction, the appropriate verdict remains conditional pending independent label validation. I do not recommend rejection: the deployment timing, component ablation, and qualitative shot evidence are real contributions, and the self-consistency hypothesis is testable.","tokens_in":17144,"tokens_out":4782,"duration_ms":44580,"concrete_test":"Choose 30–50 held-out shots with possible MARFE activity. Have at least two expert annotators independently mark MARFE onset/worsening frames from raw visible-light videos, or use bolometric/spectroscopic reconstructions as an objective reference. Recompute the 40 ms-ahead worsening label b_j(t) from this independent reference instead of from m'_j, and evaluate the frozen deployed model (and, if needed, retrain under the identical protocol). If AUC remains near 0.95 and median lead time near +36 ms, the self-consistency concern is resolved. If AUC drops materially or lead time becomes non-positive, the reported metrics largely reflect prediction of the cleaned feature's own future jump. Also report exact shot IDs and metrics for the ~70-shot test set implied by the 8:1:1 split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—40 ms-ahead MARFE worsening warning with AUC 0.969/F1 0.862—rests on the worsening label b_j(t) defined in Eq. (8) being a faithful proxy for real MARFE onset. It is not an independent ground truth. b_j(t) is 1 when the future 40 ms jump Δm'_j(t) in the cleaned visual feature m'_j exceeds a threshold θ_j (with a slope or c·θ_j alternative), and m'_j (Eq. (7)) is exactly the model input after EM-based zeroing: m'_j = m_j if the refined label ŷ_i = 1, and 0 otherwise. The EM refinement itself is driven by a Gaussian mixture over [n_e, T_e, f_G, t] with the physics score s_i as prior (Eqs. (2)–(3)); those same plasma variables are model inputs (Table 2) and enter the physics gate g(f_G,T_e) in Eq. (10). The target is therefore a deterministic transform, up to fitted thresholds, of information the model already sees. A high AUC then shows the model can anticipate a jump in its own cleaned input, not necessarily that it anticipates an externally verified MARFE. The paper states this explicitly in §3.1: 'Due to the lack of a comprehensive manually-labeled ground truth...' and the validation offered is qualitative (corrections cluster early in discharges; Shot #11522). The numeric inconsistencies between the abstract (AUC 0.981, F1 0.840, 140-shot test set) and the body (AUC 0.969, F1 0.862, 8:1:1 split of 701 shots ≈ 70 test shots) further show the reported evaluation is not stable enough to rule out protocol dependence. These points do not disprove the system, but they make the central claim conditional until an independent MARFE reference is used.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a real-time MARFE prediction system for the HL-3 tokamak. The authors propose a three-stage label refinement pipeline that converts noisy CCD-camera area features into cleaned features using a physics-scored EM algorithm, and then train a physics-gated Neural ODE to predict a 40 ms-ahead 'MARFE worsening' label per poloidal zone. The model is evaluated on a held-out shot split with reported AUC 0.969 or 0.981 (abstract and body disagree), and the deployed TensorRT service is claimed to run within a 1 ms control-cycle budget. The central claim is that the system provides an actionable 40 ms warning of MARFE worsening, with a median lead time of +36 ms reported in the abstract.","tokens_in":17715,"tokens_out":5628,"duration_ms":45528,"significance":"The paper addresses a practically important problem—real-time MARFE warning for tokamak control—and the engineering contributions are real: a deployed TensorRT service with sub-millisecond inference, a physics-gated Neural ODE that naturally treats the 40 ms horizon as continuous-time evolution, and a large HL-3 dataset. The EM-based label refinement is a thoughtful attempt to suppress visual artifacts, and the paper honestly acknowledges in §3.1 the absence of a comprehensive manually-labeled ground truth. If the label circularity is resolved—for example by independent validation against bolometry or expert labels—the approach would be a useful addition to the disruption-avoidance toolbox. However, as it stands, the headline AUC and lead-time numbers cannot be distinguished from self-consistency of the labeling pipeline, and the abstract/body inconsistencies undermine confidence in the exact performance claims.","major_comments":[{"comment":"The central claim of 40 ms-ahead MARFE prediction is not supported by an independent ground truth. The worsening label b_j(t) in Eq. (8) is defined as a thresholded future jump of the cleaned features m'_j, which are exactly the model inputs after EM-based zeroing in Eq. (7). Those cleaned features are zeroed whenever the EM refinement (Eqs. (2)–(6)), driven by the same plasma variables n_e, T_e, f_G used in the physics gate g(f_G,T_e) of Eq. (10), decides that no MARFE is present. Consequently, the reported AUC/F1 measure the model's ability to anticipate a jump in its own cleaned input, and the abstract's claim of a '+36 ms median label-aligned lead time' is a statement about self-consistency of the labeling pipeline rather than verified predictive skill. The paper explicitly acknowledges in §3.1 the absence of a comprehensive manually-labeled ground truth and offers only qualitative checks (correction times, Shot #11522). To establish the headline claim, the authors should either (i) validate the refined labels against an independent diagnostic (e.g., bolometer/radiated-power signals or expert-labeled frames for a subset of shots), or (ii) reframe the claims as predicting the refined pseudo-label and add a control baseline that directly thresholds the slope or future jump of m'_j to show the Neural ODE adds skill beyond trivial extrapolation.","section":"§2.3, §2.4.2, §3.1"},{"comment":"The numerical reporting is internally inconsistent. The abstract reports AUC=0.981, F1=0.840 on a 'frozen 140-shot held-out test set', while §3.2.1 and Figure 8 report AUC=0.969 (and the text also mentions 0.959), Table 3 reports AUC=0.9685 and F1=0.8618, and §3 describes an 8:1:1 split of 701 shots, which implies roughly 70 test shots rather than 140. The ablation text gives 'without refined labels' AUC=0.868 in one place and 0.908 in the figure caption. These discrepancies make it impossible to identify the official evaluation protocol or the size of the test set, and they must be resolved before the reported performance can be considered reproducible.","section":"Abstract vs. §3.2.1, Table 3, Figure 8"},{"comment":"Even if the label circularity were accepted as a deliberate soft-target design, the paper does not compare against a trivial predictor that uses the same 40 ms future jump directly. Since b_j(t) is, up to fitted thresholds and a slope condition, essentially an indicator that m'_j will increase by more than θ_j over the next 40 ms, a baseline that thresholds the recent slope or the current value of m'_j could achieve high AUC without any learned dynamics. Reporting such a baseline would clarify whether the Neural ODE's continuous-time modeling (and the physics gate) adds genuine predictive information beyond the label construction.","section":"§2.4.2"},{"comment":"The abstract's claim of a 'median label-aligned lead time of +36 ms' is not supported by any systematic analysis in the body. The only lead-time evidence is a single illustrative shot (#6255) with '>20 ms' lead time. If the median lead time is computed over the test set, the corresponding distribution (or at least the median and an interval) should be reported in the main text; otherwise the abstract's number is unverifiable.","section":"§3.2.1"}],"minor_comments":[{"comment":"The uncertainty weights are written as σ_j in the equation but σ_i in the surrounding text; please unify the notation.","section":"§2.4.4, Eq. (11)"},{"comment":"In Table 5, the EM feature vector is listed as [n_e, T_e, q_95, t]_i, while Eqs. (2)–(3) and the text define x_i = [n_e, T_e, f_G, t]_i. Please correct the inconsistency.","section":"Table 5 vs. §2.3"},{"comment":"The figure caption appears after a corrupted Unicode dump in the manuscript; the figure rendering must be fixed so the reader can actually see the time distribution of corrections.","section":"Figure 6"},{"comment":"The observed ranges for n_e and T_e are given as [−3, 15] and [−1, 13]; negative values are physically suspicious and the units or offset should be clarified or corrected.","section":"§2.1"},{"comment":"Please provide the hyperparameters of the sequence encoder, f_θ, f_ϕ, the ODE solver settings, and the training schedule; without these, the ablation and deployment results cannot be reproduced by the community.","section":"§3.2.1"},{"comment":"The terms 'physics-gated' and 'physics-constrained' are used interchangeably; pick one term and use it consistently.","section":"Throughout"},{"comment":"The sentence 'This work lays a very foundation' contains a typo and should read 'lays a foundation'.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":"The paper is interesting and the deployment effort is real, but the lack of independent ground truth and the inconsistent headline numbers are serious. The authors can likely fix the issues by adding external validation or clearly reframing the claims as predictions of a refined pseudo-label, and by aligning the abstract with the body. Without these changes, the reported AUC and lead time may measure self-consistency rather than predictive skill, so I cannot recommend acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a serious engineering paper and the first deployed MARFE predictor on HL-3 that I know of. The authors combine EM-based label refinement with a physics-gated Neural ODE and run it in real time within the control cycle. That is real work, and the paper is honest about a key limitation: they lack comprehensive manually labeled ground truth, so they build labels from physics priors and cleaned visual features. On its own terms, the system is coherent and the low-FPR behavior is the right thing to optimize for control.\n\nThe main soft spot is exactly where the stress-test lands. The worsening label b_j(t) is a thresholded future jump of the cleaned features m'_j, which are also model inputs, and the cleaning was driven by the same n_e, T_e, f_G used in the gate. So a high AUC can just mean the model learns to anticipate a jump in its own input. That does not disprove the system, but it means the reported AUC 0.969 and F1 0.862 are not yet evidence of predictive skill against real MARFE onset. The paper acknowledges this in §3.1 and offers qualitative validation—corrections cluster early, one good shot example—but that is not a substitute for a quantitative independent reference.\n\nThere are also numeric inconsistencies that need fixing before this is citable: the abstract reports AUC 0.981, F1 0.840, and a 140-shot held-out set, while the body says AUC 0.969 (or 0.9685), F1 0.862, and an 8:1:1 split of 701 shots (roughly 70 test shots). The gate gains k_n and k_T in Eq. (10) are never specified, which makes the physics gate hard to reproduce.\n\nIf I were the editor, I would send this to peer review—the deployment and the label-refinement pipeline are worth referee attention—but I would ask for consistent numbers, the fitted thresholds, a sensitivity analysis on the EM and labeling parameters, and ideally a comparison against bolometer or expert-labeled events. The central claim is plausible but conditional. See it as an engineering contribution with a validation gap, not as a demonstrated predictive model.\n\nI would bring it to a reading group focused on fusion control or machine learning for diagnostics, and I'd cite it as evidence that such systems are being deployed, with the caveat about label self-consistency.","headline":"A real deployed MARFE warning system on HL-3, but the headline numbers measure anticipation of the model's own cleaned video features, not independently verified MARFE onset.","tokens_in":18205,"tokens_out":1639,"would_cite":true,"duration_ms":15996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["52.55.Fa"],"model":"deepseek-v4-flash","headline":"A physics-gated Neural ODE can warn of MARFE worsening 40 ms ahead on HL-3, with reported AUC 0.969 and a 1 ms runtime.","keywords":["MARFE","disruption prediction","tokamak","HL-3","Neural ODE","physics-informed machine learning","visible-light imaging","expectation-maximization"],"falsifier":"Take the trained model and its EM label pipeline to a set of shots in which MARFE onset is marked independently—by high-field-edge peaking in bolometric radiated power, by MARFE-zone spectroscopic line emission, or by expert review of raw video—and require that prediction threshold crossings lead those markers by at least 40 ms at the claimed false-positive rate. If the aligned lead time does not reproduce, the AUC mostly certifies label self-consistency, not prediction of MARFE.","tokens_in":16904,"feed_emoji":"⚡","tokens_out":16232,"duration_ms":128356,"temperature":0.7,"pith_summary":"The paper sets out to show that a tokamak can be given a reliable, real-time warning that a MARFE—the edge radiation instability that often precedes density-limit disruptions—is about to worsen, using only a visible-light camera and routinely available plasma signals. Its pipeline first cleans noisy camera-derived MARFE features with a weighted expectation-maximization step whose prior is a physics score built from $n_e$, $T_e$, $f_G$, and time, then feeds the cleaned features into a continuous-time Neural ODE whose dynamics are gated by $f_G$ and $T_e$ and integrated 40 ms forward. On a shot-level held-out set the paper reports an Area Under the Curve (AUC) of 0.969 in the experimental section (0.981 in the abstract) and an $F_1$ score of 0.862, and the optimized deployed inference service runs within the 1 ms control-cycle budget. If these claims hold, the system offers a lead time long enough for shape-control setpoints to be computed before a MARFE worsens, which is what proactive mitigation requires.","feed_headline":"Physics-gated model flags MARFE 40 ms before worsening on HL-3","feed_subtitle":"Camera diagnostics cleaned by physics and a continuous-time ODE warn controllers within the 1 ms loop.","key_machinery":"The load-bearing machinery is the pair formed by the physics-scored EM label refiner and the physics-gated Neural ODE, where the ODE is a neural network that parameterizes the derivative of a latent state. In the refiner, a physics consistency score $s_i$ is computed from indicator thresholds on $n_e$, $T_e$, $f_G$, and discharge time, and a weighted EM algorithm fits a two-component Gaussian mixture with $s_i$ as a Bayesian prior; the converged responsibilities $\\gamma(z_i,\\mathrm{pos})$ become the refined label $\\hat{y}_i$, which is then used to zero out inconsistent visual features. The predictor evolves a latent state $h(t)$ by $dh/dt=f_\\theta(h)+g(f_G,T_e)\\,f_\\phi(h)$, with $g$ a sigmoid gate activating as $f_G$ exceeds $f_G^{\\mathrm{thr}}$ and $T_e$ falls below $T_e^{\\mathrm{thr}}$; integrating from $t_0$ to $t_0+40\\,\\mathrm{ms}$ yields the future latent state decoded into three zone-wise worsening probabilities. The 40 ms horizon is matched to the actuator-response budget, and the deployment fits the 1 ms control cycle.","core_discovery":"The central claim is that MARFE worsening on HL-3 is predictable 40 ms in advance by evolving a learned latent state in continuous time, with extra dynamics switched on by a sigmoid physics gate $g(f_G,T_e)=\\sigma\\big(k_n(f_G-f_G^{\\mathrm{thr}})-k_T(T_e-T_e^{\\mathrm{thr}})\\big)$ as the plasma approaches the MARFE-prone region. The training labels are not raw camera thresholds: an EM algorithm over a two-component Gaussian mixture, using each sample's physics score $s_i$ as a sample-specific prior, refines the noisy binary labels and then zeros out the visual features the EM judges inconsistent. The full model achieves AUC 0.969 and $F_1=0.862$; ablations show training without refined labels drops AUC to 0.908, replacing the ODE with a Bi-LSTM drops it to 0.935, and removing the gate leaves global AUC almost unchanged but reduces the true-positive rate at low false-positive rates. The paper further claims that the deployed service completes the whole pipeline in about 1 ms, matching the HL-3 control cycle, and outputs six shape-target quantities for the controller.","pith_inferences":["Because the worsening label is defined as a thresholded future jump in the same cleaned visual features the model consumes, the reported lead time measures internal label self-consistency unless the refined labels are independently confirmed; a bolometer- or spectroscopy-based MARFE-onset set would settle that.","The gate thresholds and score weights are fitted to HL-3 training data, so porting the architecture to another tokamak would require re-estimating them; the claim of improved generalization is therefore plausible but device-specific.","The paper stops at open-loop deployment; the practical payoff would be demonstrated by closed-loop tests in which the six shape targets are fed to the plasma control system and MARFE onset is suppressed or delayed relative to the open-loop baseline.","Because $f_G$ is computed from core rather than line-averaged density, its values sit above the usual density-limit-normalized literature values; cross-device comparisons should use a consistent density definition before treating a given $f_G$ threshold as transferable."],"forward_implications":["A 40 ms-ahead warning is compatible with the HL-3 control cycle: the shape-target calculation completes before the worsening interval begins, so the warning is actionable rather than retrospective.","In the low-false-alarm region that controllers demand, the physics gate provides a higher true-positive rate than the ungated model at the same false-positive rate, meaning fewer spurious interventions.","Training on EM-refined labels rather than raw camera thresholds raises the reported AUC from 0.908 to 0.969, so the label-refinement step, not the ODE alone, is responsible for much of the accuracy.","Because the core is a continuous-time ODE, the same trained model can be queried at horizons other than 40 ms without retraining, which would allow the warning horizon to be tuned to a device's actuator response.","The deployed pipeline logs raw indicators and probabilities into the HL-3 data systems, so the predictor doubles as a diagnostic archive for post-shot MARFE studies."],"supporting_citations":[{"why":"It defines MARFE as an edge plasma phenomenon and documents its link to density-limit disruptions, the target the predictor warns about.","marker":"[3]"},{"why":"It establishes the density limit used to define $f_G$, the normalized density that enters both the physics score and the ODE gate.","marker":"[4]"},{"why":"It reports TEXTOR-94 experiments showing MARFE can be suppressed by plasma displacement or gas-puff feedback, giving the actuator rationale for a 40 ms warning.","marker":"[6]"},{"why":"It provides the original EM algorithm, the maximum-likelihood machinery the label-refinement pipeline adapts with physics priors.","marker":"[24]"},{"why":"It gives automatic MARFE identification from JET visible camera videos, the visual-feature approach this work extends and stabilizes.","marker":"[13]"},{"why":"It demonstrates machine-learning MARFE detection in JET operational videos, the domain context for learned visual detection.","marker":"[16]"},{"why":"It introduces the Neural ODE formulation and adjoint sensitivity method, the continuous-time backbone the physics gate modulates.","marker":"[30]"},{"why":"It supplies continuous-time modeling of irregularly sampled time series with ODE-based hidden dynamics, supporting the gated continuous-time design.","marker":"[38]"},{"why":"It provides the uncertainty-weighted multi-task loss used to balance the three zone prediction tasks.","marker":"[39]"},{"why":"It shows deep-learning disruption prediction in tokamaks, broader evidence that learned predictors can anticipate instabilities.","marker":"[18]"}],"fun_headline_variants":["MARFE flagged 40 ms early by physics-gated neural ODE","Physics-gated ODE predicts MARFE 40 ms ahead on HL-3","Neural ODE with physics gate catches MARFE 40 ms earlier","Physics-scored EM and neural ODE spot MARFE 40 ms early","40 ms early MARFE alarm from physics-gated deep learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the EM-refined 'worsening' labels really correspond to MARFE onset; since those labels are thresholded future jumps in the same cleaned camera features the model is trained on, and the cleaning is driven by the same physics measurements that gate the model, the reported lead time and accuracy could reflect self-consistency rather than predictive skill unless the labels are independently verified.","fun_headline_variants_meta":{"raw":{"variants":["MARFE flagged 40 ms early by physics-gated neural ODE","Physics-gated ODE predicts MARFE 40 ms ahead on HL-3","Neural ODE with physics gate catches MARFE 40 ms earlier","Physics-scored EM and neural ODE spot MARFE 40 ms early","40 ms early MARFE alarm from physics-gated deep learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000784,"raw_usage":{"total_tokens":3543,"prompt_tokens":1108,"completion_tokens":2435,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":724,"completion_tokens_details":{"reasoning_tokens":2338}},"tokens_in":724,"tokens_out":2435,"duration_ms":15534,"temperature":1.0,"reasoning_tokens":2338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:41:00.936380+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained model and its EM label pipeline to a set of shots in which MARFE onset is marked independently—by high-field-edge peaking in bolometric radiated power, by MARFE-zone spectroscopic line emission, or by expert review of raw video—and require that prediction threshold crossings lead those markers by at least 40 ms at the claimed false-positive rate. If the aligned lead time does not reproduce, the AUC mostly certifies label self-consistency, not prediction of MARFE.","supporting_citations":[{"cited_title":"MARFE: An edge plasma phenomenon.Nuclear Fusion, 24(8):977, 1984","cited_arxiv_id":null,"evidence_quote":"It defines MARFE as an edge plasma phenomenon and documents its link to density-limit disruptions, the target the predictor warns about."},{"cited_title":"A new look at density limits in tokamaks.Nuclear Fusion, 28(12):2199, 1988","cited_arxiv_id":null,"evidence_quote":"It establishes the density limit used to define $f_G$, the normalized density that enters both the physics score and the ODE gate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It reports TEXTOR-94 experiments showing MARFE can be suppressed by plasma displacement or gas-puff feedback, giving the actuator rationale for a 40 ms warning."},{"cited_title":"Maximum likelihood from incomplete data via the EM algorithm.Journal of the royal statistical society: series B (methodological), 39(1):1– 22, 1977","cited_arxiv_id":null,"evidence_quote":"It provides the original EM algorithm, the maximum-likelihood machinery the label-refinement pipeline adapts with physics priors."},{"cited_title":"Algorithms for the automatic identification of MARFEs and UFOs in JET database of visible camera videos.IEEE Transactions on Plasma Science, 38(12):3409–3418, 2010","cited_arxiv_id":null,"evidence_quote":"It gives automatic MARFE identification from JET visible camera videos, the visual-feature approach this work extends and stabilizes."},{"cited_title":"Gonz ´alez Ganz´abal, G","cited_arxiv_id":null,"evidence_quote":"It demonstrates machine-learning MARFE detection in JET operational videos, the domain context for learned visual detection."},{"cited_title":"Neural ordinary differential equations","cited_arxiv_id":null,"evidence_quote":"It introduces the Neural ODE formulation and adjoint sensitivity method, the continuous-time backbone the physics gate modulates."},{"cited_title":"Gru-ode-bayes: Continuous modeling of sporadically-observed time series.Advances in Neural Information Processing Systems, 32, 2019","cited_arxiv_id":null,"evidence_quote":"It supplies continuous-time modeling of irregularly sampled time series with ODE-based hidden dynamics, supporting the gated continuous-time design."},{"cited_title":"Multi-task learning using uncertainty to weigh losses for scene geometry and semantics","cited_arxiv_id":null,"evidence_quote":"It provides the uncertainty-weighted multi-task loss used to balance the three zone prediction tasks."},{"cited_title":"Predicting disruptive instabilities in controlled fusion plasmas through deep learning.Nature, 568(7753):526–531, 2019","cited_arxiv_id":null,"evidence_quote":"It shows deep-learning disruption prediction in tokamaks, broader evidence that learned predictors can anticipate instabilities."}],"review_version":2}