{"id":"f759f0c0-f437-4d3f-bd0a-07aec778adf6","arxiv_id":"2507.12797","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A transformer encoder forecasts current quench events in ADITYA tokamak discharges up to about 10 ms ahead with recall above 0.9, beating LSTM on the same data.","lead":"Researchers trained a transformer model on diagnostic signals from the ADITYA tokamak to predict the sudden current drops that precede plasma disruptions. The model flagged disruptions up to about 8-10 ms in advance with recall above 0.9, outperforming LSTM baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8–10 ms recall claim is established only on the 725-shot filtered subset; the 682 excluded shots include exactly the slow terminations and very early quenches that a deployed predictor would encounter, so the practical-feasibility claim is unsupported on full ADITYA operations.","rationale":"The reader's weakest assumption correctly identifies the outlier filtering as the critical step. The paper's stated aim is practical early prediction for a short-pulse tokamak, and its headline number is a recall level at a specific lead time. That number is computed after removing shots the authors themselves describe as controlled shutdowns or non-disruptive terminations, plus a lower tail of very early quenches. Excluding these cases biases the task toward a clean two-class separation and removes precisely the ambiguous events that a real-time mitigation system would face. The internal comparison of transformer and LSTM may still be valid, but the central applied claim is conditional on the filtered distribution being representative. Because no code, data, or error bars are provided, the most direct way to settle the concern is to evaluate on the complete shot set or on the excluded shots. This does not weaken the transformer-versus-LSTM ranking on the filtered data; it does mean the headline should be phrased as performance on a curated subset until full-set metrics are shown.","tokens_in":13164,"tokens_out":6747,"duration_ms":79239,"concrete_test":"Re-run the entire pipeline on the full 1407-shot dataset, with fast quenches labeled disruptive and slow terminations labeled non-disruptive (or equivalently report confusion matrices separately for the 682 excluded shots and for the 725 retained shots). Use the same prediction-threshold definition (alarm >0.5 within 8–10 ms before current-quench onset) and a fixed random seed. If recall on the full operational distribution falls below 0.9 or differs materially from the filtered-set recall, the abstract's practical claim must be restricted to the filtered distribution; if it holds, the data-representativeness objection is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §2.1.3 the dataset is cut from 1407 to 725 shots by removing all current-quench times outside ±25% of the median (~72 ms). The paper explicitly says the high-quench-time tail contains non-disruptive terminations and controlled shutdowns; the same trim also removes very early quench events from the lower tail. Section 3.1 then evaluates the transformer/LSTM comparison only on the remaining 725 shots, with a 30% random split. No metric is reported for the 682 excluded shots, either as a held-out distribution or as an additional class. Therefore the central quantitative claim—recall above 0.9 up to 8–10 ms and \"practical feasibility\" for ADITYA—is demonstrated only for the artificially separated two-cluster distribution. The transformer-versus-LSTM ranking on the filtered set may be internally valid, but the load-bearing generalization from filtered evaluation to real operational data is not. The weakest assumption is data representativeness, not the model architecture.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a transformer-encoder model for early prediction of current-quench events in the ADITYA tokamak, using six diagnostic time series from 725 selected shots, and compares its performance against an LSTM baseline. The authors report that the transformer maintains recall above 0.9 for prediction thresholds up to 8–10 ms, with greater stability than LSTM as the threshold increases, and they also study the effect of biasing the training set toward disruptive shots. The claimed contribution is the first transformer-based disruption predictor for ADITYA, with practical feasibility for short-pulse tokamaks.","tokens_in":13354,"tokens_out":2243,"duration_ms":26491,"significance":"If the central claims hold, the paper is a useful incremental contribution to tokamak disruption prediction: it applies a modern transformer architecture to ADITYA data, makes a direct quantitative comparison with LSTM, and explicitly addresses dataset diversity and class-imbalance sensitivity. The use of a larger ADITYA dataset than prior work and the attention to evaluation at multiple prediction thresholds are strengths, as is the clear statement that recall, not just accuracy, is the key operational metric. The main significance depends on whether the filtered dataset and the evaluation protocol support the strong practical-feasibility claim for real ADITYA operations.","major_comments":[{"comment":"The data filtering step removes all shots with current-quench times outside ±25% of the median (~54–90 ms), reducing the dataset from 1407 to 725 shots. The paper itself notes in §2.1.2 that shots with quench times below 65 ms are often disruptive, and §2.1.3 states that high-quench-time events are often non-disruptive terminations or controlled shutdowns. Thus the evaluation in §3.1, which uses only the filtered 725-shot set, excludes both the early disruptive shots and the slow shutdowns that a deployed predictor on ADITYA would encounter. The abstract's claim of recall above 0.9 up to 8–10 ms and the claim of 'practical feasibility' are therefore established only for an artificially separated two-cluster distribution, not for the full distribution of ADITYA discharges. The authors should either report performance on the excluded shots as a separate class or distribution, or substantially qualify the practical-feasibility claim.","section":"§2.1.3, §3.1"},{"comment":"The exact label-encoding rule is never specified. Section 2.2.1 states that the target value 'encodes the proximity to a disruption event,' with values near 0 for normal operation and near 1 for imminent disruption, but no formula or algorithm is given for assigning y_t ∈ {0,1} to each time step, nor how the 80-step alignment and the 0.5 alarm threshold interact across shots of different lengths. Without a precise definition of the labeling rule, the reported precision/recall/accuracy numbers are not reproducible, and the TP/TN/FP/FN definitions in §2.2.4 are ambiguous for shots where quench occurs at intermediate times. The authors should provide the exact labeling procedure, including how the prediction horizon and the alarm threshold are applied in time.","section":"§2.1.3, §2.2.3"},{"comment":"The test set for the threshold study consists of a single 30% random split (123 disruptive and 93 non-disruptive shots), and no confidence intervals, standard deviations, or repeated-seed results are reported. The observed differences between transformer and LSTM, particularly the claimed recall advantage of 8–10 ms, could be sensitive to the particular split and to the small number of non-disruptive test shots. The authors should report error bars or multiple-seed statistics, or at minimum state the number of runs over which the reported curves are averaged.","section":"§3.1"},{"comment":"The abstract states that the transformer maintains recall 'above 0.9 even up to a prediction threshold of 8-10 ms,' but the results section does not report the actual numerical recall values; Figure 10 is the only evidence, and its values are hard to read from the figure as described. The authors should state the exact recall, precision, and accuracy values at thresholds of, say, 5, 8, 10, and 16 ms, so that the quantitative claim in the abstract is directly verifiable.","section":"Abstract and §3.1"}],"minor_comments":[{"comment":"The text says 'This removal is purely mathematical' but also argues that it 'improves the reliability of identifying truly non-disruptive cases'; these two statements sit in tension and should be clarified, since the removal is not neutral with respect to the physical interpretation.","section":"§2.1.3"},{"comment":"The notation in Equation (3) uses y_{t+80} both as the hidden state and as part of the input-output alignment; this is confusing and should be redefined, for example as h_{t+80} for the hidden state.","section":"§2.2.1"},{"comment":"The number 600 time steps is derived from 120 ms at 5 kHz, but the text says 'negative plasma current values are replaced with zero'; if this is a physical correction, it should be justified, and if it is an artifact removal, the fraction of affected shots/time steps should be reported.","section":"§2.1.4"},{"comment":"There are several typographical errors and inconsistencies: 'Covolution' in the introduction, 'ADITY A' with a space throughout, and the diagnostic name 'C-111' versus 'C-III' in different places. These should be corrected.","section":"§1, §2.1.1"},{"comment":"The diversity analysis computes Euclidean distance only on plasma current signals, but the models use six features; the claim that this measures diversity for the full feature space should be softened or supported with a comparison using the full feature set.","section":"§2.1.5"}],"recommendation":"major_revision","confidential_remarks":"The paper's central comparison on the filtered dataset may well be valid, but the practical-feasibility claim is currently overreaching relative to the evidence. The main fix is not architectural but experimental: either evaluate on the excluded shots or clearly reframe the claim as valid only for the selected distribution. The missing label-encoding details also need to be supplied for reproducibility. The paper is within scope for a plasma-physics or applied-ML journal, though the novelty is incremental relative to the existing ADITYA-AU classification work cited by the authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first transformer-based current-quench predictor for ADITYA, and the internal comparison with LSTM on the filtered data looks reasonable. But the practical claim—recall above 0.9 up to 8–10 ms, ready for mitigation—is only demonstrated on the 725 shots left after cutting out half the dataset, including exactly the slow terminations and very early quenches a deployed predictor would meet. On full ADITYA operations, we have no evidence.\n\nWhat's genuinely nice: the application is new, the diversity analysis (Euclidean distances across shots) is a useful diagnostic, and they explicitly discuss the precision/recall trade-off and the effect of biasing the training set. The architecture is standard, but that's fine for a data-driven engineering study. They also compare against LSTM on the same input/output structure, which is the right baseline.\n\nSoft spots, in order of seriousness. First, the outlier trim: removing all quench times outside ±25% of the median (~72 ms) cuts the dataset from 1407 to 725 shots. The paper itself says the high-quench-time tail contains non-disruptive terminations and controlled shutdowns; the lower tail removes very early disruptions. No metric is reported on those excluded shots, either as a held-out set or as a separate class. So the claim that the model offers 'practical feasibility' for ADITYA is unsupported—it's feasibility for a two-cluster subset.\n\nSecond, no error bars anywhere. The test set is 216 shots, and the curves in Fig. 10 have no uncertainty. Given the small test size, the reported recall advantage could be within noise. Third, the label encoding is under-specified: they mention soft labels in the abstract but the text describes binary labels and a 0.5 threshold without explaining the mapping from quench time to per-timestep targets. Fourth, no code or data release, which makes the quantitative claims unverifiable.\n\nNone of these kill the internal comparison. The transformer may well beat LSTM on this filtered, two-cluster distribution. But the headline is stated as a general capability, and that's where the paper overreaches.\n\nWho this is for: researchers working on small tokamak disruption prediction, especially short-pulse machines, and anyone interested in how data filtering can inflate apparent predictive skill. It deserves a serious referee—the application is real and the data-filtering discussion is instructive—but the revision needs to confront the excluded shots and provide uncertainty estimates.","headline":"The transformer-vs-LSTM comparison is plausible on the filtered subset, but the headline 8–10 ms recall claim doesn't survive contact with the excluded half of the data.","tokens_in":13926,"tokens_out":1992,"would_cite":false,"duration_ms":21204,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A transformer model predicts tokamak current quenches 8–10 ms ahead with recall above 0.9, outperforming LSTM on ADITYA data.","keywords":["tokamak disruption prediction","current quench","transformer encoder","LSTM baseline","ADITYA tokamak","multivariate time series","recall","early prediction"],"falsifier":"Apply the same trained models to the original 1407 shots without excluding slow-quench shots and compare recall at an 8–10 ms threshold; a drop below the reported level or below LSTM's would show the result depends on the filtering, while matching performance would support the generalization claim.","tokens_in":12944,"feed_emoji":"⚡","tokens_out":6698,"duration_ms":66241,"temperature":0.7,"pith_summary":"This paper tries to establish that a transformer encoder trained on six ADITYA tokamak diagnostic signals can warn of an upcoming current quench early enough to act. On the filtered dataset, the model keeps recall above 0.9 even when the prediction threshold is 8–10 ms, while the LSTM baseline's recall declines. The authors argue that such lead time is practical for disruption mitigation on short-pulse machines like ADITYA, where the whole discharge lasts about 120 ms. They also report that the transformer stays more stable than LSTM as the warning window widens and across datasets with different disruption-to-nondisruption ratios. This is the first application of transformer architectures to ADITYA data for early current-quench prediction.","feed_headline":"Transformer warns of tokamak quenches 8–10 ms ahead","feed_subtitle":"On ADITYA data it beats LSTM on recall across thresholds and bias levels, leaving time to act on short-pulse tokamaks.","key_machinery":"The load-bearing object is the transformer encoder with self-attention, which lets each time step in the 600-step, six-channel diagnostic sequence weigh its dependence on every other step. The model is trained with binary cross-entropy to output a disruption probability for a time 80 steps (16 ms) ahead, and an alarm triggers when the probability exceeds 0.5. The comparison is made against an LSTM with the same input structure, alarm rule, and training procedure.","core_discovery":"The central claim is that transformer-based models are better than LSTM models at early current-quench prediction on ADITYA tokamak data. Using six diagnostic signals (plasma current, soft and hard X-rays, bolometer, H-alpha, and C-III), the transformer achieves recall above 0.9 up to a prediction threshold of 8–10 ms, whereas LSTM recall degrades faster with threshold. The transformer also maintains higher accuracy and more balanced precision–recall performance across naturally balanced and bias-adjusted training datasets, which the authors take as evidence of reliable generalization.","pith_inferences":["If attention weights are examined, the six selected diagnostics may reveal which signals carry the earliest precursor information, but the paper leaves that interpretability step to future work.","The 8–10 ms recall result was obtained after removing shots with quench times outside ±25% of the median; on an unfiltered operational dataset that includes slow decays and controlled shutdowns, recall and precision may differ.","The paper's suggested 5–10 ms mitigation window is based on an electromagnetic particle injector study; if faster actuators become available, the model's useful threshold could extend further."],"forward_implications":["A practical real-time current-quench warning on ADITYA is feasible with a lead time of about 8–10 ms, which the paper argues is enough to trigger mitigation on a 120 ms discharge.","Because the transformer keeps recall above 0.9 while LSTM recall declines, fewer actual disruptions would be missed at longer warning windows.","The model's stability across biased training sets suggests it can be deployed when operational conditions make disruptive and non-disruptive shots imbalanced.","The same architecture could be a starting point for real-time disruption-avoidance systems on other short-pulse tokamaks, though the paper flags that confirmation on long-duration tokamaks is still needed."],"supporting_citations":[{"why":"Supplies the transformer encoder architecture with self-attention that the paper adapts for time-series current-quench prediction.","marker":"[53]"},{"why":"Provides the previous LSTM-based disruption prediction study on ADITYA used as the baseline and as a validation reference.","marker":"[42]"},{"why":"Correlation analysis of ADITYA diagnostic signals that selects the six model input features.","marker":"[52]"},{"why":"Electromagnetic particle injector study cited to justify that a 5–10 ms prediction threshold is sufficient for disruption mitigation.","marker":"[10]"},{"why":"Characterizes plasma current quench during ADITYA disruptions and supports the labeling of disruptive versus non-disruptive shots.","marker":"[36]"},{"why":"Real-time machine-learning disruption predictor that motivated the emphasis on high recall for tokamak safety.","marker":"[17]"}],"fun_headline_variants":["Transformer beats LSTM for ADITYA tokamak quench prediction","Transformer gives 8-10 ms quench warning on ADITYA, beats LSTM","First transformer model predicts ADITYA current quenches early","ADITYA quench early warning: Transformer beats LSTM at 8 ms lead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results rely on treating shots whose current quench times fall outside the median by more than 25% as outliers and removing them, but real ADITYA operation includes slow decays and controlled shutdowns, so performance on the filtered set may not carry over to unfiltered operation.","fun_headline_variants_meta":{"raw":{"variants":["Transformer beats LSTM for ADITYA tokamak quench prediction","Transformer gives 8-10 ms quench warning on ADITYA, beats LSTM","First transformer model predicts ADITYA current quenches early","ADITYA quench early warning: Transformer beats LSTM at 8 ms lead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001197,"raw_usage":{"total_tokens":4907,"prompt_tokens":885,"completion_tokens":4022,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":3938}},"tokens_in":501,"tokens_out":4022,"duration_ms":30104,"temperature":1.0,"reasoning_tokens":3938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:38:58.963335+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the same trained models to the original 1407 shots without excluding slow-quench shots and compare recall at an 8–10 ms threshold; a drop below the reported level or below LSTM's would show the result depends on the filtering, while matching performance would support the generalization claim.","supporting_citations":[{"cited_title":"Vaswani, N","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer encoder architecture with self-attention that the paper adapts for time-series current-quench prediction."},{"cited_title":"Agarwal, et al., Deep sequence to sequence learning- based prediction of major disruptions in ADITY A toka- mak, Plasma Physics and Controlled Fusion 63 (7) (2021) 075006","cited_arxiv_id":null,"evidence_quote":"Provides the previous LSTM-based disruption prediction study on ADITYA used as the baseline and as a validation reference."},{"cited_title":"Agarwal, B","cited_arxiv_id":null,"evidence_quote":"Correlation analysis of ADITYA diagnostic signals that selects the six model input features."},{"cited_title":"Raman, W","cited_arxiv_id":null,"evidence_quote":"Electromagnetic particle injector study cited to justify that a 5–10 ms prediction threshold is sufficient for disruption mitigation."},{"cited_title":"Purohit, et al., Characterization of the plasma current quench during disruptions in ADITY A tokamak, Nuclear Fusion 60 (7) (2020) 076018","cited_arxiv_id":null,"evidence_quote":"Characterizes plasma current quench during ADITYA disruptions and supports the labeling of disruptive versus non-disruptive shots."},{"cited_title":"Rea, et al., A real-time machine learning-based dis- ruption predictor in DIII-D, Nuclear Fusion 59 (9) (2019) 096016","cited_arxiv_id":null,"evidence_quote":"Real-time machine-learning disruption predictor that motivated the emphasis on high recall for tokamak safety."}],"review_version":1}