{"id":"2c091c17-6860-436d-a15d-3e933a410b22","arxiv_id":"2607.14190","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A transformer-based discrete-time survival model plus Cox regression predicts ALS domain decline and wheelchair timing from ALSFRS-R visits, but discrimination is near chance (mean C-index 0.549 on 28 test patients) and the wheelchair-survival model itself is never evaluated.","lead":"Using repeated ALSFRS-R clinic scores, the authors built a machine-learning pipeline that predicts ALS functional decline and time to wheelchair access and released it as an interactive web app. Predictive discrimination is modest (mean C-index about 0.55) and there is no external validation, so the tool is a prototype rather than a proven clinical aid.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All wheelchair-timing results (Table 4, Fig. 7) come from a 187/1,639-patient subset after device-log merge, with no attrition analysis; selection bias could invalidate the central claim.","rationale":"The paper is a competent, clearly written workshop-scale study with real assets: the ASTP stage-transition model is honestly evaluated on 28 held-out patients with a mean C-index of 0.549, the code and app are publicly available, and several limitations are acknowledged. However, the central claim in the abstract and the app's headline output is the wheelchair-free survival curve. That output is powered entirely by the Cox model in Section 2.5, which is fit on the 187 patients who survived the device-log merge. This is exactly the kind of data assumption that, if wrong, invalidates the result no matter how sophisticated the model architecture is. The paper's own Table 2 shows the authors know how to evaluate a time-to-event model, yet no such evaluation is reported for the wheelchair model; this makes the cohort-selection issue even more critical, because there is no internal check to fall back on. The reader's verdict of CONDITIONAL is appropriate: the framework is a plausible prototype, but the predictive and associative claims about wheelchair timing are not yet defensible. Our read does not move the verdict; it reinforces the need for the conditions the reader already stated, most importantly a transparent treatment of the 88% attrition.","tokens_in":15403,"tokens_out":8804,"duration_ms":87057,"concrete_test":"Compute baseline characteristics and 12-month ALSFRS-R slopes for the 187 included vs the 1,452 excluded patients (age, sex, phenotype, ALSFRS-R total and items, follow-up duration). Then re-fit the Table 4 Cox model using inverse-probability-of-inclusion weights (or multiple imputation for missing device-log indicators) and compare the ALSFRS8/9 coefficients and confidence intervals. If the coefficients shift materially or lose the reported p<0.005 significance, the lower-limb/wheelchair conclusion is a selection artifact rather than a clinical signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 reports that after merging assistive-device logs and demographic data the analytic cohort collapses to 1,409 cases from 187 patients, from 1,639 linked patients. Every coefficient in Table 4 and every wheelchair-free survival curve in Figure 7 is estimated inside this 11% subset of patients who happen to have device-log records. The paper never compares these 187 patients with the 1,452 excluded, nor specifies how excluded patients would be censored. If device-log availability correlates with disease severity, care access, or follow-up intensity, the Cox event times are selected not at random, and both the lower-limb coefficients (ALSFRS8/9) and the app's wheelchair-free survival output are biased regardless of model architecture. This is a structural dataset assumption, not a modeling detail: it precedes every downstream result and is acknowledged only generically in the Discussion ('observational data ... selection bias') without addressing the specific 88% attrition. A secondary but compounding issue is that, unlike the ASTP stage-transition model (Table 2, C-index 0.549 on 28 held-out patients), the Cox wheelchair model is never evaluated for discrimination or calibration, so even within the selected cohort the claim that the survival curves are informative is unsupported. The primary load-bearing risk remains the unanalyzed cohort selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a time-to-event framework for ALS that combines correlation-based clustering of ALSFRS-R items, GAMM-based longitudinal modeling, an attention-based discrete-time survival model (ASTP) for per-domain stage transitions, and a Cox proportional hazards model for time to wheelchair use. The results are deployed in a Streamlit app that outputs personalized wheelchair-free survival curves. The main claims are that lower limb function (walking and stair climbing) is the strongest predictor of earlier wheelchair use and that the framework provides dynamically updated, individualized survival predictions.","tokens_in":15619,"tokens_out":2026,"duration_ms":22346,"significance":"If valid, the framework would offer a clinically interpretable tool for anticipating assistive-device needs in ALS, with potential utility in care planning and trial stratification. The paper has genuine strengths: a publicly available app and code repository, honest reporting of a held-out test set for the ASTP model, and explicit acknowledgment of several limitations. However, the central wheelchair-related claims rest on a small, unanalyzed subcohort and an unevaluated Cox model, which substantially undercuts the significance as currently presented.","major_comments":[{"comment":"The analytic cohort for the wheelchair time-to-event analysis collapses from 1,639 linked patients to 187 patients (1,409 cases) after merging assistive-device logs. The paper never compares these 187 patients with the 1,452 excluded patients, nor does it specify censoring or missingness mechanisms for the excluded group. If device-log availability correlates with severity, care access, or follow-up intensity, all Cox coefficients and the wheelchair-free survival curves in Fig. 7 are biased by selection. This directly threatens the abstract's claim that lower limb decline is the strongest predictor of earlier wheelchair use. Please provide an attrition analysis and a sensitivity analysis (e.g., inverse-probability weighting or multiple imputation) or explicitly reframe the claim to the device-log subpopulation.","section":"§3.1, Table 4, Fig. 7"},{"comment":"The Cox proportional hazards model that drives the wheelchair-free survival curves is never evaluated for discrimination or calibration on held-out data. The only quantitative evaluation in the paper is for the ASTP model (Table 2), and it shows C-indices of 0.513–0.598 on 28 held-out patients (194 landmarks), which is barely above chance, with no confidence intervals. The Figure 7 prediction of a median wheelchair-free survival of 75.5 months for a 'preserved baseline' patient is therefore unsupported. Please provide internal validation for the Cox model (e.g., bootstrap C-index, calibration plot, time-dependent AUC) and report confidence intervals for the ASTP C-indices; otherwise the headline 'dynamically predicts wheelchair-free survival' is an overclaim.","section":"§2.5, Table 4, Fig. 7"},{"comment":"The 5-year survival curves are produced by holding the hazard constant at the mean of the last three trained bins (Eq. 9). This is a strong distributional assumption with no stated justification or validation. While the manuscript does visually shade the extrapolated region, the abstract's claim that the model 'generates individualized survival curves' conflates in-distribution 720-day predictions with an unvalidated constant-hazard tail. Please either validate the extrapolation against long follow-up patients or explicitly state that the 5-year curves are scenario projections, not empirical predictions.","section":"§2.4.3, Eq. (9)"}],"minor_comments":[{"comment":"Typo: 'we implement a this prototype' should read 'we implement this prototype' or 'we implement a prototype'.","section":"§1, last paragraph"},{"comment":"The Lower limb row is corrupted: '124 700.598169.4' should read '124 70 0.598 169.4' (three separate columns). Please fix the table formatting.","section":"Table 2"},{"comment":"Section references in §3.4 point to '§2.4–§2.4.2' when describing evaluation; these are methods sections, not results sections. Please correct the cross-references.","section":"§3.4–§3.6"},{"comment":"The app description says the Cox model has 'n=15 predictors: age at visit, follow-up since diagnosis, sex, and the 12 ALSFRS-R items,' but the abstract and §2.5 list individual ALSFRS items and the Cox model in Eq. (11) also includes ALSFRS1r–ALSFRS3r; please clarify whether the respiratory items are included in the deployed model and ensure the count matches.","section":"§2.6"},{"comment":"Reference [27] is cited in the Conclusion as a disclaimer ('not a neural sparse-regression method'); this appears to be a response to a prior reviewer comment and reads oddly in a final manuscript. Please remove or integrate it more naturally.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution with a deployable app, but the central clinical claim about wheelchair timing is built on an 88% cohort attrition that is never analyzed, and the Cox model is not validated at all. The ASTP evaluation, though honest, is underpowered and close to chance. I do not see this as reject because the issues are addressable in revision, but the authors must either provide the missing attrition/validation analyses or sharply curtail the claims. Also, the paper should be candid that the 'digital twin' is essentially a Streamlit wrapper around a Cox model plus a Transformer hazard model, not a closed-loop digital twin in the engineering sense."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe ASTP architecture is the real news: five independent discrete-time hazard heads over a transformer-encoded, cross-attention-pooled visit history, trained on per-domain first-1-point drop events. That composition is new relative to the papers they cite, and the score-conditioned heads are a sensible way to handle floor effects. The held-out evaluation is also honest: 28 patients, 194 landmark samples, mean C-index 0.549, with lower limb best at 0.598 and respiratory at 0.513. Those numbers are barely above chance, but they are reported without spin, and the authors flag proportional hazards, external validation, and a research-use-only qualifier in the app.\n\nThe soft spots are structural, not cosmetic. The wheelchair-free survival model that anchors the abstract is never evaluated—no discrimination, no calibration—and it is estimated on 187 of 1,639 linked patients after merging assistive-device logs. The paper never compares those 187 with the excluded 1,452 or states how the excluded patients would be censored. Device-log availability almost certainly correlates with severity and care access, so selection by indication is a real threat to every Table 4 coefficient and every Figure 7 curve. The generic \"observational data\" caveat in the Discussion does not address this.\n\nSeveral of the \"predictions\" are also fitted values by the paper's own equations: Eq. 9 extends the survival curves by holding the hazard at the mean of the last three bins, and Section 3.5 projects to floor by scaling the predicted slope by the current score. Those are extrapolation mechanisms, not validated forecasts. The ASTP itself has no baseline comparison against the landmark or semi-competing-risk models they cite, and Table 4's unadjusted multiple testing produces an ALSFRS5 coefficient with a positive sign at p=0.05, which is hard to interpret clinically.\n\nThe lower-limb/wheelchair association is clinically plausible and directionally consistent with prior work. I don't doubt the direction; I doubt the strength and the timing claims.\n\nWho should read this: method developers working on discrete-time survival with irregular visit data, and ALS modelers who want a concrete example of what not to do with device-log merging. It deserves a serious referee—the architecture is novel and the authors are transparent—but the revision needs Cox discrimination/calibration numbers, an attrition analysis, baseline comparisons, and clearer language about extrapolation vs. prediction. I'd send it back for major revision rather than desk reject.","headline":"Novel ASTP architecture and honest held-out numbers, but the wheelchair-survival claim sits on an unevaluated Cox fit to 187/1,639 patients with no attrition analysis.","tokens_in":16248,"tokens_out":2991,"would_cite":false,"duration_ms":30693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a temporal machine-learning model turns longitudinal ALSFRS-R scores into individualized wheelchair-free survival curves, and that lower-limb decline (walking, stair-climbing) is the strongest driver of earlier wheelc","keywords":["Amyotrophic lateral sclerosis","ALSFRS-R","time-to-event prediction","survival analysis","digital twin","wheelchair-free survival","functional decline","Cox proportional hazards"],"falsifier":"Re-estimate the Cox model after including all 1,639 linked patients, treating those without device logs as censored at their last follow-up, and compare baseline ALSFRS-R and follow-up durations between the 187 with logs and the 1,452 without; if the lower-limb coefficients (ALSFRS8 ≈ −0.30, ALSFRS9 ≈ −0.40) shrink substantially or flip, the central claim fails.","tokens_in":15149,"feed_emoji":"🦽","tokens_out":4288,"duration_ms":43406,"temperature":0.7,"pith_summary":"ALS progression is notoriously uneven, and predicting when a patient will need a wheelchair remains hard. This paper tries to establish two things: that a digital-twin-inspired time-to-event model can convert a patient's ALSFRS-R visit history into per-domain transition risks and personalized wheelchair-free survival curves; and that among all functional domains, lower-limb decline—especially walking and stair-climbing—is the strongest predictor of earlier assistive device use. A sympathetic reader would care because such dynamic, interpretable predictions are exactly what proactive care planning, clinical trial stratification, and precision medicine need. The authors test their claim on a harmonized cohort of ALS patients with longitudinal assessments and device logs, evaluating the model with C-indices, MAE, and Cox regression.","feed_headline":"Lower-limb decline is the strongest wheelchair-timing signal in ALS","feed_subtitle":"New model turns routine ALSFRS-R scores into personalized wheelchair-free survival curves for ALS patients.","key_machinery":"The load-carrying machinery is the Attention-based Stage Transition Predictor (ASTP), a transformer-encoder model with cross-attention pooling that maps each patient's irregular visit history to five domain-specific hazard heads (bulbar, upper limb, axial, lower limb, respiratory) over 24 discrete 30-day time bins; per-domain survival is the product of one minus hazards, and the current score is fed into each head to prevent impossible transitions. It is paired with a regularized Cox proportional-hazards model (elastic net, λ=0.05, α=0.2) fitted to time-to-wheelchair-access, with the same ALSFRS-R items as predictors. The transformer handles irregular spacing via elapsed-time features rather","core_discovery":"The central discovery is that functional decline in ALS is not uniform: mobility loss dominates the timeline to wheelchair use. Using Cox proportional hazards on longitudinal ALSFRS-R data, the authors find that walking (ALSFRS8, coefficient −0.30) and stair climbing (ALSFRS9, coefficient −0.40) are the strongest predictors of earlier wheelchair access (p<0.005), far exceeding respiratory, bulbar, or age effects. They then embed this finding in a five-head discrete-time survival model (ASTP) that, from a single clinic visit, emits per-domain hazard curves over 24 30-day bins, computes survival probabilities, and extrapolates to a 5-year window with a constant-hazard tail. The result is an in","pith_inferences":["Because the wheelchair event is only observed for the 11% of patients with device logs, the Cox effect sizes are likely inflated if device-logged patients are sicker on average; a fuller follow-up could weaken the walking/stairs coefficients.","The constant-hazard tail used to extend the 24-bin (720-day) horizon to 5 years is an extrapolation the paper itself shades as out-of-distribution; in very slow progressors it may under- or over-estimate long-term survival probability.","The same framework could be pointed at other neurodegenerative diseases with functional rating scales (e.g., Parkinson's or multiple sclerosis) without architectural changes, an extension the authors hint at but do not test.","A natural test: in an independent ALS registry with complete device-dates, fit only ALSFRS8 and ALSFRS9 against time-to-wheelchair; if the discrimination (C-index) remains ~0.6 or better, the claim transfers."],"forward_implications":["Clinicians could enter a patient's current ALSFRS-R items at a single visit and receive a per-domain risk readout (transition within <6 mo, <1 yr, <2 yr, >2 yr) plus a wheelchair-free survival curve.","Monitoring walking and stair-climbing scores becomes a concrete, low-cost strategy for flagging patients who need early wheelchair planning.","The model's predictions are dynamic: as new visits accrue, the latent state updates and the survival curves shift, matching the digital-twin idea.","Trial designers could use predicted lower-limb transition time as a stratification variable or a surrogate endpoint in ALS studies.","The architecture is event-agnostic and could be retrained for other care milestones (e.g., feeding-tube placement) wherever longitudinal scales and event logs exist."],"fun_headline_variants":["Leg mobility drives ALS wheelchair timeline","Walking and stairs forecast ALS wheelchair need","Lower-limb decline signals ALS wheelchair timing","ALS wheelchair risk predicted by leg function decline","Mobility loss tops ALS wheelchair timing signals"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Every wheelchair-timing result depends on the 187 patients (out of 1,639) who had assistive-device logs; if having a device record correlates with disease severity or care access, the survival curves are biased, and the paper never compares the included 187 with the excluded 1,452.","fun_headline_variants_meta":{"raw":{"variants":["Leg mobility drives ALS wheelchair timeline","Walking and stairs forecast ALS wheelchair need","Lower-limb decline signals ALS wheelchair timing","ALS wheelchair risk predicted by leg function decline","Mobility loss tops ALS wheelchair timing signals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001125,"raw_usage":{"total_tokens":4540,"prompt_tokens":791,"completion_tokens":3749,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":3686}},"tokens_in":535,"tokens_out":3749,"duration_ms":24702,"temperature":1.0,"reasoning_tokens":3686,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:08:00.018982+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-estimate the Cox model after including all 1,639 linked patients, treating those without device logs as censored at their last follow-up, and compare baseline ALSFRS-R and follow-up durations between the 187 with logs and the 1,452 without; if the lower-limb coefficients (ALSFRS8 ≈ −0.30, ALSFRS9 ≈ −0.40) shrink substantially or flip, the central claim fails.","supporting_citations":[],"review_version":1}