{"id":"a26d39b5-bcae-4ed7-879f-1cd1d6fa4aa0","arxiv_id":"2607.25120","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ML-NMR-MS jointly synthesizes PFS/OS via an illness-death model while integrating individual-level regressions over aggregate covariate distributions to yield population-adjusted conditional and marginal effects.","lead":"This paper combines multistate illness-death modeling with multilevel network meta-regression so oncology trials can jointly synthesize PFS and OS while adjusting for patient-level effect modifiers across mixed IPD and aggregate evidence. It matters because HTA decisions need population-relevant, coherent joint survival inputs that can feed state-transition cost-effectiveness models directly.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The AgD covariate reconstruction assumes the binary effect modifiers are independent of the continuous covariates (Appendix C), and the one channel that could bias the population-adjusted effects through this — dependence involving the binary effect modifier — is exactly the one the simulation never","rationale":"I agree with the reader that no-frailty is a genuine, honestly-reported limitation, and the paper deserves credit for demonstrating its failure mode rather than hiding it (simulation aim iii), for the algebraic care in justifying the ratio-of-marginals identity (10), and for the Appendix L calibration sub-study showing the two-binomial composite is near-nominal while the multinomial is overconfident on reconstructed counts. But the frailty assumption is not the most load-bearing concern for this paper's specific claim: it afflicts the full-IPD comparator equally, so it cannot distinguish ML-NMR-MS from the gold standard it is benchmarked against, and the headline claim is explicitly hedged \"under correct specification.\" The channel that is (i) specific to the novel AgD machinery, (ii) assumed rather than tested, and (iii) attached to the strongest effect modifier is the covariate-reconstruction independence of the binary effect modifier. This sharpens rather than overturns the reader's assessment: the reader already cited the composite AgD likelihood and strong transport assumptions among the limitations keeping the verdict at CONDITIONAL, and this concern lands in the same place. Hence UNCHANGED — CONDITIONAL remains right — but the conditionality should explicitly include \"dependence structure of binary effect modifiers in the AgD reconstruction uncharacterized,\" and the proposed one-scenario simulation extension would settle it cheaply since all infrastructure (DGM, fitting, scoring) already exists.","tokens_in":48389,"tokens_out":3957,"duration_ms":132572,"concrete_test":"Extend simulation scenario (d): generate x1 ~ N(μ1j, 1) and the binary effect modifier x2 | x1 ~ Bernoulli(logit^{-1}(α0j + 0.8·x1)), with α0j chosen so the reported marginal Pr(x2=1) matches the base design. Fit ML-NMR-MS exactly as in Section 4.4, with the AgD reconstruction retaining Appendix C's independent-binary default (IPD studies still see the true joint). Score bias and 95% coverage of d^SP_k(P) over 100 replicates as in Table 3. If coverage falls below ~0.90 or |bias| exceeds ~0.05 on the log-HR scale relative to base (0.013 / 0.96), the reconstruction channel is a practical vulnerability and the independence default needs replacing (e.g., a latent-Gaussian threshold copula for binary margins).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's chosen weakest assumption (no unobserved shared frailty) is real but inherited: the frailty scenario degrades the full-IPD gold standard as much as ML-NMR-MS (coverage 0.75 vs 0.84), so it threatens the illness–death Markov framework as a whole, not this paper's novel contribution. The contribution specific to this paper is the aggregate-data arm: covariate-marginal occupancies (7) computed by QMC integration over a reconstructed joint covariate distribution. Because the occupancy map is nonlinear in x, an incorrectly reconstructed joint distribution biases the marginal occupancies — the same aggregation-bias channel ML-NMR exists to eliminate, re-entering through the reconstruction. Appendix C states the reconstruction borrows \"a single correlation between the two continuous covariates, the binary covariate being drawn independently.\" Yet in both the example and the simulation the binary covariate (ISS-III) is the strongest effect modifier (β2^SP = 0.40 in the DGM; +0.26 in the example), and clinically it is implausible that ISS-III is independent of age and response. The simulation's scenario (d) perturbs only the continuous–continuous copula correlation and finds robustness; Section 4.6 concedes \"misspecification of the reported marginals themselves, and dependence involving the binary effect modifier, are not exercised here and remain uncharacterized.\" So the validity of the AgD likelihood rests on an independence assumption about precisely the covariate whose misplacement would most distort the population-adjusted S→P effects, and the one scenario that would probe it is absent. This is not a circularity or soundness flaw — the construction is internally coherent — but a correctness risk sitting directly under the strongest claim's \"from mixed IPD+AgD evidence\" clause, since AgD arms enter only through this reconstruction.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The manuscript introduces ML-NMR-MS, which embeds the multilevel network meta-regression (ML-NMR) integrated IPD+AgD likelihood inside an illness–death multistate model, thereby combining the joint PFS/OS multistate NMA of Jansen et al. with the population-adjustment machinery of Phillippo et al. For IPD studies the full multistate likelihood over linked transitions is used; for aggregate studies, covariate-marginal state-occupancy probabilities are computed by quasi-Monte-Carlo integration of the occupancies implied by the transition intensities over a reconstructed covariate distribution (Gaussian copula matched to reported marginals, correlations borrowed from IPD), and enter a conditional-survival grouped-count likelihood (two-binomial composite by default, multinomial when joint occupancy counts exist). A ratio-of-marginals identity (Eq. 10) justifies forming interval survival probabilities from marginal occupancies. The method yields conditional and marginal population-adjusted joint treatment effects for arbitrary target populations and directly parameterizes state-transition cost-effectiveness models. An illustrative multiple-myeloma network (with a semi-synthetic OS layer) demonstrates identifiability of the transition-level decomposition, and a simulation study with known truth shows small bias and near-nominal coverage under correct specification, with precision losses scaling with the AgD share and degradation localized to unobserved frailty and misspecified treat","tokens_in":48890,"tokens_out":4969,"duration_ms":86145,"significance":"If the results hold, this is a genuine and useful advance: the first population-adjusted joint PFS/OS synthesis, directly relevant to HTA practice where IPD is partial and effect-modifier imbalance across trials is common. The paper ships several concrete strengths: an exact algebraic identity justifying the ratio-of-marginals form (10); a reproducible Stan implementation (Appendix J) with careful numerical handling of the removable singularity; a simulation with fully known external truth demonstrating unbiased recovery and near-nominal coverage under correct specification; a calibration sub-study comparing composite versus multinomial aggregate likelihoods (Appendix L); and direct parameterization of a clock-forward state-transition economic model with the conditional/marginal estimand distinction handled explicitly. The honest treatment of failure modes (frailty degrades the full-IPD comparator as much as the proposed method) is to its credit. The limitations are evidential rather than derivational: the AgD arm's validity under realistic covariate dependence and the breadth of the simulation evidence.","major_comments":[{"comment":"The aggregate-data construction reconstructs each AgD arm's joint covariate distribution as reported marginals plus an IPD-borrowed Gaussian copula, with the binary covariate 'drawn independently' (Appendix C). This independence assumption concerns precisely the covariate that carries the strongest effect modification in both the simulation DGM (beta2^SP = 0.40 for ISS-III) and the example (+0.26), and clinical independence of ISS-III from age/response is implausible. Because the occupancy map is nonlinear in x (Eq. 7 and the surrounding discussion), an incorrectly reconstructed joint distribution biases the marginal occupancies — reintroducing, through the reconstruction, exactly the aggregation-bias channel ML-NMR exists to remove. Simulation scenario (d) perturbs only the continuous-continuous copula correlation, and Section 4.6 concedes this channel is unexercised. The paper's centra","section":"§2.3.2, Appendix C, §4.2 scenario (d), §4.6"},{"comment":"Of 100 replicates per scenario, only 76-94 converged (R-hat <= 1.05) for ML-NMR-MS versus 97-100 for the full-IPD comparator, and all performance measures are computed on the retained subset. Coverage and bias are thus conditional on convergence; if non-convergence correlates with weak identification (plausibly the same intervals in which estimates are extreme), reported coverage is optimistically biased, and the ML-NMR-MS/full-IPD comparisons in Table 3 are made on different replicate sets. The differential is largest exactly where conclusions are most consequential (the frailty scenario, coverage 0.84 vs 0.75). Request: report per-scenario convergence counts (currently only the range), characterize the dropped replicates, and provide a sensitivity analysis — e.g., performance including non-converged fits scored by their posterior means, or a comparison restricted to replicates where bo","section":"§4.4, Tables 2-3"},{"comment":"The two-binomial composite (9) is adopted as the default for curve-only evidence, and the paper's coverage claims rest on it. The supporting evidence (Appendix L) is a single sub-study at the base design, showing near-nominal coverage (0.96) for the composite and overconfidence (0.71) for the reconstructed-count multinomial. Composite-likelihood theory (Varin et al., ref. 11) says the pseudo-posterior is miscalibrated in general; that it happens to be calibrated here may be design-dependent (monthly bins, six studies, 200/arm, this DGM). Since the composite is the default likelihood for the novel AgD contribution, the basis for trusting its calibration beyond the tested design is thin, and no Godambe/sandwich-type adjustment or sensitivity to bin width is offered. Request: at minimum, a calibration check at one off-base design (e.g., the aggregate-dominated network) and an explicit discu","section":"§2.3.3, Appendix L, Table S12"}],"minor_comments":[{"comment":"Notation collisions: the loop counters a, l, M are reused in the Stan listing for purposes unrelated to their main-text meanings (treatment contrast a in (13)-(16), integration point l in (7)); this is acknowledged in Appendix J but would be better avoided in the code itself. In Eq. (8), the m' > m indexing (reporting intervals as unions of grid intervals) is easy to misread on first pass; a small diagram or explicit example would help.","section":"Appendix J; Eq. (8)"},{"comment":"Table 2, non-PH column, gamma^PD rows: the apparent bias of -0.11 to -0.17 is explained in the text as an artifact of scoring a u=1-month log-HR against a constant truth under a redundant P->D time slope. This caveat should appear as a table footnote, since readers consulting the table alone will misread it as a failure of recovery.","section":"Table 2"},{"comment":"Appendix D: the binomial denominator n^c does not adjust for censoring within the interval, and r^c is a rounded expectation from the KM ratio. A sentence quantifying the resulting information loss (or citing its negligibility at monthly resolution) would strengthen the construction.","section":"Appendix D"},{"comment":"The thalidomide conditional S->P HR at 48 months varies materially across baseline forms (0.89 Weibull, 1.33 FP2, 0.79 M-spline; Table S8), presumably because it is identified from a single aggregate study. The LOO preference for FP2 (Table S5) partially addresses which to trust, but a remark on the fragility of effects identified only through AgD arms would be useful for practitioners.","section":"Appendix I.3, Table S8"},{"comment":"Reference [22] is an arXiv preprint; verify status. Throughout, 'introducesmultilevel' and similar spacing artifacts suggest a macro issue in the compiled PDF worth checking before resubmission.","section":"References; passim"}],"recommendation":"major_revision","confidential_remarks":"Single-author manuscript that combines the author's own prior multistate NMA (Jansen et al. 2023) with the Phillippo et al. ML-NMR framework; the self-citation is substantive and appropriate rather than padded. Novelty relative to both parents is genuine and clearly delineated (Appendix A, Table S1). The AI-assistance acknowledgment is unusually detailed and appears consistent with the manuscript. Industry affiliation is disclosed; the work is methodological and I see no conflict affecting the science. Good fit for the journal's scope. My major comments 1–3 all request additional simulation evidence rather than new theory; the required machinery already exists in the paper's own simulation pipeline, so the revision burden is moderate."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is the natural next step in Jansen’s multistate NMA and Phillippo’s ML-NMR lines: one joint illness–death likelihood that takes full multistate IPD and, for AgD, QMC-integrates the Kolmogorov occupancy over a reconstructed covariate distribution, then feeds conditional-survival counts. That is genuinely new, and it is done carefully.\n\nThe math holds up. The individual multistate likelihood, the piecewise occupancy solution with the removable singularity handled, the ratio-of-marginals identity for the at-risk distribution, and the composite-vs-multinomial calibration sub-study are all clean. Simulation under known truth recovers the population-adjusted conditional S→P effects with small bias and near-nominal coverage when the model is right; frailty and PH misspecification degrade both ML-NMR-MS and the full-IPD gold standard, which the paper reports rather than hides. Stan code is in the supplement. Conditional and marginal estimands are separated properly, and the HTA motivation (building blocks that can parameterize a state-transition model) is coherent.\n\nSoft spots, in proportion. The illustrative OS layer is semi-synthetic, so the myeloma numbers are demonstration only. Simulation is moderate (one parameterization family, ~100 replicates). The stress-test point on covariate reconstruction is fair: Appendix C draws the binary modifier independently of the continuous covariates, ISS-III is the strongest effect modifier in both the example and the DGM, and the copula-misspecification scenario only perturbs the continuous–continuous correlation. Section 4.6 admits the binary-dependence channel is uncharacterized. That sits under the “mixed IPD+AgD” claim, but it is a correctness risk for the AgD arm, not a crack in the likelihood derivation itself, and it is the same class of reconstruction assumption every ML-NMR paper carries. No-frailty is inherited by the whole Markov framework, not invented here.\n\nWho it is for: methodologists and HTA analysts who already live in multistate NMA or ML-NMR and need joint PFS/OS with population adjustment. Not a general-statistics paper. I would send it to peer review; the contribution is real and the limitations are stated. I would cite it when I next need population-adjusted joint PFS/OS from mixed evidence.","headline":"Solid unification of multistate NMA and ML-NMR with careful likelihood work and honest simulation; the AgD reconstruction gap is real but secondary to the main contribution.","tokens_in":49554,"tokens_out":554,"would_cite":true,"duration_ms":12659,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62P10","62N02","62F15"],"pacs":[],"model":"grok-4.5","headline":"A multistate model can jointly synthesize progression and survival from mixed individual and aggregate trials while adjusting for who was in each trial.","keywords":["network meta-analysis","multistate models","population adjustment","individual participant data","aggregate-level data","effect modification","progression-free survival","overall survival"],"falsifier":"In a simulation (or trial network) with a genuine shared frailty on the transitions, check whether population-adjusted log hazard ratios stay unbiased with near-nominal coverage; the paper’s own frailty scenario already collapses coverage to roughly 0.75–0.84 for both ML-NMR-MS and full-IPD fits, so a frailty-aware fit that restores coverage would falsify the no-frailty claim as sufficient.","tokens_in":49294,"feed_emoji":"📉","tokens_out":1016,"duration_ms":22711,"temperature":0.7,"pith_summary":"Oncology network meta-analysis usually treats progression-free and overall survival as separate endpoints, and often cannot correct for patient-mix differences across trials. This paper introduces ML-NMR-MS: it puts an individual-level illness–death model (stable, progressed, dead) at the center, uses the full linked transition likelihood when patient-level data exist, and for published curves only averages the implied state occupancies over each trial’s reconstructed covariate distribution. The shared parameters then give population-adjusted joint treatment effects—conditional and marginal—and the absolute state paths needed for a state-transition economic model. On an illustrative myeloma network the method separates a treatment’s effect on progression from its effect on post-progression death; a simulation with known truth recovers those effects without bias and with near-nominal coverage when the model is correctly specified.","feed_headline":"Joint PFS/OS synthesis that fixes trial mix imbalance","feed_subtitle":"ML-NMR inside an illness–death model recovers population-adjusted effects from mixed individual and aggregate data","key_machinery":"Covariate-marginal state occupancy: the individual transition intensities imply stable/progressed/dead probabilities; those occupancies are integrated over each aggregate arm’s reconstructed covariate distribution (quasi-Monte-Carlo), then entered as interval conditional-survival probabilities. That integral is the multistate form of the ML-NMR hallmark and is what de-biases aggregate curves and restores the PFS–OS link without inventing individual progression–death pairs.","core_discovery":"Embedding the multilevel network meta-regression integral inside a clock-forward illness–death model yields unbiased, population-adjusted joint PFS/OS treatment effects from mixed individual-participant and aggregate evidence. IPD studies contribute the full multistate likelihood over linked transitions; aggregate studies contribute covariate-marginal state occupancies obtained by quasi-Monte-Carlo integration and scored with a conditional-survival likelihood. Under correct specification the method recovers conditional and marginal effects in a target population and can parameterize a state-transition economic model directly.","pith_inferences":["The same occupancy-integral idea could be pushed to a clock-reset (semi-Markov) post-progression hazard, at the cost of replacing the closed Kolmogorov step with a renewal convolution—exactly the extension the discussion flags as worthwhile.","If the no-frailty assumption is the binding failure mode in real oncology networks, formal surrogacy analyses that estimate residual progression–death association will need frailty-augmented multistate ML-NMR, not just richer covariates.","Disconnected single-arm evidence could inherit the construction only by swapping consistency for treatment-specific baselines and shared prognostic factors—an unanchored variant the paper sketches but does not build."],"forward_implications":["HTA models can be fed conditional building blocks (prognostic effects, treatment-by-covariate interactions, transition baselines) and re-marginalized for any target population rather than plugging a single trial HR into a marginal curve.","A treatment’s OS benefit can be decomposed into delay of progression versus change in post-progression mortality, giving a mechanistic read on how much PFS carries the survival signal.","Studies that report only PFS can still enter the joint synthesis; the network identifies the shared transition effects while that study’s post-progression baseline stays prior-driven.","When genuine joint occupancy counts exist, a multinomial likelihood is available; for ordinary separate KM curves the two-binomial composite remains the calibrated default."],"fun_headline_variants":["ML-NMR inside illness-death model adjusts joint PFS/OS for trial imbalance","Population-adjusted PFS/OS effects from mixed IPD and aggregate multistate data","Multistate ML-NMR separates progression from post-progression survival effects","Joint PFS/OS synthesis with covariate-adjusted multistate network meta-regression","Unbiased target-population PFS/OS effects via multistate multilevel NMR"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"Given the measured covariates, patients who progress early are not also systematically more likely to die early; any such hidden shared frailty is assumed away.","fun_headline_variants_meta":{"raw":{"variants":["ML-NMR inside illness-death model adjusts joint PFS/OS for trial imbalance","Population-adjusted PFS/OS effects from mixed IPD and aggregate multistate data","Multistate ML-NMR separates progression from post-progression survival effects","Joint PFS/OS synthesis with covariate-adjusted multistate network meta-regression","Unbiased target-population PFS/OS effects via multistate multilevel NMR"]},"model":"grok-4.5","effort":"low","cost_usd":0.002719,"raw_usage":{"total_tokens":1137,"prompt_tokens":917,"num_sources_used":0,"completion_tokens":83,"cost_in_usd_ticks":27188000,"prompt_tokens_details":{"text_tokens":917,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":137,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":917,"tokens_out":83,"duration_ms":3788,"temperature":1.0,"reasoning_tokens":137,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T01:19:53.479492+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"In a simulation (or trial network) with a genuine shared frailty on the transitions, check whether population-adjusted log hazard ratios stay unbiased with near-nominal coverage; the paper’s own frailty scenario already collapses coverage to roughly 0.75–0.84 for both ML-NMR-MS and full-IPD fits, so a frailty-aware fit that restores coverage would falsify the no-frailty claim as sufficient.","supporting_citations":[],"review_version":2}