{"id":"20fa2963-c6c8-4864-b27a-6f76fef0362d","arxiv_id":"2601.04821","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Risk-guided MCO reduces predicted radiation-pneumonitis risk by ~7.7% on 19 patients by lowering lung dose, but the endpoint is the optimized model's own prediction.","lead":"A radiotherapy-planning method embeds a patient-specific lung-damage risk model as a secondary objective in multi-criteria optimization, letting planners trade small dose deviations for predicted risk reductions. On 19 lung-cancer patients it lowered predicted pneumonitis risk by ~7.7%, but that number comes from the same model being optimized, not from observed outcomes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 7.7% RP risk reduction is computed by optimizing the same unvalidated logistic risk model used for evaluation; without a held-out cohort with observed outcomes, the clinical claim is unsupported.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the risk model's validity and the independence of the evaluation cohort. I agree that this is decisive. The paper's own limitation statement concedes that external validation is needed, but the abstract and conclusion assert a clinical benefit. The circularity—optimizing r(d) and then measuring the reduction in r(d)—means the headline quantitative result is an expected consequence of the algorithm, not evidence about real patient outcomes. Nonconvexity and plan-selection choices are secondary concerns; they affect how generalizable the front approximation is, but they do not undermine the basic algorithmic idea if reframed as a proof-of-concept for model-in-the-loop planning. As written, however, the clinical claim is unsupported, so the REJECT verdict should stand.","tokens_in":16151,"tokens_out":6672,"duration_ms":75537,"concrete_test":"Run the method on a held-out cohort where the Eq. (10) coefficients are locked before evaluation and actual grade≥2 RP outcomes are known. Compare risk-guided versus risk-agnostic plans using observed RP incidence or a proper scoring rule, not model-predicted risk; if the predicted risk reductions do not translate into lower observed RP rates (e.g., no association in a paired analysis adjusting for baseline risk), the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central clinical claim—that risk-guided MCO 'can reduce the risk of RP'—rests entirely on the risk model r(d) in Eq. (10). That model was fit by bootstrapped stepwise logistic regression on 69 patients (Section 3.1.3, Appendix 6.1), and the evaluation cohort is 19 patients whose independence from those 69 is never stated. More importantly, the evaluation is circular: Section 3.1.4 selects the risk-guided plan as the one with minimal r(d) among plans within ±30% of the risk-agnostic dose objectives, and Section 3.2.4 reports the change in this same r(d) as the outcome. Because r(d) is exactly the objective being minimized, a positive risk reduction is guaranteed whenever the optimizer can move lung V5/V20 within the constraints—even if the model has no true predictive validity. No observed grade≥2 RP outcomes are used in the 19-patient evaluation, and the paper itself states in the Discussion that external validation is still needed. If the model is overfit or the evaluation patients overlap the training cohort, the reported 7.7% mean reduction is an in-sample artifact, not clinical evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-criteria optimization (MCO) framework that incorporates a patient-specific risk model as a secondary objective while keeping conventional dose-based objectives primary. The authors encode this priority structure through a modified domination cone with a tunable parameter epsilon and apply a sandwiching algorithm to approximate the resulting front. They retrospectively test the method on 19 NSCLC patients, using a bootstrapped logistic regression model of grade>=2 radiation pneumonitis trained on 69 patients. The main reported result is a mean 7.7% reduction in model-predicted RP risk, along with reductions in lung V5/V20 and small changes in target coverage, from which the authors conclude that risk-guided MCO can reduce RP risk while preserving target coverage.","tokens_in":16520,"tokens_out":7384,"duration_ms":79369,"significance":"The algorithmic idea is interesting: replacing the standard Pareto dominance relation by a cone that makes risk strictly secondary to dose objectives, with epsilon controlling the trade-off, is a clean way to fuse bi-level prioritization with established MCO machinery. The paper gives a formal cone construction, demonstrates extension to three objectives, and provides a detailed retrospective plan comparison on 19 patients. Its clinical significance, however, is not established: the primary endpoint is the prediction of the very model that is minimized during plan generation, so the reported 7.7% risk reduction is a mathematical consequence of the lung dose changes rather than independent evidence of reduced pneumonitis. The manuscript itself concedes that external validation is still needed, and that concession should be reflected in the abstract and conclusions.","major_comments":[{"comment":"The reported primary outcome is the value r(d) of the logistic model (Eq. 10), which is exactly the secondary objective minimized in problem (9). Since Table 3 lists positive coefficients for right-lung V5 and total-lung V20, the observed reductions in these metrics mechanically lower r(d); the 7.7% mean 'RP risk reduction' is therefore a restatement of the dosimetric changes, not an independent estimate of clinical benefit. The abstract conclusion that the method 'can reduce the risk of RP' is not supported by this evaluation. Please reframe all such statements as reductions in model-predicted risk, or provide an evaluation using observed outcomes or a validated/cross-validated risk model.","section":"§3.2.4 and Eq. (10)"},{"comment":"The risk model was trained on a cohort of 69 NSCLC patients and the evaluation uses 19 patients from 'the NSCLC data set described in Section 3.1.1,' but the manuscript never states whether the 19 are a subset of the 69 or an independent cohort. This distinction is load-bearing: with overlap, the predicted risk reductions are in-sample and would be optimistically biased. The authors must state the cohort relation explicitly; if there is overlap, the risk model should be refit on a disjoint training set or evaluated by cross-validation.","section":"§3.1.1 and §3.1.3"},{"comment":"The paired-comparison protocol selects the risk-guided plan as the candidate with the lowest predicted risk among plans within ±30% of the risk-agnostic dose objectives, after a discretionary epsilon filter. This selection rule ensures a nonnegative risk difference by construction, so the mean reduction of 7.73% and range 0.27–20.07% in Table 2 reflect the selection rule more than the typical behavior of the method. Please report results under a fixed, clinically motivated selection rule or summarize the distribution over all acceptable plans, and quantify the sensitivity of the headline numbers to the ±30% window and to the choice of epsilon.","section":"§3.1.4"},{"comment":"The sandwiching algorithm and weighted-sum scalarization are used to approximate the front of a problem whose risk objective (Eq. 10) and volume-percentile functions (Eq. 6) are nonconvex. As the Discussion acknowledges, nonconvex Pareto-optimal points can then be missed, so the generated set may not be 'a good representation of the Pareto front' in the sense assumed by the sandwiching method. The possible magnitude of this effect on the reported risk/dose trade-offs should be assessed or at least stated as a formal limitation of the computed fronts.","section":"§2.1.2 and §4 (Discussion)"}],"minor_comments":[{"comment":"'logistics regression' should be 'logistic regression'.","section":"Abstract and §3.1.3"},{"comment":"Decimal separators are inconsistent: several entries use commas ('1,145', '-1,06', '2,73') while the rest of the table uses periods. These appear to be typos and should be corrected.","section":"Table 2"},{"comment":"The sentence 'This is not surprising, as our model explicitly posits a positive correlation between these dose metrics and the risk' explicitly acknowledges the circular relationship between the optimized objective and the reported outcome. It should be moved to the limitations/caveats rather than used as an explanation of the result.","section":"§3.2.4"},{"comment":"The statement 'shrinking epsilon enlarges the domination cone' is initially counterintuitive; a short illustrative explanation or reference to Figure 1 would help the reader.","section":"§2.2.2"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline reject / major revision. The methodological contribution is sufficiently novel and clearly presented to justify a revision, but the central clinical claim must be tempered to 'model-predicted risk' and the evaluation protocol needs substantial rework, especially the circular endpoint, the cohort overlap question, and the plan-selection bias. If the authors are unwilling to make those changes, rejection would be appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The optimization idea is sound and the paper is an honest proof-of-concept. What's actually new is the integration: embedding a data-driven risk model as a strictly secondary objective in MCO via a modified domination cone, and demonstrating it on 19 NSCLC patients. The front visualizations, DVH trade-offs, and epsilon sensitivity analysis are useful and show the method works as intended. The authors are also upfront in the Discussion that this is proof-of-concept and external validation is needed.\n\nThat said, the central clinical claim outruns the evidence. The endpoint in the paired comparison is the same logistic model r(d) that is minimized in problem (9). Since the model has positive coefficients on right-lung V5 and total-lung V20, any optimizer that lowers those volumes will mechanically lower r(d). The reported 7.7% mean reduction is therefore an internal consistency check, not a measured health outcome. On top of that, the plan selection protocol picks the lowest-risk plan within a ±30% window, so the reported reduction is best-case, not what a typical planner would achieve. The abstract's final sentence — 'can reduce the risk of RP' — overstates what the evaluation can support. The paper should say 'reduces predicted RP risk' and leave the clinical claim to a properly validated study.\n\nThe independence of the 19 evaluation patients from the 69 training patients is never stated. If they overlap, the already-circular result becomes an in-sample artifact. That should be explicit, even in a proof-of-concept. The nonconvexity issue is acknowledged honestly — weighted-sum scalarization only sees the convex hull — and it doesn't undermine the method as a heuristic.\n\nNone of this kills the core contribution. For readers working on MCO or risk-informed planning, the approach to encoding priorities via a cone parameter is worth knowing. I would send this to peer review; the right outcome is major revision with a softened clinical claim, a statement of cohort independence, and ideally a sensitivity analysis using a different plan selection rule. It's not a desk reject.","headline":"The optimization framework is a legitimate proof-of-concept, but the 7.7% RP risk reduction is an evaluation of the model's own objective, not clinical efficacy; the paper needs a softened claim and clearer validation status.","tokens_in":16977,"tokens_out":2612,"would_cite":true,"duration_ms":30475,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Risk-guided optimization cuts predicted radiation pneumonitis risk by 7.7% on average in a 19-patient lung cancer cohort.","keywords":["multi-criteria optimization","radiation therapy treatment planning","radiation pneumonitis","bi-level optimization","Pareto front","risk model","NSCLC","NTCP"],"falsifier":"Check whether the 19-patient evaluation set overlaps the 69-patient training set for the RP risk model; any overlap would mean the 'risk reduction' is in-sample. More decisively, run a prospective or external validation where risk-guided plans and standard plans are compared on actual observed grade 2+ radiation pneumonitis: if the risk-guided arm does not show lower toxicity, the central claim would be refuted.","tokens_in":16078,"feed_emoji":"🫁","tokens_out":3834,"duration_ms":39054,"temperature":0.7,"pith_summary":"The paper tries to establish that a patient-specific biological risk model can be embedded directly into multi-criteria optimization (MCO) for radiotherapy planning as a secondary objective, without turning the plan into a purely model-driven one. It does this by altering the order relation that defines Pareto optimality, so that dose objectives remain primary and risk is minimized only when the dose trade-off is acceptable. On 19 lung cancer patients, the resulting risk-guided plans lowered predicted radiation pneumonitis risk by a mean of 7.7% (range 0.3–20.1%) while keeping target coverage nearly intact. The value of the claim is that it offers a one-shot, interactive way to individualize treatment plans using predictive models, instead of sequential re-optimization or forcing the clinician to trust the model as an equal objective.","feed_headline":"Risk-guided plans cut pneumonitis risk 7.7%","feed_subtitle":"A bi-level MCO method keeps dose goals primary while minimizing patient-specific toxicity predictions.","key_machinery":"The load-bearing mechanism is a modified domination cone in objective space. Instead of treating the risk objective as equally important as the dose objectives, the cone CA (defined by a matrix Q with epsilon entries) imposes a lexicographic-like ordering: a plan is only considered better if it improves risk without worsening dose objectives beyond a relative fraction epsilon. Because the cone is polyhedral, the standard Sandwiching algorithm for convex multi-criteria optimization can approximate the risk-guided Pareto front directly, which turns the bi-level problem into a one-shot computation. The planner tunes epsilon to choose how aggressively to chase risk reduction.","core_discovery":"The central claim is that any risk model that maps a dose distribution to a patient-specific outcome probability can be integrated into conventional multi-criteria radiotherapy planning as a strictly secondary objective. The authors define a domination cone CA with a parameter epsilon that encodes the maximum acceptable relative worsening of each dose objective for a given risk improvement, and they show, building on prior work, that the resulting bi-level problem can be solved as a single MCO problem. Applying this to 19 NSCLC patients using a logistic regression model for grade 2+ radiation pneumonitis, they report a mean predicted risk reduction of 7.7%, achieved mainly by lowering right-","pith_inferences":["If the risk model were replaced by a validated multimodal predictor (e.g., imaging or molecular markers), the same machinery would carry patient-specific biology into planning; the method's value lies in being model-agnostic.","The same bi-level trick could balance multiple competing risk outcomes (e.g., pneumonitis vs. esophagitis) by making the secondary layer an MCO problem itself — the authors note this is feasible but do not test it.","A testable extension: measure whether the epsilon-tuned fronts actually shift the chosen clinical plan in the direction of lower predicted risk for patients who are predicted high-risk; that would give a direct decision-support use.","A risky consequence left implicit: if clinicians trust the secondary risk model too much and raise epsilon, plans can drift from evidence-based dose constraints; the paper's selection protocol for the 19-patient comparison partially masks this."],"forward_implications":["For patients whose risk model responds to lung dose, the method automatically finds plans with lower predicted pneumonitis risk while keeping dose objectives near the conventional Pareto front.","The planner can tune epsilon to reflect trust in the model: smaller epsilon keeps plans close to established dose protocols; larger epsilon allows wider risk–dose trade-offs.","Conventional MCO is recovered in the limit epsilon to infinity, so the method is a strict generalization rather than a replacement.","The framework extends to any number of dose objectives, with the caveat that the Sandwiching algorithm's complexity grows quickly with dimensionality.","Because it produces a full front in one run, it supports navigation and interpolation between plans rather than a single sequential re-optimization."],"fun_headline_variants":["Radiotherapy planning that cuts pneumonitis risk 7.7%","Risk-guided MCO trims pneumonitis risk by 7.7%","Personalized RT plan cuts lung toxicity by 7.7%","Bi-level MCO reduces pneumonitis risk with same coverage","One-shot risk-informed planning: 7.7% less pneumonitis"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reported risk reduction is only as valid as the logistic regression model that predicts radiation pneumonitis from lung V5/V20, smoking, and breathing function; if that model is overfit or the 19 evaluation patients are not independent of the 69 training patients, the improvement could be an artifact of optimizing the same function used to score the plans.","fun_headline_variants_meta":{"raw":{"variants":["Radiotherapy planning that cuts pneumonitis risk 7.7%","Risk-guided MCO trims pneumonitis risk by 7.7%","Personalized RT plan cuts lung toxicity by 7.7%","Bi-level MCO reduces pneumonitis risk with same coverage","One-shot risk-informed planning: 7.7% less pneumonitis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000895,"raw_usage":{"total_tokens":3742,"prompt_tokens":840,"completion_tokens":2902,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":2806}},"tokens_in":584,"tokens_out":2902,"duration_ms":17618,"temperature":1.0,"reasoning_tokens":2806,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T11:53:26.792779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether the 19-patient evaluation set overlaps the 69-patient training set for the RP risk model; any overlap would mean the 'risk reduction' is in-sample. More decisively, run a prospective or external validation where risk-guided plans and standard plans are compared on actual observed grade 2+ radiation pneumonitis: if the risk-guided arm does not show lower toxicity, the central claim would be refuted.","supporting_citations":[],"review_version":1}