{"id":"235b5f49-96e8-4b6f-af1e-296162cc2265","arxiv_id":"2505.03633","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CUI-MET picks the optimal dose by maximizing a weighted average of several endpoint response probabilities, estimated from observed data or simple dose-response models, with bootstrap uncertainty.","lead":"Cancer trials often need to choose a drug dose that balances benefit and side effects, and this paper proposes a simple scoring system that combines several outcomes into one number per dose. The method comes with an interactive web app, making it easier for clinical teams to apply structured dose optimization in early-phase trials.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The stated Shiny default forcing monotone increase in modeled no-toxicity probability is clinically backwards and contradicts Example 2; if enforced, it can bias OBD selection toward overly toxic doses.","rationale":"The reader's weakest_assumption targeted the marginal-versus-joint limitation, which is real and acknowledged. My concern is different and more concrete: an internal inconsistency between the stated default monotonicity constraint for toxicity and the example results. The paper's Section 2.4 says toxicity is modeled as 1−Toxicity and that its probability is always assumed to increase monotonically with dose, which would mean no-toxicity increases with dose. Example 2, however, shows no-toxicity decreasing with dose under a logit-linear toxicity model. This contradiction cannot be resolved from the text. If the app enforces the stated constraint, the method will produce clinically implausible toxicity curves and may select an excessively high dose, undermining the central claim that CUI-MET identifies the OBD. If the constraint is not actually enforced, the methods text is incorrect and needs correction. Either way, the paper requires a fix, and the absence of code or a simulation study prevents verification. I therefore maintain the reader's CONDITIONAL verdict, as the core idea is reasonable but the implementation and evidence need strengthening. The concrete test would settle whether the concern lands by checking the app's actual fitted values against the paper's own reported results.","tokens_in":11913,"tokens_out":11297,"duration_ms":108608,"concrete_test":"Run the CUI-MET Shiny app on Example 2 data, selecting a logit-linear model for toxicity and leaving monotonicity options at their defaults. Export the fitted 1−Toxicity probabilities for doses 1–5. If they are non-decreasing, the stated constraint is enforced and the paper's Table 2 results (decreasing 1−Toxicity) are not reproducible; if they are decreasing, the text's 'always assumed' default is inaccurate. Independently, simulate a dataset with true toxicity increasing in dose, fit logit linear to 1−Toxicity under the stated monotone constraint, and compare the selected OBD against the unconstrained fit to quantify any bias toward higher, more toxic doses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.4 states that the app converts toxicity to 1−Toxicity and that for toxicity 'a monotonic increase in probability with dose is always assumed.' Since the modeled outcome is the probability of no toxicity, this default forces no-toxicity to increase with dose, i.e., toxicity decreases with dose—the opposite of the usual dose-toxicity relationship. This is not merely a wording issue: Example 2 uses a logit-linear model for toxicity, yet Table 2 shows 1−P(Tox=1) decreasing monotonically from 0.905 to 0.558 across doses. Those results cannot be produced if the stated monotone-increasing constraint is enforced on the modeled 1−Toxicity outcome. Either the app does not enforce the stated default, or the example results were generated with a different procedure. If the app does enforce it, CUI-MET will systematically underestimate toxicity at higher doses, potentially selecting a dose above the true optimal biological dose. This directly threatens the central claim that CUI-MET identifies the OBD in standard oncology settings where toxicity increases with dose. The paper provides no code or simulation to resolve the discrepancy, so the implementation's behavior is unverifiable from the manuscript alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CUI-MET, a dose-optimization framework for early-phase oncology trials with multiple dose levels and multiple binary endpoints. The clinical utility of each dose is defined as a weighted arithmetic mean of marginal endpoint probabilities, with weights specified by the user. The framework offers both empirical estimation and four parametric dose-response models (logit linear, logit quadratic, Emax, exponential), and selects the dose with the highest CUI as the optimal biological dose. A stratified bootstrap procedure provides percentile confidence intervals for the CUI and the probability that each dose is selected as optimal. The methods are implemented in an R Shiny application and illustrated with three simulated example datasets of 150 patients each.","tokens_in":12166,"tokens_out":5860,"duration_ms":53929,"significance":"If validated, CUI-MET would provide a simple, transparent, and accessible alternative to more complex dose-finding designs, and it directly addresses the practical need to trade off efficacy, toxicity, and tolerability when selecting a dose. The paper is clearly written, the CUI definition is explicit, and the bootstrap approach is a sensible way to communicate uncertainty in small samples. The authors are commendably candid about limitations: they explicitly acknowledge the marginal-probability assumption, the lack of formal dose-comparison tests, and the absence of a formal simulation study. However, the current manuscript does not yet support the central claims because of an internal inconsistency in the Shiny app's stated monotonicity default, the lack of operating-characteristic simulations, and a notational error in the core empirical estimator.","major_comments":[{"comment":"There is an internal contradiction about the monotonicity constraint for toxicity. Section 2.4 states that for toxicity 'a monotonic increase in probability with dose is always assumed,' but because the app models 1-Toxicity, this constraint would force the probability of no toxicity to increase with dose, i.e., toxicity would be forced to decrease with dose. That is clinically backwards and is contradicted by the paper's own results: Table 2 for Example 2 shows 1-P(Tox=1) decreasing from 0.905 to 0.558 across doses 1-5 under a logit-linear model. Either the app does not enforce the stated default, in which case Section 2.4 is incorrect, or the example results were not produced by the app as described, in which case the examples are not reproducible. An enforced monotone increasing no-toxicity constraint would bias CUI-MET toward selecting overly toxic doses, so this issue directly affects the validity of the proposed tool. Please correct the description and/or the implementation, and provide the app code or a link so the default behavior can be verified.","section":"Section 2.4 and Table 2"},{"comment":"The Discussion explicitly states that 'we have not conducted a formal simulation study to assess operating characteristics of CUI-MET.' The examples in Section 3 consist of three hand-specified datasets, and the model for each endpoint is selected by visual inspection of the same data used for fitting (see 'Modeling Approach Selection' in Section 3). Consequently, the statement on page 19 that 'the CUI-MET framework reliably captures the trade-offs between efficacy, toxicity, and tolerability' is not supported by the presented evidence. For a methods paper proposing a dose-selection tool, at least a modest simulation study is needed to quantify the probability of correct OBD selection under known true dose-response scenarios, sensitivity to weight misspecification, and the impact of model selection. Without such a study, the central practical claim that CUI-MET identifies the OBD remains unverified.","section":"Section 4 and Section 3"},{"comment":"The empirical estimator is written as P(EP_k=1|dose j) = (1/N) sum_{i=1}^N Y_ijk, but N was introduced earlier as the total number of patients randomized across all J dose levels, not the number of patients assigned to dose j. The denominator should be the per-dose sample size, say n_j, and the summation should be over the n_j patients at that dose. As written, the formula would average over all patients and is not what the text describes. This notation should be fixed and used consistently in Section 2.3, where the bootstrap is said to resample 'within each dose level group, maintaining the same sample size per dose level.'","section":"Section 2.1, Eq. (1)"},{"comment":"The choice of dose-response model for each endpoint is made after viewing the observed data in Figure 2, with no pre-specification, model diagnostics, or sensitivity analysis. Because the same data are used both to select the model and to estimate the CUI, the reported OBD may reflect overfitting to noise. The paper acknowledges that these selections are 'relatively subjective,' but it does not assess how sensitive the OBD recommendation is to reasonable alternative model choices. This is especially relevant for Examples 1 and 3, where the UWM differences between adjacent doses are small and the bootstrap selection probabilities are near 50%.","section":"Section 3, Modeling Approach Selection"}],"minor_comments":[{"comment":"There are several typographical errors: 'mulitple' in Section 1, 'feasibile' in Section 2.3, 'ablity' in Section 2.2, 'mariginal' in the note to Table 2, and 'plausability' in Section 3. A careful proofread is needed.","section":"Throughout"},{"comment":"No URL, repository, or package name is provided for the R Shiny application, even though the app is central to the paper's usability claim. Providing the code would also help resolve the monotonicity question raised above.","section":"Section 2.4"},{"comment":"The weights are described as ranging from 0 to 5, but only the normalized relative weights enter the CUI formula. It would be clearer to state explicitly that the 0-5 range is an arbitrary scale and that the normalization makes only the relative weights meaningful.","section":"Section 2"},{"comment":"The paper uses N both for the total number of patients and for the per-dose sample size; for example, Figure 2 says N=30 subjects for each dose level, while the methods section defines N as the total sample. Using n_j for the per-dose sample size throughout would remove ambiguity.","section":"Figure 2 and Table 2"},{"comment":"The bootstrap confidence intervals are percentile-based with no mention of bias correction or assessment of coverage in small samples. Given that the examples have only 30 patients per dose, a sentence noting that coverage may be imperfect in small samples would be appropriate.","section":"Section 2.3"}],"recommendation":"major_revision","confidential_remarks":"The central ideas are simple and potentially useful, but the manuscript currently has a reproducibility-blocking inconsistency in the Shiny app's toxicity monotonicity default and no simulation evidence for the main operating-characteristic claims. These are fixable, but they require substantive revision rather than copy-editing. I would encourage the editor to request a revised version with a clarified and corrected app description, code availability, and a small simulation study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clearly written, low-complexity methods paper that packages a known weighted-average utility index into a user-friendly Shiny app. The statistical content is thin but honest. The main thing to know: the central formula is correct and simply a weighted mean of marginal endpoint probabilities; there's no new statistical machinery. What's potentially useful is the accessible implementation and the bootstrap-based selection probabilities.\n\nThe paper does a good job of outlining the framework, describing four dose-response models, and giving three simulated examples that show how weights change the optimal dose. The writing is clear, the limitations section is candid, and the authors acknowledge that the core CUI idea comes from Winzenborg and U-MET. Credit where due: the bootstrap procedure is standard and appropriately used.\n\nSoft spots: the claimed novelty is packaging. More importantly, there's an internal inconsistency around the monotonicity constraint for toxicity. Section 2.4 says the app converts toxicity to 1−Toxicity and that for toxicity a monotonic increase in probability with dose is always assumed. Read literally, that forces the probability of no toxicity to increase with dose, which is clinically backwards. And Example 2's Table 2 shows 1−P(Tox) decreasing from 0.905 to 0.558 across doses, which cannot be produced under that constraint. Either the stated default is wrong, the examples were generated under a different procedure, or the text is ambiguous. Without code, this can't be resolved. That needs fixing before I'd trust the tool.\n\nOther soft spots: model selection is done by visual inspection of the same data used for selection, which invites overfitting; the paper admits there's no formal simulation study; and no code or data are provided, so the Shiny app's behavior is unverifiable. The marginal-vs-joint limitation is real but acknowledged.\n\nOverall: this is a reasonable methods note for practitioners who want a quick, transparent way to compute a utility-weighted mean and see bootstrap uncertainty. It's not a methodological breakthrough. With the monotonicity issue resolved and reproducibility artifacts added, I'd be comfortable seeing it in print. I'd send it to peer review because the device could be useful, but I'd expect substantial revision.","headline":"A clearly written but low-novelty methods paper that packages a known weighted-average utility index into a Shiny app; a potentially serious internal inconsistency about the toxicity monotonicity default needs resolution before I'd trust the tool.","tokens_in":12696,"tokens_out":3680,"would_cite":false,"duration_ms":33940,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CUI-MET identifies the optimal biological dose as the dose with the largest clinical utility index, a weighted average of marginal endpoint probabilities, and attaches bootstrap-based confidence to that choice.","keywords":["CUI-MET","clinical utility index","optimal biological dose","dose optimization","multiple binary endpoints","early-phase oncology trials","bootstrap selection probability"],"falsifier":"Generate a simulated trial with strong positive within-patient correlation between efficacy and toxicity, choose weights so the marginal weighted mean favors one dose, and define a patient-level utility that penalizes joint occurrence; if CUI-MET selects the dose favored marginally while a joint-utility evaluation favors another dose, the marginal simplification's limit is demonstrated.","tokens_in":11722,"feed_emoji":"⚖️","tokens_out":9172,"duration_ms":83012,"temperature":0.7,"pith_summary":"This paper proposes CUI-MET, a decision rule for early-phase oncology trials that randomize patients across several dose levels. The rule assigns each dose a Clinical Utility Index: a weighted average of the probabilities that each binary endpoint, such as efficacy, toxicity, or tolerability, turns out positive, with the weights encoding clinical priorities. The dose with the largest index is declared the Optimal Biological Dose. Both empirical response rates and fitted dose-response models can supply the probabilities, and a bootstrap resampling procedure attaches confidence intervals to each dose's score and estimates how often each dose would be selected as optimal. The paper argues this gives a transparent, flexible, and computationally light alternative to complex joint-model dose-finding designs.","feed_headline":"Weighted endpoint scores pick the best dose in small trials","feed_subtitle":"CUI-MET ranks doses by weighted response probabilities; resampling shows how often each dose wins despite tiny samples.","key_machinery":"The central object is the clinical utility index, $CUI(j) = \\sum_{k=1}^K \\tilde{w}_k P(EP_k=1 \\mid \\mathrm{dose}\\, j)$, with normalized weights $\\tilde{w}_k = w_k/\\sum_l w_l$; it collapses any number of binary endpoints into a single desirability score per dose, and the optimal dose is the one with the largest score. The marginal probabilities can be estimated empirically as observed response means or parametrically from four dose-response models (logit-linear, logit-quadratic, Emax, and exponential), and a stratified bootstrap over patients produces percentile confidence intervals and selection probabilities for each dose. This combination of a simple weighted score, flexible probability estimates, and bootstrap uncertainty is what carries the dose-selection argument.","core_discovery":"The central claim is that dose optimization in a randomized multi-dose trial can be reduced to comparing one number per dose, the clinical utility index $CUI(j) = \\sum_{k=1}^K \\tilde{w}_k P(EP_k=1 \\mid \\mathrm{dose}\\, j)$, a weighted mean of marginal endpoint probabilities. Because only marginal probabilities are needed, the method remains workable with many endpoints and small samples, and the endpoint weights make the benefit-risk trade-off explicit and adjustable. The paper demonstrates, through simulated examples with five dose levels and three endpoints, that different weighting schemes select different doses in clinically sensible ways, and that bootstrap selection percentages quantify confidence in the top-ranked dose, providing the robustness information that formal hypothesis testing cannot meaningfully offer at early-phase sample sizes.","pith_inferences":["A natural extension the paper leaves implicit: the same utility score could drive adaptive randomization or early stopping, using bootstrap selection probabilities to drop poorly performing doses during the trial rather than only ranking them at the end.","The marginal simplification could be stress-tested by comparing CUI-MET's chosen dose against a joint-model-based choice in simulated trials with strong within-patient correlations; the paper reports generally consistent results under correlation but does not quantify when the ranking would diverge.","The weighting sensitivity could be turned into a formal robustness metric: report the range of weights over which a given dose remains optimal, making the subjectivity of weights explicit rather than hidden."],"forward_implications":["The same trial data can support different dosing recommendations: when efficacy is weighted heavily, higher doses tend to win, while shifting weight to toxicity or tolerability can move the optimal dose downward.","Bootstrap selection frequencies give an interpretable measure of confidence; a trial team could require the top dose's selection probability to exceed a chosen threshold before committing to it as the optimal biological dose.","The index remains computable with many binary endpoints because it avoids specifying a joint distribution, which becomes impractical beyond two or three outcomes with small samples.","Empirical and parametric estimates can be mixed across endpoints, letting users exploit dose-response structure where it is credible and stay agnostic where it is not."],"supporting_citations":[{"why":"Establishes the clinical utility index as an integrated measure of risk-benefit over an exposure range, the conceptual foundation of CUI-MET.","marker":"Rowland M. 1980"},{"why":"Introduces clinical utility assessments in early clinical development, motivating the use of a utility score for dose decisions.","marker":"Khan, Perlstein and Krishna 2009"},{"why":"Applies a clinical utility index to compare compounds on benefit-risk, validating the composite-score approach.","marker":"Ouellet, Werth et al. 2009"},{"why":"Supplies desirability-function rescaling that lets independent expected outcomes be combined on a 0-1 scale.","marker":"Coffey, Gennings and Moser 2007"},{"why":"Provides the specific weighting and normalization scheme (weights from 0 to 5 normalized to sum to one) that CUI-MET adopts.","marker":"Winzenborg, Soliman and Shebley 2021"},{"why":"Reports similar operating characteristics for independent versus jointly modelled outcomes, the stated justification for modeling each endpoint marginally.","marker":"Cunanan and Koopmeiners 2014"},{"why":"Describes the generalized MCP-Mod framework used to fit the parametric dose-response models for endpoint probabilities.","marker":"Pinheiro, Bornkamp et al. 2014"},{"why":"Supplies the software implementation used for the Emax and exponential model fits in the modeling method.","marker":"Bornkamp, Pinheiro et al. 2025"},{"why":"Provides the bootstrap implementation used for confidence intervals and optimal-dose selection probabilities.","marker":"Canty and Ripley 2024"}],"fun_headline_variants":["One weighted score per dose guides early oncology dose choice","CUI-MET: Resampling confirms top dose from weighted endpoints","Simplify multi-outcome trials with a clinical utility index","Bootstrap-validated dose ranking from weighted endpoint probabilities","Interactive tool computes utility index for optimal dose selection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the clinical value of a dose is fully captured by a weighted arithmetic mean of marginal endpoint probabilities, so that correlations among outcomes within individual patients do not change which dose is best.","fun_headline_variants_meta":{"raw":{"variants":["One weighted score per dose guides early oncology dose choice","CUI-MET: Resampling confirms top dose from weighted endpoints","Simplify multi-outcome trials with a clinical utility index","Bootstrap-validated dose ranking from weighted endpoint probabilities","Interactive tool computes utility index for optimal dose selection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1490,"prompt_tokens":969,"completion_tokens":521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":442}},"tokens_in":585,"tokens_out":521,"duration_ms":5144,"temperature":1.0,"reasoning_tokens":442,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:46:41.651924+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a simulated trial with strong positive within-patient correlation between efficacy and toxicity, choose weights so the marginal weighted mean favors one dose, and define a patient-level utility that penalizes joint occurrence; if CUI-MET selects the dose favored marginally while a joint-utility evaluation favors another dose, the marginal simplification's limit is demonstrated.","supporting_citations":[{"cited_title":"We introduce the clinical utility index dose optimization approach for multiple-dose randomized trial designs (CUI-MET)","cited_arxiv_id":null,"evidence_quote":"Supplies desirability-function rescaling that lets independent expected outcomes be combined on a 0-1 scale."}],"review_version":1}