Pith. sign in

REVIEW 3 minor 35 references

Subgroup analysis in randomized controlled trials with binary outcomes: dilution and logic-respecting properties

T0 review · 0 major / 3 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Odds ratio for overall population can change in magnitude and direction when two subgroups are combined in binary outcome trials.

desk verdict The paper adds new theorems showing the odds ratio can shift direction or dilute toward 1 when subgroups combine in binary RCTs, while relative response stays logic-respecting. read the letter →

arxiv 2606.17841 v1 pith:Y7VITWEM submitted 2026-06-16 stat.ME

Long-Hao Xu , Yang Han , Tim Friede This is my paper
classification stat.ME
keywords subgroupanalysisoddsratiorelativeresponsebinaryoutcomesrandomizedcontrolledtrialslogic-respectingpropertydilution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proves new theorems showing how the odds ratio behaves when subgroups are pooled in randomized controlled trials with binary outcomes. It demonstrates that the odds ratio can increase, decrease, or reverse direction upon combination, violating the logic-respecting property. By contrast, the relative response always lies between the values for the individual subgroups. These results establish that the odds ratio is inappropriate as an efficacy measure in this setting, while the relative response is appropriate. The work also clarifies the dilution property, where mixing subgroups tends to move the odds ratio toward 1.

What carries the argument

New theorems on the change in odds ratio and relative response when the overall population is formed by a fixed mixture of two subgroups.

What would settle it

An empirical check on trial data showing the overall odds ratio lying strictly between the two subgroup odds ratios in a case where the theorems predict it cannot.

Watch

Extended reading notes

Core claim

The central claim is that several new theorems characterize how the odds ratio for the overall population changes in both magnitude and direction when two subgroups are combined. These results confirm that the odds ratio is inappropriate as an efficacy measure in subgroup analysis with binary outcomes, whereas the relative response is appropriate. The paper presents the formal relationship between the odds ratio and the relative response and clarifies their differences in terms of the logic-respecting property and the dilution property.

Load-bearing premise

The overall population is a fixed mixture of the two subgroups with subgroup-specific response probabilities that are independent of treatment assignment.

Editorial extensions

If this is right

  • The overall odds ratio need not lie between the subgroup-specific odds ratios.
  • Combining subgroups can move the odds ratio toward 1 even when both subgroup odds ratios are away from 1.
  • The relative response always satisfies the logic-respecting property that the overall value lies between the subgroup values.
  • Under some conditions the odds ratio may approximate logic-respecting behavior.
  • Efficacy conclusions drawn from pooled odds ratios can differ in direction from those in the subgroups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Trial reports should report relative response alongside or instead of odds ratio when subgroup analyses are presented.
  • Meta-analyses that pool odds ratios across trials with differing subgroup compositions may inherit the same directional inconsistencies.
  • Software for subgroup analysis could flag cases where the pooled odds ratio falls outside the subgroup range.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper claims to establish several new theorems characterizing how the odds ratio for the overall population changes in both magnitude and direction when two subgroups are combined under a fixed-mixture RCT model with binary outcomes and treatment-independent conditional probabilities. It argues that these properties confirm the odds ratio is inappropriate as an efficacy measure in subgroup analysis, while the relative response is appropriate because it is logic-respecting (overall efficacy lies between subgroup efficacies) and exhibits dilution (mixing moves the measure toward 1). The work also derives the formal relationship between odds ratio and relative response, notes conditions under which the odds ratio may approximately behave as logic-respecting, and illustrates the findings with an example from clinical trial data.

Significance. If the theorems hold, the manuscript provides precise characterizations of the non-collapsibility and dilution behavior of the odds ratio (contrasted with the logic-respecting properties of relative response) under standard RCT assumptions. This strengthens existing literature on marginal versus conditional measures and could guide better practice in reporting subgroup analyses for binary outcomes in randomized trials.

minor comments (3)
  1. [§3] §3 (or wherever the main theorems are stated): the modeling assumptions (fixed mixture proportions, treatment-independent conditional probabilities) should be stated explicitly as a numbered assumption or definition to make the scope of the theorems immediately clear.
  2. [Illustrative example] The illustrative example would benefit from a table listing the subgroup-specific response probabilities, mixture weights, and computed OR and RR values so readers can directly verify the claimed changes in magnitude and direction.
  3. [Abstract] Abstract: the phrase 'new theorems' is used without cross-references; adding '(Theorems 1–3)' or similar would improve traceability.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the careful reading and positive assessment of our manuscript. The summary accurately captures the main contributions regarding the logic-respecting and dilution properties of the odds ratio versus relative response under the fixed-mixture RCT model. We appreciate the recognition that the theorems strengthen the literature on marginal versus conditional measures for binary outcomes in subgroup analysis. As no major comments were raised, we will proceed with a minor revision to address any editorial or presentational suggestions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper derives new theorems on the non-collapsibility, dilution, and logic-respecting properties of the odds ratio versus relative response under the standard fixed-mixture RCT model with treatment-independent subgroup probabilities. These are direct mathematical characterizations from the conditional response probabilities; no parameters are fitted to data subsets, no predictions reduce to inputs by construction, and no load-bearing self-citations or imported uniqueness theorems are invoked to force the central results. The derivations remain self-contained against the stated modeling assumptions.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The work is a theoretical statistical analysis. It relies on standard definitions of conditional probabilities and mixture distributions but introduces no new fitted parameters or invented entities.

assumptions (2)
  • domain assumption Binary outcomes are modeled by subgroup-specific success probabilities that combine via fixed mixture weights to form the overall population.
    Invoked when defining overall odds ratio and relative response from subgroup quantities.
  • domain assumption Treatment assignment is independent of subgroup membership (standard RCT randomization).
    Required for the efficacy measures to be well-defined without confounding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Subgroup analysis in randomized controlled trials with binary outcomes: dilution and logic-respecting properties." pith.science (2026). https://pith.science/paper/Y7VITWEM

@misc{pith2026260617841,
  author       = {Pith},
  title        = {Pith review of: Subgroup analysis in randomized controlled trials with binary outcomes: dilution and logic-respecting properties},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7VITWEM}},
  note         = {Machine review of arXiv:2606.17841}
}
read the original abstract

Subgroup analysis is routinely used in randomized controlled trials to examine whether treatment effects are homogeneous across patient subgroups or differ because of treatment-effect heterogeneity. In this paper, we investigate the properties of the odds ratio and the relative response in subgroup analyses with binary outcomes, extending previous work with new theoretical insights and methodological developments. We establish several new theorems that characterize how the odds ratio for the overall population changes in both magnitude and direction when two subgroups are combined. These results further confirm that the odds ratio is inappropriate as an efficacy measure in this subgroup setting, whereas the relative response is appropriate. We also present the formal relationship between the odds ratio and the relative response, and clarify their differences in terms of the logic-respecting property, that is, whether the overall efficacy lies between the subgroup efficacies, and the dilution property, that is, whether mixing subgroups moves the overall odds ratio toward 1. Although the odds ratio is generally not logic-respecting, it may behave approximately like a logic-respecting efficacy measure under certain conditions. To illustrate our findings, we present an illustrative example based on clinical trial data and discuss its implications for subgroup analysis in randomized controlled trials.

Figures

Figures reproduced from arXiv: 2606.17841 by the authors.

Figure 1
Figure 1. The function g(x) under different combination of (p2, p4, γ+) 12 [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. The relationship between RR and OR Pennello and Xu (2020) show that the odds ratio behaves as a logic￾respecting measure when patients in the control (C) treatment are homo￾geneous across the g + and g − subgroups. In other words, under certain conditions, the odds ratio appears to satisfy the logic-respecting property. However, this phenomenon can be misleading, potentially creating an illu￾sion of consistency. It … view at source ↗
Figure 3
Figure 3. The relationship between OR{g+,g−} and p1 in a REMAP-CAP￾motivated illustrative configuration with γ + = 0.5 the g − subgroup at (p2, p4) = (0.3178, 0.2362), and vary the configuration of the g + subgroup as in Theorem 1. Because the prevalence of the g + subgroup in the target population is not available for this illustrative construction, we set γ + = 0.5 for presentation purposes. Accordingly, the curve of the ov… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 2 canonical work pages

  1. [1]

    R., Steins, M., Ready, N

    Borghaei, H., Paz-Ares, L., Horn, L., Spigel, D. R., Steins, M., Ready, N. E., ... and Brahmer, J. R. (2015). Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer. New England Journal of Medicine , 373(17) , 1627--1639

    2015

  2. [2]

    Colnet, B., Josse, J., Varoquaux, G., and Scornet, E. (2024). Risk ratio, odds ratio, risk difference... Which causal measure is easier to generalize? arXiv preprint , arXiv:2303.16008

    work page arXiv 2024

  3. [3]

    Didelez, V., and Stensrud, M. J. (2022). On the logic of collapsibility for causal effect measures. Biometrical Journal , 64(2) , 235--242

    2022

  4. [4]

    M., and Hsu, J

    Ding, Y., Lin, H. M., and Hsu, J. C. (2016). Subgroup mixable inference on treatment efficacy in mixture populations, with an application to time-to-event outcomes. Statistics in Medicine , 35(10) , 1580--1594

    2016

  5. [5]

    Ding, Y., Wei, Y., and Wang, X. (2020). Logical Inference on Treatment Efficacy When Subgroups Exist. In Design and Analysis of Subgroups with Biopharmaceutical Applications (pp. 209-228). Springer, Cham

    2020

  6. [6]

    A., Platt, A., and Goldstein, D

    Dugger, S. A., Platt, A., and Goldstein, D. B. (2018). Drug development in the era of precision medicine. Nature Reviews Drug discovery , 17(3) , 183--196

    2018

  7. [7]

    Freedman, D. A. (2008). Randomization does not justify logistic regression. Statistical Science , 237-249

    2008

  8. [8]

    J., White, I

    Godolphin, P. J., White, I. R., Tierney, J. F., and Fisher, D. J. (2023). Estimating interactions and subgroup-specific treatment effects in meta-analysis without aggregation bias: A within-trial framework. Research Synthesis Methods , 14(1) , 68-78

    2023

Show all 35 references
  1. [9]

    Good, I. J. (1985). Mathematically natural generalizations of some measures of association and dependence for contingency tables. Journal of Statistical Computation and Simulation , 22 , 93-97

    1985

  2. [10]

    J., and Mittal, Y

    Good, I. J., and Mittal, Y. (1987). The amalgamation and geometry of two-by-two contingency tables. The Annals of Statistics , 15(2) , 694--711

    1987

  3. [11]

    Greenland, S. (2011). Collapsibility. In: Lovric M. (eds) International Encyclopedia of Statistical Science . Springer, Berlin, Heidelberg

    2011

  4. [12]

    Y., Lin, H

    Han, Y., Tang, S. Y., Lin, H. M., and Hsu, J. C. (2022). Exact simultaneous confidence intervals for logical selection of a biomarker cut-point. Biometrical Journal , 64(2) , 272--289

    2022

  5. [13]

    A., and Robins, J

    Hern\' a n, M. A., and Robins, J. M. (2020). Causal Inference: What If. Boca Raton: Chapman & Hall/CRC

    2020

  6. [14]

    J., and Suzuki, E

    Huitfeldt, A., Stensrud, M. J., and Suzuki, E. (2019). On the collapsibility of measures of effect in the counterfactual causal framework. Emerging themes in epidemiology , 16(1) , 1-5

    2019

  7. [15]

    Y., and Hsu, J

    Kil, S., Kaizar, E., Tang, S. Y., and Hsu, J. C. (2022). Confident Statistical Inference with Multiple Outcomes, Subgroups, and Other Issues of Multiplicity

    2022

  8. [16]

    Y., Man, M., Li, Y

    Liu, Y., Tang, S. Y., Man, M., Li, Y. G., Ruberg, S. J., Kaizar, E., and Hsu, J. C. (2016). Thresholding of a continuous companion diagnostic test confident of efficacy in targeted population. Statistics in Biopharmaceutical Research , 8(3) , 325--333

    2016

  9. [17]

    Liu, Y., Tian H., Hsu, J. C. (2021). Subgroups analysis for personalized and precision medicine development. Handbook of Multiple Comparisons (Editors: Xinping Cui, Thorsten Dickhaus, Ying Ding, and Jason C. Hsu) . Chapman & Hall / CRC Handbooks of Modern Statistical Methods

    2021

  10. [18]

    Liu, Y., Wang, B., Tian, H., and Hsu, J. C. (2021). Rejoinder for discussions on correct and logical causal inference for binary and time-to-event outcomes in randomized controlled trials. Biometrical Journal , 64(2) , 246--255

    2021

  11. [19]

    Liu, Y., Wang, B., Yang, M., Hui, J., Xu, H., Kil, S., and Hsu, J. C. (2020). Correct and logical causal inference for binary and time-to-event outcomes in randomized controlled trials. Biometrical Journal , 64(2) , 198--224

    2020

  12. [20]

    M., Xu, H., Ding, Y., and Hsu, J

    Lin, H. M., Xu, H., Ding, Y., and Hsu, J. C. (2019). Correct and logical inference on efficacy in subgroups and their mixture for binary outcomes. Biometrical Journal , 61(1) , 8--26

    2019

  13. [21]

    Ondra, T., Dmitrienko, A., Friede, T., Graf, A., Miller, F., Stallard, N., and Posch, M. (2016). Methods for identification and confirmation of targeted subgroups in clinical trials: a systematic review. Journal of Biopharmaceutical Statistics , 26(1) , 99--119

    2016

  14. [22]

    Pearl, J. (2009). Causality . Cambridge university press

    2009

  15. [23]

    Pennello, G., and Xu, D. (2020). Discussion on ``Correct and logical causal inference for binary and time-to-event outcomes in randomized controlled trials'' by Yi Liu, Bushi Wang, Miao Yang, Jianan Hui, Heng Xu, Siyoen Kil, and Jason C. Hsu. Biometrical Journal , 64(2) , 225--234

    2020

  16. [24]

    Remap-Cap Investigators. (2021). Interleukin-6 receptor antagonists in critically ill patients with Covid-19. New England Journal of Medicine , 384(16) , 1491--1502

    2021

  17. [25]

    V., Parikh, P., Von Pawel, J., Biesma, B., Vansteenkiste, J., Manegold, C.,

    Scagliotti, G. V., Parikh, P., Von Pawel, J., Biesma, B., Vansteenkiste, J., Manegold, C., ... and Gandara, D. (2008). Phase III study comparing cisplatin plus gemcitabine with cisplatin plus pemetrexed in chemotherapy-naive patients with advanced-stage non-small-cell lung can...

    2008

  18. [26]

    O., and Kohlmann, T

    Schmidt, C. O., and Kohlmann, T. (2008). When to use the odds ratio or the relative risk?. International Journal of Public Health , 53(3) , 165--167

    2008

  19. [27]

    Szklo, M., and Nieto, F. J. (2014). Epidemiology: beyond the basics . Jones & Bartlett Publishers

    2014

  20. [28]

    Seltman, H. (2012). Approximations for mean and variance of a ratio. unpublished note

    2012

  21. [29]

    Y., Kaizar, E., and Hsu, J

    Tang, S. Y., Kaizar, E., and Hsu, J. C. (2013). Designing studies for assessing efficacy in mixture populations. In Topics in Applied Statistics (pp. 63-73). Springer, New York, NY

    2013

  22. [30]

    The WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group. (2021). Association between administration of IL-6 antagonists and mortality among patients hospitalized for COVID-19: a meta-analysis. JAMA , 326(6) , 499--518

    2021

  23. [31]

    C., Chen, W., Chew, E

    Wei, Y., Hsu, J. C., Chen, W., Chew, E. Y., and Ding, Y. (2020). A simultaneous inference procedure to identify subgroups from RCTs with survival outcomes: application to analysis of AMD progression studies. arXiv preprint arXiv:2003.10528

    2020

  24. [32]

    C., Chen, W., Chew, E

    Wei, Y., Hsu, J. C., Chen, W., Chew, E. Y., and Ding, Y. (2021). Identification and inference for subgroups with differential treatment efficacy from randomized controlled trials with survival outcomes through multiple testing. Statistics in Medicine , 40(29) , 6523--6540

    2021

  25. [33]

    Woodcock, J. (2016). ``Precision'' drug development?. Clinical Pharmacology & Therapeutics , 99(2) , 152--154

    2016

  26. [34]

    Xi, D., and Bretz, F. (2021). Discussion on 'Correct and logical causal inference for binary and time-to-event outcomes in randomized controlled trials'. Biometrical Journal , 64(2) , 243--245

    2021

  27. [35]

    Zhang, J., and Kai, F. Y. (1998). What's the relative risk?: A method of correcting the odds ratio in cohort studies of common outcomes. JAMA , 280(19) , 1690--1691

    1998

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.