Evaluating the Reproducibility of Research in Obstetrics and Gynecology
Pith reviewed 2026-05-24 20:43 UTC · model grok-4.3
The pith
Trials in obstetrics and gynecology rarely include data availability statements, preregistration, protocols, or conflict disclosures.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By examining 300 trials, the authors found that none supplied links to protocols or statements on material availability, none were replication studies, only 10.58% provided data availability statements, 5.82% addressed preregistration, 25.85% omitted conflict of interest reporting, and 54.08% did not state funding origins. This leads to the conclusion that research in obstetrics and gynecology is not consistently reproducible and frequently lacks conflict of interest disclosure.
What carries the argument
A checklist of fourteen indicators including data availability, analysis scripts, pre-registration, study protocols, open access, conflict of interest statements, and funding sources, applied to a sample of 300 empirical trials.
Load-bearing premise
The fourteen chosen indicators adequately capture reproducibility and the three-hundred-trial sample represents the broader literature in the field.
What would settle it
A larger or differently selected sample of trials where most include data links, protocols, preregistration statements, and conflict disclosures would contradict the central claim.
Figures
read the original abstract
Objective: Reproducibility is a core tenet of scientific research. A reproducible study is one where the results can be recreated by different investigators in different circumstances using the same methodology and materials. Unfortunately, reproducibility is not a standard to which the majority of research is currently adherent. Methods: We objectively evaluated 300 trials in the field of Obstetrics and Gynecology for fourteen indicators of reproducibility. These indicators include availability of data, analysis scripts, pre-registration information, study protocols and whether or not the study was available via Open Access. We also assessed the trials for financial conflict of interest statements and source of funding. Results: Of the 300 trials in our sample, 208 contained empirical data that could be assessed for reproducibility. None of the trials in our sample provided a link to their protocols or provided a statement on availability of materials. None were replication studies. Just 10.58% provided a statement regarding their data availability, while only 5.82% provided a statement on preregistration. 25.85% failed to report the presence or absence of conflicts of interest and 54.08% did not state the origin of their funding. Conclusion: Research in the field of Obstetrics and Gynecology is not consistently reproducible and frequently lacks conflict of interest disclosure. Consequences of this could be far-reaching and include increased research waste, widespread acceptance of misleading results and erroneous conclusions guiding clinical decision-making.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript evaluates 300 trials in Obstetrics and Gynecology against 14 indicators of reproducibility (data availability, analysis scripts, preregistration, protocols, materials, open access, COI statements, funding source). Among the 208 empirical trials, none provided protocol links or materials-availability statements, none were replication studies, 10.58% included data-availability statements, 5.82% mentioned preregistration, 25.85% omitted COI statements, and 54.08% did not report funding origin. The paper concludes that research in the field is not consistently reproducible and frequently lacks conflict-of-interest disclosure.
Significance. If the chosen indicators are accepted as valid proxies, the concrete empirical counts (explicit n=300, 208 empirical subset, specific percentages) provide a field-specific snapshot of reporting practices that could motivate transparency initiatives in clinical research. The work supplies direct, countable data rather than fitted models or derivations.
major comments (2)
- [Abstract, Conclusion] Abstract, Conclusion: The central claim that 'Research in the field of Obstetrics and Gynecology is not consistently reproducible' is not directly supported by the reported data, which consist exclusively of presence/absence of reporting statements; the manuscript contains no reproduction attempts, no verification of actual data or protocol availability beyond statements, and no test of whether the 14 indicators predict replicability. This proxy-to-conclusion link is load-bearing for the title and final sentence.
- [Methods] Methods (implied by abstract and reader's assessment): The full selection criteria used to identify the 300 trials, the exact operational definitions and coding rules for each of the 14 indicators, and any inter-rater reliability statistics are not visible, preventing assessment of whether the reported percentages (e.g., 10.58% data statements) are reproducible or representative.
Simulated Author's Rebuttal
We thank the referee for the detailed review and the opportunity to clarify our work on reproducibility indicators in Obstetrics and Gynecology. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract, Conclusion] Abstract, Conclusion: The central claim that 'Research in the field of Obstetrics and Gynecology is not consistently reproducible' is not directly supported by the reported data, which consist exclusively of presence/absence of reporting statements; the manuscript contains no reproduction attempts, no verification of actual data or protocol availability beyond statements, and no test of whether the 14 indicators predict replicability. This proxy-to-conclusion link is load-bearing for the title and final sentence.
Authors: We agree that the study measures the presence or absence of reporting statements and other indicators widely viewed as necessary for reproducibility, rather than conducting reproduction attempts or testing predictive validity of the indicators. These indicators are drawn from established reproducibility frameworks, and their near-absence supports the inference that most studies in the sample are not positioned for consistent reproduction. To strengthen precision, we will revise the abstract and conclusion to state that the research 'lacks key indicators of reproducibility' and 'frequently omits elements required for reproducibility' while retaining the overall assessment. This constitutes a partial revision. revision: partial
-
Referee: [Methods] Methods (implied by abstract and reader's assessment): The full selection criteria used to identify the 300 trials, the exact operational definitions and coding rules for each of the 14 indicators, and any inter-rater reliability statistics are not visible, preventing assessment of whether the reported percentages (e.g., 10.58% data statements) are reproducible or representative.
Authors: The Methods section describes the sample of 300 trials and the 14 indicators evaluated, but we accept that greater detail on selection criteria, exact operational definitions, and coding rules would improve transparency and allow readers to assess reproducibility of the percentages. We will expand the Methods section with these specifics. If multiple independent coders were employed, inter-rater reliability statistics will be added; if a single coder was used, this will be stated explicitly as a methodological detail. revision: yes
Circularity Check
Empirical indicator count with no derivation chain
full rationale
The paper conducts a direct empirical audit: it samples 300 trials, applies 14 pre-specified binary indicators (data availability, preregistration, COI statements, etc.), and reports observed frequencies. No equations, fitted parameters, model predictions, or self-citation chains are used to derive the central claims. The conclusion that reproducibility indicators are infrequently reported follows immediately from the tabulated counts in the 208 empirical trials; the analysis is therefore self-contained and contains no circular reduction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The 300 trials constitute a representative sample of obstetrics and gynecology research and the 14 indicators are sufficient to assess reproducibility.
Reference graph
Works this paper leans on
-
[1]
Plesser HE. Reproducibility vs. Replicability: A Brief History of a Confused Terminology. Frontiers in Neuroinformatics . 2018;11. doi: 10.3389/fninf.2017.00076
-
[2]
Impact of data transparency: Scientific publications
Joshi M, Bhardwaj P. Impact of data transparency: Scientific publications. Perspect Clin Res . 2018;9(1):31-36
work page 2018
-
[3]
Novelty in science should not come at the cost of reproducibility
Holding AN. Novelty in science should not come at the cost of reproducibility. FEBS J . 2019;103:843
work page 2019
-
[4]
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis JPA, Greenland S, Hlatky MA, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet . 2014;383(9912):166-175
work page 2014
-
[5]
Jack A. Tamiflu: “a nice little earner.” BMJ . 2014;348(apr09 2):g2524-g2524. doi: 10.1136/bmj.g2524
-
[6]
Tamiflu: the battle for secret drug data
Payne D. Tamiflu: the battle for secret drug data. BMJ . 2012;345:e7303
work page 2012
-
[7]
Complications: tracking down the data on oseltamivir
Cohen D. Complications: tracking down the data on oseltamivir. BMJ . 2009;339:b5387
work page 2009
-
[8]
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD008965.pub4/full
-
[9]
Hardwicke TE, Wallach JD, Kidwell M, Ioannidis J. An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). April
work page 2014
-
[10]
doi: 10.31222/osf.io/6uhg5
-
[11]
Genome-wide Estrogen Receptor-α activation is sustained, not cyclical
Holding AN, Cullen AE, Markowetz F. Genome-wide Estrogen Receptor-α activation is sustained, not cyclical. Elife . 2018;7. doi: 10.7554/eLife.40854
-
[12]
Cofactor Dynamics and Sufficiency in Estrogen Receptor–Regulated Transcription
Shang Y, Hu X, DiRenzo J, Lazar MA, Brown M. Cofactor Dynamics and Sufficiency in Estrogen Receptor–Regulated Transcription. Cell . 2000;103(6):843-852
work page 2000
-
[13]
Métivier R, Penot G, Hübner MR, et al. Estrogen receptor-alpha directs ordered, cyclical, and combinatorial recruitment of cofactors on a natural target promoter. Cell . 2003;115(6):751-763
work page 2003
-
[14]
Guidelines for reporting meta-epidemiological methodology research
Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med . 2017;22(4):139-142
work page 2017
-
[15]
Repeatability of published microarray gene expression analyses
Ioannidis JPA, Allison DB, Ball CA, et al. Repeatability of published microarray gene expression analyses. Nat Genet . 2009;41(2):149-155
work page 2009
-
[16]
Public availability of published research data in high-impact journals
Alsheikh-Ali AA, Qureshi W, Al-Mallah MH, Ioannidis JPA. Public availability of published research data in high-impact journals. PLoS One . 2011;6(9):e24357
work page 2011
-
[17]
Hardwicke TE, Mathur MB, MacDonald K, et al. Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition. R Soc Open Sci . 2018;5(8):180448
work page 2018
-
[18]
Data sharing in PLOS ONE: An analysis of Data Availability Statements
Federer LM, Belter CW, Joubert DJ, et al. Data sharing in PLOS ONE: An analysis of Data Availability Statements. PLoS One . 2018;13(5):e0194768
work page 2018
-
[19]
Data availability for industry sponsored trials: what should medical journals require? BMJ
Steinbrook R, Kassirer JP. Data availability for industry sponsored trials: what should medical journals require? BMJ . 2010;341:c5391
work page 2010
-
[20]
1,500 scientists lift the lid on reproducibility
Baker M. 1,500 scientists lift the lid on reproducibility. Nature . 2016;533(7604):452-454
work page 2016
-
[21]
Preparing and Publishing a Scientific Manuscript
Jirge PR. Preparing and Publishing a Scientific Manuscript. J Hum Reprod Sci . 2017;10(1):3-9
work page 2017
-
[22]
Tools and techniques for computational reproducibility
Piccolo SR, Frampton MB. Tools and techniques for computational reproducibility. Gigascience . 2016;5(1):30
work page 2016
-
[23]
The preregistration revolution
Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci U S A . 2018;115(11):2600-2606
work page 2018
-
[24]
Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals
Howard B, Scott JT, Blubaugh M, Roepke B, Scheckel C, Vassar M. Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals. PLoS One . 2017;12(7):e0180986
work page 2017
-
[25]
Dodson TB. The Problem With P-Hacking. Journal of Oral and Maxillofacial Surgery . 2019;77(3):459-460. doi: 10.1016/j.joms.2018.12.034
- [26]
-
[27]
Zou CX, Becker JE, Phillips AT, et al. Registration, results reporting, and publication bias of clinical trials supporting FDA approval of neuropsychiatric drugs before and after FDAAA: a retrospective cohort study. Trials . 2018;19(1). doi: 10.1186/s13063-018-2957-0
-
[28]
Association of the FDA Amendment Act with trial registration, publication, and outcome reporting
Phillips AT, Desai NR, Krumholz HM, Zou CX, Miller JE, Ross JS. Association of the FDA Amendment Act with trial registration, publication, and outcome reporting. Trials . 2017;18(1):333
work page 2017
-
[29]
Under-reporting of conflicts of interest among trialists: a cross-sectional study
Rasmussen K, Schroll J, Gøtzsche PC, Lundh A. Under-reporting of conflicts of interest among trialists: a cross-sectional study. J R Soc Med . 2015;108(3):101-107
work page 2015
-
[30]
Dunn AG, Coiera E, Mandl KD, Bourgeois FT. Conflict of interest disclosure in biomedical research: A review of current practices, biases, and the role of public registries in improving transparency. Res Integr Peer Rev . 2016;1. doi: 10.1186/s41073-016-0006-7
-
[31]
Kesselheim AS, Wang B, Studdert DM, Avorn J. Conflict of interest reporting by authors involved in promotion of off-label drug use: an analysis of journal disclosures. PLoS Med . 2012;9(8):e1001280
work page 2012
-
[32]
Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study
Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD. Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ . 2012;345:e5155
work page 2012
-
[33]
Sunshine Act: shedding light on inaccurate disclosures at a gynecologic annual meeting
Thompson JC, Volpe KA, Bridgewater LK, et al. Sunshine Act: shedding light on inaccurate disclosures at a gynecologic annual meeting. Am J Obstet Gynecol . 2016;215(5):661.e1-e661.e7
work page 2016
-
[34]
Transparency of industry payments needed in clinical practice guidelines
Vassar M, Bibens M, Wayant C. Transparency of industry payments needed in clinical practice guidelines. BMJ Evid Based Med . 2019;24(1):8-9
work page 2019
-
[35]
Financial Conflicts of Interest Among Authors of Urology Clinical Practice Guidelines
Carlisle A, Bowers A, Wayant C, Meyer C, Vassar M. Financial Conflicts of Interest Among Authors of Urology Clinical Practice Guidelines. Eur Urol . 2018;74(3):348-354
work page 2018
-
[36]
Evaluating Industry Payments Among Dermatology Clinical Practice Guidelines Authors
Checketts JX, Sims MT, Vassar M. Evaluating Industry Payments Among Dermatology Clinical Practice Guidelines Authors. JAMA Dermatol . 2017;153(12):1229-1235
work page 2017
-
[37]
doi: 10.1126/science.aab2374
-
[38]
Repeat: a framework to assess empirical reproducibility in biomedical research
McIntosh LD, Juehne A, Vitale CRH, et al. Repeat: a framework to assess empirical reproducibility in biomedical research. BMC Med Res Methodol . 2017;17(1):143
work page 2017
-
[39]
doi: 10.1097/SAP.0000000000001476
-
[40]
Jorski A, Scott J, Heavener T, Vassar M. Reporting guideline and clinical trial registration requirements in gastroenterology and hepatology journals. Int J Evid Based Healthc . 2018;16(2):119-127
work page 2018
-
[41]
Trial registration and adherence to reporting guidelines in cardiovascular journals
Sims MT, Bowers AM, Fernan JM, Dormire KD, Herrington JM, Vassar M. Trial registration and adherence to reporting guidelines in cardiovascular journals. Heart . 2018;104(9):753-759
work page 2018
-
[42]
The availability of research data declines rapidly with article age
Vines TH, Albert AYK, Andrew RL, et al. The availability of research data declines rapidly with article age. Curr Biol . 2014;24(1):94-97
work page 2014
-
[43]
Hardwicke TE, Ioannidis JPA. Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLoS One . 2018;13(8):e0201856. Figure
work page 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.