Pith. sign in

REVIEW 3 major objections 5 minor 26 references

My Statistics is Better than Yours

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The essay argues that no statistical school can be rationally preferred over all others, so methodological choice should be context-dependent.

desk verdict A clear, honest philosophy-of-statistics essay that borrows 'operational objectivity' and applies it to the choice of statistical schools; the argument's formal core rests on a premise the author admits to not defending, so the conclusion runs ahead of the support. read the letter →

arxiv 2412.10296 v3 pith:ALLQUQIR submitted 2024-12-13 stat.OT

Simon Benhaïem This is my paper · ORCID
classification stat.OT MSC 62A0162C05
keywords operationalobjectivitynormativetheoryforstatisticsBayesianismFrequentismDutchbookargumentsEllsbergparadoxcontext-dependentmethodologydecision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This essay argues that the choice between statistical schools—Bayesianism, Frequentism, Likelihoodism, and others—cannot be settled by rationality alone, because each school rests on value judgments that empirical evidence cannot refute. The author uses the Ellsberg paradox to show that a descriptive violation of Bayesian norms does not falsify Bayesianism as a normative theory. From that, the essay concludes that universalist attempts to crown one school as the single correct foundation fail, and that methodologists should instead choose a statistical framework context by context, aligning the school with the value judgments inherent in the research question. The proposed standard is 'operational objectivity,' borrowed from recent philosophy of science, in which objectivity and method choice are shaped by the particular context of application. A short case study of a procrastination experiment illustrates that mixing subjective Bayesian modelling with frequentist hypothesis testing can be coherent when each part matches its context.

What carries the argument

The argument's load-bearing machinery is the premise chain P1–P4, with the premise that a school of statistics follows from the choice of a normative decision theory doing the heavy lifting. The essay supplements this with two conceptual tools: Dutch Book Arguments, which establish only a necessary condition for rationality since multiple schools can avoid Dutch Books, and Ellsberg-style ambiguity, which provides a descriptive violation of Bayesian axioms without touching the normative claim. The positive mechanism is the concept of 'operational objectivity' from philosophy of science: objectivity is not a single property but is achieved when methods are chosen in light of the context and value judgments of the field, so the same standard warrants different statistical schools in different settings. The universalist alternative is attacked through the likelihood-function existence problem: for some statistical endeavours no dominating reference measure exists, so a universal foundation such as Likelihoodism either leaves tasks undefined or must rely on future fixes.

What would settle it

A refutation would be a worked example in which a single normative decision theory, accepted by both camps, uniquely prescribes a complete statistical protocol for a non-trivial class of tasks—for instance, deriving a uniformly most powerful test and a Bayesian posterior from the same axioms. If such a derivation exists for the composite-testing setting that the essay uses as its main universalist failure, the impossibility claim collapses. A weaker falsifier would be a research context in which all context features and value judgments agree but the context-dependent rule still fails to single out a school.

Watch

Extended reading notes

Core claim

The central claim is a premise-conclusion argument: empirical falsification cannot refute a normative school; there are multiple normative decision theories; a school of statistics is implied by the choice of a normative decision theory; and researchers must choose a normative theory for statistics. The essay argues that methodologists therefore need a way to choose between normative systems, and that empirical tests cannot supply it. Its positive discovery is that the universalist response—pick one foundation and declare everything else irrational—cannot resolve the multiplicity, because each school can dismiss ill-defined tasks as irrational, leaving an arbitrary choice intact. The proposed resolution is the context-dependent approach: for each research context, the value judgments built into the question warrant one school over another, so different parts of a single study may legitimately use different schools. The case study shows a Bayesian model for belief updating alongside frequentist significance tests, with the warrant coming from the respective contexts.

Load-bearing premise

The load-bearing premise is that choosing a decision-theoretic foundation necessarily dictates which school of statistics a researcher must use; the author admits this premise is 'too difficult to defend in this essay,' yet without it the multiplicity of normative theories does not force a meta-choice.

Editorial extensions

If this is right

  • Methodologists should stop searching for a single correct statistical framework and instead justify each protocol choice by the research context.
  • Researchers can legitimately mix schools within one study, as long as each school is warranted by the context of the specific sub-task.
  • Descriptive failures like the Ellsberg paradox cannot by themselves refute a normative statistical theory.
  • Universalist defences must give a non-arbitrary account of why tasks outside the chosen school are 'irrational'; otherwise the multiplicity of schools remains.
  • The context-dependent approach makes the value judgments behind statistical choices explicit, which should make multiple-testing and other protocol errors more visible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The context-dependent rule is under-specified as stated; it needs an operational criterion for when a research context 'warrants' a school, which the essay leaves open.
  • If correct, the argument extends beyond schools of statistics to any normative choice among statistical methodologies, such as model-selection criteria or causal-inference frameworks, wherever empirical success underdetermines the normative rule.
  • A testable extension would be a decision procedure that takes context features—sample size, prior information, decision stakes, conventions of the field—and outputs a warranted statistical school; the essay does not provide one.
  • The essay's P3 is the hinge: showing that a chosen decision-theoretic foundation can yield multiple schools, or that a school can be founded on multiple decision theories, would turn the meta-problem into a classification exercise rather than an impossibility.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This essay argues that no single statistical school (Bayesian, Frequentist, Likelihoodist, etc.) can be rationally selected as a universally binding normative foundation, because empirical falsification cannot refute normative claims and multiple normative decision systems (NSDs) exist. The author proposes that researchers choose a statistical school context-dependently, drawing on Douglas's "operational objectivity" and the earlier application by Worsdale and Wright. The argument is presented as a premise-conclusion structure in Section 3, a critique of the universalist approach in Section 4.1, and an illustrative case study in Section 4.3.

Significance. The essay is clearly written and commendably states its own limitations, including the Appendix admission that premise P3 is undefended. It brings a relevant philosophical framework (operational objectivity) into the statistics methodology debate and illustrates it with a concrete empirical study of motivated procrastination. If the central premise could be supported, the paper would provide a substantial argument against methodological universalism. In its current form, however, the main conclusion rests on an admitted gap, so the contribution is at this stage an insightful position piece rather than a complete argument.

major comments (3)
  1. [Section 3, premise (P3); Appendix A] The inference from (P2) to (I1) requires that each normative decision system (NSD) implies a distinct statistical school. The Appendix states verbatim that P3 is "too difficult to defend in this essay." Without a defense of P3, the multiplicity of NSDs does not establish a multiplicity of statistical schools, and the cited results of Savage, Wald, and Lehmann do not fill the gap: they show sufficiency or admissibility of Bayesian rules under decision-theoretic axioms, but not that an NSD uniquely determines a school. In fact, minimax theory is listed as an NSD and can justify both frequentist minimax tests and Bayes rules for least-favorable priors, which are different schools reporting different quantities. The central conclusion therefore needs either a defense of P3 or a reformulation of the thesis to a claim about multiplicity of NSDs only.
  2. [Section 4.1] The rejection of the universalist approach relies on examples in which Likelihoodism yields "silly" tests or no well-defined procedure (e.g., composite testing without a dominating reference measure). These examples show that one particular universalist foundation has gaps, not that no universalist foundation could exist. The author acknowledges that a universalist can declare such endeavours outside the scope of statistics, but the response that "the multiplicity of schools remains" is insufficient: a universalist may defend a single school plus a demarcation criterion for legitimate statistical endeavours. The essay should either provide a general argument that any such demarcation must be arbitrary or restrict its conclusion to a critique of specific universalist proposals.
  3. [Section 1 and Section 5] The claim that "it becomes impossible to choose between the two schools" is stronger than what the argument supports. The premises (P1)-(P4) only imply that empirical falsification cannot settle the choice; they do not rule out non-empirical rational criteria such as axiomatic derivations, pragmatic adequacy, or ethical and political values. The context-dependent approach itself relies on such non-empirical judgments when deciding whether a context "fits" a school. To sustain the impossibility claim, the essay must address whether any non-empirical adjudication is possible, or must be weakened to "empirical evidence alone cannot decide."
minor comments (5)
  1. [Section 3.1] The abbreviation "GRLT" should be "GLRT" (generalized likelihood ratio test).
  2. [References] The Italian title of de Finetti (1931) contains a rendering artifact, "probabilitytextà," which should be corrected to the proper spelling of the original title.
  3. [Section 4.3] The name "Bonferonni" is misspelled; it should be "Bonferroni."
  4. [Section 2.1] The sentence "A rational agent is one that can never be Dutch-Booked" conflates rationality with Dutch-book avoidance; a brief clarifying remark would avoid a potential misreading of the normative claim.
  5. [Biographical note] The first-person biographical note is unusual for a journal article; if the target venue is not an essay-oriented outlet, this section should be removed or moved to acknowledgments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the essay derives no quantitative result, fits no parameter, and its conclusion does not reduce to its premises by definition; the conceded gap on P3 is an unsupported premise, not a circular step.

full rationale

The paper is a philosophical essay arguing that empirical falsification cannot adjudicate between normative statistical schools, so a context-dependent selection of norms is warranted. The derivation chain is not formal-equational: no parameter is fitted, no quantity is predicted from fitted inputs, and no known result is renamed as a new organization. The author's stated inspirations (Worsdale & Wright 2021; Douglas 2004) are external philosophical sources, and the decision-theoretic authorities (Savage 1954; Wald 1947; Lehmann et al. 1986) are cited as external support rather than as the author's own prior results. Inspecting the premise-conclusion block in Section 3, (I1) follows from (P2)-(P4) and (C) follows from (P1) and (I1), with no premise equivalent to the conclusion by construction. The real vulnerability is that premise (P3), which transfers multiplicity of decision-theoretic foundations to multiplicity of statistical schools, is conceded in Appendix A: 'P3 is too difficult to defend in this essay.' That is an admitted gap in support, not circularity: the essay does not assume the conclusion to prove the conclusion, and it does not disguise a fitted value as a prediction. The absence of any self-citation chain or definitional equivalence makes the appropriate circularity score 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The essay is a non-mathematical position paper. It has no fitted parameters or invented entities. Its argument rests on four explicit premises, of which P3 is the most fragile and is admitted as undefended.

assumptions (4)
  • domain assumption Empirical falsification does not lead to normative failing (P1).
    Normative systems prescribe how one ought to act, so observed deviations do not refute the prescription. This premise is defended in the Appendix.
  • domain assumption There exist multiple normative schools for decision theory (P2).
    The essay lists Bayesianism, Minimax theory, Frequentism, Fiducial probabilities, and Likelihoodism as existing alternatives.
  • ad hoc to paper A school of statistics is implied by the choice of the normative system (P3).
    This premise is needed to move from decision-theoretic pluralism to statistical pluralism. The author states in the Appendix that it is too difficult to defend.
  • domain assumption Researchers ought to choose a normative theory of statistics (P4).
    The essay argues that communication and scientific integrity require shared methodological standards, citing the Dutch code of conduct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of My Statistics is Better than Yours." pith.science (2026). https://pith.science/paper/ALLQUQIR

@misc{pith2026241210296,
  author       = {Pith},
  title        = {Pith review of: My Statistics is Better than Yours},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ALLQUQIR}},
  note         = {Machine review of arXiv:2412.10296}
}
read the original abstract

Statistical schools-such as Bayesianism and Frequentism-are often presented as competing frameworks, each claiming technical rigour and superiority. Frequentism emphasizes objective inferences through repeated sampling, while Bayesianism incorporates prior beliefs and updates them with new evidence. Despite their strengths, neither school proves universally applicable, and the pursuit of a single "correct" statistical framework is ultimately misguided. Instead, this essay advocates for a context-dependent approach to statistical norms, drawing on Douglas (2004)'s concept of "operational objectivity". The idea is that by aligning the context of the research question with the value judgments inherent to its field, a certain statistical paradigm is warranted. This essay explores the decision-theoretic foundations of Bayesianism, examines its descriptive limitations as highlighted by the Ellsberg paradox, and addresses the challenges of comparing different normative systems.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 23 canonical work pages

  1. [1]

    , Aliyev, S A O

    asli2017handbook APACrefauthors Asli, K H. , Aliyev, S A O. , Thomas, S. \ Gopakumar, D A. APACrefauthors \ 2017 . Handbook of research for fluid and solid mechanics: theory, simulation, and experiment Handbook of research for fluid and solid mechanics: theory, simulation, and experiment . CRC Press

    work page 2017

  2. [2]

    \ Braun, H

    benjamini2002john APACrefauthors Benjamini, Y. \ Braun, H. APACrefauthors \ 2002 . John W. Tukey's contributions to multiple comparisons John w. tukey's contributions to multiple comparisons . Annals of Statistics 1576--1594

    work page 2002

  3. [3]

    \ Wolpert, R L

    berger1988likelihood APACrefauthors Berger, J O. \ Wolpert, R L. APACrefauthors \ 1988 . The likelihood principle The likelihood principle

    work page 1988

  4. [4]

    APACrefauthors \ 1962

    birnbaum1962foundations APACrefauthors Birnbaum, A. APACrefauthors \ 1962 . On the foundations of statistical inference On the foundations of statistical inference . Journal of the American Statistical Association 57 298 269--306

    work page 1962

  5. [5]

    , Friedrichsen, J

    cordes2024motivated APACrefauthors Cordes, C. , Friedrichsen, J. \ Schudy, S. APACrefauthors \ 2024 . Motivated procrastination Motivated procrastination

    work page 2024

  6. [6]

    APACrefauthors \ 1931

    de1931sul APACrefauthors de Finetti, B. APACrefauthors \ 1931 . Sul S ignificato S oggettivo della P robabilittext \`a Sul S ignificato S oggettivo della P robabilittext \`a . Fundamenta mathematicae 17

    work page 1931

  7. [7]

    APACrefauthors \ 2004

    douglas2004irreducible APACrefauthors Douglas, H. APACrefauthors \ 2004 . The irreducible complexity of objectivity The irreducible complexity of objectivity . Synthese 138 453--473

    work page 2004

  8. [8]

    APACrefauthors \ 1961

    ellsberg1961risk APACrefauthors Ellsberg, D. APACrefauthors \ 1961 . Risk, ambiguity, and the S avage axioms Risk, ambiguity, and the S avage axioms . The quarterly journal of economics 75 4 643--669

    work page 1961

Show all 26 references
  1. [9]

    \ Le Breton, M

    epstein1993dynamically APACrefauthors Epstein, L G. \ Le Breton, M. APACrefauthors \ 1993 . Dynamically consistent beliefs must be Bayesian Dynamically consistent beliefs must be bayesian . Journal of Economic theory 61 1 1--22

    1993

  2. [10]

    \ Savage, L J

    halmos1949application APACrefauthors Halmos, P R. \ Savage, L J. APACrefauthors \ 1949 . Application of the Radon-Nikodym theorem to the theory of sufficient statistics Application of the radon-nikodym theorem to the theory of sufficient statistics . The Annals of Mathematical...

    1949

  3. [11]

    APACrefauthors \ 1982

    jeffrey1982sure APACrefauthors Jeffrey, R. APACrefauthors \ 1982 . The sure thing principle The sure thing principle . PSA: Proceedings of the biennial meeting of the philosophy of science association Psa: Proceedings of the biennial meeting of the philosophy of science associ...

    1982

  4. [12]

    APACrefauthors \ 1921

    keynes2013treatise APACrefauthors Keynes, J M. APACrefauthors \ 1921 . A treatise on probability A treatise on probability . Courier Corporation

    1921

  5. [13]

    APACrefauthors \ 2020

    kleijn2020frequentist APACrefauthors Kleijn, B. APACrefauthors \ 2020 . The frequentist theory of Bayesian statistics. The frequentist theory of bayesian statistics. New York, NY: Springer-Verlag New York

    2020

  6. [14]

    APACrefauthors \ 2024

    koning2024continuous APACrefauthors Koning, N W. APACrefauthors \ 2024 . Continuous Testing Continuous testing . arXiv preprint arXiv:2409.05654

    2024 arXiv

  7. [15]

    , Ramdas, A

    larsson2024numeraire APACrefauthors Larsson, M. , Ramdas, A. \ Ruf, J. APACrefauthors \ 2024 . The numeraire e-variable The numeraire e-variable . arXiv preprint arXiv:2402.18810

    2024 arXiv

  8. [16]

    , Romano, J P

    lehmann1986testing APACrefauthors Lehmann, E L. , Romano, J P. \ Casella, G. APACrefauthors \ 1986 . Testing statistical hypotheses Testing statistical hypotheses \ ( 3). Springer

    1986

  9. [17]

    APACrefauthors \ 2024

    sep-epistemology-bayesian APACrefauthors Lin, H. APACrefauthors \ 2024 . Bayesian E pistemology Bayesian E pistemology . E N. Zalta\ U. Nodelman\ ( ), The Stanford Encyclopedia of Philosophy The Stanford encyclopedia of philosophy \ ( S ummer 2024 \ ). Metaphysics Research Lab...

    2024

  10. [18]

    APACrefauthors \ 1994

    megill1994rethinking APACrefauthors Megill, A. APACrefauthors \ 1994 . Rethinking objectivity Rethinking objectivity . Duke University Press

    1994

  11. [19]

    APACrefauthors \ 1989

    nagel1989view APACrefauthors Nagel, T. APACrefauthors \ 1989 . The view from nowhere The view from nowhere . Oxford University Press

    1989

  12. [20]

    \ Wang, R

    ramdas2024hypothesis APACrefauthors Ramdas, A. \ Wang, R. APACrefauthors \ 2024 . Hypothesis Testing with E-values Hypothesis testing with e-values . arXiv preprint arXiv:2410.23614

    2024 arXiv

  13. [21]

    , Sprenger, J

    reiss2014scientific APACrefauthors Reiss, J. , Sprenger, J. \ . APACrefauthors \ 2014 . Scientific objectivity Scientific objectivity . The Stanford encyclopedia of philosophy The stanford encyclopedia of philosophy \ ( \ 0--0). Zalta, Ed

    2014

  14. [22]

    APACrefauthors \ 1954

    savage1972foundations APACrefauthors Savage, L J. APACrefauthors \ 1954 . The foundations of statistics The foundations of statistics . Courier Corporation

    1954

  15. [23]

    APACrefauthors \ 1961

    savage1961foundations APACrefauthors Savage, L J. APACrefauthors \ 1961 . The foundations of statistics reconsidered The foundations of statistics reconsidered . Proceedings of the F ourth B erkeley S ymposium on M athematical S tatistics and P robability, V olume 1: C ontribu...

    1961

  16. [24]

    \ Geary, D C

    stoet2019simplified APACrefauthors Stoet, G. \ Geary, D C. APACrefauthors \ 2019 . A simplified approach to measuring national gender inequality A simplified approach to measuring national gender inequality . PloS one 14 1 e0205349

    2019

  17. [25]

    APACrefauthors \ 1947

    wald1947foundations APACrefauthors Wald, A. APACrefauthors \ 1947 . Foundations of a general theory of sequential decision functions Foundations of a general theory of sequential decision functions . Econometrica, Journal of the Econometric Society 279--313

    1947

  18. [26]

    \ Wright, J

    worsdale2021my APACrefauthors Worsdale, R. \ Wright, J. APACrefauthors \ 2021 . My objectivity is better than yours: contextualising debates about gender inequality My objectivity is better than yours: contextualising debates about gender inequality . Synthese 199 1 1659--1683

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.