REVIEW
Expertises : proc\'{e}dures statistiques d'aide \`{a} la d\'{e}cision
Statistics Theory · Submitted 27 Feb 2006 · 22 references
Pith's verdict: A genuinely novel decision-theoretic frame that reduces p-values and flat-prior posteriors to a vote over conditionally unbiased experts—but the central identification is fixed by an unaxiomatized 1/2 atom-split convention.
T0 · AI review
Why Pith says this
Highlight any text to comment on it.
Referee report
The paper
The paper claims
The paper claims that an 'expert' decision rule is exactly a common Neyman test, and the vote of all experts is the mid-p-value.
The reading
What carries the argument
The machinery is the totally ordered ladder $\Phi_s=\{\varphi(0,1);\ \varphi(k,\beta)=1_{\{K<k\}}+\beta 1_{\{K=k\}},\ k\in\mathbb{R}_+^*, \beta\in\{0,1\};\ \varphi(\infty,0)\}$, with lexicographic order on $(k,\beta)$. Proposition 2.3.1 shows every expert is almost surely inside this ladder, and Definition 2.4.1 turns the ladder into a ballot box: the two measures $m_\theta(]\leftarrow,\varphi[)=E_\theta(\varphi)$ and $m'_\theta(]\leftarrow,\varphi])=E_\theta(\varphi)$ agree everywhere except on the atoms $\{K=k\}$, and the equitable split of each atom gives the vote $Q^\omega_\theta(\{1\})=1-E_\theta(\varphi(k,1/2))$. The same ladder, re-indexed through the essential statistic $K(T)$ of Definition 4.3.1, carries the extension to stable hypotheses, one-sided and two-sided hypotheses, and finally to a probability on the parameter space.
Extended reading notes
Core claim
The central discovery is Proposition 2.3.1: a deterministic rule $\varphi$ is an expert for the choice between $P_0$ and $P_1$, meaning $P_1(\varphi=1)\ge P_0(\varphi=1)$ and the same inequality holds conditionally on every event, if and only if it is almost surely sandwiched between consecutive simple Neyman test functions $\varphi(k,0)\le\varphi\le\varphi(k,1)$ of the likelihood ratio $K=p_0/p_1$, with boundary cases at $k=0$ and $k=\infty$. When the experts vote, using the probability $m_\theta$ on the ordered space of simple test functions defined by $m_\theta(]\leftarrow,\varphi[)=E_\theta(\varphi)$ and splitting the mass at any atom $\{K=k\}$ evenly between the two adjacent test functions, the vote for decision 1 is $1-E_\theta(\varphi(k,1/2))$ when $K(\omega)=k\in(0,\infty)$, with boundary values 1 and 0. Thus the vote is the mid-p-value. In monotone-likelihood models the compatible votes of Section 5.2 extend to a probability on the parameter space; in the normal example that probability is $N(\bar{x},\sigma^2/n)$, the same object a Bayesian obtains from the improper flat prior.
Load-bearing premise
The load-bearing premise is the convention that when the data lands exactly on a boundary value of the likelihood ratio that has positive probability, the experts on that atom split their votes equally between the two adjacent decisions; if that fifty-fifty split is replaced, the vote stops being the mid-p-value.
Editorial extensions
If this is right
- For two simple hypotheses, the output a user receives from the vote is the mid-p-value, a quantity already familiar from testing practice, so the vote can be reported and compared with standard rejection thresholds.
- In monotone-likelihood models the vote for $\{\theta<\theta_0\}$ against $\{\theta\ge\theta_0\}$, evaluated at the boundary $\theta_0$, is both the minimum rejection threshold of the UMP test and a neutral vote: its mean under $P_{\theta_0}$ is $1/2$.
- The compatible-vote construction yields a genuine probability on the order $\sigma$-algebra of $\Theta$; for one-sided hypotheses this probability is not a Bayes posterior from a prior on $\Theta$, and in the normal illustration it is the flat-prior posterior.
- Pure two-sided hypotheses admit only trivial experts, so the practical route for two-sided problems is to reconstruct votes from compatible one-sided votes, not to search for a two-sided expert.
Reading between the lines
- The equitable $1/2$ split is the single convention that makes the vote coincide with the mid-p-value. This is my inference: any other split of the atom defines a family of beta-votes that are not mid-p-values, so the paper's novelty lies in the democratic interpretation of the mid-p rather than in a new numerical quantity.
- The compatibility axioms can be read as a general recipe for constructing evidence measures on ordered parameter spaces: a coherent vote map is a probability on the order $\sigma$-algebra, and the paper's conditions are continuity and boundary conditions for that probability. This recipe could be transferred to partially ordered or product parameter spaces by building transition probabilities, tho
- For discrete models the difference between $m_\theta$ and $m'_\theta$ at atoms means the vote is deliberately non-conservative, using the mid-p rather than the usual upper p. A user who prefers conservative testing would choose the atom split $\beta=1$ or $\beta=0$, so the paper's choice is a substantive small-sample decision, not merely a technical device.
Original abstract
In this study, we introduce a new approach to statistical decision theory. Without using a loss function, we select good decision rules to choice between two hypotheses. We call them "experts". They are globally unbiased but also conditionally unbiased on a family of events. We do not try to define the best expert. We define a probability distribution on the space of "experts". The measure of evidence for a hypothesis is the inductive probability of experts that decide this hypothesis, we call this measure: a "vote". We compare this point of view with the p-values. For some family of hypotheses, the "votes" can define a probability on the space of parameters. We compare these results with the Bayes posterior distributions. We study in detail real-parameter families of distributions with monotone likelihood ratio and multiparameter exponential families.
The compiled record
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
The abstract, sentence by sentence
In this study, we introduce a new approach to statistical decision theory. Without using a loss function, we select good decision rules to choice between two hypotheses. We call them "experts". They are globally unbiased but also conditionally unbiased on a family of events. We do not try to define the best expert. We define a probability distribution on the space of "experts". The measure of evidence for a hypothesis is the inductive probability of experts that decide this hypothesis, we call this measure: a "vote". We compare this point of view with the p-values. For some family of hypotheses, the "votes" can define a probability on the space of parameters. We compare these results with the Bayes posterior distributions. We study in detail real-parameter families of distributions with monotone likelihood ratio and multiparameter exponential families.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | In this study, we introduce a new approach to statistical decision theory. | contribution 1.00 | 0.90 | 0.06 | asserted 0.99 |
| s1 | Without using a loss function, we select good decision rules to choice between two hypotheses. We call them "experts". | method 0.64 | 0.92 | 0.15 | asserted 1.00 |
| s2 | They are globally unbiased but also conditionally unbiased on a family of events. | result 0.71 | 0.79 | 0.09 | asserted 1.00 |
| s3 | We do not try to define the best expert. | limitation 0.66 | 0.87 | 0.07 | asserted 0.95 |
| s4 | We define a probability distribution on the space of "experts". | method 0.82 | 0.94 | 0.06 | asserted 1.00 |
| s5 | The measure of evidence for a hypothesis is the inductive probability of experts that decide this hypothesis, we call this measure: a "vote". | method 0.79 | 0.88 | 0.20 | asserted 1.00 |
| s6 | We compare this point of view with the p-values. | contribution 0.47 | 0.75 | 0.04 | asserted 0.68 |
| s7 | For some family of hypotheses, the "votes" can define a probability on the space of parameters. | result 0.87 | 0.85 | 0.15 | hedged possibility 0.87 |
| s8 | We compare these results with the Bayes posterior distributions. | method 0.40 | 0.84 | 0.07 | asserted 0.76 |
| s9 | We study in detail real-parameter families of distributions with monotone likelihood ratio and multiparameter exponential families. | method 0.61 | 0.90 | 0.05 | asserted 0.96 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
References
Discussion
Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.