Maximum Likelihood Criterion for Non-nested Model Selection

Min Tsao

Maximum Likelihood Criterion for Non-nested Model Selection

T0 review · 0 major / 3 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The model with the highest maximum likelihood is the consistent choice among non-nested candidates.

desk verdict This paper proposes a penalty-free max-likelihood rule for non-nested models and proves consistency via the KL property. read the letter →

arxiv 2606.22403 v1 pith:2EBVCA2R submitted 2026-06-21 stat.ME

Min Tsao This is my paper · ORCID

classification stat.ME

keywords modelselectionnon-nestedmodelsmaximumlikelihoodcriterionconsistencypenalizationinformationcriteriastatisticalmodeling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper addresses model selection when candidate models are non-nested, meaning none is a special case of another. In this setting, standard penalization methods that favor simpler models can lead to poor choices because they penalize without reason. Instead, the authors introduce a criterion that simply chooses the model whose maximum likelihood is largest, ignoring the number of parameters entirely. They prove that this approach is consistent, so it will select the true model with probability going to one as the data size grows. This makes it appropriate when all models are considered on equal footing without any built-in preference for parsimony.

What carries the argument

The maximum likelihood criterion, which ranks candidate models solely by their maximized likelihood value without adding any penalty for model dimension.

What would settle it

Simulations or data examples with non-nested models in which the maximum likelihood criterion fails to select the true model with probability approaching one as sample size grows, while a penalized criterion succeeds more often.

Watch

Extended reading notes

Core claim

We propose a Maximum Likelihood Criterion for this non-nested setting that selects the candidate model with the highest maximum likelihood. This criterion does not take into consideration the number of parameters of a candidate model. It is well-suited for situations where all candidate models are regarded as equal with no preference for models having fewer parameters. We establish the consistency of this criterion and compare its performance with that of existing penalization-based criteria.

Load-bearing premise

Penalization is counterproductive for non-nested candidate models and that all candidate models are regarded as equal with no preference for models having fewer parameters.

Editorial extensions

If this is right

The criterion consistently selects the true model as sample size increases.
It avoids the bias toward simpler models that penalization introduces in non-nested cases.
It outperforms existing penalization-based criteria such as AIC and BIC when the models are non-nested.
The approach applies when the modeling goal is to identify the best-fitting model without regard to the number of parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

When the data-generating process lies among the non-nested candidates, the criterion identifies it reliably without requiring a nested structure.
Settings where added parameters carry no extra cost, such as certain high-dimensional prediction tasks, can drop the penalty term without loss of consistency.
Finite-sample behavior could be examined by comparing the criterion against cross-validation on the same non-nested collection.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main takeaway is that when models are non-nested and you have no reason to favor simpler ones, just choose the candidate with the highest maximized log-likelihood. The paper shows this rule is consistent because the true model has strictly smaller Kullback-Leibler divergence to the data-generating distribution.

What is new is the explicit framing for the case where penalization is counterproductive. Standard criteria like AIC or BIC add penalties to encourage parsimony, but here the authors drop that entirely and treat all candidates as equal. They supply a consistency argument and run performance comparisons against penalized methods.

The consistency part follows directly from the definition of the rule and standard large-sample behavior, so it holds up on its own terms. The comparisons give some practical sense of when the approach differs from existing tools.

The main limitation is the narrow target. This only applies when you genuinely want to ignore parameter count, which is uncommon even in non-nested settings. Most applications still value some form of simplicity for prediction or interpretation. The paper does not explore how often this no-preference condition actually occurs in real data problems.

A second minor point is that the consistency result assumes the true model is among the candidates, which is standard but restricts the claim. The simulations or examples would need to be checked for whether they cover realistic misspecification patterns.

This paper is for statisticians who work on non-nested selection problems and want a simple alternative when penalties do not fit. A reader already familiar with information criteria could extract the targeted use case quickly.

It deserves peer review. The claim is narrow but cleanly stated, the math is straightforward, and referees can verify the proof and the comparison setups without much trouble.

Referee Report

0 major / 3 minor

Summary. The manuscript proposes a Maximum Likelihood Criterion (MLC) for non-nested model selection that selects the candidate model maximizing the maximized log-likelihood, without any penalty on the number of parameters. It asserts that this rule is consistent because the Kullback-Leibler divergence to the true distribution is strictly smaller for a correctly specified model, and it compares the finite-sample performance of MLC against penalization-based criteria.

Significance. If the consistency result holds, the criterion supplies a simple, penalty-free selection rule for the specific setting in which all candidate models are regarded as equally plausible and parsimony is not desired. The manuscript ships an explicit consistency argument resting on KL divergence ordering; this is a strength that distinguishes it from purely heuristic proposals.

minor comments (3)

[Abstract / Introduction] The abstract states that penalization is 'counterproductive' for non-nested models but does not define the precise sense in which this occurs; a one-sentence clarification in the introduction would help readers locate the intended use case.
[Consistency section] The consistency claim is stated without an explicit list of regularity conditions (e.g., compactness of parameter spaces, identifiability, or moment conditions on the log-likelihood); adding a short 'Assumptions' paragraph before the theorem would make the result easier to verify.
[Numerical results] Simulation comparisons would benefit from reporting the exact sample sizes, number of Monte Carlo replications, and the precise non-nested model pairs used, so that the performance advantage can be reproduced.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their review and for recommending minor revision. The referee's summary accurately captures the manuscript's proposal of the Maximum Likelihood Criterion (MLC) for non-nested model selection and its consistency argument based on Kullback-Leibler divergence ordering. No specific major comments were provided in the report.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The paper's central proposal is the Maximum Likelihood Criterion, defined directly as selecting the candidate model with the highest maximized log-likelihood (no penalty term). Consistency is asserted via the standard fact that the Kullback-Leibler divergence to the true distribution is minimized by the correctly specified model among non-nested candidates. No equations, self-definitions, fitted inputs renamed as predictions, or load-bearing self-citations appear in the abstract or described derivation; the construction is self-contained against external benchmarks of asymptotic model selection theory and does not reduce to its inputs by construction.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Review performed on abstract only; full text unavailable so ledger entries are minimal and provisional.

assumptions (1)

standard math Standard regularity conditions for consistency of maximum likelihood estimators
Required to establish consistency of the proposed criterion.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Maximum Likelihood Criterion for Non-nested Model Selection." pith.science (2026). https://pith.science/paper/2EBVCA2R

@misc{pith2026260622403,
  author       = {Pith},
  title        = {Pith review of: Maximum Likelihood Criterion for Non-nested Model Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2EBVCA2R}},
  note         = {Machine review of arXiv:2606.22403}
}

read the original abstract

Penalization is a widely used approach to model selection with roots in information theory and Bayesian inference. We study a model selection problem involving non-nested candidate models for which penalization is counterproductive. We propose a Maximum Likelihood Criterion for this non-nested setting that selects the candidate model with the highest maximum likelihood. This criterion does not take into consideration the number of parameters of a candidate model. It is well-suited for situations where all candidate models are regarded as equal with no preference for models having fewer parameters. We establish the consistency of this criterion and compare its performance with that of existing penalization-based criteria.

Figures

Figures reproduced from arXiv: 2606.22403 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references

[1]

Akaike, H. (1974). A new look at the statistical model identification.IEEE Transac- tions on Automatic Control, 19, 716–723

1974
[2]

Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Annals of Mathematical Statistics, 40(2):633–643

1969
[3]

Schwarz, G. E. (1978). Estimating the dimension of a model.Annals of Statistics, 6, 461–464

1978
[4]

& Leibler, R

Kullback, S. & Leibler, R. A. (1951). On Information and Sufficiency.Annals of Mathematical Statistics, 22(1), 79–86

1951
[5]

Tsao, M. (2025). Sparse maximum likelihood estimation of regression modelsCana- dian Journal of Statistics, in press. 9

2025

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.

[1] [1]

Akaike, H. (1974). A new look at the statistical model identification.IEEE Transac- tions on Automatic Control, 19, 716–723

1974

[2] [2]

Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Annals of Mathematical Statistics, 40(2):633–643

1969

[3] [3]

Schwarz, G. E. (1978). Estimating the dimension of a model.Annals of Statistics, 6, 461–464

1978

[4] [4]

& Leibler, R

Kullback, S. & Leibler, R. A. (1951). On Information and Sufficiency.Annals of Mathematical Statistics, 22(1), 79–86

1951

[5] [5]

Tsao, M. (2025). Sparse maximum likelihood estimation of regression modelsCana- dian Journal of Statistics, in press. 9

2025