Pith. sign in

REVIEW 2 major objections 5 minor 11 references

Size independence of consistency index for pairwise comparison matrices in analytic hierarchy process

T0 review · 2 major / 5 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A consistency index for pairwise comparison matrices is size-independent exactly when it equals the average of the indices of all its one-item-smaller submatrices.

desk verdict Clean combinatorial axiomatization of size-independence that correctly selects the already-known CI*; modest but solid AHP contribution. read the letter →

arxiv 2607.07950 v1 pith:WV2RP6ZJ submitted 2026-07-08 stat.ME cs.GTmath.OC

Tsuneshi Obata , Shunsuke Shiraishi This is my paper
classification stat.MEcs.GTmath.OC MSC 90B5015A1862C99
keywords analytichierarchyprocesspairwisecomparisonmatrixconsistencyindexsizeindependenceaverage-preservingpropertycharacteristicpolynomialsub-PCMCI*
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In the analytic hierarchy process, people judge how much more important one item is than another and assemble those judgments into a matrix. Because human judgments are never perfectly consistent, every method needs a number that says how far the matrix is from perfect consistency. Saaty’s classic numbers use a fixed threshold of 0.1 no matter how many items are being compared; it has never been clear whether that threshold means the same thing for three items as for ten. This paper answers the question by relating a full matrix (the super-matrix) to the smaller matrices obtained by dropping one item at a time (the sub-matrices). It proposes that a size-independent index must equal the average of the indices of its sub-matrices. The authors then show that a simple rescaling of the coefficient of the characteristic polynomial satisfies this average-preserving property and is identical to an already-known index called CI*. Random-matrix experiments confirm that CI* sits exactly on the equality line while Saaty’s indices only approach it for larger sizes. The result gives a clear mathematical criterion for deciding whether any future consistency number can be used across different numbers of alternatives.

What carries the argument

The average-preserving property: for every n ≥ 4, φ(A) equals (1/n) times the sum of φ over the n submatrices obtained by deleting one row and column. The property is proved by counting how many times each triple of entries appears when the characteristic-polynomial coefficients of all submatrices are summed.

What would settle it

Construct or sample a family of pairwise comparison matrices for which the average of the submatrix indices systematically differs from the super-matrix index under CI*, or exhibit another natural axiom of size-independence that CI* fails while another index succeeds.

Watch

Extended reading notes

Core claim

The consistency index defined by φ(A) = −6/(n(n−1)(n−2)) times the degree-(n−3) coefficient of the characteristic polynomial of a pairwise comparison matrix A equals the average of the same index on every principal submatrix of size n−1. That average-preserving identity characterises size-independence and, by a known algebraic relation, shows that φ coincides with Peláez and Lamata’s CI*.

Load-bearing premise

The claim that the right way to compare consistency across sizes is to require that a matrix’s index equals the average of the indices of its one-item-smaller submatrices, rather than matching every submatrix or the worst one.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies size-independence of consistency indices for pairwise comparison matrices (PCMs) in AHP. It introduces sub-PCMs (leave-one-out principal submatrices) and super-PCMs, motivates an average-preserving property φ(A)=(1/n)∑ φ(A(k)), and proves that the scaled coefficient φ(A)=−6/(n(n−1)(n−2))c3(A) of the characteristic polynomial is a consistency index satisfying that property (Lemma 7, Theorem 8). By the known identity of Brunelli et al., this index coincides with Peláez–Lamata CI*. Bounds relating Saaty’s CI of a super-PCM to its sub-PCMs are derived from Horn–Johnson (Theorem 10). Numerical experiments with random Saaty-scale PCMs for n=4…7 visualise the super/sub relationship for CI*, CI and CR.

Significance. If accepted, the average-preserving property supplies a clean, size-independent axiom that singles out CI* among common indices and gives a rigorous reason to prefer it (or an equivalent scaling of −c3) when thresholds must be comparable across matrix orders. The combinatorial identity of Lemma 7 is elementary, self-contained and correct; the reduction to CI* is immediate. The sub-/super-PCM viewpoint also opens a practical diagnostic route (identify items whose removal most improves consistency). Strengths include an explicit, parameter-free derivation, transparent counting argument, and reproducible random-matrix experiments. The contribution is incremental rather than transformative, but it is a solid, usable clarification of a long-standing practical question in AHP.

major comments (2)
  1. Claim 2 / Definition 8 (Section 3.2) is presented as the normative foundation of size-independence, yet it is only one modelling choice among several (e.g., worst-case sub-PCM, median, or max-min). The paper correctly notes that individual sub-PCMs can differ substantially from the super-PCM, but never tests whether average-preservation is the property practitioners actually need when they apply a fixed threshold such as 0.1. A short discussion or counter-example showing when the average and the worst sub-PCM diverge would strengthen the claim that CI* is the privileged size-independent index.
  2. Section 4 and Figure 3: the numerical comparison of CR with the average-preserving line is suggestive but incomplete. For n=4 the black dots deviate markedly; for n≥5 they appear closer. Because RI itself is an empirical average of CI over random matrices, the apparent improvement may be an artefact of that normalisation rather than evidence that CR inherits average-preservation. A quantitative summary (mean absolute deviation of the black points from the diagonal, or a formal test) is needed before concluding that “normalisation by the RI has achieved its intended effect.”
minor comments (5)
  1. Abstract and Introduction: the phrase “refine our previously proposed index … demonstrating that it coincides with the existing consistency index” is accurate but under-states that the coincidence is already known (Brunelli et al., Prop. 5). A single clarifying sentence would avoid any impression of rediscovery.
  2. Table 2 caption and footnote: the RI values for n=12 are flagged as possibly erroneous in Saaty; the source (Tone) should be cited more prominently in the table itself.
  3. Figure 3: grey and black dots are hard to distinguish in monochrome print; a different marker or transparency would improve readability.
  4. Theorem 10: the second inequality is simply the algebraic rearrangement of the first; stating both is redundant and can be condensed.
  5. Notation: the symbol φ is used for the new index while c3 (or a3) is used for the characteristic-polynomial coefficient; a short glossary or consistent Greek/Latin choice would help readers.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity; central result is a self-contained combinatorial counting argument plus elementary scaling, with only non-load-bearing self-citation to a re-derived algebraic identity.

  1. self citation load bearing [Section 2.3, Theorems 3-4 and abstract]
    "In our previous study, we introduced an alternative index derived from the relationship between the coefficient of the characteristic polynomial and the consistency of comparisons. ... Theorem 3 (Shiraishi, Obata, and Daigo, 1998). ... Theorem 4 (Shiraishi, Obata, and Daigo, 1998)."

    The raw consistency index -c3 is imported from the authors' own prior paper. However the citation is not load-bearing for the new result: the combinatorial identity of Lemma 7 and the scaling that produces the average-preserving property are proved from scratch in the present manuscript, and the same algebraic expression for c3 is also available via the independent Brunelli et al. reference. The self-citation therefore remains minor and does not force the size-independence claim.

full rationale

The paper's strongest claim (Theorem 8) follows directly from Lemma 7's exact identity sum_k c3(A(k)) = (n-3)c3(A), which is proved by a pure double-counting of the triples in C3 versus the C3(k) sets, followed by the algebraic choice of the prefactor 6/(n(n-1)(n-2)) that cancels the (n-3)/n factor. This identity and the subsequent verification that the scaled index equals the average of the sub-PCM indices are internal to the manuscript and do not rely on any fitted constant, external uniqueness theorem, or definition that already encodes the average-preserving property. The only self-citations are to the authors' 1998 expression for the coefficient c3 (Theorems 3-4) and to the known scaling relation -c3 = (n choose 3)CI* (Proposition 5, attributed to Brunelli et al.). Both facts are restated with proofs or explicit formulae inside the paper and are independently available in the non-overlapping literature; they supply the raw material that is then scaled, not a circular premise that forces the average-preserving conclusion. Claim 2 is openly labelled a modelling preference that motivates the axiom, not a derived necessity. Consequently the derivation chain contains no reduction-by-construction and no load-bearing self-reference.

Assumptions & free parameters 0 free parameters · 3 assumptions · 2 invented entities

The paper rests on standard linear-algebra facts about reciprocal matrices and on one normative modeling choice (average consistency of leave-one-out submatrices). No free parameters are fitted; the only invented notions are the named sub-/super-PCM terminology and the average-preserving property itself.

assumptions (3)
  • domain assumption A consistency index must be non-negative and vanish if and only if the matrix is consistent (Definition 4).
    Standard definition used throughout the AHP literature; invoked from Section 2.2 onward.
  • ad hoc to paper The average consistency of the n leave-one-out sub-PCMs is the correct notion of size-independent consistency (Claim 2 / Definition 8).
    Normative modeling choice introduced in Section 3.2; not forced by earlier axioms and not independently validated outside the paper.
  • standard math For a non-negative matrix the spectral radius of any principal submatrix is at most the spectral radius of the whole matrix (Horn–Johnson Corollary 8.1.20).
    Used to prove the CI bounds of Theorem 10.
invented entities (2)
  • average-preserving property
    purpose: Serves as the formal definition of size-independence for a consistency index.
    Introduced in Definition 8; no independent experimental or axiomatic justification is supplied beyond the authors’ Claim 2.
  • sub-PCM / super-PCM terminology independent evidence
    purpose: Names the leave-one-out principal submatrices used to state the average-preserving property.
    Purely terminological; the objects themselves are ordinary principal submatrices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Size independence of consistency index for pairwise comparison matrices in analytic hierarchy process." pith.science (2026). https://pith.science/paper/WV2RP6ZJ

@misc{pith2026260707950,
  author       = {Pith},
  title        = {Pith review of: Size independence of consistency index for pairwise comparison matrices in analytic hierarchy process},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WV2RP6ZJ}},
  note         = {Machine review of arXiv:2607.07950}
}
read the original abstract

Pairwise comparisons are fundamental in the analytic hierarchy process. Various consistency indices have been proposed to assess inconsistencies in these comparisons. Since Saaty first proposed his consistency index, the assessment of the degree of consistency in pairwise comparison matrices has remained an open and hot topic in the study of the analytic hierarchy process. The consistency indices CI and CR proposed by Saaty are defined using the principal eigenvalue of the pairwise comparison matrix. In our previous study, we introduced an alternative index derived from the relationship between the coefficient of the characteristic polynomial and the consistency of comparisons. Saaty proposed a fixed threshold of 0.1 for CI or CR as a guideline for an acceptable level of consistency, regardless of the matrix size. However, whether this threshold represents an equivalent level of consistency across different matrix sizes, that is, across different numbers of evaluation items, remains unclear. This study analysed the relationship between consistency and matrix size by examining pairwise comparison matrices constructed from subsets of evaluation items. Based on this analysis, we propose the fundamental property to be satisfied by a size-independent consistency index. Furthermore, we refine our previously proposed index to ensure that it satisfies this property, demonstrating that it coincides with the existing consistency index. Finally, we visualise the relationship between the matrix size and consistency index values using randomly generated pairwise comparison matrices, thereby providing insights into the impact of matrix size on consistency evaluation.

Figures

Figures reproduced from arXiv: 2607.07950 by the authors.

Figure 1
Figure 1. Consistent and inconsistent judgements Proposition 1. PCM = () is consistent if and only if = , for all , , ∈ {1, 2, . . . , } with < < . (4) Proof. Consider the case < < . Then = 1 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Omitting one item from pairwise comparisons [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Super-PCM index values (horizontal) vs. sub-PCM i [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Scatter plots of CI values for super-PCMs and their s [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages

  1. [1]

    Springer, Cham (2015) doi: 10.1007/978-3-319-12502-2

    Brunelli, M.: Introduction to the Analytic Hierarchy Pr ocess. Springer, Cham (2015) doi: 10.1007/978-3-319-12502-2

    work page doi:10.1007/978-3-319-12502-2 2015

  2. [2]

    Brunelli, M.: A survey of inconsistency indices for pair wise comparisons. Int. J. Gen. Syst. 47(8), 751–771 (2018) doi: 10.1080/03081079.2018.1523156

    work page doi:10.1080/03081079.2018.1523156 2018

  3. [3]

    Brunelli, M., Critch, A., Fedrizzi, M.: A note on the prop ortionality between some consistency indices in the AHP . Appl. Math. Comput. 219(14), 7901–7906 (2013) doi: 10.1016/j.amc.2013.01.036

    work page doi:10.1016/j.amc.2013.01.036 2013

  4. [4]

    Cambri dge University Press, Cambridge (2012) doi: 10.1017/CBO9781139020411

    Horn, R., Johnson, C.: Matrix Analysis (2nd ed.). Cambri dge University Press, Cambridge (2012) doi: 10.1017/CBO9781139020411

    work page doi:10.1017/cbo9781139020411 2012

  5. [5]

    Springer, Cham (2023) doi: 10.1007/978-3- 031-23884-0

    Mazurek, J.: Advances in Pairwise Comparisons: Detecti on, Evaluation and Reduction of Inconsistency. Springer, Cham (2023) doi: 10.1007/978-3- 031-23884-0

    work page doi:10.1007/978-3- 2023

  6. [6]

    Obata, T., Shiraishi, S.: Computational study of charac teristic polynomial of 4th order PCM in AHP . Bull. Inform. Cybern. 53(3), 1–12 (2021) doi: 10.5109/4372243

    work page doi:10.5109/4372243 2021

  7. [7]

    Peláez, J., Lamata, M.: A new measure of consistency for p ositive reciprocal matrices. Comput. Math. Appl. 46, 1839–1845 (2003) doi: 10.1016/S0898-1221(03)90240-9

    work page doi:10.1016/s0898-1221(03)90240-9 2003

  8. [8]

    McGraw-Hi ll, New Y ork (1980)

    Saaty , T.L.: The Analytic Hierarchy Process. McGraw-Hi ll, New Y ork (1980)

    work page 1980

Show all 11 references
  1. [9]

    RWS Publications, Pittsburgh (1996)

    Saaty , T.L.: The Analytic Hierarchy Process, Planning, Priority Setting, Resource Alloca- tion, 2nd ed. RWS Publications, Pittsburgh (1996)

    1996

  2. [10]

    Shiraishi, S., Obata, T., Daigo, M.: Properties of posi tive reciprocal matrix and their application to AHP . J. Oper. Res. Soc. Jpn. 41(3), 404–414 (1998) doi: 10.15807/jorsj.41.404

    1998 doi

  3. [11]

    Japanese Science and T echnology Press, T okyo (1986) 16

    T one, K.: The Analytic Hierarchy Process: Decision Making (in Japanese). Japanese Science and T echnology Press, T okyo (1986) 16

    1986

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.