Pith. sign in

REVIEW 3 major objections 4 minor 7 cited by

Sparse pairwise comparisons can yield entrywise-accurate preference scores and valid ranking confidence intervals for groups and individual users, under a heterogeneous Bradley-Terry-Luce model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A debiased regularized estimator provides entrywise-accurate scores and uncertainty quantification for heterogeneous Bradley-Terry-Luce rankings.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Genuinely new indirect-regularization idea and plausible entrywise UQ results for heterogeneous BTL, but the main theorem currently rests on an omitted proof section and the real-data experiment is actually a simulation. the 3 major comments →

arxiv 2509.01847 v1 pith:UVDAJ2OD submitted 2025-09-02 stat.ME math.STstat.TH

Uncertainty Quantification for Ranking with Heterogeneous Preferences

Jianqing Fan , Hyukjun Kwon , Xiaonan Zhu This is my paper
classification stat.ME math.STstat.TH
keywords Bradley–Terry–Luce modelpreference learningcollaborative rankingsieve approximationnonconvex optimizationℓ∞ error bounduncertainty quantificationnuclear norm regularization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to prove that heterogeneous human preferences can be learned from sparse pairwise comparisons with uniform entrywise accuracy, and that the learned score gaps can support valid confidence intervals for rankings, at the group level and even for an individual user. It models each user's preference as a smooth function of low-dimensional item features, which makes the underlying score matrix approximately low-rank, and then "indirectly regularizes" the induced pairwise-choice probability matrix instead of the score matrix itself. The argument combines sieve approximation, a reparameterized nonconvex surrogate of the regularized least-squares problem, leave-one-out gradient descent, and a single Newton-Raphson debiasing step to remove regularization bias. If the theory is right, this is the first entrywise error guarantee and uncertainty-quantification framework for heterogeneous Bradley-Terry-Luce preference learning, and it requires far fewer comparisons per user than learning each user's preferences separately.

Core claim

The paper's central claim is that the score matrix Θ⋆, whose (i,j) entry is user i's score for item j, can be estimated entrywise uniformly from a small random sample of item comparisons, and that the debiased estimator is asymptotically normal for both averaged and individual score gaps. The key move is to regularize the probability matrix σ(M⋆), where M⋆ is the score-gap matrix, rather than the score matrix itself: because σ is smooth and the preference functions are assumed smooth, σ(M⋆) is approximately low-rank with a sieve representation σ(M⋆)=ΛΦ⊤+E whose approximation error is o(1/√max{d1,d2(d2−1)/2}). Theorem 4.1 gives an ℓ∞ error bound of order κ²μR/(min{d1,d2(d2−1)/2}) √(d̄ log d̄/

What carries the argument

The carrying mechanism is indirect regularization: instead of penalizing the score matrix Θ⋆ or the score-gap matrix M⋆, the estimator solves a nuclear-norm-regularized least-squares problem for the induced choice-probability matrix σ(M⋆). Smoothness of the logistic link and of the nonparametric preference functions transfers approximate low-rank structure to the probability matrix, with sieve rank R. The reparameterized nonconvex surrogate f(X,Y) has a loss whose gradient is linear in X and Y, so it lies in the low-rank subspace and the gap between the convex and nonconvex solutions can be controlled; leave-one-out gradient-descent iterates then yield the entrywise bound. A one-step Newton-

Load-bearing premise

The load-bearing premise is that real preferences are smooth functions of a low-dimensional item feature vector, so the pairwise-choice probability matrix is close to a low-rank matrix with manageable sieve rank; if preferences are not smooth or items do not share low-dimensional features, the entrywise error bound has no low-rank structure to exploit.

What would settle it

Take a synthetic population where each user's preference over items is an arbitrary permutation (no smooth low-dimensional structure) and items have no shared latent features; run the proposed estimator with increasing d1 and d2 and check whether the entrywise error ∥Θ̂−Θ⋆∥∞ shrinks at the rate predicted by Theorem 4.1. If it does not, or if the sieve approximation error ∥E∥∞ fails to satisfy Assumption 3.1(i) when R grows, the low-rank premise underlying the theorem fails. A less extreme check is to generate preferences from g_i(η_j) with high-frequency, non-smooth oscillations and compare th

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Every entry of the estimated score matrix is simultaneously close to the truth, not just on average, so recommendations and top-K item sets for each user inherit uniform guarantees.
  • A single Newton-Raphson step removes the regularization bias, so score-gap differences and item rankings can be reported with confidence intervals at the aggregated level.
  • Individual-user inference is also available after rank-q spectral projection, so personalized ranking and selection statements can be quantified, not just population averages.
  • The per-user sample requirement depends on the sieve dimension and conditioning parameters rather than directly on the number of items, a major reduction compared with learning each user's preferences in isolation.
  • The same indirect-regularization analysis extends to any binary choice model with a smooth link function, not only the logistic BTL model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same indirect-regularization idea could debias other nonlinear low-rank problems, such as low-rank reward models trained from human feedback, where the likelihood gradient is not aligned with the factorization subspaces; a direct test would be to replace the logistic link with a probit or neural link and check whether entrywise control survives.
  • Because the procedure never estimates or validates the item feature vectors or the sieve basis, a practical diagnostic would be to compare observed entrywise errors against the rate predicted by Assumption 3.1; a data-driven sieve dimension R might make the method less sensitive to misspecification.
  • The sample-splitting step in individual inference is introduced for technical independence; the paper's own simulations suggest splitting costs little, so a theory avoiding the split may be within reach.
  • The ranking confidence intervals are built from simultaneous score-gap intervals and are likely conservative; sharper intervals could come from directly bootstrapping the rank statistic rather than passing through all pairwise gaps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies heterogeneous Bradley–Terry–Luce (BTL) preference learning with user-specific nonparametric preference functions and low-dimensional item features. The underlying score-gap matrix M* is estimated by an 'indirect regularization' scheme: a nuclear-norm penalized least squares problem on the induced probability matrix σ(M*), followed by inverse-link transformation and averaging over score gaps to recover Θ*. The central theoretical claims are an entrywise ℓ∞ error bound for bΘ (Theorem 4.1), debiased asymptotic normality for aggregated and individual score-gap estimates (Theorems 5.1 and 5.3), and simultaneous ranking confidence intervals via Gaussian multiplier bootstrap (Theorem 5.5). The proof strategy relies on a nonconvex surrogate of the regularized problem, a convex–nonconvex equivalence argument, and leave-one-out gradient-descent iterations (Appendices F–H).

Significance. If the theorems are correct, the paper makes a substantive contribution: entrywise error control and uncertainty quantification for heterogeneous BTL preferences have not been established before. The indirect regularization idea is elegant and potentially transferable to other binary choice models. The appendix contains substantial detailed proof material, including the convex–nonconvex equivalence (Lemma G.3) and sieve-based structural arguments. However, the most delicate part of the proof—the leave-one-out analysis of the nonconvex iterates—is omitted, so the central entrywise bound is not currently verifiable. The numerical study is also self-referential: it simulates BTL comparisons from a constructed 'quasi-true' score matrix rather than using real pairwise choices, so it does not validate the key structural assumptions.

major comments (3)
  1. [Section H; Theorem F.1] The proof of Theorem F.1 (and hence Theorem 4.1, Corollary F.2, and the debiasing results) invokes (H.2), Lemma H.1, and Lemma H.2 directly. Section H states that the proofs are omitted 'for brevity' and 'available upon request.' These lemmas are not routine: Lemma H.1 supplies the leave-one-out gradient-descent bounds (H.4)–(H.8), and Lemma H.2 supplies the incoherence and closeness properties used to control ||X_t Y_t^T − L*||∞ and ||bL − X_t Y_t^T||∞. Without a complete Section H, the central entrywise error claim is unverifiable. Please include full proofs, or restructure the claims so that they do not depend on unproved lemmas.
  2. [Section 6; Assumption 3.1] The real-data experiment does not use actual pairwise choices. A 780×40 'quasi-true' score matrix is constructed from discretized watch ratios, and then BTL comparisons are simulated (Section 6). Thus the numerical study can only demonstrate algorithmic self-consistency under the assumed generative model; it cannot validate the load-bearing structural premises of Assumption 3.1 (smoothness of preference functions, low-dimensional item features, small sieve error, spiked/incoherent σ(M*)). The paper should state this limitation explicitly and, ideally, add a diagnostic of approximate low-rankness or analyze a real pairwise-comparison dataset.
  3. [Section 5.3 vs. Section A] Individual-level inference assumes the rank q of the dominant component of M* is known ('For simplicity, this paper assumes that the rank q is known'), but the numerical experiments estimate q by a 10% singular-value threshold (Section A). No theorem shows that this estimator is consistent under Assumptions C.1–C.2, nor that substituting estimated singular vectors into the debiasing procedure preserves the error bounds needed in Theorem 5.3 and Proposition 5.4. The implemented procedure is therefore not covered by the stated guarantees. Either prove consistency of the threshold estimator, or label this part as heuristic and run the UQ simulations with a known q.
minor comments (4)
  1. [Eq. (3.9), Step 2 of Section 3.4] The convex estimator bL is not constrained to [0,1], yet Step 2 applies σ^{−1} entrywise. The analysis assumes bL entries are bounded away from 0 and 1; a clipping step would make the procedure well-defined and should be mentioned.
  2. [Section 5.4] The notation \(\bar L(j,j')\) is used for both orientations in Eq. (5.2) and Theorem 5.5, but it is introduced only informally. Please define it at first use and make the orientation in the simultaneous CIs explicit.
  3. [Section A and Theorem 4.1] Simulations set λ = sqrt(0.5 dbar/pbar), while Theorem 4.1 requires λ = C_λ sqrt(dbar/pbar) for a large constant C_λ. The value 0.5 is not 'large'; please clarify whether the constant can be taken small in practice or justify the choice.
  4. [Section 6] The paragraph 'All proofs are provided in the appendix' in the introduction is contradicted by Section H. Please revise the wording after completing the appendix.

Circularity Check

0 steps flagged

No circular reduction: the theoretical results are derived from explicit structural assumptions, and the omitted Section H proofs, while a verifiability gap, do not make the derivation equivalent to its inputs.

full rationale

The paper's derivation chain is not circular in the sense of the review criteria. The estimator is defined by solving (3.9), then transforming via cM = sigma^{-1}(bL) and linear averaging to bTheta; the entrywise bound of Theorem 4.1 is proved from Theorem F.1 and Corollary F.2 using Taylor expansion and smoothness of sigma^{-1}, with no constant fitted to the target rate. The regularization parameter is set to a generic constant (lambda = C_lambda sqrt(dbar/pbar)), and Assumption 3.1 is a stated structural condition rather than an output of the estimation procedure. The debiasing results (Theorems 5.1, 5.3, 5.5) follow from Taylor expansions, leave-one-out bounds, and spectral projector representations, again without fitting parameters to achieve normality. The main caveat is that Section H, containing the crucial leave-one-out lemmas H.1 and H.2, says 'we omit them for brevity' and cites Chen et al. (2020b) for 'similar' proofs. Chen et al. (2020b) shares an author (Jianqing Fan) and supplies the proof template, but the target result — entrywise error control for heterogeneous BTL models — is not itself imported from that paper; the citation is for a general technique in matrix completion, and the manuscript states the lemmas it requires. This is an incompleteness/verifiability concern, not a circular reduction of the prediction to its inputs. No fitted parameter is renamed as a prediction, and no uniqueness theorem from the authors' prior work is used to forbid alternatives.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The theory rests on the smooth low-rank latent structure of the preference model, the random graph sampling assumption, and a substantial list of technical conditions on the sieve approximation, condition numbers, incoherence, and signal strength. No new physical entities are introduced. The free parameters are the regularization tuning constant, the unestimated sieve dimension R, and the rank q of the score gap matrix, which is assumed known in the main individual inference theory. The preprint does not supply the omitted proofs in Appendix H, so the ledger cannot be fully certified from the available text.

free parameters (3)
  • Regularization parameter lambda = lambda = C_lambda sqrt(dbar/pbar) with large constant; simulations set lambda = sqrt(0.5 * dbar/pbar)
    Tuning parameter for the nuclear norm penalty; chosen by the authors and not derived from the data or leading to a fitted prediction in the theorems.
  • Sieve dimension R = Not estimated; assumed in Assumption 3.1
    Rank of the sieve representation L* = Lambda Phi^T. The theory requires R to grow slowly, but no data-driven selection rule is given for the main estimator.
  • Rank q of M* for individual inference = Assumed known in Section 5.3; in simulations estimated by thresholding singular values at 10% of the largest
    The individual-level projection and variance formula require the rank q of the score gap matrix; the paper explicitly assumes q known and only uses a heuristic estimator in simulations.
axioms (6)
  • domain assumption Each item has a low-dimensional latent feature vector eta_j and each user has a smooth nonparametric preference function g_i, so the induced pair-comparison functions are smooth.
    Introduced in Sections 2 and 3.3. Without this, the score matrix and probability matrix have no approximate low-rank structure.
  • domain assumption The sieve approximation error is small and L* has spiked singular values, bounded incoherence, and slowly growing condition number (Assumption 3.1).
    These conditions make nuclear norm regularization effective for estimating sigma(M*) and are stated as assumptions rather than consequences of the model.
  • domain assumption Pairwise comparisons are sampled according to heterogeneous Erdos-Renyi graphs with pmin > c pmax and pmin/pmax bounded away from zero (Assumption 2.2).
    Defines the observation model used throughout the proofs and in simulations.
  • standard math Row sums of the score matrix are zero (Assumption 2.1).
    Identifies the BTL model up to row-wise shifts; standard in ranking models.
  • domain assumption The score gap matrix M* has a low-rank sieve approximation with known rank q and spiked singular values (Assumptions C.1, C.2, F.1-F.3).
    Needed only for individual-level UQ and ranking intervals; it is an additional structural assumption parallel to Assumption 3.1.
  • standard math Sieve approximation theory: smooth functions can be approximated by polynomial, trigonometric, or spline bases with error O(R^{-s}).
    Invoked in Section 3.3 and borrowed from Chen (2007) and related references.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncertainty Quantification for Ranking with Heterogeneous Preferences." pith.science (2026). https://pith.science/paper/UVDAJ2OD

@misc{pith2026250901847,
  author       = {Pith},
  title        = {Pith review of: Uncertainty Quantification for Ranking with Heterogeneous Preferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVDAJ2OD}},
  note         = {Machine review of arXiv:2509.01847}
}
Share X Bluesky LinkedIn Reddit HN
abstract

This paper studies human preference learning based on partially revealed choice behavior and formulates the problem as a generalized Bradley-Terry-Luce (BTL) ranking model that accounts for heterogeneous preferences. Specifically, we assume that each user is associated with a nonparametric preference function, and each item is characterized by a low-dimensional latent feature vector - their interaction defines the underlying low-rank score matrix. In this formulation, we propose an indirect regularization method for collaboratively learning the score matrix, which ensures entrywise $\ell_\infty$-norm error control - a novel contribution to the heterogeneous preference learning literature. This technique is based on sieve approximation and can be extended to a broader class of binary choice models where a smooth link function is adopted. In addition, by applying a single step of the Newton-Raphson method, we debias the regularized estimator and establish uncertainty quantification for item scores and rankings of items, both for the aggregated and individual preferences. Extensive simulation results from synthetic and real datasets corroborate our theoretical findings.

Figures

Figures reproduced from arXiv: 2509.01847 by Hyukjun Kwon, Jianqing Fan, Xiaonan Zhu.

Figure 1
Figure 1. Figure 1: Histogram of the watch ratios and their assigned baskets and scores. [PITH_FULL_IMAGE:figures/full_fig_p029_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The left panel corresponds to the first experiment with higher probabilities (0 [PITH_FULL_IMAGE:figures/full_fig_p038_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Histograms for the aggregated score gap. The first and second rows corre [PITH_FULL_IMAGE:figures/full_fig_p039_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Histograms for the individual score gap. As in Figure [PITH_FULL_IMAGE:figures/full_fig_p039_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The left panel corresponds to the first experiment with higher probability [PITH_FULL_IMAGE:figures/full_fig_p041_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency

    stat.ME 2026-04 unverdicted novelty 8.0

    LLM pairwise evaluation is recast as low-rank tensor completion, yielding semiparametric efficient estimators and asymptotic normality for ability functionals via a score-whitening correction for anisotropic operators.

  2. Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisons

    stat.ME 2026-05 unverdicted novelty 6.0

    Low-rank matrix modeling with cross-fitted debiased estimators and multiplier bootstrap yields stable task-specific LLM rankings and asymptotically valid simultaneous confidence sets from sparse pairwise data.

  3. Learning Perturbations to Extrapolate Your LLM

    stat.ML 2026-05 unverdicted novelty 6.0

    A learnable continuous perturbation framework for LLM token prefixes via latent vector transformations, optimized through unbiased estimating equations, yields gains in out-of-domain performance.

  4. Heterogeneous Judge-Aware Ranking with Sensitivity, Disagreement, and Confidence

    stat.ME 2026-05 unverdicted novelty 6.0

    HJA ranking separates consensus ranking, judge sensitivity, and residual disagreement as distinct inferential targets with identifiability conditions and an anchored alternating algorithm, yielding better recovery and...

  5. Perturbation is All You Need for Extrapolating Language Models

    stat.ML 2026-05 unverdicted novelty 6.0

    Perturbing prefixes to semantic neighbors during training creates a hierarchical noise model that improves language model predictions on token sequences outside the training corpus support.

  6. A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth

    stat.ML 2026-01 conditional novelty 6.0

    A judge-aware Bradley–Terry model with per-judge discrimination parameters estimates LLM quality and judge reliability from unlabeled pairwise comparisons, with identifiability and asymptotic normality guarantees.

  7. Perturbation is All You Need for Extrapolating Language Models

    stat.ML 2026-05 conditional novelty 5.0

    Perturbing the prefix before next-token prediction, during both training and inference, improves out-of-distribution language-model generation and yields a conditional extrapolation guarantee.

Reference graph

Works this paper leans on

70 extracted references · 58 canonical work pages · cited by 6 Pith papers · 3 internal anchors

  1. [1]

    Abbe, E., Fan, J., Wang, K., and Zhong, Y. (2020). Entrywise eigenvector analysis of random matrices with low expected rank. Annals of statistics , 48(3):1452

    2020

  2. [2]

    Ahn, S. C. and Horenstein, A. R. (2013). Eigenvalue ratio test for the number of factors. Econometrica , 81(3):1203--1227

    2013

  3. [3]

    Aouad, A., Farias, V., Levi, R., and Segev, D. (2018). The approximability of assortment optimization under ranking preferences. Operations Research , 66(6):1661--1669

    work page 2018

  4. [4]

    N., Glickman, M

    Avery, C. N., Glickman, M. E., Hoxby, C. M., and Metrick, A. (2013). A revealed preference ranking of us colleges and universities. The Quarterly Journal of Economics , 128(1):425--467

    work page 2013

  5. [5]

    and Ng, S

    Bai, J. and Ng, S. (2002). Determining the number of factors in approximate factor models. Econometrica , 70(1):191--221

    2002

  6. [6]

    Baltrunas, L., Makcinskas, T., and Ricci, F. (2010). Group recommendations with rank aggregation and collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems , pages 119--126

    work page 2010

  7. [7]

    S., Van Handel, R., et al

    Bandeira, A. S., Van Handel, R., et al. (2016). Sharp nonasymptotic bounds on the norm of random matrices with independent entries. The Annals of Probability , 44(4):2479--2506

    work page 2016

  8. [8]

    and Teboulle, M

    Beck, A. and Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences , 2(1):183--202

    work page 2009

  9. [9]

    Bradley, R. A. and Terry, M. E. (1952). Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika , 39(3/4):324--345

    work page 1952

  10. [10]

    J., and Shen, Z

    Cai, J.-F., Cand \`e s, E. J., and Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on optimization , 20(4):1956--1982

    work page 2010

  11. [11]

    Cand\` e s, E. J. and Plan, Y. (2010). Matrix completion with noise. Proceedings of the IEEE , 98(6):925--936

    work page 2010

  12. [12]

    Cand\` e s, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational mathematics , 9(6):717

    work page 2009

  13. [13]

    Chatzi, I., Straitouri, E., Thejaswi, S., and Rodriguez, M. (2024). Prediction-powered ranking of large language models. Advances in Neural Information Processing Systems , 37:113096--113133

    work page 2024

  14. [14]

    Model-free Nonconvex Matrix Completion: Local Minima Analysis and Applications in Memory-efficient Kernel PCA

    Chen, J. and Li, X. (2017). Memory-efficient kernel pca via partial matrix sampling and nonconvex optimization: a model-free analysis of local minima. arXiv preprint arXiv:1711.01742

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  15. [15]

    Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. Handbook of econometrics , 6:5549--5632

    2007

  16. [16]

    Chen, X., Wang, Y., and Zhou, Y. (2020a). Dynamic assortment optimization with changing contextual information. The Journal of Machine Learning Research , 21(1):8918--8961

    work page

  17. [17]

    Chen, Y., Chi, Y., Fan, J., Ma, C., and Yan, Y. (2020b). Noisy matrix completion: Understanding statistical guarantees for convex relaxation via nonconvex optimization. SIAM journal on optimization , 30(4):3098--3121

    work page

  18. [18]

    Chen, Y., Fan, J., Ma, C., and Wang, K. (2019a). Spectral method and regularized mle are both optimal for top-k ranking. Annals of statistics , 47(4):2204

    work page

  19. [19]

    Chen, Y., Fan, J., Ma, C., and Yan, Y. (2019b). Inference and uncertainty quantification for noisy matrix completion. Proceedings of the National Academy of Sciences , 116(46):22931--22937

    work page

  20. [20]

    Chen, Y., Li, C., Ouyang, J., and Xu, G. (2023). Statistical inference for noisy incomplete binary matrix. Journal of Machine Learning Research , 24(95):1--66

    work page 2023

  21. [21]

    and Suh, C

    Chen, Y. and Suh, C. (2015). Spectral mle: Top-k rank aggregation from pairwise comparisons. In International Conference on Machine Learning , pages 371--380. PMLR

    work page 2015

  22. [22]

    Chernozhukov, V., Hansen, C., Liao, Y., and Zhu, Y. (2023). Inference for low-rank models. The Annals of statistics , 51(3):1309--1330

    work page 2023

  23. [23]

    Chernozhuokov, V., Chetverikov, D., Kato, K., and Koike, Y. (2022). Improved central limit theorem and bootstrap approximations in high dimensions. The Annals of Statistics , 50(5):2562--2586

    work page 2022

  24. [24]

    Choi, J., Kwon, H., and Liao, Y. (2023). Inference for low-rank models without estimating the rank. arXiv preprint arXiv:2311.16440

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  25. [25]

    Choi, J., Kwon, H., and Liao, Y. (2024). Inference for low-rank completion without sample splitting with application to treatment effect estimation. Journal of Econometrics , 240(1):105682

    work page 2024

  26. [26]

    and Yuan, M

    Choi, J. and Yuan, M. (2024). Matrix completion when missing is not at random and its applications in causal panel data models. Journal of the American Statistical Association , pages 1--15

    work page 2024

  27. [27]

    Dhurandhar, A., Nair, R., Singh, M., Daly, E., and Ramamurthy, K. N. (2024). Ranking large language models without ground truth. arXiv preprint arXiv:2402.14860

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    Dwork, C., Kumar, R., Naor, M., and Sivakumar, D. (2001). Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web , pages 613--622

    work page 2001

  29. [29]

    Fan, J., Ge, J., and Hou, J. (2025a). Covariates-adjusted mixed-membership estimation: A novel network model with optimal guarantees. arXiv preprint arXiv:2502.06671

    Pith/arXiv arXiv

  30. [30]

    Fan, J., Hou, J., and Yu, M. (2024a). Covariate assisted entity ranking with sparse intrinsic scores. arXiv preprint arXiv:2407.08814

    Pith/arXiv arXiv

  31. [31]

    Fan, J., Hou, J., and Yu, M. (2024b). Uncertainty quantification of mle for entity ranking with covariates. Journal of Machine Learning Research , 25(358):1--83

    work page

  32. [32]

    Fan, J., Liao, Y., and Wang, W. (2016). Projected principal component analysis in factor models. Annals of statistics , 44(1):219

    work page 2016

  33. [33]

    Fan, J., Lou, Z., Wang, W., and Yu, M. (2025b). Ranking inferences based on the top choice of multiway comparisons. Journal of the American Statistical Association , 120:237--250

    work page

  34. [34]

    Fan, J., Lou, Z., Wang, W., and Yu, M. (2025c). Spectral ranking inferences based on general multiway comparisons. Operations Research , page to appear

    work page

  35. [35]

    Fernandes, C., Siderius, J., and Singal, R. (2025). Peer review market design: Effort-based matching and admission control. Available at SSRN

    work page 2025

  36. [36]

    Gao, C., Li, S., Lei, W., Chen, J., Li, B., Jiang, P., He, X., Mao, J., and Chua, T.-S. (2022). Kuairec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management , CIKM '22, page 540–550

    work page 2022

  37. [37]

    Gao, C., Shen, Y., and Zhang, A. Y. (2023). Uncertainty quantification in the bradley--terry--luce model. Information and Inference: A Journal of the IMA , 12(2):1073--1140

    work page 2023

  38. [38]

    and Montanari, A

    Javanmard, A. and Montanari, A. (2013). Confidence intervals and hypothesis testing for high-dimensional statistical models. Advances in neural information processing systems , 26

    work page 2013

  39. [39]

    and Montanari, A

    Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. The Journal of Machine Learning Research , 15(1):2869--2909

    work page 2014

  40. [40]

    and Scott, C

    Katz-Samuels, J. and Scott, C. (2018). Nonparametric preference completion. In International Conference on Artificial Intelligence and Statistics , pages 632--641. PMLR

    work page 2018

  41. [41]

    H., Montanari, A., and Oh, S

    Keshavan, R. H., Montanari, A., and Oh, S. (2010). Matrix completion from a few entries. IEEE transactions on information theory , 56(6):2980--2998

    work page 2010

  42. [42]

    B., et al

    Koltchinskii, V., Lounici, K., Tsybakov, A. B., et al. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. The Annals of Statistics , 39(5):2302--2329

    work page 2011

  43. [43]

    J., Sun, W

    Lee, S. J., Sun, W. W., and Liu, Y. (2024a). Low-rank contextual reinforcement learning from heterogeneous human feedback. arXiv preprint arXiv:2412.19436

    arXiv

  44. [44]

    J., Sun, W

    Lee, S. J., Sun, W. W., and Liu, Y. (2024b). Low-rank online dynamic assortment with dual contextual information. arXiv preprint arXiv:2404.17592

    work page arXiv

  45. [45]

    X., and Zhu, W

    Li, H., Simchi-Levi, D., Wu, M. X., and Zhu, W. (2023). Estimating and exploiting the impact of photo layout: A structural approach. Management Science , 69(9):5209--5233

    work page 2023

  46. [46]

    Li, Z., Xu, Q., Jiang, Y., Ma, K., Cao, X., and Huang, Q. (2020). Neural collaborative preference learning with pairwise comparisons. IEEE Transactions on Multimedia , 23:1977--1989

    work page 2020

  47. [47]

    Luce, R. D. (1959). Individual choice behavior , volume 4. Wiley New York

    1959

  48. [48]

    Ma, S., Goldfarb, D., and Chen, L. (2011). Fixed point and bregman iterative methods for matrix rank minimization. Mathematical Programming , 128(1-2):321--353

    work page 2011

  49. [49]

    Maia Polo, F., Xu, R., Weber, L., Silva, M., Bhardwaj, O., Choshen, L., de Oliveira, A., Sun, Y., and Yurochkin, M. (2024). Efficient multi-prompt evaluation of llms. Advances in Neural Information Processing Systems , 37:22483--22512

    work page 2024

  50. [50]

    and Walsh, T

    Mattei, N. and Walsh, T. (2013). Preflib: A library for preferences http://www.preflib.org. In International conference on algorithmic decision theory , pages 259--270. Springer

    work page 2013

  51. [51]

    Mazumder, R., Hastie, T., and Tibshirani, R. (2010). Spectral regularization algorithms for learning large incomplete matrices. Journal of machine learning research , 11(Aug):2287--2322

    work page 2010

  52. [52]

    McFadden, D. (1972). Conditional logit analysis of qualitative choice behavior

    work page 1972

  53. [53]

    K., and Xu, J

    Negahban, S., Oh, S., Thekumparampil, K. K., and Xu, J. (2018). Learning from comparisons and choices. Journal of Machine Learning Research , 19(40):1--95

    work page 2018

  54. [54]

    and Wainwright, M

    Negahban, S. and Wainwright, M. J. (2012). Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. The Journal of Machine Learning Research , 13(1):1665--1697

    work page 2012

  55. [55]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training language models to follow instructions with human feedback. Advances in N eural I nformation P rocessing S ystems , 35:27730--27744

    work page 2022

  56. [56]

    and Boyd, S

    Parikh, N. and Boyd, S. (2014). Proximal algorithms. Foundations and Trends in optimization , 1(3):127--239

    work page 2014

  57. [57]

    Park, C., Liu, M., Kong, D., Zhang, K., and Ozdaglar, A. (2024). Rlhf from heterogeneous feedback via personalization and preference aggregation. arXiv preprint arXiv:2405.00254

    Pith/arXiv arXiv 2024

  58. [58]

    Park, D., Neeman, J., Zhang, J., Sanghavi, S., and Dhillon, I. (2015). Preference completion: Large-scale collaborative ranking from pairwise comparisons. In International Conference on Machine Learning , pages 1907--1916. PMLR

    work page 2015

  59. [59]

    Plackett, R. L. (1975). The analysis of permutations. Journal of the Royal Statistical Society: Series C (Applied Statistics) , 24(2):193--202

    work page 1975

  60. [60]

    Shah, N. B. (2022). Challenges, experiments, and computational solutions in peer review. Communications of the ACM , 65(6):76--87

    work page 2022

  61. [61]

    Su, B., Zhang, J., Collina, N., Yan, Y., Li, D., Cho, K., Fan, J., Roth, A., and Su, W. J. (2024). Analysis of the icml 2023 ranking data: Can authors' opinions of their own papers assist peer review in machine learning? arXiv preprint arXiv:2408.13430

    work page arXiv 2024

  62. [62]

    Su, W. (2021). You are the best reviewer of your own papers: An owner-assisted scoring mechanism. Advances in Neural Information Processing Systems , 34:27929--27939

    work page 2021

  63. [63]

    Tropp, J. A. (2015). An introduction to matrix concentration inequalities. arXiv preprint arXiv:1501.01571

    Pith/arXiv arXiv 2015

  64. [64]

    and Fan, J

    Wang, B. and Fan, J. (2025). Robust matrix completion with heavy-tailed noise. Journal of the American Statistical Association , 120(550):922--934

    work page 2025

  65. [65]

    Wang, X., Bendersky, M., Metzler, D., and Najork, M. (2016). Learning to rank with selection bias in personal search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval , pages 115--124

    work page 2016

  66. [66]

    X., Wang, L., and Lu, J

    Wang, Z., Han, Y., Fang, E. X., Wang, L., and Lu, J. (2024). Ranking of large language model with nonparametric prompts. arXiv preprint arXiv:2412.05506

    Pith/arXiv arXiv 2024

  67. [67]

    Xia, D. (2021). Normal approximation and confidence region of singular subspaces. Electronic Journal of Statistics , 15(2):3798--3851

    work page 2021

  68. [68]

    and Yuan, M

    Xia, D. and Yuan, M. (2021). Statistical inferences of linear forms for noisy matrix completion. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 83(1):58--77

    work page 2021

  69. [69]

    J., and Fan, J

    Yan, Y., Su, W. J., and Fan, J. (2024). Isotonic mechanism for exponential family estimation in machine learning peer review

    work page 2024

  70. [70]

    and Lafferty, J

    Zheng, Q. and Lafferty, J. (2016). Convergence analysis for rectangular matrix completion using burer-monteiro factorization and gradient descent. arXiv preprint arXiv:1605.07051

    Pith/arXiv arXiv 2016

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.