{"id":"c40db66f-cfa9-4850-9641-d8aef4063934","arxiv_id":"2509.01847","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A debiased regularized estimator provides entrywise-accurate scores and uncertainty quantification for heterogeneous Bradley-Terry-Luce rankings.","lead":"This paper introduces a regularized least squares method for learning user-specific item preferences from sparse pairwise comparisons, with proven error bounds and confidence intervals. It targets applications like conference reviewing, recommendation systems, and ranking large language models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.1's entrywise bound rests on Section H's leave-one-out lemmas, which the manuscript explicitly omits; the central claim is currently unverifiable.","rationale":"The reader's weakest assumption—Assumption 3.1 and the unverified smooth low-dimensional structure—is genuinely load-bearing, and I agree it is a serious concern. But the manuscript itself flags a more immediate and concrete gap: Section H, the technical core of the entrywise error bound, is explicitly omitted and available only upon request. This is not an external objection; it is a missing proof for the central theorem, and the review instructions require explicitly weighing such a passage. I also agree with the reader that the 'real data' experiment is actually a simulation from a constructed score matrix and thus does not supply empirical support. I do not see an internal contradiction or an obviously false step in the provided arguments, so rejection is not warranted. However, acceptance should require the full Section H proof and a genuinely observed-choice experiment. Since the reader already issued a CONDITIONAL verdict covering these gaps, my stress-test does not move the verdict; it sharpens the reason by locating the omitted proof precisely.","tokens_in":76534,"tokens_out":7295,"duration_ms":87057,"concrete_test":"Obtain the complete proof of Section H and independently re-derive Lemmas H.1 and H.2, together with Lemmas G.3–G.7, from Assumptions 2.2 and 3.1 alone. Check the stated dependence on κ, μ, R, and ¯p in the displayed bounds, and verify that Theorem F.1's stated κ² rate follows from the proof's factors involving σ*_max/(σ*_min)². If any lemma requires an unstated additional assumption, or if the displayed rate does not match the derivation, then Theorem 4.1 and the subsequent debiasing theorems are not established as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's main entrywise error claim, Theorem 4.1, is justified through Theorem F.1. The proof of Theorem F.1 invokes Lemma H.1 and Lemma H.2 from Section H, along with Lemmas G.3–G.7. Section H states: 'As the proofs of the lemmas in this section are similar to those in Chen et al. (2020b), we omit them for brevity. The complete proofs are available upon request.' These are not peripheral technicalities: Lemma H.1 supplies the leave-one-out gradient-descent bounds (H.4)–(H.8), and Lemma H.2 supplies the incoherence and closeness properties, used directly to control ||X_t Y_t^T − L*||_∞ and ||bL − X_t Y_t^T||_∞. Without a complete Section H, the proof of Theorem F.1—and therefore Theorem 4.1 and the debiasing results built on Corollary F.2—has a black box at its central step. The real-data section does not observe actual pairwise choices; it constructs a quasi-true score matrix from watch ratios and simulates BTL comparisons, so it does not independently validate the structural assumptions or the claimed rate. This is a verifiability gap, not evidence of a false theorem: the claims may be true, but the manuscript does not currently provide enough material to certify them.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies heterogeneous Bradley–Terry–Luce (BTL) preference learning with user-specific nonparametric preference functions and low-dimensional item features. The underlying score-gap matrix M* is estimated by an 'indirect regularization' scheme: a nuclear-norm penalized least squares problem on the induced probability matrix σ(M*), followed by inverse-link transformation and averaging over score gaps to recover Θ*. The central theoretical claims are an entrywise ℓ∞ error bound for bΘ (Theorem 4.1), debiased asymptotic normality for aggregated and individual score-gap estimates (Theorems 5.1 and 5.3), and simultaneous ranking confidence intervals via Gaussian multiplier bootstrap (Theorem 5.5). The proof strategy relies on a nonconvex surrogate of the regularized problem, a convex–nonconvex equivalence argument, and leave-one-out gradient-descent iterations (Appendices F–H).","tokens_in":76813,"tokens_out":8682,"duration_ms":88732,"significance":"If the theorems are correct, the paper makes a substantive contribution: entrywise error control and uncertainty quantification for heterogeneous BTL preferences have not been established before. The indirect regularization idea is elegant and potentially transferable to other binary choice models. The appendix contains substantial detailed proof material, including the convex–nonconvex equivalence (Lemma G.3) and sieve-based structural arguments. However, the most delicate part of the proof—the leave-one-out analysis of the nonconvex iterates—is omitted, so the central entrywise bound is not currently verifiable. The numerical study is also self-referential: it simulates BTL comparisons from a constructed 'quasi-true' score matrix rather than using real pairwise choices, so it does not validate the key structural assumptions.","major_comments":[{"comment":"The proof of Theorem F.1 (and hence Theorem 4.1, Corollary F.2, and the debiasing results) invokes (H.2), Lemma H.1, and Lemma H.2 directly. Section H states that the proofs are omitted 'for brevity' and 'available upon request.' These lemmas are not routine: Lemma H.1 supplies the leave-one-out gradient-descent bounds (H.4)–(H.8), and Lemma H.2 supplies the incoherence and closeness properties used to control ||X_t Y_t^T − L*||∞ and ||bL − X_t Y_t^T||∞. Without a complete Section H, the central entrywise error claim is unverifiable. Please include full proofs, or restructure the claims so that they do not depend on unproved lemmas.","section":"Section H; Theorem F.1"},{"comment":"The real-data experiment does not use actual pairwise choices. A 780×40 'quasi-true' score matrix is constructed from discretized watch ratios, and then BTL comparisons are simulated (Section 6). Thus the numerical study can only demonstrate algorithmic self-consistency under the assumed generative model; it cannot validate the load-bearing structural premises of Assumption 3.1 (smoothness of preference functions, low-dimensional item features, small sieve error, spiked/incoherent σ(M*)). The paper should state this limitation explicitly and, ideally, add a diagnostic of approximate low-rankness or analyze a real pairwise-comparison dataset.","section":"Section 6; Assumption 3.1"},{"comment":"Individual-level inference assumes the rank q of the dominant component of M* is known ('For simplicity, this paper assumes that the rank q is known'), but the numerical experiments estimate q by a 10% singular-value threshold (Section A). No theorem shows that this estimator is consistent under Assumptions C.1–C.2, nor that substituting estimated singular vectors into the debiasing procedure preserves the error bounds needed in Theorem 5.3 and Proposition 5.4. The implemented procedure is therefore not covered by the stated guarantees. Either prove consistency of the threshold estimator, or label this part as heuristic and run the UQ simulations with a known q.","section":"Section 5.3 vs. Section A"}],"minor_comments":[{"comment":"The convex estimator bL is not constrained to [0,1], yet Step 2 applies σ^{−1} entrywise. The analysis assumes bL entries are bounded away from 0 and 1; a clipping step would make the procedure well-defined and should be mentioned.","section":"Eq. (3.9), Step 2 of Section 3.4"},{"comment":"The notation \\(\\bar L(j,j')\\) is used for both orientations in Eq. (5.2) and Theorem 5.5, but it is introduced only informally. Please define it at first use and make the orientation in the simultaneous CIs explicit.","section":"Section 5.4"},{"comment":"Simulations set λ = sqrt(0.5 dbar/pbar), while Theorem 4.1 requires λ = C_λ sqrt(dbar/pbar) for a large constant C_λ. The value 0.5 is not 'large'; please clarify whether the constant can be taken small in practice or justify the choice.","section":"Section A and Theorem 4.1"},{"comment":"The paragraph 'All proofs are provided in the appendix' in the introduction is contradicted by Section H. Please revise the wording after completing the appendix.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and a plausible proof strategy, but the omission of Section H is currently a certification blocker. If the authors can supply complete proofs of Lemmas H.1–H.2 and reconcile the rank-estimation practice with the theory, the manuscript may be appropriate for publication. I would not recommend acceptance until those points are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. The indirect regularization idea — regularizing the induced probability matrix sigma(M*) rather than the score matrix — is a real innovation, and the sieve-based argument for why sigma(M*) is approximately low-rank is a genuine step beyond the usual homogeneous-preference BTL literature. If the theorems are right, this is the first entrywise l_infinity error control plus individual-level UQ for heterogeneous preference learning, and the debiasing via one Newton-Raphson step is a clean, natural extension. The authors also cite the relevant low-rank inference literature honestly, and the self-citations are to standard tools rather than to themselves.\n\nThe soft spots are real, though. The stress-test note lands: Lemma H.1 and H.2 are load-bearing for Theorem F.1, hence for Theorem 4.1 and everything built on Corollary F.2. The appendix says the proofs are 'similar to Chen et al. (2020b)' and available on request. That is not acceptable for the central step. The claims may be true, but the manuscript as posted is not verifiable. This is a verifiability gap, not evidence of a false theorem. The other issue is the 'real data' section: it constructs a quasi-true score matrix from watch ratios and then simulates BTL comparisons. That is a useful synthetic experiment built from real covariates, but it does not validate the model on actual observed pairwise choices, and the paper should label it as such. No code or data is released, which makes the simulations irreproducible as-is.\n\nThe assumptions in 3.1 are strong — smooth preference functions and low-dimensional item features — but they are the standard price of sieve-based low-rank inference, and the paper is honest about what it needs. The rest of the appendix is substantial; the omission of Section H is the main defect.\n\nThis paper should go to a serious referee, but conditionally: the referee must be able to inspect the full leave-one-out proof. I would not cite it yet, and I would not put it in a reading group until the omitted lemmas are public. If the authors supply Section H, the question becomes whether the rates are sharp and whether the simulations actually match the theory.","headline":"Genuinely new indirect-regularization idea and plausible entrywise UQ results for heterogeneous BTL, but the main theorem currently rests on an omitted proof section and the real-data experiment is actually a simulation.","tokens_in":77291,"tokens_out":991,"would_cite":false,"duration_ms":16686,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse pairwise comparisons can yield entrywise-accurate preference scores and valid ranking confidence intervals for groups and individual users, under a heterogeneous Bradley-Terry-Luce model.","keywords":["Bradley–Terry–Luce model","preference learning","collaborative ranking","sieve approximation","nonconvex optimization","ℓ∞ error bound","uncertainty quantification","nuclear norm regularization"],"falsifier":"Take a synthetic population where each user's preference over items is an arbitrary permutation (no smooth low-dimensional structure) and items have no shared latent features; run the proposed estimator with increasing d1 and d2 and check whether the entrywise error ∥Θ̂−Θ⋆∥∞ shrinks at the rate predicted by Theorem 4.1. If it does not, or if the sieve approximation error ∥E∥∞ fails to satisfy Assumption 3.1(i) when R grows, the low-rank premise underlying the theorem fails. A less extreme check is to generate preferences from g_i(η_j) with high-frequency, non-smooth oscillations and compare th","tokens_in":76400,"feed_emoji":"🏆","tokens_out":7581,"duration_ms":80510,"temperature":0.7,"pith_summary":"This paper aims to prove that heterogeneous human preferences can be learned from sparse pairwise comparisons with uniform entrywise accuracy, and that the learned score gaps can support valid confidence intervals for rankings, at the group level and even for an individual user. It models each user's preference as a smooth function of low-dimensional item features, which makes the underlying score matrix approximately low-rank, and then \"indirectly regularizes\" the induced pairwise-choice probability matrix instead of the score matrix itself. The argument combines sieve approximation, a reparameterized nonconvex surrogate of the regularized least-squares problem, leave-one-out gradient descent, and a single Newton-Raphson debiasing step to remove regularization bias. If the theory is right, this is the first entrywise error guarantee and uncertainty-quantification framework for heterogeneous Bradley-Terry-Luce preference learning, and it requires far fewer comparisons per user than learning each user's preferences separately.","feed_headline":"Heterogeneous rankings get entrywise-accurate scores from sparse comparisons","feed_subtitle":"A new estimator controls every score entry and, after one Newton step, yields confidence intervals for group and individual rankings.","key_machinery":"The carrying mechanism is indirect regularization: instead of penalizing the score matrix Θ⋆ or the score-gap matrix M⋆, the estimator solves a nuclear-norm-regularized least-squares problem for the induced choice-probability matrix σ(M⋆). Smoothness of the logistic link and of the nonparametric preference functions transfers approximate low-rank structure to the probability matrix, with sieve rank R. The reparameterized nonconvex surrogate f(X,Y) has a loss whose gradient is linear in X and Y, so it lies in the low-rank subspace and the gap between the convex and nonconvex solutions can be controlled; leave-one-out gradient-descent iterates then yield the entrywise bound. A one-step Newton-","core_discovery":"The paper's central claim is that the score matrix Θ⋆, whose (i,j) entry is user i's score for item j, can be estimated entrywise uniformly from a small random sample of item comparisons, and that the debiased estimator is asymptotically normal for both averaged and individual score gaps. The key move is to regularize the probability matrix σ(M⋆), where M⋆ is the score-gap matrix, rather than the score matrix itself: because σ is smooth and the preference functions are assumed smooth, σ(M⋆) is approximately low-rank with a sieve representation σ(M⋆)=ΛΦ⊤+E whose approximation error is o(1/√max{d1,d2(d2−1)/2}). Theorem 4.1 gives an ℓ∞ error bound of order κ²μR/(min{d1,d2(d2−1)/2}) √(d̄ log d̄/","pith_inferences":["The paper leaves implicit that the same indirect-regularization idea could debias other nonlinear low-rank problems, such as low-rank reward models trained from human feedback, where the likelihood gradient is not aligned with the factorization subspaces; a direct test would be to replace the logistic link with a probit or neural link and check whether entrywise control survives.","Because the procedure never estimates or validates the item feature vectors or the sieve basis, a practical diagnostic would be to compare observed entrywise errors against the rate predicted by Assumption 3.1; a data-driven sieve dimension R might make the method less sensitive to misspecification.","The sample-splitting step in individual inference is introduced for technical independence; the paper's own simulations suggest splitting costs little, so a theory avoiding the split may be within reach.","The ranking confidence intervals are built from simultaneous score-gap intervals and are likely conservative; sharper intervals could come from directly bootstrapping the rank statistic rather than passing through all pairwise gaps."],"forward_implications":["Every entry of the estimated score matrix is simultaneously close to the truth, not just on average, so recommendations and top-K item sets for each user inherit uniform guarantees.","A single Newton-Raphson step removes the regularization bias, so score-gap differences and item rankings can be reported with confidence intervals at the aggregated level.","Individual-user inference is also available after rank-q spectral projection, so personalized ranking and selection statements can be quantified, not just population averages.","The per-user sample requirement depends on the sieve dimension and conditioning parameters rather than directly on the number of items, a major reduction compared with learning each user's preferences in isolation.","The same indirect-regularization analysis extends to any binary choice model with a smooth link function, not only the logistic BTL model."],"supporting_citations":[{"why":"Supplies the leave-one-out nonconvex surrogate technique for entrywise error in low-rank matrix completion that the indirect-regularization analysis adapts.","marker":"Chen et al., 2019b"},{"why":"Shows it suffices to bound the convex-nonconvex gap on the low-dimensional subspace when the gradient is small, the key transfer step for the regularized least-squares estimator.","marker":"Chen et al., 2020b"},{"why":"Provides sieve low-rank assumptions, incoherence bounds, and low-rank inference tools the paper borrows for the probability matrix.","marker":"Chernozhukov et al., 2023"},{"why":"Establishes the low-rank completion inference framework and assumptions used for individual-score inference.","marker":"Choi et al., 2024"},{"why":"Documents the technical difficulty of nonconvex surrogates for regularized MLE that motivates indirect regularization.","marker":"Fan et al., 2025a"},{"why":"Supplies the representation formula for spectral projectors used to control singular-subspace estimation error in individual inference.","marker":"Xia and Yuan, 2021"},{"why":"Provides the matrix Bernstein inequalities used throughout the proof of entrywise and debiasing error bounds.","marker":"Tropp, 2015"},{"why":"Gives the homogeneous top-K ranking baseline and entrywise/uncertainty-quantification results that the paper extends to heterogeneous preferences.","marker":"Chen et al., 2019a"}],"fun_headline_variants":["Entrywise-accurate heterogeneous ranking scores from sparse data","One Newton step debiases ranking scores for confident comparisons","Individual and group ranking CIs from indirect regularization","Sparse choices, precise scores: entrywise error bounds for rankings","Debiased heterogeneous rankings: error control and uncertainty"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that real preferences are smooth functions of a low-dimensional item feature vector, so the pairwise-choice probability matrix is close to a low-rank matrix with manageable sieve rank; if preferences are not smooth or items do not share low-dimensional features, the entrywise error bound has no low-rank structure to exploit.","fun_headline_variants_meta":{"raw":{"variants":["Entrywise-accurate heterogeneous ranking scores from sparse data","One Newton step debiases ranking scores for confident comparisons","Individual and group ranking CIs from indirect regularization","Sparse choices, precise scores: entrywise error bounds for rankings","Debiased heterogeneous rankings: error control and uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00028,"raw_usage":{"total_tokens":1501,"prompt_tokens":751,"completion_tokens":750,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":670}},"tokens_in":495,"tokens_out":750,"duration_ms":9101,"temperature":1.0,"reasoning_tokens":670,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:07:31.043664+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a synthetic population where each user's preference over items is an arbitrary permutation (no smooth low-dimensional structure) and items have no shared latent features; run the proposed estimator with increasing d1 and d2 and check whether the entrywise error ∥Θ̂−Θ⋆∥∞ shrinks at the rate predicted by Theorem 4.1. If it does not, or if the sieve approximation error ∥E∥∞ fails to satisfy Assumption 3.1(i) when R grows, the low-rank premise underlying the theorem fails. A less extreme check is to generate preferences from g_i(η_j) with high-frequency, non-smooth oscillations and compare th","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides sieve low-rank assumptions, incoherence bounds, and low-rank inference tools the paper borrows for the probability matrix."}],"review_version":1}