{"id":"2e5accdf-145b-4e4a-b0ea-b8266f391536","arxiv_id":"2607.03515","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Relevance vectors to support items/queries, mapped by light networks, uniformly approximate any continuous ranker and improve lightweight candidate retrieval over dual encoders.","lead":"Queries and items are represented by their scores against a small fixed support set from an expensive ranker; light networks turn those vectors into embeddings that approximate any continuous relevance function. The method yields smaller models that beat dual encoders and prior CUR baselines on academic and production retrieval tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"I agree with the reader that Theorem 3.2 is the strongest formal claim and that continuity/compactness is its explicit hypothesis. I disagree that this hypothesis is a material soft spot for the paper. The theorem is carefully scoped ('under mild conditions'), the proof is standard and correct, and the practical method (relevance vectors + light MLPs + support selection) does not rely on the uniform guarantee holding for every conceivable R. The production experiments already use discontinuous-looking models (gradient-boosted trees) yet still show clear gains, which is consistent with the weaker L2 guarantee of Theorem 3.1. Support-selection ablations and public code further reduce residual risk. Consequently the reader's CONDITIONAL verdict with low correctness risk needs no adjustment; the continuity caveat is worth a clarifying sentence but does not change the overall assessment.","tokens_in":26045,"tokens_out":491,"duration_ms":4726,"concrete_test":"Verify that the heavy rankers used in the experiments (CatBoost on RecGames/RecMusic, the ZESHEL cross-encoder, all-mpnet-base-v2) are continuous (or continuous almost everywhere) on the feature domains that appear at inference; if any is shown to be discontinuous on a positive-measure set that affects ranking, re-evaluate HitRate after a continuous approximation of that ranker.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (continuity of R and compactness of I, Q) is correctly identified as the formal precondition of Theorem 3.2, but it is not load-bearing for the paper's central claim. The claim is an existence result under those mild conditions (Stone-Weierstrass + universal approximation of MLPs, Appendix A.1); the paper never asserts that every practical ranker is continuous. In the regimes that matter (CatBoost, cross-encoders, SentenceTransformers) the functions are continuous or Lipschitz almost everywhere, so the guarantee applies. The L2 result of Theorem 3.1 further covers discontinuous but square-integrable cases. Empirical gains over dual encoders and AnnCUR remain independent of the uniform-approximation statement. Thus the continuity/compactness hypothesis does not undermine either the theory as stated or the practical contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes relevance-based embeddings (RBE) for candidate retrieval: each query (item) is represented by its scores under a heavy ranker R against a fixed support set of items (queries); lightweight maps f_I and f_Q then turn those vectors into embeddings whose inner product approximates R. Theorem 3.1 shows that a regularized CUR factorization of the relevance matrix converges in L2 under mild integrability; Theorem 3.2 shows that, for continuous R on compact spaces, neural maps of the relevance vectors can uniformly approximate R to arbitrary precision (Stone-Weierstrass + universal approximation). Support-set selection is studied (random, popular, clustering, diversity, and an l2-greedy CUR-MSE procedure). Experiments on five ZESHEL domains, MS MARCO, and two production recommenders (RecGames, RecMusic) show that RBE with non-random supports and trainable maps improves HitRate over AnnCUR and, under fair heavy-ranker budgets, over production dual encoders and AXN.","tokens_in":26193,"tokens_out":959,"duration_ms":8079,"significance":"If the claims hold, the work supplies a theoretically grounded, lightweight alternative to dual-encoder candidate selection that can exploit pairwise features unavailable to pure two-tower models, while requiring far fewer trainable parameters (~50K vs hundreds of millions). The uniform-approximation guarantee (Theorem 3.2) and the L2 guarantee for regularized CUR (Theorem 3.1) are clean existence results under standard hypotheses; the empirical gains on both academic and production data, with explicit accounting for the m support calls in the re-ranking budget, make the method immediately usable. Code and data release further strengthen reproducibility. The contribution is therefore both conceptual (relevance vectors as universal features) and practical (better candidate quality at lower model size).","major_comments":[{"comment":"Section 5 / Tables 4–5: the production comparison is the strongest practical claim, yet only point estimates are reported for DE, AXN_DE and RBE. Appendix E.4 supplies means and standard deviations solely for random AnnCUR. Without analogous variability (or at least multiple random seeds / support-set draws) for the production HitRate curves, it is hard to judge whether the reported crossovers (X=200 on RecGames1, X=100 on RecMusic) are stable. Adding error bars or a short multi-seed table would make the claim load-bearing rather than suggestive.","section":null},{"comment":"Section 3.4 and Appendix B: the l2-greedy procedure optimizes the CUR MSE on train queries (Eq. 3), which is not the final HitRate metric. While Table 1 shows it is the best among the tested heuristics, the paper never verifies that the selected supports remain near-optimal once the trainable maps f_I, f_Q are fitted (Table 2). A short ablation that re-optimizes or re-ranks supports under the final listwise loss would confirm that the theoretically motivated selection still drives the neural RBE gains.","section":null}],"minor_comments":[{"comment":"Notation: R(i,SQ) and R(SI,q) are introduced cleanly, but the matrix R(SI,SQ) is sometimes written without parentheses; a single consistent notation would help.","section":null},{"comment":"Figure 1 caption is terse; a one-sentence description of the color coding (support vs test) would make the visualization self-contained.","section":null},{"comment":"Appendix A.2 (sphere reduction) is interesting but unused in the experiments; either cite it in the main text when discussing ANN indexes or move it to a remark.","section":null},{"comment":"Table 3 reports approximate parameter counts; giving exact figures (or a short breakdown of the MLP widths) would make the “lightweight” claim sharper.","section":null},{"comment":"A few typos: “proceesing” (p. 3), “the the” (Appendix A.1), and inconsistent capitalization of HitRate vs HR.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The continuity/compactness hypothesis flagged by the reader is correctly stated as a precondition of Theorem 3.2 and does not undermine the practical contribution; the L2 result covers the discontinuous case. I see no novelty or citation issues. The paper is a solid fit for a machine-learning / IR venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean, practical paper that takes Yadav et al. 2022’s CUR idea and makes it both theoretically tighter and more useful. The core new pieces are Theorem 3.2 (any continuous R on compact spaces is uniformly approximable by the inner product of two light nets on finite support relevance vectors, via Stone-Weierstrass + UAT) and the systematic support-selection work, especially the l2-greedy algorithm that actually optimizes the CUR residual. They also show that once you have the relevance vectors you can train a ~50k-parameter RBE that beats both AnnCUR and production dual encoders (hundreds of millions of parameters) on RecGames and RecMusic once the re-ranking budget is equalized for the 100 support calls.\n\nWhat they do well: the theory is short and correct; the L2 result for regularized CUR is standard but cleanly stated; the experiments cover five ZESHEL domains, MS MARCO, and two live recommenders with honest budget accounting; code and academic data are public. Support selection is ablated thoroughly—clustering and l2-greedy clearly beat random, and popular works when the data has natural categories. The production comparison is the part that will matter to practitioners: same HitRate@K curves, far smaller model, no feature engineering beyond the black-box ranker.\n\nSoft spots are minor and proportional. Continuity/compactness is required for the uniform guarantee, but the paper never claims every practical ranker satisfies it; CatBoost and the cross-encoders they use are continuous or Lipschitz a.e., and Theorem 3.1 already covers the L2 case. Error bars appear only for the random baseline; the other methods look stable enough that this is not load-bearing. Free parameters (support size 100, MLP widths) are standard and ablated. No circularity: the approximation theorems stand alone, and HitRate is measured on held-out queries.\n\nThis is for people who already run a heavy ranker and need a lightweight candidate stage that can still use pairwise features. It is not a paradigm shift, but it is a real, checkable improvement over the previous CUR line and over production DEs. I would send it to peer review without hesitation; the contribution is clear enough that a referee can decide how much novelty is enough.","headline":"Solid, usable extension of Yadav CUR: uniform-approx theorem + better support selection + real production wins with a tiny model.","tokens_in":26835,"tokens_out":554,"would_cite":true,"duration_ms":7378,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Any continuous relevance function can be recovered from finite support-item scores alone, turning a heavy ranker into lightweight embeddings for candidate retrieval.","keywords":["candidate retrieval","relevance-based embeddings","cross-encoder approximation","support items","dual encoders","approximate nearest neighbor","CUR decomposition","information retrieval"],"falsifier":"On a compact continuous relevance task, measure whether HitRate of RBE with a few hundred support items approaches the HitRate of exhaustive heavy-ranker search as the support size grows; if the gap stays large the claimed approximation power is false.","tokens_in":26918,"feed_emoji":"🔍","tokens_out":882,"duration_ms":7116,"temperature":0.7,"pith_summary":"When you must find the best items for a query under an expensive relevance model, exhaustive search is impossible and ordinary dual-encoder embeddings lose pairwise features. This paper shows you can instead describe every query by its scores against a small fixed set of support items, and every item by its scores against a small set of support queries. Those relevance vectors become the inputs to two light neural nets whose inner product approximates the original heavy ranker arbitrarily closely. The authors prove the approximation is uniform for any continuous relevance on compact spaces, and they show that carefully chosen support sets (especially a greedy L2 method or cluster centers) beat random selection. On entity-linking, question-answering and production recommendation data the resulting lightweight embeddings match or exceed strong dual encoders while using far fewer trainable parameters and no hand-engineered features.","feed_headline":"Heavy ranker scores alone recover any continuous relevance","feed_subtitle":"Finite support vectors turn expensive models into lightweight, ANN-searchable embeddings","key_machinery":"Relevance-based embeddings (RBE): the maps e_I(i)=f_I(R(i,S_Q)) and e_Q(q)=f_Q(R(S_I,q)) whose inner product approximates R; the uniform-approximation theorem (Theorem 3.2) that guarantees such maps exist via the Stone–Weierstrass algebra generated by the elementary relevance functions.","core_discovery":"Under mild topological conditions any continuous relevance function R can be uniformly approximated to arbitrary precision by the inner product of two neural networks that see only the relevance vectors of a finite support set of items and of queries. Consequently a heavy black-box ranker can be turned into high-quality, ANN-searchable embeddings without ever training a separate dual encoder from scratch.","pith_inferences":["The same support-relevance construction could serve as a drop-in feature layer for any downstream model that currently relies on dual-encoder embeddings (e.g., category prediction or diversity re-ranking).","Because the heavy ranker is treated as a black box, the method is immediately portable to any domain where a strong pairwise scorer already exists but a fast first-stage index does not.","If support items are chosen once offline, the online cost is only a few dozen heavy-ranker calls per query—comparable to the re-ranking budget already paid by dual-encoder pipelines."],"forward_implications":["Any existing black-box ranker can be converted into an ANN index without training a second heavy dual-encoder model.","Pairwise features that dual encoders cannot use become automatically available through the relevance vectors.","Support-set choice (cluster centers or L2-greedy) becomes a first-class design lever that measurably improves retrieval quality.","Model size collapses from hundreds of millions of parameters to tens of thousands while still matching production dual-encoder quality on recommendation tasks.","New items can be embedded on the fly by scoring them against the fixed support queries, without re-training."],"fun_headline_variants":["Support relevances alone recover any continuous ranker","Finite support scores turn black-box rankers into ANN embeddings","Heavy ranker calls on supports yield approximable dual embeddings","Relevance vectors of supports uniformly approximate continuous R","Support-set relevances make expensive models lightweight embeddings"],"cache_read_input_tokens":20480,"weakest_assumption_plain":"The relevance function must be continuous and the spaces of queries and items must be compact; if either fails the uniform-approximation guarantee no longer holds.","fun_headline_variants_meta":{"raw":{"variants":["Support relevances alone recover any continuous ranker","Finite support scores turn black-box rankers into ANN embeddings","Heavy ranker calls on supports yield approximable dual embeddings","Relevance vectors of supports uniformly approximate continuous R","Support-set relevances make expensive models lightweight embeddings"]},"model":"grok-4.5","effort":"low","cost_usd":0.003722,"raw_usage":{"total_tokens":1147,"prompt_tokens":742,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":37220000,"prompt_tokens_details":{"text_tokens":742,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":342,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":742,"tokens_out":63,"duration_ms":3032,"temperature":1.0,"reasoning_tokens":342,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T01:56:25.969458+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a compact continuous relevance task, measure whether HitRate of RBE with a few hundred support items approaches the HitRate of exhaustive heavy-ranker search as the support size grows; if the gap stays large the claimed approximation power is false.","supporting_citations":[],"review_version":1}