Pith. sign in

REVIEW 3 minor 25 references

CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM

T0 review · 0 major / 3 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A belief-aware dueling bandit framework matches users to LLMs using iterative pairwise feedback under budget limits.

desk verdict CUPID frames LLM selection as dueling bandits with a belief-aware UCB and claims lower-cost matching via human studies, but the size of any real gain is unclear without the full experimental details. read the letter →

arxiv 2606.00846 v1 pith:N7H6DRX7 submitted 2026-05-30 cs.LG

Son Nguyen , Xinyuan Liu , Ransalu Senanayake This is my paper · ORCID
classification cs.LG
keywords LLMselectionduelingbanditsactivelearningpreferenceonlinealgorithmsusermodelingmodelzoo
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes an active learning approach to help users select suitable LLMs from a large pool without needing to articulate their preferences explicitly. It uses a dueling bandit algorithm to choose pairs of LLMs for users to compare based on responses, then updates beliefs about what the user values. A novel belief-aware upper confidence bound method helps decide when to explore new models versus exploit known good ones. Experiments with LLMs and human participants demonstrate that this leads to well-aligned recommendations at reduced interaction cost compared to other methods.

What carries the argument

Dueling bandit algorithm employing a belief-aware upper confidence bound strategy for selecting LLM pairs based on updated preference beliefs.

What would settle it

A user study where the proposed method fails to achieve higher alignment scores than a baseline random pairing method within the same budget would falsify the efficiency claim.

Watch

Extended reading notes

Core claim

The framework enables efficient alignment between user needs and LLM capabilities by iteratively selecting LLM pairs, collecting feedback, and using a belief-aware UCB strategy to balance exploration and exploitation within user-specified cost and time budgets.

Load-bearing premise

User pairwise feedback on LLM responses accurately reflects their latent preferences.

Editorial extensions

If this is right

  • Users receive LLM recommendations that better match their implicit preferences with fewer comparisons.
  • The method operates effectively within given cost and time budgets.
  • Human studies confirm lower cost for achieving alignment.
  • Applicable to growing pools of LLMs with opaque properties.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If user feedback is inconsistent, the belief updates may lead to suboptimal matches.
  • This approach could be adapted for selecting other types of AI models or services.
  • Future work might incorporate multi-turn interactions or richer feedback types.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper proposes CUPID, an interaction-efficient active learning framework based on dueling bandits that iteratively selects pairs of LLMs, collects user pairwise feedback on responses, and updates beliefs over latent user preferences using a novel belief-aware UCB strategy. The central claim is that this procedure enables efficient matching of users to well-aligned LLMs under explicit cost and time budgets, with experimental verification through diverse LLM experiments and human studies showing lower cost than alternatives.

Significance. If the empirical results hold under the stated assumptions about feedback fidelity and UCB performance, the work addresses a practical and timely problem in LLM selection amid growing model pools with opaque properties. The budget-aware active learning approach could offer a scalable alternative to explicit preference articulation, with potential impact on user-facing LLM deployment tools.

minor comments (3)
  1. The abstract and introduction would benefit from explicit enumeration of the baseline methods compared in the experiments (e.g., random selection, standard UCB without belief awareness) to allow readers to assess the claimed cost reductions.
  2. Clarify the precise definition of 'cost' and 'budget' (monetary, query count, or wall-clock time) in the experimental setup section, as this directly affects interpretation of the efficiency claims.
  3. Add a limitations paragraph discussing potential biases in human pairwise feedback (e.g., position bias or fatigue) and how they were mitigated in the human studies.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the careful summary of our work and the positive assessment of its significance for practical LLM selection. We are pleased with the recommendation for minor revision and will address any editorial or minor points in the revised version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The paper describes an empirical active-learning framework (dueling-bandit selection of LLM pairs, belief updates from pairwise feedback, and a belief-aware UCB rule) whose central claim is verified by experiments on LLMs and human studies under explicit budgets. No derivation chain, uniqueness theorem, fitted-parameter prediction, or self-citation load-bearing step is present in the provided text; the method is presented as a novel but directly testable procedure rather than a result that reduces by construction to its own inputs.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Based solely on the abstract, no explicit free parameters, axioms, or invented entities are identifiable; the framework description implies standard bandit assumptions but provides no details for auditing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM." pith.science (2026). https://pith.science/paper/N7H6DRX7

@misc{pith2026260600846,
  author       = {Pith},
  title        = {Pith review of: CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N7H6DRX7}},
  note         = {Machine review of arXiv:2606.00846}
}
read the original abstract

Users increasingly face the challenge of selecting an appropriate LLM for a given task from a rapidly growing pool of LLMs, each with distinct but often opaque latent properties. Compounding this challenge, users may lack the vocabulary or awareness to explicitly articulate the characteristics they value in an LLM's responses or deployment. We propose an interaction-efficient active learning framework in which a dueling bandit algorithm iteratively selects pairs of LLMs, collects user feedback about their responses, and updates its belief about the user's latent preferences. We introduce a novel belief-aware upper confidence bound strategy that balances exploration of the model pool with exploitation of inferred preferences, enabling efficient alignment between user needs and LLM capabilities under user-specified cost and time budgets. Through diverse experiments on LLMs and human studies, we experimentally verify that our model can efficiently match well-aligned LLMs to users at a lower cost.

Figures

Figures reproduced from arXiv: 2606.00846 by the authors.

Figure 1
Figure 1. To help users with latent objectives to find matching LLMs with matching capabilities, CUPID efficiently navigates a latent preference space. This targeted navigation results in lower cumulative regret, allowing the system to quickly identify the optimal model at a lower cost. latency, safety, and response characteristics—which are of￾ten difficult to formalize upfront as a traditional software requirements specific… view at source ↗
Figure 2
Figure 2. Based on pairwise comparisons of LLM responses from the model zoo and optional language feedback provided by a user with hidden objectives, a posterior distribution over model preferences and a latent belief is iteratively updated (red loop), while meeting user specified cost and round budget constraints. supplement preference signals (Section 3.2.4). CUPID, the matchmaker. CUPID progressively “matches” a user to an… view at source ↗
Figure 3
Figure 3. Convergence rounds and convergence percentages (left) and mean cost and standard deviation across all algorithms. All CUPID variants attained the maximum CP while converging at competitive cost and performance. convergence but higher cost. Overall, CUPID provides the strongest balance of cost and convergence across all experiments. 2. CUPID achieves reliable convergence across all experiments. Across A1–A4 in [PITH… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Robustness To verify CUPID’s perfor￾mance over a range of di￾verse providers in a com￾petitive setting, we further use the same protocol as Experiment 4.1 over 17 models with the highest MMLU Pro score, and use the best one as the ground truth. Robustness to noisy lan￾…
Figure 5
Figure 5. Figure 5: Human study results. (a) Combined win rates across all studies show CUPID outperforming the LMA baseline in both budget and quality. (b) Final matched model examples: (1) Text example; (2) Image example. (c) Rating results with different levels of abstractness. uninfor…
Figure 6
Figure 6. Figure 6: Language Feedback Modulation. The auxiliary model processes user inputs (blue) and the model information (purple). The bias is calculated and added to the initial eUCB (yellow), yielding an adjusted immediate bias for the subsequent turn (red). o4-mini, initially being…
Figure 7
Figure 7. Figure 7: Mean cost and standard deviation at final rounds. System Prompt: User Simulator You are simulating how a human user refines their preferences over language models. You will be given: - A list of formal constraints / objectives they have (possibly empty). - Metadata for…
Figure 8
Figure 8. Figure 8: User study interface for the interaction phase (within-system duels). 36 [PITH_FULL_IMAGE:figures/full_fig_p036_8.png]
Figure 9
Figure 9. Figure 9: Final-model evaluation interface. Participants rate each system’s final matched model. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_9.png]
Figure 10
Figure 10. Figure 10: Cumulative cost growth during the interaction phase. The y-axis reports cumulative spend. The x-axis is interaction round. This diagnostic provides a time-resolved view of budget consumption, complementing aggregate cost outcomes. 1 2 3 4 5 Budget Rating (1 = Poor, 5 …
Figure 11
Figure 11. Figure 11: Objective spending vs. subjective budget rating. Each point corresponds to one participant session. The x-axis is the budget rating (Likert 1-5), and the y-axis is the participant’s budget-compliance rating for the corresponding system. This figure links the subjectiv…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 6 canonical work pages

  1. [1]

    Bandits with knapsacks

    doi: 10.1145/3164539. URL https://doi. org/10.1145/3164539. Bouneffouf, D. and Feraud, R. Multi-armed bandits meet large language models.arXiv preprint arXiv:2505.13355, 2025. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-V oss, A., Krueger, G., Henighan, ...

    work page doi:10.1145/3164539 2025

  2. [2]

    Llm routing with dueling feedback.arXiv preprint arXiv:2510.00841,

    URL https://proceedings.neurips. cc/paper_files/paper/2020/file/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper. pdf. Chen, L., Zaharia, M., and Zou, J. FrugalGPT: How to use large language models while reducing cost and im- proving performance.Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https:// openreview.net/forum?id=cSimKw5p6R. Fea-...

    work page arXiv 2020

  3. [3]

    URL https://openreview.net/forum? id=3MW8GKNyzI. Chu, W. and Ghahramani, Z. Preference learning with gaus- sian processes. InProceedings of the 22nd International Conference on Machine Learning, pp. 137–144, 2005. Deng, Y ., Zhao, N., and Huang, X. Early ChatGPT User Portrait through the Lens of Data . In2023 IEEE International Conference on Big Data (Big...

    work page doi:10.1109/bigdata59044.2023.10386415 2005

  4. [4]

    cc/paper_files/paper/2020/file/ 9b7c8d13e4b2f08895fb7bcead930b46-Paper

    URL https://proceedings.neurips. cc/paper_files/paper/2020/file/ 9b7c8d13e4b2f08895fb7bcead930b46-Paper. pdf. Hutagalung, D., Sancayawati, N., Aryanty, R., Hutabarat, T., and Almumin, W. Factors influencing user satisfaction with generative artificial intelligence power chat system. International Research Journal of Business Studies, 18: 67–84, 2025. doi:...

    work page doi:10.21632/irjbs.18.1.67-84 2020

  5. [5]

    URL https: //doi.org/10.24963/ijcai.2018/776

    doi: 10.24963/ijcai.2018/776. URL https: //doi.org/10.24963/ijcai.2018/776. Sutton, R. S. and Barto, A. G.Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA,

    work page doi:10.24963/ijcai.2018/776 2018

  6. [6]

    Tsiourvas, A., Sun, W., and Perakis, G

    ISBN 0262039249. Tsiourvas, A., Sun, W., and Perakis, G. Causal llm routing: End-to-end regret minimization from observational data. In Belgrave, D., Zhang, C., Lin, H., Pascanu, R., Koniusz, P., Ghassemi, M., and Chen, N. (eds.), Advances in Neural Information Processing Systems, volume 38, pp. 37289–37314. Curran Associates, Inc.,

  7. [7]

    doi:10.52202/079017-3018 , shorttitle =

    URL https://proceedings.neurips. cc/paper_files/paper/2025/file/ 357774d53e5ee21c5f08ba779e3b5dd9-Paper-Conference. pdf. Tversky, A. and Shafir, E. Choice under conflict: The dynamics of deferred decision.Psychological Sci- ence, 3:358 – 361, 1992. URL https://api. semanticscholar.org/CorpusID:2167374. Urvoy, T., Clerot, F., Féraud, R., and Naamane, S. Ge...

    work page doi:10.52202/079017-3018 2025

  8. [8]

    Zhang, Y .-K., Lu, S., Chen, Q., Luo, W., Zhan, D.-C., and Ye, H.-J

    URL https://openreview.net/forum? id=haa457jwjw. Zhang, Y .-K., Lu, S., Chen, Q., Luo, W., Zhan, D.-C., and Ye, H.-J. Let the llm stick to its strengths: Learning to route economical llm. In Belgrave, D., Zhang, C., Lin, H., Pascanu, R., Koniusz, P., Ghassemi, M., and Chen, N. (eds.),Advances in Neural Information Processing Systems, vol- ume 38, pp. 3355...

Show all 25 references
  1. [9]

    cc/paper_files/paper/2025/file/ 305a5630c29aaca434e3415fc4638eb2-Paper-Conference

    URL https://proceedings.neurips. cc/paper_files/paper/2025/file/ 305a5630c29aaca434e3415fc4638eb2-Paper-Conference. pdf. Zheng, L., Chiang, W.-L., Sheng, Y ., Zhuang, S., Wu, Z., Zhuang, Y ., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I. Judg...

    2025

  2. [10]

    Curran Associates Inc. Zhou, L. and Brunskill, E. Latent contextual bandits and their application to personalized recommendations for new users. InProceedings of the Twenty-Fifth Interna- tional Joint Conference on Artificial Intelligence (IJCAI- 16), 2016. 12 Cupid in the Mod...

    2016

  3. [11]

    URL https://proceedings.mlr

    PMLR. URL https://proceedings.mlr. press/v32/zoghi14.html. 13 Cupid in the Model Zoo A. Theoretical Analysis and Derivations A.1. Why probit? Several likelihood models have been proposed for preference learning, including the Bradley–Terry (logistic) model, temperature-scaled ...

  4. [12]

    Laplace Approximation of the Posterior.Since the probit likelihood is non-Gaussian, the true posterior p(f| D) is analytically intractable. We apply the Laplace approximation to approximate the posterior as a multivariate Gaussian centered at the Maximum A Posteriori (MAP) est...

  5. [13]

    Distribution of the Utility Difference.Let δ=f(a (r))−f(a (s)) be the difference in latent utility between the two items. Sincefis approximated as Gaussian, the linear transformationδis also Gaussian distributed: p(δ| D) =N(δ|µ δ, σ2 δ),(15) where the mean and variance are giv...

  6. [14]

    exchange rate

    Solving the Convolution.Substituting the probit likelihood P(a (r) ≻a (s) |f) = Φ fr−fs√ 2σ into the integral, we seek to solve: P(a (r) ≻a (s) | D) = Z ∞ −∞ Φ δ√ 2σ N(δ|µ δ, σ2 δ)dδ.(18) We utilize the standard identity for the convolution of a Gaussian Cumulative Distributio...

    2005

  7. [15]

    Output no other words

    Output exactly one of: ’arm_A’ or ’arm_B’. Output no other words. The two candidate models (in CSV form) are: [Candidate Models CSV] The constraint is: [User Constraints] B.1.2. ROUTINGMODEL This agent filters the global pool of models based on the user’s directional feedback,...

  8. [16]

    For each model (each row), decide if it matches the user’s preferences and DIRECTION, possibly relative to the last chosen model

  9. [17]

    Use ’1’ if the model is acceptable, ’0’ if it is not

    Output exactly [N] integers (space-separated), one per row in the CSV, in the same order. Use ’1’ if the model is acceptable, ’0’ if it is not

  10. [18]

    Model Zoo

    Output no other words. Last chosen model (CSV; may be ’(none yet’): [Last Model Metadata] CSV of ALL models: [Full Model Zoo CSV] B.2. Hyperparameter Settings Table 4.Hyperparameter Settings Hyperparameter Feature Setting βExploration Weight 1.5 ηBudget Penalty Learning Rate 0...

    2025

  11. [19]

    Do not combine multiple items

    From the list below, focus on at most ONE main change or preference that the constraint did not meet. Do not combine multiple items. [List of Attributes: intelligence, speed, input_price, etc...]

  12. [20]

    For example, if the last model is expensive AND slow, the direction might ask for a cheaper model

    You SHOULD consider the last chosen model’s metadata and any explicit constraints. For example, if the last model is expensive AND slow, the direction might ask for a cheaper model. And if the next iteration it is still slow the direction might ask for a faster model. Only giv...

  13. [21]

    No explanations

    Output ONLY the DIRECTION sentence, without quotes, without any prefix like ’DIRECTION:’. No explanations

  14. [22]

    intelligence

    If all constraints are met, output ’NONE’ D.3. User Types (A1–A4) and Constraint/Budget Construction Budget and round limits.We simulate a strict interaction budget with a maximum number of roundsTand a monetary budgetB, and use the main paper pacing rulec target =B/(T+τ)for t...

  15. [23]

    Battle phase:participants submitted prompts and provided within-system pairwise preferences for each system at each round

  16. [24]

    Participants then evaluated the two final matched models by playing with it and seeing their responses side by side

    Final model Evaluation phase:after the interaction phase ended (budget exhausted or rounds completed), each system nominated its final matched model. Participants then evaluated the two final matched models by playing with it and seeing their responses side by side

  17. [25]

    Give me a model that is bad at math

    User Rating phase:participants rated each system’s final matched model on two 5-point Likert scales (shown in Table 10). Participants also saw each system’stotal accrued cost,rounds completedand the final matched model details before submitting ratings. Participants, ethics, a...

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.