{"id":"f87c2252-2066-4f72-8d52-b0ffca249703","arxiv_id":"2606.00846","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A dueling bandit algorithm with belief-aware upper confidence bound is introduced for efficient, interaction-based selection of LLMs matching user latent preferences.","lead":"The paper proposes an active learning framework using dueling bandits to iteratively select pairs of LLMs, gather user preference feedback, and match users to suitable models under cost and time constraints. A smart generalist might read it to see a practical approach for navigating the growing number of opaque AI models without needing to describe exact preferences upfront.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest assumption directly targets the two premises required for the experimental claim to hold. Because the full manuscript is stated to be available yet the reader's verdict remains UNVERDICTED solely for lack of access, the skeptic pass finds no additional load-bearing gap beyond what the reader already flagged.","tokens_in":1660,"tokens_out":291,"duration_ms":18809,"concrete_test":"Re-run the human-study protocol from the full paper (Section 5) with an independent cohort of 30 participants, recording both pairwise choices and a post-hoc ranking of all models; if the inferred ranking from the bandit matches the explicit ranking in fewer than 70 % of cases within the stated budget, the feedback-to-preference mapping fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract presents a dueling-bandit active-learning setup that iteratively queries pairwise LLM responses and updates a belief over latent user preferences via a novel belief-aware UCB rule. The central claim is that this procedure empirically matches users to suitable LLMs at lower cost than alternatives, under explicit budgets. No internal contradiction, missing identifiability condition, or unstated modeling assumption that would invalidate the claim is visible in the given description. The reader's weakest assumption correctly isolates the two empirical premises (feedback fidelity and UCB efficacy) that the claimed experiments are intended to test.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes CUPID, an interaction-efficient active learning framework based on dueling bandits that iteratively selects pairs of LLMs, collects user pairwise feedback on responses, and updates beliefs over latent user preferences using a novel belief-aware UCB strategy. The central claim is that this procedure enables efficient matching of users to well-aligned LLMs under explicit cost and time budgets, with experimental verification through diverse LLM experiments and human studies showing lower cost than alternatives.","tokens_in":1744,"tokens_out":334,"duration_ms":13863,"significance":"If the empirical results hold under the stated assumptions about feedback fidelity and UCB performance, the work addresses a practical and timely problem in LLM selection amid growing model pools with opaque properties. The budget-aware active learning approach could offer a scalable alternative to explicit preference articulation, with potential impact on user-facing LLM deployment tools.","major_comments":[],"minor_comments":[{"comment":"The abstract and introduction would benefit from explicit enumeration of the baseline methods compared in the experiments (e.g., random selection, standard UCB without belief awareness) to allow readers to assess the claimed cost reductions.","section":null},{"comment":"Clarify the precise definition of 'cost' and 'budget' (monetary, query count, or wall-clock time) in the experimental setup section, as this directly affects interpretation of the efficiency claims.","section":null},{"comment":"Add a limitations paragraph discussing potential biases in human pairwise feedback (e.g., position bias or fatigue) and how they were mitigated in the human studies.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful summary of our work and the positive assessment of its significance for practical LLM selection. We are pleased with the recommendation for minor revision and will address any editorial or minor points in the revised version.","responses":[],"tokens_in":1130,"tokens_out":64,"duration_ms":7257,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"CUPID treats the growing pool of LLMs as a bandit problem where the system picks pairs of models, shows users their responses, and updates a belief over what the user actually prefers. The new piece is the belief-aware UCB rule that tries to balance trying new models against using what it has already learned, all while staying inside a user-set budget for queries or time.\n\nThe paper does a reasonable job of stating the practical problem: most users cannot name the exact properties they want in an LLM response. Running both synthetic LLM experiments and actual human studies is the right move for this kind of work, and the abstract says the method finds suitable models at lower cost than the alternatives they tested.\n\nThe soft spot is that everything depends on pairwise feedback being a reliable signal of latent preference. If users are inconsistent or if the responses are too similar to compare, the belief update will not help much. The abstract also does not spell out how the belief is represented or exactly how the new UCB differs from standard dueling-bandit UCBs already used in recommendation settings, so it is hard to judge whether the technical step is incremental or more substantial.\n\nThis is the kind of paper that would interest people building model-selection tools or running internal LLM evaluations for teams. A practitioner who has to pick from dozens of models for a specific task might get a usable algorithm if the cost numbers hold up under scrutiny.\n\nI would send it to peer review. The framing is clear, the human-study component is appropriate, and referees can check whether the reported savings survive proper baselines and statistical tests.","headline":"CUPID frames LLM selection as dueling bandits with a belief-aware UCB and claims lower-cost matching via human studies, but the size of any real gain is unclear without the full experimental details.","tokens_in":2243,"tokens_out":409,"would_cite":false,"duration_ms":21316,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A belief-aware dueling bandit framework matches users to LLMs using iterative pairwise feedback under budget limits.","keywords":["LLM selection","dueling bandits","active learning","preference learning","online algorithms","user modeling","model zoo"],"falsifier":"A user study where the proposed method fails to achieve higher alignment scores than a baseline random pairing method within the same budget would falsify the efficiency claim.","tokens_in":2544,"feed_emoji":"🤖","tokens_out":497,"duration_ms":21675,"temperature":0.7,"pith_summary":"This paper proposes an active learning approach to help users select suitable LLMs from a large pool without needing to articulate their preferences explicitly. It uses a dueling bandit algorithm to choose pairs of LLMs for users to compare based on responses, then updates beliefs about what the user values. A novel belief-aware upper confidence bound method helps decide when to explore new models versus exploit known good ones. Experiments with LLMs and human participants demonstrate that this leads to well-aligned recommendations at reduced interaction cost compared to other methods.","feed_headline":"Pairwise feedback finds users' preferred LLM at lower cost","feed_subtitle":"Collects comparisons to infer hidden preferences and match users to aligned models under budget.","key_machinery":"Dueling bandit algorithm employing a belief-aware upper confidence bound strategy for selecting LLM pairs based on updated preference beliefs.","core_discovery":"The framework enables efficient alignment between user needs and LLM capabilities by iteratively selecting LLM pairs, collecting feedback, and using a belief-aware UCB strategy to balance exploration and exploitation within user-specified cost and time budgets.","pith_inferences":["If user feedback is inconsistent, the belief updates may lead to suboptimal matches.","This approach could be adapted for selecting other types of AI models or services.","Future work might incorporate multi-turn interactions or richer feedback types."],"forward_implications":["Users receive LLM recommendations that better match their implicit preferences with fewer comparisons.","The method operates effectively within given cost and time budgets.","Human studies confirm lower cost for achieving alignment.","Applicable to growing pools of LLMs with opaque properties."],"fun_headline_variants":["Bandit algorithm uses LLM pairs to learn user preferences","Iterative comparisons align LLMs with hidden user needs","Belief aware method selects models efficiently under constraints","Active learning matches users to LLMs with cost control","Pairwise feedback enables budget friendly LLM selection"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"User pairwise feedback on LLM responses accurately reflects their latent preferences.","fun_headline_variants_meta":{"raw":{"variants":["Bandit algorithm uses LLM pairs to learn user preferences","Iterative comparisons align LLMs with hidden user needs","Belief aware method selects models efficiently under constraints","Active learning matches users to LLMs with cost control","Pairwise feedback enables budget friendly LLM selection"]},"model":"grok-4.3","cost_usd":0.004589,"raw_usage":{"total_tokens":2220,"prompt_tokens":554,"num_sources_used":0,"completion_tokens":71,"cost_in_usd_ticks":45887000,"prompt_tokens_details":{"text_tokens":554,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1595,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":554,"tokens_out":71,"duration_ms":10462,"temperature":1.0,"reasoning_tokens":1595,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T19:04:20.139696+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A user study where the proposed method fails to achieve higher alignment scores than a baseline random pairing method within the same budget would falsify the efficiency claim.","supporting_citations":[],"review_version":1}