{"id":"22c05e73-b911-48e6-847a-a58a646d33d2","arxiv_id":"2601.12078","paper_version":2,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"PURPLE uses contextual bandits and a Plackett-Luce model to optimize the selection and ordering of user records for retrieval-augmented LLMs, directly aligning retrieval with generation quality via reference likelihood feedback.","lead":"PURPLE is a contextual bandit system that builds user profiles for LLMs by learning which history records and in what order actually improve answer quality instead of picking by semantic similarity. This could make personalized LLM responses more reliable and efficient without retraining the model.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Reference likelihood reward risks overfitting to fixed references rather than true generation utility","rationale":"The identified concern is identical to the reader's weakest assumption about the reference-likelihood reward. No stronger internal inconsistency (e.g., in the Plackett-Luce formulation or bandit regret analysis) is visible from the abstract and described method; the risk is empirical and would be settled by the proposed check.","tokens_in":1678,"tokens_out":358,"duration_ms":12651,"concrete_test":"On one of the nine tasks, replace the reference-likelihood reward with an alternative (e.g., ROUGE-L or BERTScore against the reference, or a small human preference set) and retrain the bandit; if the relative improvement over the strongest baseline drops by >15% or loses statistical significance, the original reward signal is the load-bearing weakness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PURPLE outperforms baselines rests on training the Plackett-Luce contextual bandit with reward = log P(reference response | query, selected profile). This implicitly assumes the reference is an unbiased proxy for downstream quality. However, the reward can be gamed by profiles that boost probability mass on the exact reference tokens (e.g., via lexical overlap or stylistic mimicry) without improving robustness to paraphrases, conflicting facts, or novel queries. Because the ranking model is order-sensitive and the bandit updates directly on this scalar, any reference-specific bias propagates into the learned policy. The abstract and reader's weakest assumption both flag this, but without reported correlation between the reward and independent metrics (human preference, zero-shot on new references, or adversarial paraphrases), the reported gains on nine tasks could be artifacts of the proxy rather than genuine alignment.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes PURPLE, a contextual bandit framework for optimizing user profiles in retrieval-augmented LLM personalization. It models profile construction as an order-sensitive ranking process via a Plackett-Luce model and trains the policy using the log-likelihood of generating a fixed reference response as the reward signal. Experiments on nine personalization tasks are reported to show consistent outperformance over heuristic and retrieval-augmented baselines in both effectiveness and efficiency.","tokens_in":1859,"tokens_out":516,"duration_ms":24126,"significance":"If the reported gains prove robust to the concerns below, the work supplies a direct optimization route from retrieval selection to generation quality that goes beyond semantic similarity. The Plackett-Luce treatment of inter-record dependencies and the emphasis on efficiency constitute concrete technical strengths that could influence future retrieval-augmented personalization systems.","major_comments":[{"comment":"§4 (Experiments): The abstract claims consistent gains across nine tasks, yet the manuscript supplies no exact baseline specifications, no statistical significance tests, and no ablation isolating the Plackett-Luce component from simpler ranking or greedy selection. Without these elements the central empirical claim cannot be evaluated at the required level of rigor.","section":"§4 (Experiments)"},{"comment":"§3.2 (Reward definition): The training objective uses log P(reference response | query, selected profile) as the scalar reward. This proxy can be gamed by profiles that increase probability mass on the exact reference tokens (lexical overlap or stylistic mimicry) without improving robustness on paraphrases or novel queries; no correlation analysis with independent metrics (human preference, zero-shot new references, or adversarial paraphrases) is provided to support the alignment assumption.","section":"§3.2 (Reward definition)"}],"minor_comments":[{"comment":"The Plackett-Luce parameterization is introduced without a worked numerical example or pseudocode, making the order-sensitive update rule harder to follow on first reading.","section":"§3.1"},{"comment":"A small number of recent bandit-for-retrieval papers are omitted from the related-work discussion; adding them would sharpen the novelty statement.","section":"§2"}],"recommendation":"major_revision","confidential_remarks":"The experimental protocol is underdeveloped relative to the strength of the abstract claims; a revision that adds the missing ablations, significance tests, and reward-validation experiments would materially strengthen the submission."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on our manuscript. We address each major comment below and commit to a revised version that incorporates the suggested improvements for greater rigor and validation.","responses":[{"response":"We agree that the experimental section requires additional detail for full reproducibility and rigor. In the revision we will: (1) provide exact hyperparameter settings, dataset splits, and implementation details for all baselines (including the specific retrieval-augmented and heuristic methods); (2) report statistical significance via paired t-tests or Wilcoxon signed-rank tests with p-values across the nine tasks; and (3) add an ablation study that isolates the Plackett-Luce ranking model against simpler greedy selection and non-order-sensitive ranking variants. These additions will directly support the central empirical claims.","revision_made":"yes","referee_comment":"[§4 (Experiments)] §4 (Experiments): The abstract claims consistent gains across nine tasks, yet the manuscript supplies no exact baseline specifications, no statistical significance tests, and no ablation isolating the Plackett-Luce component from simpler ranking or greedy selection. Without these elements the central empirical claim cannot be evaluated at the required level of rigor."},{"response":"We acknowledge that the reference-likelihood reward could in principle be influenced by surface-level overlap. However, because the reward is computed from the full generative likelihood under the target LLM (rather than token-level matching), it already encodes semantic and contextual utility. To strengthen the alignment claim we will add, in the revision, a correlation analysis between the learned reward and (a) human preference ratings on a held-out subset and (b) performance on paraphrased and zero-shot novel queries. This will provide empirical support for the proxy while preserving the direct optimization objective.","revision_made":"partial","referee_comment":"[§3.2 (Reward definition)] §3.2 (Reward definition): The training objective uses log P(reference response | query, selected profile) as the scalar reward. This proxy can be gamed by profiles that increase probability mass on the exact reference tokens (lexical overlap or stylistic mimicry) without improving robustness on paraphrases or novel queries; no correlation analysis with independent metrics (human preference, zero-shot new references, or adversarial paraphrases) is provided to support the alignment assumption."}],"tokens_in":1362,"tokens_out":494,"duration_ms":14243,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is framing user profile construction as an order-sensitive ranking task solved via contextual bandits and a Plackett-Luce model, with the policy trained directly on the log-likelihood of generating a reference response given the selected records. This moves past simple semantic similarity and tries to optimize for downstream generation utility instead. The abstract makes a clear case that relevance alone can introduce redundancy or conflicts that hurt the LLM output, and the bandit approach is a reasonable way to capture inter-record dependencies during selection and ordering. That part lands as a clean, practical idea for anyone scaling personalized retrieval without fine-tuning. The experiments claim consistent wins over heuristics and other retrieval baselines across nine tasks, which suggests the method is at least competitive in their setup. The soft spot is the reward signal. Using log P(reference | query, profile) as the training objective assumes this scalar is an unbiased stand-in for real generation quality. In practice it can reward profiles that simply increase overlap with the fixed reference tokens or mimic its style, without improving robustness on paraphrased queries or new user contexts. The abstract does not report ablations isolating the Plackett-Luce component, statistical significance, or correlation checks against independent metrics like human ratings or zero-shot performance on held-out references. Without those, the reported gains could partly reflect proxy overfitting rather than genuine alignment. This work is aimed at researchers and engineers building production RAG systems that adapt to user history. It is worth sending to peer review because the problem framing is honest and the bandit formulation is well-motivated, even though the current evidence leaves room for the reward concern to be tested more rigorously.","headline":"PURPLE uses a contextual bandit with Plackett-Luce to order user records for RAG, trained on reference likelihood, but the reward proxy needs tighter validation before the gains look solid.","tokens_in":2362,"tokens_out":408,"would_cite":false,"duration_ms":40009,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"PURPLE treats profile construction as an order-sensitive generation process and utilizes a Plackett-Luce ranking model to capture complex inter-record dependencies. By training with ... the likelihood of the reference response"},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/RealityFromDistinction.lean","rs_theorem":"reality_from_one_distinction","paper_passage":"We formulate retrieval-augmented LLM personalization as a contextual bandit problem ... reward R(LLM(P ∥ x), y) = log p_ϕ(y | P, x)"}],"headline":"PURPLE bandit + Plackett-Luce ranking has no structural overlap with RS J-cost or distinction forcing","alignment":"orthogonal","rationale":"The paper's core machinery (contextual bandit policy gradient, Plackett-Luce ranking over ordered profiles, reward = log P(reference | profile, query)) operates entirely in the domain of NLP retrieval optimization. It contains no J-cost functional, no ratio-symmetric cost, no golden-ratio ladder, no 8-tick periodicity, and no parameter-free derivation of constants. RS theorems such as reality_from_one_distinction, washburn_uniqueness_aczel, and J_uniquely_calibrated_via_higher_derivative are therefore neither matched nor contradicted.","tokens_in":56436,"confidence":"high","tokens_out":346,"duration_ms":15902,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"PURPLE uses contextual bandits and Plackett-Luce ranking to select user records that directly raise LLM generation quality rather than semantic relevance.","keywords":["contextual bandits","LLM personalization","retrieval augmentation","user profiles","Plackett-Luce ranking","personalization tasks","reference likelihood reward"],"falsifier":"A controlled experiment on held-out queries in which profiles chosen by PURPLE produce lower reference-response likelihood than profiles chosen by the strongest semantic-relevance baseline.","tokens_in":2598,"feed_emoji":"🤖","tokens_out":574,"duration_ms":23778,"temperature":0.7,"pith_summary":"The paper claims that relevance-based retrieval often picks records that add redundancy or conflict and therefore fail to help downstream generation. It introduces PURPLE to model profile construction as an order-sensitive ranking problem solved by a Plackett-Luce bandit whose reward is the likelihood of producing a reference response. This training signal is intended to make selection decisions optimize actual output quality instead of proxy similarity. Experiments across nine personalization tasks report consistent gains in both effectiveness and efficiency over heuristic and retrieval baselines.","feed_headline":"Bandit ranking selects user records to raise LLM output quality","feed_subtitle":"PURPLE replaces relevance matching with Plackett-Luce bandits trained on reference-response likelihood and improves personalization across 9","key_machinery":"PURPLE contextual bandit with Plackett-Luce ranking model that treats profile construction as order-sensitive selection and uses reference-response likelihood as the reward signal.","core_discovery":"PURPLE frames user-profile construction as a contextual bandit problem solved with a Plackett-Luce ranking model; the model is trained on the likelihood of generating reference responses so that retrieval selection is aligned directly with downstream generation quality rather than semantic similarity.","pith_inferences":["The approach could be tested on tasks without reference responses by substituting other cheap feedback signals such as user click-through or self-consistency scores.","Similar utility-versus-similarity gaps may appear in non-personalization retrieval settings such as document summarization or multi-hop QA.","If reference responses are narrow, the learned policy may overfit to particular answer styles and require periodic retraining on fresh data."],"forward_implications":["Profile construction shifts from greedy top-k relevance to order-aware selection that accounts for inter-record dependencies.","Retrieval decisions become directly optimized for generation quality through the reference-likelihood reward.","The same bandit formulation scales across multiple personalization tasks without requiring LLM fine-tuning."],"fun_headline_variants":["PURPLE uses Plackett-Luce bandits for LLM user profile optimization","Contextual bandits train on reference likelihood to rank user records","Bandit framework aligns profile selection with LLM generation quality","PURPLE replaces relevance with Plackett-Luce ranking for LLM personalization"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The likelihood of generating the reference response gives an unbiased and sufficiently rich reward that aligns record selection with actual generation quality.","fun_headline_variants_meta":{"raw":{"variants":["PURPLE uses Plackett-Luce bandits for LLM user profile optimization","Contextual bandits train on reference likelihood to rank user records","Bandit framework aligns profile selection with LLM generation quality","PURPLE replaces relevance with Plackett-Luce ranking for LLM personalization"]},"model":"grok-4.3","cost_usd":0.005994,"raw_usage":{"total_tokens":2736,"prompt_tokens":623,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":59940500,"prompt_tokens_details":{"text_tokens":623,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2043,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":623,"tokens_out":70,"duration_ms":22913,"temperature":1.0,"reasoning_tokens":2043,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-16T12:58:36.553252+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled experiment on held-out queries in which profiles chosen by PURPLE produce lower reference-response likelihood than profiles chosen by the strongest semantic-relevance baseline.","supporting_citations":[],"review_version":1}