{"id":"06dcb1fe-31bb-4a29-bc8d-143410f7b8f8","arxiv_id":"2505.12334","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"UPC uses LLM-judge feedback to generate better training dialogues and an easy-to-hard curriculum, improving user-oriented proactivity in open-domain chatbots.","lead":"This paper introduces UPC, a chatbot training method where an LLM critic scores each response for user background relevance, interest, and value, and the chatbot regenerates low-scoring answers. Training proceeds on simulated users ordered from easy to hard, using a new 800-persona background dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Optimizing against the GPT-3.5 critic during training makes Table 1's critic scores a measure of critic alignment, not user experience; the 62-user pilot is too small to support 'outperforms all other methods'.","rationale":"I agree with the reader's weakest_assumption. The paper has real contributions (ISCO-800, curriculum learning, a pilot user study), but the central evidence is a closed loop: the same critic that selects and regenerates training responses also reports the test scores. This is not internal inconsistency; it is an external-validity risk that is fully testable. The concrete human-evaluation check above would settle whether the critic-based gains transfer to real users. The reader's conditional verdict is appropriate; my stress-test does not change it.","tokens_in":16092,"tokens_out":5579,"duration_ms":57813,"concrete_test":"Conduct a preregistered human evaluation on a held-out set of 300+ dialogues, drawing evenly from UPC and the strongest Table 1 baselines (Qwen1.5-72B-Chat, Llama-3-70B-Instruct, MemoChat). Annotators see only the dialogue transcript and the user background, are blind to system identity, and rate relevance, interest, and value on the same 1-5 scale. If the human ranking and margins do not reproduce the critic-based ordering in Table 1, the central claim is an artifact of critic optimization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the identity between the critic used to build the training corpus and the critic used to score the headline results. Section 4.1 defines S = J(u, U, D) (Eq. 1), and Algorithm 1 (lines 12-15) regenerates any response until the GPT-3.5 critic gives scores >= 4 on relevance, interest, and value. Section 5.1 then evaluates Rel./Int./Val. with gpt-3.5-turbo-0125 or gpt-4-turbo using the same prompt and rubric. The reported 4.858/3.943/3.925 therefore measure how well the trained chatbot satisfies the same judge that filtered its training data; this is direct optimization against the evaluation metric, and can reflect critic-specific stylistic alignment rather than genuine user-oriented proactivity. The GPT-4 column reduces but does not remove this problem: it uses the same rubric and is a close sibling of the training judge. PPL is not a user-experience metric and can improve simply from fine-tuning on a narrow simulated-user corpus. The only external check, the 62-participant study in Section 5.4, compares UPC against a single baseline (Llama-3-70B), reports aggregate preferences without error bars or significance tests, and cannot establish the headline claim of outperforming 'all other methods' in Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UPC (User-oriented Proactive Chatbot), a training framework for open-domain dialogue that enhances user-oriented proactivity. The method has three components: an LLM-based critic that scores chatbot responses on relevance to user background, user interest, and response value; a critic-guided dialogue corpus generation procedure in which low-scoring responses are regenerated until the critic is satisfied; and a communication-difficulty-aware iterative curriculum that trains the chatbot first on easy-to-serve users from a new ISCO-800 dataset. Experiments on ISCO-800 report that a 32B-parameter Qwen1.5 model fine-tuned with UPC outperforms larger baselines on the critic-based metrics and on perplexity. A small online study with 62 participants compares UPC with Llama-3-70B-Instruct and reports that over 75% of participants preferred UPC on overall quality and on each of the three dimensions.","tokens_in":16411,"tokens_out":5476,"duration_ms":57742,"significance":"The paper addresses a real and underexplored problem: making open-domain chatbots proactively attend to the user's background and interests rather than merely following the user or steering to system-chosen targets. The ISCO-800 dataset is a potentially useful resource, and the idea of using critic-guided regeneration together with a communication-difficulty curriculum is plausible and reasonably motivated. The authors also provide ablations, apply the method to multiple base LLMs, and include a real-user pilot, which is more than many dialogue papers do. However, the headline claim that UPC 'outperforms all other methods' is not currently established because the central evaluation metric is the same class of LLM judge that filtered the training data, and the sole external human study is too small and too narrow to carry the claim. If the authors can close this gap with a rigorous human evaluation or a convincing critic-human agreement analysis, the paper could be a solid contribution.","major_comments":[{"comment":"","section":"§4.2, Algorithm 1 vs. §5.1–5.2"},{"comment":"","section":"§5.4, Figure 5"},{"comment":"","section":"§5.1, Table 1"},{"comment":"","section":"§4.3, Eq. (3) and Appendix B"}],"minor_comments":[{"comment":"","section":"Table 2"},{"comment":"","section":"Figure 5"},{"comment":"","section":"Algorithm 1, line 12"},{"comment":"","section":"§5.1, Compared Methods"},{"comment":"","section":"Algorithm 1"},{"comment":"","section":"Abstract and Contributions"}],"recommendation":"major_revision","confidential_remarks":"The central evaluation circularity is the main risk: the training and evaluation judges are too closely related, and the human study is too small to bridge the gap. If the authors can provide a rigorous human evaluation with proper statistics, or a strong critic–human correlation analysis on a held-out set, the paper may become acceptable. I would not reject on novelty grounds, as the ISCO-800 dataset and the curriculum idea are potentially useful, but the current evidence does not support the headline claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's worth keeping: the ISCO-800 dataset is a genuinely useful resource. 800 synthetic but diverse user backgrounds from ISCO-08 occupation groups, with negative traits and varied hobbies, is nontrivial and will probably get reused. The training procedure is also a sensible engineering combination: critic-guided regeneration plus iterative curriculum learning over user difficulty. The paper is clearly written, positions itself correctly against persona- and target-oriented dialogue work, and the case study shows the intended behavior.\n\nThe soft spot is the one the stress test flags: the same family of LLM judges that filters the training data also produces the headline evaluation numbers. Algorithm 1 regenerates responses until the GPT-3.5 critic gives scores >= 4; Table 1 then reports Rel./Int./Val. from GPT-3.5 or GPT-4 using the same rubric and prompt. That is direct optimization against the measurement instrument. The improvements over baselines are real in a narrow sense—the model is better at satisfying that judge—but the claim that it 'outperforms all other methods' in user-oriented proactivity is not established by Table 1. GPT-4 is a sibling model with similar biases, so switching judges doesn't fix the circle.\n\nThe 62-participant online study is the right kind of check, but it's too weak to carry the weight. No significance tests, no error bars, one baseline, aggregate win rates only. It's suggestive, not conclusive.\n\nThere are also some presentation problems: Table 2's rows are ambiguous, Algorithm 1 has a garbled while condition and a suspicious duplicate in the stored dialogue, and PPL is used as if it were a user-experience metric. Those are minor fixes.\n\nMy overall read: this is a competent engineering paper with a real dataset and a plausible method, but the central quantitative claim is inflated by evaluation circularity. That's a common failure mode in LLM-as-judge work, and the authors made an honest attempt at external validation, so I wouldn't reject it on those grounds. A serious referee should ask for tighter human evaluation: larger sample, multiple baselines, significance tests, and ideally a validation of the critic against human ratings on the test outputs.\n\nFor peer review: yes, this deserves referee time. It's not going to reshape the field, but the dataset and the empirical comparison of training strategies will be useful to people working on proactive dialogue.\n\nRecommendation: send to peer review, with the clear expectation that the evaluation claims be scaled back or backed by stronger human evidence.","headline":"A useful dataset and a plausible training recipe, but the headline numbers measure how well the model satisfies the same judge that filtered its training data.","tokens_in":16930,"tokens_out":2641,"would_cite":true,"duration_ms":25201,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a 32B-parameter chatbot, trained with an LLM critic that scores and corrects responses, outperforms larger universal LLMs and existing proactive-dialogue methods on user-oriented proactivity.","keywords":["user-oriented proactivity","open-domain dialogue","critic-guided corpus generation","LLM-as-a-judge","curriculum learning","user background dataset","ISCO-800","dialogue system"],"falsifier":"Collect a held-out set of chatbot responses from UPC and a strong baseline, have independent human raters score them on the same relevance, interest, and value rubrics, and check the rank correlation with the LLM critic's scores. The central claim collapses if the critic's ordering of responses agrees no better than chance with human ordering, or if human preference for UPC over the baseline is absent in a larger preregistered study.","tokens_in":15936,"feed_emoji":"💬","tokens_out":6729,"duration_ms":60512,"temperature":0.7,"pith_summary":"The paper argues that open-domain chatbots fall short not because they cannot hold a conversation but because they do not proactively learn who the user is and steer the dialogue toward the user's own interests and background. To fix this, the authors build a User-oriented Proactive Chatbot (UPC) trained without human-written proactive dialogue data: an LLM critic, prompted as the user, scores each response on background relevance, user interest, and response value, and weak responses are regenerated from the critic's written feedback until they pass. The resulting high-quality dialogues are used to fine-tune the chatbot, and training proceeds in an easy-to-hard curriculum across 800 diverse user personas (ISCO-800) so the model gradually adapts to harder users. The paper reports that a 32B-parameter UPC outperforms larger universal LLMs, target-oriented proactive methods, and other LLM-based dialogue systems on relevance, interest, value, and perplexity, and that 62 real users preferred it over a 70B baseline more than 75% of the time. If these results hold, user-oriented proactivity can be instilled in any open-domain LLM cheaply, using a judge model as the data engine.","feed_headline":"A 32B chatbot beats 70B rivals by steering chat to user interests","feed_subtitle":"LLM-as-judge feedback and easy-to-hard training lift relevance, interest, and value scores.","key_machinery":"The load-bearing mechanism is the critic-guided regeneration loop: an LLM judge role-playing the user scores each candidate response on background relevance, interest, and value, and its written criticism is fed back to the chatbot so the response is regenerated until it satisfies the judge. This loop converts an ordinary base model into a source of its own high-quality proactive training data, removing the need for human-annotated proactive dialogues. Three components support it: the ISCO-800 dataset of 800 persona descriptions built from 40 ISCO-08 occupational groups; the difficulty measurer, which labels a user's dialogue as easy only if every scored facet stays above a threshold and enough facets improve under regeneration; and the iterative curriculum scheduler, which fine-tunes on easy users first and admits harder users as the model improves.","core_discovery":"The central claim is that user-oriented proactivity—the chatbot's ability to discover the user's background and evolving interests and lead the conversation onto user-centered topics—can be built into an open-domain chatbot through a closed loop of LLM judging and regeneration. The paper defines the target behavior as three scoreable facets: background relevance, user interest, and response value. A critic prompted with the user's persona and dialogue history rates each chatbot turn from 1 to 5 on these facets; turns scoring below 4 on any facet are sent back to the chatbot with the critic's reasons and regenerated until they clear the bar or the retry budget runs out. The filtered corpus fine-tunes the chatbot, and the loop repeats across iterations while a difficulty measurer admits only users whose dialogues are easy for the current model, gradually widening to harder users. On the ISCO-800 test set the resulting 32B-parameter UPC beats all compared methods on all reported metrics, and a 62-participant blind comparison finds that more than three quarters of users prefer it to Llama-3-70B-Instruct.","pith_inferences":["If the judge's scores are trustworthy, the same critic-guided regeneration loop could be applied to task-oriented assistants (recommendation, tutoring, customer support) where personalization is also judged through a role-played user.","ISCO-800 is a reusable benchmark asset: future chatbots can be tested for user-oriented proactivity against the same 800 personas, making results across systems directly comparable.","A natural stress test the paper does not run is replacing the proprietary judge with a smaller open-weights critic during training, which would show whether the gains depend on judge quality or on the loop itself.","Because the same critic filters the training data and computes the headline metrics, part of the reported gain could be judge self-consistency; an independent human-rated benchmark would separate genuine proactivity improvements from judge artifacts."],"forward_implications":["A 32B-parameter model can beat 70B and larger models on user-focused conversational quality, so raw parameter count is not the main lever for proactivity.","The critic-guided loop is model-agnostic: the paper reports consistent gains on Qwen1.5-14B, Qwen1.5-32B, and Vicuna-33B, with larger gains on stronger base models.","Training no longer depends on a human-annotated proactive dialogue corpus; the bottleneck shifts to judge reliability and persona diversity.","Curriculum order matters for efficiency as well as final quality: the regeneration rate falls from about 37.7% to 23.2% over four iterations as the model learns easier users first."],"supporting_citations":[{"why":"Supplies the LLM-as-a-judge strategy on which the critic is built.","marker":"[Zheng et al., 2024]"},{"why":"Defines the ISCO-08 occupational classification from which 40 occupation groups are sampled for ISCO-800.","marker":"[Organization, 2008]"},{"why":"Provides the Qwen1.5 base models used as the chatbot and as the user agent.","marker":"[Bai et al., 2023]"},{"why":"Supplies the curriculum learning rationale for training from easy to difficult users.","marker":"[Wang et al., 2021]"},{"why":"Supports the claim that fine-tuning on small high-quality corpora can improve LLM alignment.","marker":"[Zhou et al., 2024]"},{"why":"Target-oriented proactive dialogue method (ProCoT) that UPC is compared against.","marker":"[Deng et al., 2023c]"},{"why":"Target-oriented bidirectional planning method (TRIP) that serves as the strongest target-oriented baseline.","marker":"[Wang et al., 2024]"},{"why":"Defines perplexity, the corpus-quality metric reported in the main results.","marker":"[Jelinek et al., 1977]"}],"fun_headline_variants":["Critic-guided LLM loop makes chatbots steer chats to user interests","Small chatbot outshines big rivals with critic-guided proactivity training","LLM-as-judge feedback points chatbots to user-centered topics","Critic-guided curriculum lifts chatbot's user-oriented conversational skill"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's load-bearing premise is that the LLM critic, when given a user's background and chat history, scores responses the way the real user would, and this same critic both filters the training data and computes the headline evaluation numbers; if the judge's scores are biased or gameable, the measured gains may not show up with real users.","fun_headline_variants_meta":{"raw":{"variants":["Critic-guided LLM loop makes chatbots steer chats to user interests","Small chatbot outshines big rivals with critic-guided proactivity training","LLM-as-judge feedback points chatbots to user-centered topics","Critic-guided curriculum lifts chatbot's user-oriented conversational skill"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000686,"raw_usage":{"total_tokens":3151,"prompt_tokens":1029,"completion_tokens":2122,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":2050}},"tokens_in":645,"tokens_out":2122,"duration_ms":17053,"temperature":1.0,"reasoning_tokens":2050,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:35:26.285561+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a held-out set of chatbot responses from UPC and a strong baseline, have independent human raters score them on the same relevance, interest, and value rubrics, and check the rank correlation with the LLM critic's scores. The central claim collapses if the critic's ordering of responses agrees no better than chance with human ordering, or if human preference for UPC over the baseline is absent in a larger preregistered study.","supporting_citations":[{"cited_title":"Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36,","cited_arxiv_id":null,"evidence_quote":"Supplies the LLM-as-a-judge strategy on which the critic is built."},{"cited_title":"A survey on curriculum learning","cited_arxiv_id":null,"evidence_quote":"Supplies the curriculum learning rationale for training from easy to difficult users."},{"cited_title":"Lima: Less is more for align- ment","cited_arxiv_id":null,"evidence_quote":"Supports the claim that fine-tuning on small high-quality corpora can improve LLM alignment."},{"cited_title":"Target-constrained bidirectional planning for genera- tion of target-oriented proactive dialogue","cited_arxiv_id":null,"evidence_quote":"Target-oriented bidirectional planning method (TRIP) that serves as the strongest target-oriented baseline."},{"cited_title":"Perplexity—a measure of the difficulty of speech recognition tasks","cited_arxiv_id":null,"evidence_quote":"Defines perplexity, the corpus-quality metric reported in the main results."}],"review_version":1}