{"id":"82db9c70-f412-4f21-8dd2-06db5a3869b0","arxiv_id":"2504.20458","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A generative-reward-model-based simulated user that scores and critiques recommendations lets LLM conversational recommenders improve via beam-search interaction, surpassing prior methods on ReDial and INSPIRED.","lead":"This paper trains a simulated user, GRSU, that automatically scores and critiques movie recommendations so a conversational recommender can improve through multi-turn search. On two public datasets, adding this simulated user lifted LLM-based recommenders above existing trained conversational recommenders.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reward sign in Eq. 2 is ambiguous: if p('No') is the score, beam search maximizes rejection; the paper never states p('Yes') vs p('No') or a signed transform, making the central empirical claim uninterpretable without code.","rationale":"The reader's weakest-assumption analysis correctly identifies the reward-sign ambiguity in Eq. 2 as the most load-bearing issue. The paper's central claim—that GRSU enables general LLM-based CRSs without fine-tuning to outperform trained CRS models—depends entirely on the reward used for beam search and candidate ranking being a monotone function of user acceptance. As written, Section 3.2.1 defines the score as the probability of the selected option token, and Figure 1(c) shows p('No') for rejected items being used as the reward. If that is literal, the search objective is inverted and the reported improvements cannot be explained by the described mechanism. However, the empirical results are strong and the ablation confirms the importance of this component, so the most plausible resolution is a documentation error in the paper rather than a fabricated result. The concrete test of rerunning with explicit p('Yes') and p('No') variants, or inspecting the released code, settles this directly. Because the concern is real but likely addressable by clarification, the reader's CONDITIONAL verdict remains appropriate; no change to the verdict is needed.","tokens_in":17913,"tokens_out":6373,"duration_ms":60479,"concrete_test":"Run the GRSU pipeline on INSPIRED with the reward in Eq. 2 computed as p('Yes') versus p('No') versus 1 - p('No'), all else unchanged, and compare against the reported GRSU row in Table 3 (Recall@10 = 0.352). Only the variant matching the implementation's actual post-processor Φ_s will reproduce the number; if the matching variant is not p('No'), the text and Figure 1(c) are internally inconsistent and the reward semantics must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2.1 defines the generative item score as 'the predicted probability of the selected option token' (Eq. 2). At inference the model outputs either 'Yes' or 'No' for 'Accept the recommendation?'. If the selected token is 'No', the extracted score is p('No'). Figure 1(c) indeed labels a rejected list with scores 0.14 and 0.08, averaging 0.11, and calls this the reward. Section 3.3 then uses this score as the reward in beam search, retaining the top-B states with the highest rewards. Maximizing p('No') would prefer confidently rejected items, which is the opposite of user acceptance. The paper never specifies whether the reward is p('Yes'), p('No'), 1 - p('No'), or a signed log-probability. If the implementation actually uses p('Yes') (as a sound reward would require), the text and figure are mis-specified; if it literally uses p('No'), the central claim that GRSU improves recommendations through reward-guided search is unsupported. This ambiguity is load-bearing because the ablation in Section 4.3.1 shows generative item scoring to be the single most important component.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GRSU, a generative reward model based simulated user for conversational recommendation systems (CRSs). The simulated user is trained via instruction tuning to perform two feedback behaviors: generative item scoring (outputting 'Accept the recommendation? Yes/No' with token probabilities) and attribute-based item critique (producing natural-language critiques of recommended items). The interaction between a zero-shot LLM-based CRS and the simulated user is cast as a Markov decision process and searched with beam search; a final efficient ranking step aggregates item scores across the searched candidate lists. Experiments on ReDial and INSPIRED report consistent improvements over trained CRS baselines and multiple zero-shot LLMs, with ablations showing each component contributes, and a transferability study showing a simulated user trained on high-resource ReDial helps on low-resource INSPIRED.","tokens_in":18220,"tokens_out":7039,"duration_ms":68150,"significance":"If the mechanism is sound, the paper makes a useful contribution: it shows that a fine-tuned simulated user can provide label-free feedback at inference time to guide zero-shot LLM CRSs, achieving strong gains over trained CRSs on two benchmarks. The experimental design is extensive: main results across six metrics, ablations over components and implementation variants, transferability analysis, robustness across four LLM backbones, and a comparison of search strategies. The code link is provided, which aids reproducibility. The central weakness is the ambiguity of the scalar reward extracted from the generative item scoring, which touches the core search-and-ranking mechanism; because the ablation in Section 4.3.1 identifies this component as the most important, the ambiguity must be resolved before the empirical claims can be fully interpreted.","major_comments":[{"comment":"The definition of the generative item score is ambiguous and potentially inverted. The text states that 'the predicted probability of the selected option token is then interpreted as the item score,' and Figure 1(c) explicitly labels the scores of rejected items as p('No') = 0.14 and 0.08, with an average of 0.11. However, Section 3.3 uses this score as the reward in beam search, retaining states with the highest rewards, and the candidate ranking method sorts items by the same scores. If the selected token is 'No', then a high p('No') corresponds to confident rejection, so maximizing the reward would select the most strongly rejected items and rank them at the top. The paper never states whether the implemented score is p('Yes'), p('No'), 1 - p('No'), or a signed transform, nor does it reconcile the figure with the text. Since the ablation in Section 4.3.1 shows that generative item scoring is the single most important component, this ambiguity is load-bearing for the central claim. Please specify the exact post-processor Phi_s, correct the text and Figure 1(c) accordingly, and confirm which quantity is maximized in beam search and sorted in the final ranking.","section":"Section 3.2.1, Eq. (2), Figure 1(c), Section 3.3"},{"comment":"The main results in Table 3 do not state which data are used to train the simulated user for each evaluation dataset. For INSPIRED, the main result (Recall@10 = 0.352) exactly matches the 'ReDial+INSPIRED' row of Table 5, indicating that the simulated user is trained on both datasets for that experiment. For ReDial, it is unclear whether the simulated user is trained on ReDial only or also on INSPIRED. Please state explicitly the training data configuration used for each row of Table 3, and discuss how the transferability claim is affected if the low-resource INSPIRED evaluation already includes training on the high-resource ReDial data (which is legitimate for transfer but should be transparent).","section":"Section 4.1 and Section 4.3.3, Tables 3 and 5"},{"comment":"The same simulated user generates both the critique actions (Eq. 3) and the reward scores (Eq. 2). Because the reward and the action come from the same model, there is a risk that the reward rewards states that the model itself steers toward, rather than states a real user would accept. This potential self-bias is not discussed. Please provide an analysis or argument showing that the reward is a faithful proxy for real user acceptance despite being produced by the same model that generates the critiques, for example by comparing reward-based rankings with held-out ground-truth acceptance.","section":"Section 3.3"}],"minor_comments":[{"comment":"The baseline name is misspelled as 'Populalarity' in both the table and the baselines list; it should be 'Popularity'.","section":"Table 3 and Section 4.1"},{"comment":"The text says 'we sample k negative items from the set I - I_k' but the implementation section says 'we sample one additional item as a negative sample for each ground-truth item.' The symbol k is already used for the turn index; please clarify the number of negative samples and avoid reusing k.","section":"Section 3.2.2"},{"comment":"The caption states that the probability of the reward token (i.e., 'No') is extracted as the score. If the intended score is p('Yes') or a transformed quantity, the caption should be corrected; otherwise, the example should be reconciled with the claimed reward objective.","section":"Figure 1(c)"},{"comment":"The phrase 'bf16 16-bit mixed precision training' is redundant; consider rewriting as 'bf16 mixed precision training'.","section":"Section 4.1"},{"comment":"The code link is given, but no commit hash or direct pointer to the reward extraction function is provided. Adding a commit hash and a reference to the Phi_s implementation would resolve the ambiguity in Eq. (2) for readers and reviewers.","section":"Implementation details"}],"recommendation":"major_revision","confidential_remarks":"The sign ambiguity in Eq. (2) / Figure 1(c) is the main risk: if the implementation literally uses p('No') as a positive reward, the method's central mechanism is likely flawed; if it uses p('Yes') as intended, the text and figure need correction. The paper provides a code link, so the authors can settle this quickly. The contribution is otherwise solid and within the scope of SIGIR. I recommend major revision rather than rejection because the issue is likely fixable by clarification or correction of the exposition, and the experimental evidence is extensive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a genuinely useful integration of generative reward models into conversational recommendation, with strong and consistent empirical results, but the reward sign in Section 3.2.1 is ambiguous in a way that goes to the core mechanism, and the paper as written is not interpretable until it is fixed.\n\nWhat is new: the paper's contribution is a concrete combination: an instruction-tuned simulated user that produces both generative yes/no item scores and attribute-based critiques, a beam-search interaction loop with a generic LLM-based CRS, and a cheap candidate ranking step. That combination is not just X-applied-to-Y. The experiments are extensive: four LLMs as CRSs, two datasets, ablations on both feedback behaviors, search-strategy comparisons, and a transfer study. The ablations are consistent with the design story, and the reported gains over strong baselines are large. The transfer result—a simulated user trained on ReDial and applied on INSPIRED—is a nice practical result. The 'label-free' claim is fair: no ground-truth preferences are needed at inference time; the training uses ground-truth labels from the training split, which is ordinary supervised learning, not circularity.\n\nThe soft spot is the reward definition. Eq. 2 says the item score is 'the predicted probability of the selected option token.' The prompt makes the model answer 'Accept the recommendation? Yes/No.' Figure 1(c) shows a rejected movie scored by p('No') = 0.14, and the state reward is the average of those p('No') values. Beam search then keeps states with the highest reward. If the reward is literally p('No'), search maximizes rejection. The paper never states whether the reward is p('Yes'), 1−p('No'), or a signed log-probability. The ablation says generative item scoring is the most important component, so this ambiguity is not cosmetic. Either the text and figure are mis-specified, or the search is backwards. The code link is given, but no commit hash or detailed specification appears, so a referee cannot resolve this from the manuscript. This is the kind of issue that should be fixable in revision, but it needs to be fixed.\n\nWho this is for: people working on LLM-based conversational recommenders and user simulation. The idea of a generative reward model as a simulated user is worth taking seriously. But I would not cite it in my own work until the reward mapping is clarified and ideally the code is released in reproducible form. The paper deserves a serious referee: the problem is important, the experiments are solid, and the flaw is a specification gap rather than a fatal one. My verdict is conditional, matching the reader's.","headline":"Solid empirical integration of generative reward models into conversational recommendation, but the reward sign in Eq. 2 is ambiguous and cuts the central mechanism.","tokens_in":18705,"tokens_out":4557,"would_cite":false,"duration_ms":42536,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simulated user that scores and critiques recommendations lets an untuned LLM conversational recommender beat trained baselines.","keywords":["conversational recommendation","simulated user","generative reward model","beam search","instruction tuning","attribute-based critiquing","LLM-based recommender systems"],"falsifier":"Run the same beam-search pipeline on the test conversations with the reward taken as the probability of 'Yes', the probability of 'No', and their difference; if the final rankings and Recall/NDCG values change materially, the reported gains come from an unstated sign convention rather than from learned preference, whereas stable rankings would confirm the reward direction is not the deciding factor.","tokens_in":17746,"feed_emoji":"🎬","tokens_out":8672,"duration_ms":81619,"temperature":0.7,"pith_summary":"The paper sets out to show that a conversational recommender system can be improved without fine-tuning its recommender at all: the missing piece is a simulated user that behaves like a generative reward model, scoring each candidate item by the probability of an acceptance token and writing attribute-level critiques. This simulated user, called GRSU, is trained by instruction tuning on synthesized data, then used to drive multi-turn interaction with an off-the-shelf LLM recommender through beam search. The paper claims that on the ReDial and INSPIRED datasets, this interaction consistently beats all baselines, including trained conversational recommender systems, and that a general LLM paired with GRSU outperforms several fine-tuned recommender models. The reason to care is that it moves the hard part of preference understanding out of the recommender and into a feedback module that does not need ground-truth user preferences at inference time.","feed_headline":"Simulated user feedback lifts LLM recommenders past trained models","feed_subtitle":"A label-free critique-and-score loop with beam search beats fine-tuned conversational recommenders on ReDial and INSPIRED.","key_machinery":"The load-bearing object is GRSU, a single instruction-tuned language model that acts as both reward model and critic. Its two behaviors are unified by the same instruction format: generative item scoring asks it to answer 'Accept the recommendation (Yes/No)?' and the probability of the selected option token is extracted as the item's score; attribute-based item critiquing asks it to give a natural-language reaction to the recommended item list with item attributes supplied. The interaction is cast as a Markov decision process with the item list as state, the critique as action, the CRS regenerating the list as the transition, and the average generative score as the reward, and beam search over this process, followed by aggregating item scores from all visited lists, produces the final recommendation.","core_discovery":"On the paper's own terms, the discovery is that reward-guided search, powered by a simulated user, is enough to turn a general LLM into a strong conversational recommender. The simulated user GRSU is trained to provide two feedback actions: generative item scoring, where the probability of the chosen 'Yes' or 'No' token becomes the item score and the average over the list is the reward; and attribute-based item critiquing, where it says what it wants more of, such as more comedy elements. Treating an item list as a state and a critique as an action, the paper runs beam search over the interaction and then ranks all scored items across the candidate lists. GRSU reports consistent gains on ReDial and INSPIRED across Recall@k, NDCG@k, and MRR@k, with the largest gains on the low-resource INSPIRED dataset, and a simulated user trained on ReDial transfers to INSPIRED.","pith_inferences":["Beyond the paper: the same critique-plus-token-score recipe could be applied to other interactive LLM settings, such as question answering, tool use, or document drafting, where an automatic critic can guide revision under beam search.","Beyond the paper: because the evaluation uses held-out ground-truth items rather than real human satisfaction, a human study would be the next test of whether the simulated user's scores reflect actual acceptance; the paper itself does not report such a study.","Beyond the paper: resolving the sign convention of the extracted token probability is a cheap robustness check that would make the method easier to adopt, since the current write-up does not specify whether higher scores mean 'Yes' or 'No'."],"forward_implications":["An off-the-shelf LLM can act as a competitive conversational recommender when paired with a trained simulated user, so fine-tuning the recommender itself is not necessary.","The simulated user transfers across datasets: a user trained on the high-resource ReDial dataset improves results on the low-resource INSPIRED dataset, reducing the need for per-dataset recommender training.","The benefit appears with different LLMs of different sizes, each improving substantially when GRSU provides feedback, so the mechanism is not tied to one model.","Because one or a few interaction rounds already surpass the best trained baseline on INSPIRED, the approach offers a controllable trade-off between interaction cost and recommendation quality."],"supporting_citations":[{"why":"Supplies the zero-shot LLM-based CRS formulation and the dataset split convention used to evaluate GRSU.","marker":"[13]"},{"why":"Introduces generative reward models that represent reward as the probability of a yes/no token, which GRSU turns into item scoring.","marker":"[26, 38]"},{"why":"Motivates reward-guided beam search over candidate generations, which GRSU applies to the CRS interaction.","marker":"[29, 30]"},{"why":"Provides a trained neighborhood-based CRS baseline and the split convention GRSU must beat.","marker":"[34]"},{"why":"Provides the ReFICR instruction-tuned CRS baseline, the strongest trained comparison on ReDial.","marker":"[36]"},{"why":"Supplies the ReDial conversational recommendation dataset used for training and evaluation.","marker":"[22]"},{"why":"Supplies the INSPIRED dataset, the low-resource testbed for transfer and main experiments.","marker":"[12]"},{"why":"Argues that LLM-based simulated users for CRS evaluation depend on ground-truth preferences, motivating GRSU's label-free design.","marker":"[31]"}],"fun_headline_variants":["Search-based interaction with simulated user lifts LLM recommenders","Generative reward model steers simulated user for better recs","Beam search on simulated critiques sharpens conversational recommenders","LLM recommenders gain from label-free simulated user feedback","Simulated user plus beam search beats fine-tuned conversational models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the token probability extracted from generative item scoring is a valid reward, so that higher scores correspond to items a real user would accept; the paper does not state whether the extracted probability is the probability of 'Yes', the probability of 'No', or a signed combination, even though its illustration shows the 'No' probability used for rejected movies.","fun_headline_variants_meta":{"raw":{"variants":["Search-based interaction with simulated user lifts LLM recommenders","Generative reward model steers simulated user for better recs","Beam search on simulated critiques sharpens conversational recommenders","LLM recommenders gain from label-free simulated user feedback","Simulated user plus beam search beats fine-tuned conversational models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001005,"raw_usage":{"total_tokens":4280,"prompt_tokens":1006,"completion_tokens":3274,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":3191}},"tokens_in":622,"tokens_out":3274,"duration_ms":23966,"temperature":1.0,"reasoning_tokens":3191,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:28:41.331963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same beam-search pipeline on the test conversations with the reward taken as the probability of 'Yes', the probability of 'No', and their difference; if the final rankings and Recall/NDCG values change materially, the reported gains come from an unstated sign convention rather than from learned preference, whereas stable rankings would confirm the reward direction is not the deciding factor.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides a trained neighborhood-based CRS baseline and the split convention GRSU must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ReFICR instruction-tuned CRS baseline, the strongest trained comparison on ReDial."}],"review_version":1}