Pith. sign in

REVIEW 4 major objections 5 minor 73 references

A decision-aware simulator that reasons from cross-domain user history beats action-only imitation for advertising evaluation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 17:53 UTC pith:2UCGZMCA

load-bearing objection Solid industrial pipeline for joint thinking+action ad simulation; the “decision-aware” claim is useful engineering, not proven cognitive fidelity. the 4 major comments →

arxiv 2607.26893 v1 pith:2UCGZMCA submitted 2026-07-29 cs.IR

Beyond Action Imitation: Learning a Decision-Aware User Simulator for Online Advertising

Zipeng Chen , Jiaer Zheng , Xiangyang Xu , Xinyu Lin , Zhaobin Wang , Zhaohui Liu , Qianjin Xiang , Xiaoyu Zhao
show 10 more authors
Zhuozhen Yu Guangshuo Wang Daxing Chen Junwei Pan Zhangbin Zhu Chengguo Yin Hao Chen Tat-Seng Chua Haijie Gu Jie Jiang
This is my paper
classification cs.IR
keywords User Behavior SimulatorLarge Language ModelsOnline AdvertisingReinforcement LearningCross-domain User HistoryThinking TracesContext EngineeringRubric-based Reward
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Existing LLM user simulators for ads and recommendations usually learn from one domain’s clicks and only try to copy the final action. That gives a thin picture of preference and invites shortcuts that look right on labels but explain little. This paper argues that a useful simulator must fold heterogeneous histories—ads plus five content domains—into decision-relevant context, then jointly produce a thinking trace and an action. DASH does that in three stages: hierarchical context compression and prompt optimization, distillation of teacher thinking trajectories, and reinforcement learning with a hybrid reward that scores both action match and thinking quality along form, content, and logic. On real Tencent ad requests the smaller trained model outperforms larger prompted models on both action fidelity and thinking quality, and the traces are intended to diagnose why a recommendation fails a user.

Core claim

DASH claims that jointly generating thinking traces and behavioral actions from compressed cross-domain histories, trained first by distilling teacher trajectories and then by GRPO with a rubric-based thinking reward plus action reward, produces more accurate and diagnostically useful advertising user simulation than action-only or single-domain baselines.

What carries the argument

DASH’s progressive CE–SFT–RL pipeline: hierarchical item- and stream-level context compression plus closed-loop prompt optimization; easy/hard teacher-trace distillation with quality filtering; and GRPO driven by a hybrid reward that adds a form/content/logic rubric score to binary action correctness.

Load-bearing premise

Teacher-written and ground-truth-conditioned thinking traces, judged by the same LLM rubrics used as reward, are treated as faithful stand-ins for how real users actually decide—not just post-hoc stories that help copy the logged action.

What would settle it

If human judges, given the same inputs, systematically reject DASH’s thinking traces as unfaithful to how they would decide, or if removing the thinking reward and traces leaves offline A/B ranking of ad strategies unchanged on held-out live traffic, the central claim that decision-aware traces add fidelity and diagnostic value fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Cross-domain content history plus ad history should be standard inputs for advertising simulators, not optional extras.
  • Action-only fine-tuning is insufficient; rubric scores on form, content, and logic can measurably raise both action F1 and thinking quality.
  • A smaller student model with CE–SFT–RL can outperform much larger prompted LLMs under industrial latency constraints.
  • Thinking traces can serve as offline diagnostics for why an ad is skipped, clicked, converted, or rejected before live A/B tests.
  • The paper’s stated next uses—pre-screening strategies, pre-testing ads, and creative optimization—become practical if the simulator’s dual metrics hold.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same hierarchical compression and focus–draft–verify loop may transfer to other long-horizon, multi-stream behavior logs beyond ads (e.g., feed ranking or conversational recommenders).
  • If hard-sample regeneration mainly teaches post-hoc rationalization, scaling teacher size may inflate thinking scores without improving true counterfactual user responses.
  • Separating the reward-model LLM from the evaluation LLM is a practical pattern other industrial simulators can copy to limit reward hacking.
  • Diagnostic value will matter most when product teams can map specific rubric failures (hallucinated need, lost negative feedback) to concrete creative or targeting fixes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DASH, an LLM-based advertising user simulator that jointly emits thinking traces and discrete actions (skip/click/conversion/negative-feedback) from heterogeneous cross-domain histories. It addresses two stated limits of prior simulators—single-domain preference modeling and action-only supervision—via a three-stage pipeline: (1) Context Engineering with hierarchical item/stream compression and closed-loop prompt optimization; (2) SFT on teacher-distilled trajectories, including hard samples regenerated conditioned on ground-truth actions (Eqs. 3–6); and (3) GRPO with a hybrid reward combining action accuracy and a rubric-based thinking score over Form/Content/Logic (Eqs. 7–10, Table 1). On temporally split Tencent ad-request data with five content domains, a Qwen3.5-35B student with small-scale SFT+RL reports the best Weighted-F1 (62.15) and Think. score (92.01) among prompted larger models and ablations (Tables 2–6), with additional head/long-tail, evaluator-robustness, and qualitative case analyses.

Significance. Offline evaluation of advertising policies is a high-value industrial problem, and the paper makes a concrete systems contribution: folding multi-million-token cross-domain logs into a deployable context budget, adapting a smaller student model, and reporting multi-axis ablations (CE, history sources, prompt optimization, SFT quality/scale, reward composition) on real production traffic. The dual action/thinking evaluation protocol and head vs. long-tail breakdown are useful for the community. If the diagnostic-value claim holds—that traces are not merely post-hoc rationalizations but usable decision proxies—the work would meaningfully extend user simulation beyond action imitation. Even under a weaker reading (better regularized action imitation plus interpretable rationales), the engineering results and industrial-scale evidence remain of interest to recommender/advertising systems venues.

major comments (4)
  1. [§4.2–4.3, Eqs. 4, 7–9, Table 1, Fig. 7, Table 6] The central “decision-aware / diagnostic” claim is only weakly supported relative to how strongly it is stated in the abstract and §1. Stage-2 hard samples (Eq. 4) inject the ground-truth action a* and ask the teacher to regenerate a trace that explains it; Stage-3 then rewards Thinking–Action Alignment and related logic/content rubrics (Table 1, Eqs. 7–9) with LLM judges from the same rubric family used for reported Think. metrics. Human validation (Fig. 7, §5.3.6) shows only that GLM-4.7 agrees with annotators when scoring DASH outputs—not that traces match real users’ decision factors. Under this pipeline, Table 6’s Action+Think lift (W-F1 62.15 vs 60.56) is equally consistent with better post-hoc rationalization that regularizes action imitation. The paper should either (i) reframe claims as “interpretable rationales aligned with logged actions,” or (ii) add evidence that traces reco
  2. [§6, Appendix C.3] Diagnostic value is asserted as a primary contribution (§1, §6) but the only direct evidence is three hand-selected qualitative cases in Appendix C.3. These illustrate plausible failure modes (missing content history, lost-in-the-middle negatives, unsupported claims) and how CE/Verify help, yet they do not quantify how often such diagnostics are correct, actionable, or preferred by practitioners over action-only error analysis. A small structured study—e.g., ranking-error triage with vs. without traces, or agreement that cited evidence was decision-relevant—would make the diagnostic claim load-bearing rather than anecdotal.
  3. [§5.1–5.2, Table 2, Table 8, Appendix A.3] Action evaluation is on a 1,000-sample held-out set with a skewed label distribution (~45% skip, 35% click, 15% neg-feedback, 5% conversion; Appendix A.3). Weighted-F1 is appropriate, and the head/long-tail split (Table 8) is welcome, but confidence intervals, multiple random seeds/runs, and statistical tests against the strongest baseline (Qwen3.5-397B, W-F1 60.23) are not reported. Given that Small-scale SFT alone already reaches 59.07–60.48 and RL adds a few points, uncertainty quantification is needed before concluding clear superiority of the full pipeline for offline evaluation use.
  4. [§5.1.2, Tables 3 and 6] Baseline coverage for the joint thinking+action setting is limited to prompting general LLMs (§5.1.2). The paper reasonably notes that prior action-only simulators are not directly comparable, but it does not include strong alternatives such as (a) action-only fine-tuning of the same student with identical CE context, (b) SFT without hard-sample label injection, or (c) retrieval/summarization baselines instead of the hand-designed hierarchical compression. Table 3’s “w/o CE” and Table 6’s action-only RL help, yet a same-backbone action-only SFT+RL control would isolate whether thinking supervision is necessary for the reported action gains or whether CE+action RL would suffice for the fidelity half of the claim.
minor comments (5)
  1. [§5.1.3, Appendix B] Free parameters (token quotas q_ad/q_content/q_profile, dimension weights w1–w3 with Logic at 0.60, η_sft=80, GRPO hyperparameters) are stated in §5.1.3 and Appendix B but sensitivity is largely unreported beyond SFT scale. A short sensitivity note would strengthen reproducibility claims.
  2. [§3, Appendix D, F] Satisfaction is generated in prompts (Appendix D, F) but explicitly excluded from the task formulation and evaluation. Clarifying earlier in §3 that it is an internal auxiliary field would avoid reader confusion when inspecting prompt templates.
  3. [Figure 5, §5.3.3] Figure 5’s “Ratio to Real” for Vanilla conversion at 0.00 is striking; a brief note on whether this is zero predicted conversions or a plotting floor would help interpretation.
  4. [§2.1] Related work could more sharply position against recent RL-aligned user simulators cited (e.g., Shop-R1, Customer-R1, Mirroring Users) on the specific axis of cross-domain advertising context folding vs. single-domain shopping agents.
  5. [§3, Table 2] Minor polish: consistent notation for H_ad / H_content, and ensure Table 2’s “best/second best” markup is complete for all columns (Form already bolded for DeepSeek in places).

Circularity Check

2 steps flagged

Action W-F1 is externally anchored in logs; partial circularity is limited to thinking-quality claims trained and scored on the same rubric family plus GT-conditioned hard traces.

specific steps
  1. self definitional [§4.3.1 Eqs. 7–9, Table 1; §5.1.1 thinking metrics]
    "We therefore define a hybrid reward: R_total = R_action + R_think ... R_think is a rubric-based reward that decomposes quality into form, content, and logic dimensions. ... For thinking quality, we report Form, Content, and Logic scores, together with their weighted aggregate, denoted as Think. These dimensions are consistent with the rubric-based reward design described in Section 4.3.1."

    The reported Think. metric is the same Form/Content/Logic hierarchy used as R_think during GRPO (only judge model and scaling differ). Optimizing R_think then claiming high Think. largely restates the training objective rather than an independently defined property of user decision processes. Action metrics stay external; this circularity is confined to thinking-quality claims.

  2. fitted input called prediction [§4.2.1 Eqs. 3–4; Table 1 Thinking–Action Alignment; Appendix F.4 hard-sample prompt]
    "When the teacher’s action prediction differs from the ground truth (â_i ≠ a*_i), we construct hard samples by providing the teacher with the ground-truth action and prompting it to regenerate a thinking trace τ̃_i: D_hard = {(x_i, τ̃_i, a*_i) | π_teacher(x_i, a*_i) → τ̃_i}. ... Thinking–Action Alignment (R_behav) Behavior Alignment. ... Your task is: from the user’s perspective ... generate a genuine inner thought process that reasonably explains and supports the occurrence of this action—not to question or deny the action itself. FINAL.action must strictly equal ground_truth."

    For hard samples, the thinking target is generated conditioned on a* and instructed to support that action. Supervision and R_behav then reward thinking–action alignment. High alignment/logic on this slice is partly forced by construction of the distilled targets, not an independent recovery of latent user reasoning. This inflates “decision-aware” thinking claims; it does not make log-based action F1 circular.

full rationale

DASH’s load-bearing empirical claim on action fidelity is not circular: R_action and W-Prec./W-Rec./W-F1 compare predicted actions to held-out real log labels (Eq. 8; §5.1.1 temporal split), so those metrics are independent of the model’s own definitions. Context engineering, cross-domain ablations, and prompt optimization are also checked against that external action benchmark (Tables 3–5). Circularity appears only in the secondary “thinking quality / decision-aware” chain. Stage-2 hard samples condition the teacher on a* and ask it to regenerate a trace that explains that action (Eq. 4); quality filtering and the Logic sub-reward Thinking–Action Alignment then score traces for supporting the predicted action (Table 1; Eq. 9). Stage-3 optimizes R_think along Form/Content/Logic, and §5 reports Think. from the same three dimensions (rescaled), with a different LLM judge but the same rubric hierarchy—so high Think. after hybrid RL is partly the training objective restated. Human agreement (Fig. 7) only checks that GLM-4.7 matches annotators on scoring model outputs, not that traces match real user reasoning. This does not collapse the main action results by construction; it weakens treating Think. and selected case studies as independent proof of faithful decision processes. No self-citation uniqueness chain or renamed known law drives the claims. Score 3: mild train/eval and GT-conditioned-trace circularity on thinking metrics only; central action benchmark remains external.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The work is empirical systems ML. Load-bearing commitments are modeling choices and evaluation conventions, not physical axioms. Free parameters are context budgets, rubric weights, SFT filter threshold, and RL hyperparameters. Domain assumptions include that logged actions plus compressed multi-domain context suffice for simulation and that LLM judges track thinking quality. Invented entities are methodological constructs (DASH stages, hybrid reward hierarchy), not new physical objects.

free parameters (4)
  • Context token quotas (L_max, q_ad, q_content, q_profile) = L_max=32K; q_ad=20K; q_content=7K; q_profile=5K
    Hand-set budgets (32K; 20K/7K/5K) that determine which history survives compression and thus what the simulator can condition on.
  • Thinking-reward dimension and sub-dimension weights = w=(0.10,0.30,0.60); detailed sub-weights in Appendix B
    w1/w2/w3 and nested a_i,b_i,c_i, alpha/beta/gamma weights fully specify R_think; authors emphasize logic (w3=0.60) for advertising.
  • SFT quality threshold eta_sft = 80 / 100
    Judge score cutoff that defines the filtered distillation set used for student SFT.
  • GRPO / SFT optimization hyperparameters = SFT lr 1e-4, 4 epochs; GRPO lr 3e-7, G=8, beta_KL=0.03, 500 steps
    Learning rates, group size, KL coefficient, steps, epochs control the final policy and are not derived from theory.
axioms (5)
  • domain assumption Cross-domain content behaviors plus ad history jointly determine responses to a target ad better than single-domain history.
    Stated as motivation and supported by Table 4 ablation; still an empirical modeling assumption about preference formation.
  • domain assumption Observable logged actions are valid ground truth for simulator action reward and evaluation.
    Eq. 8 and all W-F1 metrics treat skip/click/conversion/negative-feedback labels as correct user responses despite possible position/selection bias in serving logs.
  • domain assumption LLM-as-judge rubric scores on form/content/logic are valid dense supervision and evaluation of thinking quality.
    Section 4.3.1 and Appendix E; partially checked vs humans in Fig. 7 on 100 samples.
  • ad hoc to paper Teacher trajectories (including label-conditioned regenerations) are acceptable token-level targets for learning user thinking.
    Stage 2 easy/hard curation (Eqs. 3–5); hard samples reverse-engineer thinking from the true action.
  • domain assumption Next-token likelihood over (thinking, action) is an adequate training objective for user simulation under latency constraints favoring a smaller student.
    Problem formulation Eq. 1 and footnote on deployable student size.
invented entities (2)
  • DASH CE–SFT–RL pipeline (hierarchical compression + optimized self-refinement prompt + hybrid action/thinking GRPO) no independent evidence
    purpose: End-to-end decision-aware advertising user simulator producing traces and actions.
    Primary proposed system; evaluated only inside authors' stack.
  • Rubric hierarchy R_think (Form/Content/Logic with seven sub-dimensions) no independent evidence
    purpose: Provide dense trace-level reward and dual-metric evaluation beyond action labels.
    Table 1 and Appendix E; inspired by prior rubric-as-reward work but specialized here.

pith-pipeline@v1.2.0-daily-grok45 · 40588 in / 3721 out tokens · 72651 ms · 2026-07-30T17:53:10.884081+00:00 · methodology

0 comments
read the original abstract

Recent advances in LLM-based user simulation have shown promise for offline evaluation of recommendation and advertising systems. However, existing simulators typically infer user preferences from single-domain interaction histories and are primarily optimized to reproduce observable actions such as clicks. Consequently, they capture only a partial view of user preferences, while action-only prediction easily induces model shortcuts and limits both the fidelity and diagnostic value of simulation. To address these challenges, we propose DASH, a decision-aware user simulator that jointly generates thinking traces and predicts behavioral actions from heterogeneous cross-domain histories. DASH first introduces a Context Engineering stage that folds heterogeneous cross-domain histories into decision-relevant context, together with prompt optimization for effective reasoning over the folded context. To train a user simulator, DASH distills thinking trajectories from strong LLMs as SFT data, and further tailors a rubric-based reward model that evaluates thinking traces along form, content, and logic for RL training. Combined with the action reward, these signals jointly improve action prediction and thinking quality. Extensive experiments on real-world Tencent advertising data spanning five heterogeneous content domains demonstrate the effectiveness, efficiency, fidelity, and diagnostic value of DASH.

Figures

Figures reproduced from arXiv: 2607.26893 by Chengguo Yin, Daxing Chen, Guangshuo Wang, Haijie Gu, Hao Chen, Jiaer Zheng, Jie Jiang, Junwei Pan, Qianjin Xiang, Tat-Seng Chua, Xiangyang Xu, Xiaoyu Zhao, Xinyu Lin, Zhangbin Zhu, Zhaobin Wang, ZhaoHui Liu, Zhuozhen Yu, Zipeng Chen.

Figure 1
Figure 1. Figure 1: Overview of DASH, a progressive CE-SFT-RL framework for advertising user simulation. Stage 1 uses context [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Hierarchical context compression for preserving [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Prompt optimization: simulator outputs are evalu [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ratio of predicted to real action frequency under [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: SFT ablation across data quality and scale. Base: no task-specific SFT; NonFilt.(4K): 4K unfiltered trajectories; [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Alignment between LLM-based evaluator scores [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Lost-in-the-middle on a strong negative signal. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 8
Figure 8. Figure 8: Missing current-intent signal without content [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Unsupported claims. As Stage 2 makes visible, DASH’s draft (post-Focus) already grounds two of the strong cues—the geographic match and the habitual negative-feedback pattern—but still introduces two unsupported claims—an imagined dental discomfort and an unfounded brand￾quality assumption—in order to override the negative-feedback habit. Two complementary mechanisms then suppress this post￾hoc fabricatio… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 17 linked inside Pith

  1. [1]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073(2022)

    Pith/arXiv arXiv 2022

  2. [2]

    Jiawei Chen, Ruoxi Xu, Boxi Cao, Ruotong Pan, Yunfei Zhang, Yifei Hu, Yong Du, Tingting Gao, Yaojie Lu, Yingfei Sun, et al. 2026. Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces.arXiv preprint arXiv:2604.08362 (2026)

    Pith/arXiv arXiv 2026

  3. [3]

    Luyu Chen, Quanyu Dai, Zeyu Zhang, Xueyang Feng, Mingyu Zhang, Pengcheng Tang, Xu Chen, Yue Zhu, and Zhenhua Dong. 2025. RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems. InCompanion Proceedings of the ACM on Web Conference 2025(Sydney NSW, Australia)(WWW ’25). Association for Computing Machinery, New Yo...

    arXiv 2025

  4. [4]

    Siran Chen, Boyu Chen, Yuxiao Luo, Chenyun Yu, Yi Ouyang, Lei Cheng, Chengx- iang Zhuo, Zang Li, and Yali Wang. 2026. Vragent-r1: Boosting video recommen- dation with mllm-based agents via reinforcement learning. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 1739–1747

    2026

  5. [5]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al

  6. [6]

    Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. InProceedings of the 10th ACM conference on recommender systems. 191–198

    2016

  7. [7]

    DeepSeek-AI. 2026. DeepSeek-V4: Towards Highly Efficient Million-Token Con- text Intelligence

    2026

  8. [8]

    Engel, Roger D

    James F. Engel, Roger D. Blackwell, and Paul W. Miniard. 1995.Consumer Behavior (8th ed.). Dryden Press, Fort Worth, TX

    1995

  9. [9]

    Yi Fang, Wenjie Wang, Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, and Xiangnan He. 2025. Reason4rec: Large language models for recommendation with deliberative user preference alignment.arXiv preprint arXiv:2502.02061 (2025)

    Pith/arXiv arXiv 2025

  10. [10]

    Dayuan Fu, Keqing He, Yejie Wang, Wentao Hong, Zhuoma Gongque, Weihao Zeng, Wei Wang, Jingang Wang, Xunliang Cai, and Weiran Xu. 2025. Agentrefine: Enhancing agent generalization through refinement tuning.arXiv preprint arXiv:2501.01702(2025)

    Pith/arXiv arXiv 2025

  11. [11]

    GLM Team et al. 2025. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Founda- tion Models. arXiv:2508.06471 [cs.CL] https://arxiv.org/abs/2508.06471

    Pith/arXiv arXiv 2025

  12. [12]

    Avi Goldfarb and Catherine Tucker. 2011. Online display advertising: Targeting and obtrusiveness.Marketing Science30, 3 (2011), 389–404

    2011

  13. [13]

    Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. 2025. Rubrics as rewards: Reinforcement learning beyond verifiable domains.arXiv preprint arXiv:2507.17746(2025)

    Pith/arXiv arXiv 2025

  14. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. 2025. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    Pith/arXiv arXiv 2025

  15. [15]

    Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247(2017)

    Pith/arXiv arXiv 2017

  16. [16]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. RULER: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654(2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Eugene Ie, Chih-wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, and Craig Boutilier. 2019. Recsim: A configurable simulation platform for recommender systems.arXiv preprint arXiv:1909.04847(2019)

    Pith/arXiv arXiv 2019

  18. [18]

    Sunghwan Kim, Tongyoung Kim, Kwangwook Seo, Jinyoung Yeo, and Dongha Lee. 2024. Stop playing the guessing game! target-free user simulation for evaluating conversational recommender systems.arXiv preprint arXiv:2411.16160 (2024)

    arXiv 2024

  19. [19]

    Kimi Team et al . 2026. Kimi K2.5: Visual Agentic Intelligence. arXiv:2602.02276 [cs.CL] https://arxiv.org/abs/2602.02276

    Pith/arXiv arXiv 2026

  20. [20]

    2020.Trustworthy online controlled experi- ments: A practical guide to a/b testing

    Ron Kohavi, Diane Tang, and Ya Xu. 2020.Trustworthy online controlled experi- ments: A practical guide to a/b testing. Cambridge University Press

    2020

  21. [21]

    Xinyu Lin, Yashar Deldjoo, Sunhao Dai, Honghui Bao, Xiaopeng Ye, Fatemeh Nazary, Wenjie Wang, Tommaso Di Noia, Jun Xu, and Tat-Seng Chua. 2026. Au- tonomous Information Seeking: A Roadmap for Agentic Recommender Systems. arXiv preprint arXiv:2607.04433(2026)

    Pith/arXiv arXiv 2026

  22. [22]

    Xinyu Lin, Pengyuan Liu, Wenjie Wang, Yicheng Hu, Chen Xu, Fuli Feng, Qifan Wang, and Tat-Seng Chua. 2026. Bringing reasoning to generative recommen- dation through the lens of cascaded ranking. InProceedings of the ACM Web Conference 2026. 8939–8949

    2026

  23. [23]

    Xinyu Lin, Hanqing Zeng, Hanchao Yu, Yinglong Xia, Jiang Zhang, Aashu Singh, Fei Liu, Wenjie Wang, Fuli Feng, Tat-Seng Chua, et al. 2026. Verifiable Reasoning for LLM-based Generative Recommendation.arXiv preprint arXiv:2603.07725 (2026)

    arXiv 2026

  24. [24]

    Fei Liu, Xinyu Lin, Hanchao Yu, Mingyuan Wu, Jianyu Wang, Qiang Zhang, Zhuokai Zhao, Yinglong Xia, Yao Zhang, Weiwei Li, et al . 2025. Recoworld: Building simulated environments for agentic recommender systems.arXiv preprint arXiv:2509.10397(2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts.Transactions of the association for computational linguistics12 (2024), 157–173

    2024

  26. [26]

    Tianci Liu, Ran Xu, Tony Yu, Ilgee Hong, Carl Yang, Tuo Zhao, and Haoyu Wang

  27. [27]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al

  28. [28]

    MiniMaxAI. 2025. MiniMax-M2.5 Technical Report. https://minimaxi.com/news/ minimax-m25. Accessed: 2026-03-22

    2025

  29. [29]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (2022), 27730–27744

    2022

  30. [30]

    Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng

  31. [31]

    Qwen Team. 2026. Qwen3.5: Towards Native Multimodal Agents. https://qwen. ai/blog?id=qwen3.5

    2026

  32. [32]

    David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. 2018. RecoGym: A reinforcement learning environment for the problem of product recommendation in online advertising.arXiv preprint arXiv:1808.00720(2018)

    Pith/arXiv arXiv 2018

  33. [33]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

    Pith/arXiv arXiv 2024

  34. [34]

    gradient descent

    Automatic prompt optimization with “gradient descent” and beam search. InProceedings of the 2023 conference on empirical methods in natural language processing. 7957–7968

    2023

  35. [35]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learn- ing.Advances in neural information processing systems36 (2023), 8634–8652

    2023

  36. [36]

    Tencent. 2026. Hy3 preview: The First Step in Rebuilding the Hy model. https: //hunyuan.tencent.com/research/hy3. Accessed: 2026-05-03. Chen et al

    2026

  37. [37]

    Lei Wang, Jingsen Zhang, Hao Yang, Zhi-Yuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Sun, Ruihua Song, et al. 2025. User behavior simulation with large language model-based agents.ACM Transactions on Information Systems43, 2 (2025), 1–37

    2025

  38. [38]

    Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, and An-Xiang Zeng. 2019. Virtual-taobao: Virtualizing real-world online retail environment for reinforce- ment learning. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4902–4909

    2019

  39. [39]

    Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Jing Huang, and Dakuo Wang. 2025. Customer-R1: Personalized simulation of human behaviors via RL-based LLM agent in online shopping.arXiv preprint arXiv:2510.07230(2025)

    arXiv 2025

  40. [40]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reason- ing in large language models.Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  41. [41]

    Tianjun Wei, Huizhong Guo, Yingpeng Du, Zhu Sun, Huang Chen, Dongxia Wang, and Jie Zhang. 2025. Mirroring Users: Towards Building Preference- aligned User Simulator with User Feedback in Recommendation.arXiv preprint arXiv:2508.18142(2025)

    Pith/arXiv arXiv 2025

  42. [42]

    Xiaolei Wang, Xinyu Tang, Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. 2023. Rethinking the evaluation for conversational recommendation in the era of large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 10052–10065

    2023

  43. [43]

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. InThe Twelfth International Conference on Learning Representations

    2023

  44. [44]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems36 (2023), 11809–11822

    2023

  45. [45]

    Runyang You, Yongqi Li, Xinyu Lin, Xin Zhang, Wenjie Wang, Wenjie Li, and Liqiang Nie. 2026. R 2ec: Towards Large Recommender Models with Reasoning. Advances in Neural Information Processing Systems38 (2026), 62376–62405

    2026

  46. [46]

    Chen Xu, Zhipeng Yi, Ruizi Wang, Wenjie Wang, Jun Xu, and Maarten de Rijke

  47. [47]

    Wenlin Zhang, Xiangyang Li, Qiyuan Ge, Kuicai Dong, Pengyue Jia, Xiaopeng Li, Zijian Zhang, Maolin Wang, Yichao Wang, Huifeng Guo, Ruiming Tang, and Xiangyu Zhao. 2026. Exploring Recommender System Evaluation: A Multi- Modal LLM Agent Framework for A/B Testing. InProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1(Repu...

    arXiv 2026

  48. [48]

    Yimeng Zhang, Tian Wang, Jiri Gesi, Ziyi Wang, Yuxuan Lu, Jiacheng Lin, Sinong Zhan, Vianne Gao, Ruochen Jiao, Junze Liu, et al . 2025. Shop-r1: Rewarding llms to simulate human behavior in online shopping via reinforcement learning. arXiv preprint arXiv:2507.17842(2025)

    arXiv 2025

  49. [49]

    Kesen Zhao, Shuchang Liu, Qingpeng Cai, Xiangyu Zhao, Ziru Liu, Dong Zheng, Peng Jiang, and Kun Gai. 2023. KuaiSim: A comprehensive simulator for recom- mender systems.Advances in Neural Information Processing Systems36 (2023), 44880–44897

    2023

  50. [50]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems36 (2023), 46595–46623

    2023

  51. [51]

    An Zhang, Leheng Sheng, Yuxin Chen, Hanzhi Li, Yuxuan Lu, and Tat-Seng Chua. 2024. On generative agents in recommendation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1807–1817

    2024

  52. [52]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. InThe eleventh international conference on learning representations

    2022

  53. [53]

    Lixi Zhu, Xiaowen Huang, and Jitao Sang. 2024. How reliable is your simulator? analysis on the limitations of current llm-based user simulators for conversational recommendation. InCompanion Proceedings of the ACM Web Conference 2024. 1726–1732

    2024

  54. [54]

    Lixi Zhu, Xiaowen Huang, and Jitao Sang. 2025. A LLM-based Controllable, Scalable, Human-Involved User Simulator Framework for Conversational Rec- ommender Systems. InProceedings of the ACM on Web Conference 2025(Sydney NSW, Australia)(WWW ’25). Association for Computing Machinery, New York, NY, USA, 4653–4661. doi:10.1145/3696410.3714858

    arXiv 2025

  55. [55]

    Small-scale SFT + RL

    Ming Zhu, Juntao Tan, Rithesh Murthy, Jielin Qiu, Liangwei Yang, Wenting Zhao, Silvio Savarese, Shelby Heinecke, and Huan Wang. 2026. RealUserSim: Bridging the reality gap in agent benchmarking via grounded user simulation.arXiv preprint arXiv:2605.20204(2026). A Dataset Details A.1 Data Collection We organize data collection on the Tencent platform at th...

    Pith/arXiv arXiv 2026

  56. [56]

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1059–1068

    2018

  57. [61]

    grammar — grammatical correctness.Is the grammar sound, and do the sentences read cleanly? •1: Riddled with errors; often unreadable and hard to follow; • 2: Plenty of grammar problems; you have to reread it to make sense of it; • 3: The odd small slip (a stray punctuation mark, an awkward pair- ing), but understandable overall; • 4: No real grammar error...

  58. [62]

    fluency — how smoothly it reads.Does it flow, or does it stumble and break up? •1: Very awkward or choppy; the sentences barely connect; •2: Trips up in several places; the transitions feel forced; •3: Reads well for the most part, with the occasional stiff patch; • 4: Flows nicely and hangs together, with just a spot or two that could be tightened; • 5: ...

  59. [63]

    written for a system to read

    naturalness — how human it sounds.Does this read like what a real person would actually think to themselves? •1: Very templated and robotic, with almost no personal voice; •2: Clearly “written for a system to read”; not very lifelike; • 3: Somewhere in between — part boilerplate, part everyday voice; • 4: Natural on the whole, like a real person thinking ...

  60. [64]

    Grammatically clean and smooth throughout, very conversational, like a real user thinking to themselves, and the sentences are reasonably varied

    diversity — variety of expression.Are the sentence patterns and word choices overly repetitive? • 1: Nearly every sentence follows the same mold, with the same phrases cropping up again and again; • 2: A hint of variation, but the phrasing and vocabulary stay largely the same; • 3: Some repetition, but not badly templated; the sentences do shift around a ...

  61. [65]

    a friend recommended it

    factuality (1–5): • 1: Most content seriously conflicts with the inputs; factual reliability is extremely poor; • 2: Key decisions rest on fabricated premises, or 1–2 clear contra- dictions are present; • 3: Broadly credible overall, but contains invented details (unsup- ported),e.g.,, “a friend recommended it” (not in the inputs); • 4: Core arguments are...

  62. [66]

    pregnant

    hallucination_level (1–5, higher = fewer hallucinations): Hard-fail conditions(any one of these caps the score at≤2): • Profile hard conflict: gender, age group, pregnancy status, parental status, etc. directly contradict user_profile (e.g.,, mar- riage_list=“pregnant”, but COT says “I don’t have kids”); • Behavior hard conflict: COT says “I never play ga...

  63. [67]

    user_alignment (1–5).Does the COT’s self-description of the user match what the user profile says? •1: Multiple serious mismatches; key attributes are the opposite of the profile — feels like a completely different person; •2: Clear mismatches, though some alignment remains; • 3: Broadly aligned; 1–2 vague or slightly off descriptions, but no hard contrad...

  64. [68]

    ad_alignment (1–5).Does the COT’s description of the ad match the actual ad information? Are there mix-ups or serious misreadings (category, brand, features, price range, target audience)? • 1: Serious errors — wrong category, completely wrong brand; reads like a description of a different ad entirely; • 2: Several key attributes are inaccurate, though yo...

  65. [69]

    pregnant

    context_alignment (1–5).Does the COT’s description of time, location, device, app, or usage scenario match context? • 1: Scene description seriously conflicts with context; almost com- pletely detached from the actual situation; •2: Some clear mismatches, alongside some correct details; •3: Only partial context details mentioned; the general direction is ...

  66. [70]

    Key factors

    key_factor_coverage (1–5).When the COT explains the user’s current behavior, does it actually cover the factors that matter most? “Key factors” are the dimensions that most heavily drive the decision in this particular situation — typically things like interest fit, cur- rent task/context, price/budget, ad fatigue, parenting/family needs, safety/risk conc...

  67. [71]

    I’m pregnant, in a few months I’ll need it

    evidence_support (1–5).Are the key reasons the COT uses to explain the user’s behavior backed by the inputs and common sense? Internal annotation taxonomy (for reasoning only, do not output): • Evidence source: user_profile / ad / context / history / com- mon_sense / none • Support level: strong_support (directly evidenced by an explicit field), weak_supp...

  68. [72]

    sees X→ connects it to a need/risk/constraint→ weighs up→ decides

    causal_coherence (1–5).Does the overall thought process flow logically — from seeing the ad, to forming an impression, to arriving at a behavioral intention — with clear causal connections and no obvious jumps or hard pivots? • 1: Cause-and-effect is muddled; the conclusion arrives out of nowhere with no build-up; • 2: A few causal connections are visible...

  69. [73]

    although . . . but

    internal_consistency (1–5).Does the COT contradict itself — fac- tually, emotionally, temporally, or in terms of stated position? Note: expressing ambivalence or inner conflict is fine, as long as it’s framed with a “although . . . but . . . ” type of transition. What matters is catching unexplained self-contradiction. • 1: Contradictions appear throughou...

  70. [2016]

    InProceedings of the 1st workshop on deep learning for recommender systems

    Wide & deep learning for recommender systems. InProceedings of the 1st workshop on deep learning for recommender systems. 7–10

  71. [2023]

    Self-refine: Iterative refinement with self-feedback.Advances in neural information processing systems36 (2023), 46534–46594

    2023

  72. [2025]

    Openrubrics: Towards scalable synthetic rubric generation for reward modeling and llm alignment.arXiv preprint arXiv:2510.07743(2025)

    arXiv 2025

  73. [2026]

    InProceedings of the ACM Web Conference 2026(United Arab Emirates)(WWW ’26)

    Unveiling and Simulating Short-Video Addiction Behaviors via Economic Addiction Theory. InProceedings of the ACM Web Conference 2026(United Arab Emirates)(WWW ’26). Association for Computing Machinery, New York, NY, USA, 5987–5997. doi:10.1145/3774904.3792216

    arXiv 2026