Pith. sign in

REVIEW 4 major objections 6 minor 56 references

Generative Optimization for Incentivized Advertising with Global Level Constraints

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GOAL: one generative policy allocates continuous ad incentives under strict ROI constraints and adapts its tightness at serving time.

desk verdict A plausible industrial system with a genuinely new constraint-conditioned generative combination, but the no-retraining adaptation claim needs a monotonicity proof and the SCPO pseudocode has an importance-sampling bug. read the letter →

arxiv 2608.04421 v1 pith:VVIXMUCP submitted 2026-08-05 cs.LG cs.AI

Gege Chen , Ning Luo , Hao Jiang , Da Li , Wenzheng Shu , Teng Sha , Yanxiang Zeng , Wenxin Tai
show 2 more authors
Fan Zhou Xialong Liu
This is my paper
classification cs.LGcs.AI
keywords AutoregressiveGenerationIncentivizedAdvertisementPolicyOptimizationConstrainedReinforcementLearningReturnonInvestmentMixture-of-ExpertsIncentiveAllocationSequence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that continuous incentive allocation in incentivized advertising is better solved as conditional sequence generation than as uplift modeling or constrained reinforcement learning. It introduces GOAL, which tokenizes reward amounts into ordered value units and autoregressively generates them conditioned on user history and a system-level Lagrange multiplier $\lambda$ that encodes ROI pressure. The paper argues that a hierarchical causal encoder plus a $\lambda$-conditioned Mixture-of-Experts decoder captures user fatigue and long-range dependencies that pure attention dilutes, while a safe constrained policy optimization (SCPO) stage trains one policy over a range of $\lambda$ values so a single model can serve many ROI targets without retraining. On the IA dataset with $\lambda=0.5$, GOAL reports revenue 0.32 versus 0.27 for the best baseline, ROI 4.93 versus 4.74, and an ROI violation rate of 12.24% versus 15.00%; an online A/B test reports +2.184% ROI and +2.559% revenue with $p=0.03$.

What carries the argument

The central object is the constraint-conditioned generative policy $\pi_\theta(a|s,\lambda)$, where $\lambda$ is the Lagrange multiplier in the linearized ROI objective $r_i - \lambda c_i$. Supporting machinery: a data-driven vocabulary that decomposes continuous incentive values into monotonically decreasing token sequences via dynamic quantile adjustment; a hierarchical causal state encoder whose dilated causal convolution (DCC) block captures localized behavioral dynamics before a self-attention layer models global dependencies; a decoder whose Mixture-of-Experts layers route by $\lambda$ so experts specialize into aggressive and conservative strategies; and SCPO, which augments GRPO-style group sampling with a Cartesian product of states and $\lambda$ values, uses constraint-conditional normalization for advantages, and finalizes with a PPO-style clipped surrogate and KL regularization. At serving time, Algorithm 2 picks $\lambda^*$ by binary search under the monotonicity assumption. The $\lambda$-conditioned MoE routing is what lets one set of parameters realize different incentive strategies, and the DCC is what prevents fatigue signals from being diluted in long histories.

What would settle it

Run the trained policy on the IA test set for a fine sweep of $\lambda$, compute realized ROI $\widehat{R}=\sum_i r_i/\sum_i c_i$ at each $\lambda$; if the resulting curve is not monotone, Algorithm 2's binary search can return a $\lambda^*$ whose realized ROI is below the target threshold. The paper currently reports only average cost and average revenue separately (Fig. 4), which does not settle the ratio's monotonicity.

Watch

Extended reading notes

Core claim

The paper's discovery is that a generative policy $\pi_\theta(a|s,\lambda)$ trained by SCPO over a distribution of Lagrange multipliers spans the revenue-ROI trade-off frontier in one model. The Lagrangian decoupling $\max_\pi \sum_i (r_i - \lambda c_i)$ turns the global ROI constraint into a per-decision reward, and SCPO's constraint-conditional normalization prevents high-$\lambda$ groups from looking worse merely because the constraint is stricter. At inference, a binary search over $\lambda$ selects the value whose realized ROI meets the target threshold, so constraining is no longer a retraining event but a control input. The reported results—higher revenue, higher ROI, lower violation rate on the industrial and synthetic sets, and positive online lifts—are what the paper offers as evidence that this constraint-aware generative formulation works at industrial scale.

Load-bearing premise

The load-bearing premise is that the trained policy's realized return-on-investment rises monotonically as the Lagrange multiplier $\lambda$ increases, since the serving-time binary search assumes this monotonicity to pick $\lambda^*$; the paper neither proves it nor plots the realized ROI curve.

Editorial extensions

If this is right

  • If GOAL is right, continuous incentive allocation no longer needs a separate uplift model plus allocator: a single generative model outputs the reward amount directly.
  • One trained policy can enforce a spectrum of ROI targets at serving time by inputting different $\lambda^*$ values, eliminating retraining per constraint.
  • The weighted next-token objective plus data-driven vocabulary improves precision on heavy-tailed, wide-range reward values compared with direct regression.
  • The online A/B result suggests the gains are not merely offline artifacts; reported lifts are +2.184% ROI and +2.559% revenue with $p=0.03$.
  • Constraint-aware MoE experts specialize by $\lambda$, so the same parameters implement aggressive and conservative strategies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our editorial extension: if realized ROI is not monotone in $\lambda$ on a shifted traffic mix, the binary-search calibration can pick a $\lambda^*$ that violates the target; re-validating the ROI-versus-$\lambda$ curve after model or traffic updates would settle this.
  • Our editorial extension: the tokenization-plus-MoE recipe should transfer to other heavy-tailed continuous actions such as bidding prices or coupon amounts; a second deployment would tell whether the gains come from the generative token head or from SCPO's $\lambda$-generalization.
  • Our editorial extension: conditioning on a vector of multipliers rather than a scalar $\lambda$ is a plausible route to the multi-objective case the paper names as future work, but the paper does not demonstrate it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GOAL, a generative sequence-modeling framework for continuous incentive allocation in incentivized advertising under global ROI constraints. It combines a hierarchical causal encoder, a λ-conditioned Mixture-of-Experts decoder, and SCPO, which trains a single policy over a distribution of Lagrange multipliers so that one model can be deployed under different ROI targets without retraining. Evaluation includes a large industrial dataset (IA), a synthetic fatigue-aware environment, and a four-week online A/B test. The paper reports revenue, ROI, and ROI-violation-rate improvements over strong baselines on the IA dataset, and ROI/RVR gains on the synthetic dataset.

Significance. If the claims hold, this is a practically important contribution: an industrial-scale generative framework for incentive allocation that can adapt to changing global ROI constraints without retraining, with supporting online evidence. The paper has real strengths: a large real-world dataset, a detailed synthetic testbed, explicit algorithmic appendices, and an online A/B test with a reported p-value. However, the central theoretical and algorithmic claims rest on unsubstantiated strong duality for a non-convex, non-Markovian setting and on an unverified monotonicity assumption in the λ-calibration procedure. In addition, the synthetic results do not support the revenue-improvement claim. These issues must be resolved before the paper's headline claims can be accepted.

major comments (4)
  1. [Section 2.2, Eq. (3)] The assertion that 'under the mild assumption that a feasible policy exists, strong duality holds' is not justified for the policy class considered in this paper. The policy is an autoregressive neural network over a discrete token vocabulary, which is a non-convex set, and the state includes interaction histories with fatigue-like dynamics, so the setting is not Markovian. The standard CMDP strong-duality argument via occupation measures and linear programming requires a convex policy set and Markovian transitions. Please either provide a proof that applies to this generative policy class, or weaken the claim to describe the Lagrangian objective as a heuristic surrogate, and add an empirical check that the SCPO policy at the calibrated λ* actually satisfies the ROI constraint.
  2. [Appendix B, Algorithm 2] Algorithm 2's binary search explicitly relies on 'the monotonicity of the realized ROI with respect to λ', but the paper gives neither a proof nor an empirical ROI-versus-λ curve. Fig. 4 plots average cost and average revenue separately; both decrease with λ, but the ratio of two decreasing functions need not be monotone. If realized ROI is non-monotone over [0, λ_max], the binary search can terminate at a λ whose realized ROI is below the target τ, directly undermining the no-retraining adaptation mechanism and the reported RVR improvement. Please add a plot of realized ROI (and RVR) versus λ on the calibration set, or provide a theoretical guarantee under stated assumptions, and discuss the behavior of Algorithm 2 if monotonicity fails.
  3. [Table 1, Synthetic IA row] On the Synthetic IA dataset, GOAL reports REV 0.29, which is below CDT's 0.36, yet the text reports only the ROI (+2.40%) and RVR (+9.86% relative) improvements, and the abstract claims GOAL 'improves long-term revenue'. This selective reporting weakens the revenue-improvement claim. Please qualify the revenue claim to the IA dataset, discuss the synthetic trade-off explicitly, or provide additional evidence that the REV gap is not a systematic weakness of the approach.
  4. [Section 4.3, Fig. 4] The demonstration that λ is an 'inference-time control knob' is partly circular. The decrease of cost and revenue with λ is a direct consequence of the training objective r_λ = R − λC (Eq. 17), so the figure does not independently establish that the model can meet a specified ROI target. The paper should show that for each target τ, the calibrated λ* from Algorithm 2 yields realized ROI ≥ τ with low RVR, ideally compared against an unconstrained or fixed-λ baseline, to substantiate the adaptation claim.
minor comments (6)
  1. [Section 4.4, Table 2 discussion] The text says the '-w/o SCPO' variant shows 'an increase in CVR', but the table reports RVR, not CVR; please correct the metric name.
  2. [Appendix C, Algorithm 3] In line 19, the clipping expression 'clip(ρ_{i,j},1−ϵ,1+ϵ)A' is missing a closing parenthesis; please fix the notation.
  3. [Appendix C heading] The heading 'Algorith 3' should be 'Algorithm 3'.
  4. [Section 4.5] Please report confidence intervals or standard errors for the online A/B test in addition to the p-value, and describe the production baseline and group-balance checks.
  5. [Section 3.3, Eq. (17)] Since the policy is optimized for the learned P-Score R, please include a correlation or calibration analysis between R and the actual revenue used in Tables 1 and 2, so that the transfer from the training objective to the reported business metric is explicit.
  6. [Conclusion] The phrase 'the first generative framework' is too strong given the existing body of generative sequence models for recommendation and decision-making; please soften the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SCPO's lambda-conditioned objective is a training design, not a disguised prediction, and the central claims are externally supported by the online A/B test.

full rationale

The paper's derivation chain is self-contained. The Lagrangian objective in Eq. (3) and the composite reward r_lambda = R - lambda*C in Eq. (17) define the training objective; the policy is explicitly conditioned on lambda as an input state variable. Fig. 4's observation that cost and revenue decrease with lambda is a consistency check on this objective, not a fitted parameter being relabeled as a prediction. Algorithm 2's binary-search calibration relies on an unproven monotonicity premise about realized ROI versus lambda; that is a correctness and robustness gap, not circularity. The paper does not explicitly state that offline IA evaluation reuses the P-Score model of Section 3.3 as the revenue oracle, so the offline-evaluation-overlap concern remains speculative; the reported online A/B test (Section 4.5) provides independent, externally grounded evidence of ROI and revenue gains. No load-bearing self-citations or imported uniqueness theorems were found in the derivation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

GOAL's central claim rests on three fitted or assumed components: the learned P-Score reward R (weights unspecified), the hand-chosen lambda spectrum with a unitless penalty scale, and the unproven monotonicity of ROI in lambda that makes the calibration search correct. None of these are externally benchmarked artifacts. Only the synthetic environment (Appendix E) is fully specified.

free parameters (5)
  • P-Score multi-objective weights w_k = Not reported in paper
    Eq. 14 fuses revenue, retention, scroll depth, and task-level engagement with 'adaptive weights' whose setting or fitting procedure is never described; the reward model R dominates the SCPO training signal in Eq. 17.
  • Lambda sampling ceiling lambda_max and step Delta_lambda = lambda_max = 3.0, Delta_lambda = 0.5 (7 levels) in experiments
    The constraint spectrum [0, lambda_max] and its discretization are hand-chosen. Appendix A asserts tau_max 'typically exceeds' the threshold lambda_bar but gives no check. Coverage of the spectrum determines whether the policy can reach the conservative regime needed by Algorithm 2.
  • Preference-vs-cost scale of lambda (R vs C units) = Unitless; no calibration described
    Eq. 17 subtracts lambda times a monetary cost C from a learned preference score R. Mixing a learned utility score with dollars through a unitless coefficient requires a hidden scale conversion that is neither estimated nor reported.
  • Vocabulary construction hyperparameters (q_start, q_end, delta, eps1, eps2) = No values reported
    Algorithm 1's percentile decay, restoration error, and precision thresholds determine token granularity and reconstruction error, but no values are given anywhere in the paper.
  • Position-weight smoothing constant delta (Eq. 13) = Unspecified
    Eq. 13 uses delta to keep a baseline gradient for small-value tokens; its value is not reported and it directly shapes the weighted next-token prediction loss.
assumptions (5)
  • ad hoc to paper Strong duality holds for the constrained generative-policy problem, giving min_lambda max_pi L(pi,lambda) with L(pi,lambda) = sum (r_i - lambda*c_i).
    Section 2.2 asserts 'Under the mild assumption that a feasible policy exists, strong duality holds.' The policy class is a non-convex neural generative model and the decision problem is non-Markovian, so the CMDP occupancy-measure LP argument does not apply. This is the load-bearing justification for the Lagrangian machinery in SCPO.
  • domain assumption Realized ROI is monotone in the Lagrange multiplier lambda.
    Algorithm 2 (Appendix B) performs binary search over lambda 'leveraging the monotonicity of the realized ROI with respect to lambda'. No proof or empirical curve is given; Fig. 4 plots cost and revenue, not ROI. If false, the calibration procedure can select the wrong lambda* and the constraint-control claim breaks.
  • domain assumption A policy feasible with respect to the ROI constraint exists and lambda* is attained by the trained family.
    Section 2.2 assumes feasibility for duality; Appendix B assumes the trained policy family contains a lambda with realized ROI >= tau. The observed RVR of 12.24% (Table 1) indicates the presumed feasible policy still violates the target in about one of eight sliding windows.
  • domain assumption The P-Score reward model R is an unbiased proxy for true revenue on the same scale as cost C.
    Section 3.3 trains R from auxiliary behavioral signals; Eq. 17 treats R as the revenue term of the Lagrangian. If R is mis-calibrated relative to actual monetary revenue, policies maximizing R - lambda*C optimize a surrogate that need not match the REV/ROI evaluation metrics.
  • domain assumption Offline counterfactual evaluation of new policies on the logged IA data is unbiased.
    Section 4.1 evaluates policies on historical data, but never specifies the off-policy estimator (IPS, DR, or response model) used to score actions that differ from the logged incentive. Without it, the reported REV/ROI/RVR for GOAL and baselines cannot be reproduced or trusted.
invented entities (1)
  • P-Score (personalized preference reward R)
    purpose: Dense reward signal for the generative policy; fuses revenue contribution, scroll depth, retention, and task-level engagement into one scalar score used in Eq. 17.
    No standalone validation of P-Score is provided, no calibration against realized revenue, and no ablation of its components. The online A/B test validates the whole system, not this reward. If R is misspecified, the core training signal in Eq. 17 is misspecified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Optimization for Incentivized Advertising with Global Level Constraints." pith.science (2026). https://pith.science/paper/VVIXMUCP

@misc{pith2026260804421,
  author       = {Pith},
  title        = {Pith review of: Generative Optimization for Incentivized Advertising with Global Level Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVIXMUCP}},
  note         = {Machine review of arXiv:2608.04421}
}
read the original abstract

Incentivized advertising allocates monetary or virtual rewards to drive user engagement, where a key challenge is optimizing continuous incentive magnitudes under strict global constraints. This problem is complicated by high-frequency interactions, delayed feedback, and non-Markovian user dynamics such as fatigue, which limit the effectiveness of existing uplift modeling and constrained reinforcement learning approaches. To address these challenges, we propose GOAL, a constraint-aware generative framework that formulates incentive allocation as a conditional sequence generation problem. GOAL directly generates incentive magnitudes conditioned on user histories and system-level global pressure, and integrates a hierarchical causal state encoder to capture both local behavioral dynamics and long-range dependencies. To enable flexible constraint control, we introduce \textbf{S}afe \textbf{C}onstrained \textbf{P}olicy \textbf{O}ptimization (SCPO), which learns a single generative policy that generalizes across a spectrum of ROI constraints without retraining. Experiments on large-scale real-world data and a synthetic fatigue-aware environment show that GOAL improves long-term revenue and user retention while substantially reducing ROI violation rates compared to strong baselines.

Figures

Figures reproduced from arXiv: 2608.04421 by the authors.

Figure 1
Figure 1. Overview of the incentivized advertising frame [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall architecture of GOAL. (1) Hierarchical Causal Encoder: synergizes DCC and self-attention to capture both [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The workflow of Safe Constrained Policy Optimiza [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effect of the Lagrangian multiplier 𝜆 on incentive cost and revenue. In the low-𝜆 regime, the model follows an aggressive strategy by allocating larger incentives to stimulate user engagement, resulting in higher cost and revenue. As 𝜆 increases, the policy gradually s…
Figure 5
Figure 5. Figure 5: Online Deployment of GOAL To validate the effectiveness of our model on live traffic, we adapt our framework to the online serving environment, as shown in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Expert specialization induced by the constraint [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 26 canonical work pages

  1. [27]

    Hongxu Ma, Kai Tian, Tao Zhang, Xuefeng Zhang, Han Zhou, Chunjie Chen, Han Li, Jihong Guan, and Shuigeng Zhou. 2024. Generative Regression Based Watch Time Prediction for Short-Video Recommendation.arXiv preprint arXiv:2412.20211(2024)

    arXiv 2024

  2. [33]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

    arXiv 2024

  3. [44]

    Yang Zhang, Bo Tang, Qingyu Yang, Dou An, Hongyin Tang, Chenyang Xi, Xueying Li, and Feiyu Xiong. 2021. BCORLE ( 𝜆): An Offline Reinforcement Learning and Evaluation Framework for Coupons Allocation in E-commerce Market.Advances in Neural Information Processing Systems34 (2021), 20410– 20422

    work page 2021

  4. [8]

    Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al

  5. [1]

    Meng Ai, Biao Li, Heyang Gong, Qingwei Yu, Shengjie Xue, Yuan Zhang, Yunzhou Zhang, and Peng Jiang. 2022. Lbcf: A large-scale budget-constrained causal forest algorithm. InProceedings of the ACM Web Conference 2022. 2310–2319

    work page 2022

  6. [2]

    Javier Albert and Dmitri Goldenberg. 2022. E-commerce promotions personaliza- tion via online multiple-choice knapsack with uplift modeling. InProceedings of the 31st ACM International Conference on Information & Knowledge Management. 2863–2872

    work page 2022

  7. [3]

    Shaojie Bai. 2018. An Empirical Evaluation of Generic Convolutional and Recur- rent Networks for Sequence Modeling.arXiv preprint arXiv:1803.01271(2018)

    arXiv 2018

  8. [4]

    Jiangxia Cao, Pengbo Xu, Yin Cheng, Kaiwei Guo, Jian Tang, Shijun Wang, Dewei Leng, Shuang Yang, Zhaojie Liu, Yanan Niu, et al. 2025. Pantheon: Personalized multi-objective ensemble sort via iterative pareto policy optimization. InPro- ceedings of the 34th ACM International Conference on Information and Knowledge Management. 5575–5582

    2025

Show all 56 references
  1. [5]

    Fanglin Chen, Xiao Liu, Bo Tang, Feiyu Xiong, Serim Hwang, and Guomian Zhuang. 2022. BCRLSP: An offline reinforcement learning framework for sequen- tial targeted promotion.arXiv preprint arXiv:2207.07790(2022)

    2022 arXiv

  2. [6]

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. 2021. Decision transformer: Reinforcement learning via sequence modeling.Advances in neural information processing systems34 (2021), 15084–15097

    2021

  3. [7]

    Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 2025. Onerec: Unifying retrieve and rank with generative recommender and iterative preference alignment.arXiv preprint arXiv:2502.18965 (2025)

    2025 arXiv

  4. [9]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). InProceedings of the 16th ACM conference on recommender systems. 299–315

    2022

  5. [10]

    Dmitri Goldenberg, Javier Albert, Lucas Bernardi, and Pablo Estevez. 2020. Free lunch! retrospective uplift modeling for dynamic promotions recommendation within roi constraints. InProceedings of the 14th ACM Conference on Recommender Systems. 486–491

    2020

  6. [11]

    Xian Guo, Ben Chen, Siyuan Wang, Ying Yang, Chenyi Lei, Yuqing Ding, and Han Li. 2025. OneSug: The Unified End-to-End Generative Framework for E-commerce Query Suggestion.arXiv preprint arXiv:2506.06913(2025)

    2025 arXiv

  7. [12]

    Zijian Guo, Weichao Zhou, Shengao Wang, and Wenchao Li. 2025. Constraint- conditioned actor-critic for offline safe reinforcement learning. InThe Thirteenth International Conference on Learning Representations

    2025

  8. [13]

    Behram J Hansotia and Bradley Rukstales. 2002. Direct marketing for multichan- nel retailers: Issues, challenges and solutions.Journal of Database Marketing & Customer Strategy Management9, 3 (2002), 259–266

    2002

  9. [14]

    Matthew J Hausknecht and Peter Stone. 2015. Deep Recurrent Q-Learning for Partially Observable MDPs.. InAAAI fall symposia, Vol. 45. 141. KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea Gege Chen et al

    2015

  10. [15]

    Hiroshi Iida, Dung Thai, Varun Manjunatha, and Mohit Iyyer. 2021. Tabbie: Pretrained representations of tabular data.arXiv preprint arXiv:2105.02584(2021)

    2021 arXiv

  11. [16]

    Zhanming Jie, Jierui Li, and Wei Lu. 2022. Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 5944–5955

    2022

  12. [17]

    Zhihua Jin, Xin Jiang, Xingbo Wang, Qun Liu, Yong Wang, Xiaozhe Ren, and Huamin Qu. 2021. Numgpt: Improving numeracy ability of generative pre-trained models.arXiv preprint arXiv:2109.03137(2021)

    2021 arXiv

  13. [18]

    Haruka Kiyohara, Kosuke Kawakami, and Yuta Saito. 2021. Accelerating offline reinforcement learning application in real-time bidding and recommendation: Potential use of simulation.arXiv preprint arXiv:2109.08331(2021)

    2021 arXiv

  14. [19]

    Dmytro Korenkevych, Frank Cheng, Artsiom Balakir, Alex Nikulkov, Lingnan Gao, Zhihao Cen, Zuobing Xu, and Zheqing Zhu. 2024. Offline reinforcement learning for optimizing production bidding policies. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Da...

    2024

  15. [20]

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. 2021. Offline reinforcement learning with implicit q-learning.arXiv preprint arXiv:2110.06169(2021)

    2021 arXiv

  16. [21]

    Nian Li, Xin Ban, Cheng Ling, Chen Gao, Lantao Hu, Peng Jiang, Kun Gai, Yong Li, and Qingmin Liao. 2024. Modeling user fatigue for sequential recommendation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 996–1005

    2024

  17. [22]

    Wenzhe Li, Hao Luo, Zichuan Lin, Chongjie Zhang, Zongqing Lu, and Deheng Ye. 2023. A survey on transformers in reinforcement learning.arXiv preprint arXiv:2301.03044(2023)

    2023 arXiv

  18. [23]

    Qian Lin, Bo Tang, Zifan Wu, Chao Yu, Shangqin Mao, Qianlong Xie, Xingxing Wang, and Dong Wang. 2023. Safe offline reinforcement learning with real- time budget constraints. InInternational Conference on Machine Learning. PMLR, 21127–21152

    2023

  19. [24]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics 12 (2024), 157–173

    2024

  20. [25]

    Tao Liu, Qi Xu, Wei Shi, Zhigang Hua, and Shuang Yang. 2025. Session-Level Dynamic Ad Load Optimization using Offline Robust Reinforcement Learning. arXiv preprint arXiv:2501.05591(2025)

    2025 arXiv

  21. [26]

    Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, and Ding Zhao. 2023. Constrained decision transformer for offline safe reinforcement learning. InInternational conference on machine learning. PMLR, 21611–21630

    2023

  22. [28]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

    2022

  23. [29]

    Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, and Yongfeng Zhang. 2019. Value-aware recommendation based on reinforcement profit maximization. InThe World Wide Web Conference. 3123–3129

    2019

  24. [30]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

    2023

  25. [31]

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al

  26. [32]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  27. [34]

    Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022. Transformer memory as a differentiable search index.Advances in Neural Information Processing Systems 35 (2022), 21831–21843

    2022

  28. [35]

    Arun Venkatraman, Martial Hebert, and J Bagnell. 2015. Improving multi-step prediction of learned time series models. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 29

    2015

  29. [36]

    Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019. Do NLP models know numbers? probing numeracy in embeddings.arXiv preprint arXiv:1909.07940(2019)

    2019 arXiv

  30. [37]

    Chao Wang, Xiaowei Shi, Shuai Xu, Zhe Wang, Zhiqiang Fan, Yan Feng, An You, and Yu Chen. 2023. A Multi-stage Framework for Online Bonus Allocation Based on Constrained User Intent Detection. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining...

    2023

  31. [38]

    Zhiruo Wang, Haoyu Dong, Ran Jia, Jia Li, Zhiyi Fu, Shi Han, and Dongmei Zhang

  32. [39]

    Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, and Kun Gai. 2018. Budget constrained bidding by model-free reinforcement learning in display advertising. InProceedings of the 27th ACM International Conference on Information and Knowledge Management. ...

    2018

  33. [40]

    Zifan Wu, Bo Tang, Qian Lin, Chao Yu, Shangqin Mao, Qianlong Xie, Xingxing Wang, and Dong Wang. 2024. Off-policy primal-dual safe reinforcement learning. arXiv preprint arXiv:2401.14758(2024)

    2024 arXiv

  34. [41]

    Shuai Xiao, Le Guo, Zaifan Jiang, Lei Lv, Yuanbo Chen, Jun Zhu, and Shuang Yang. 2019. Model-based constrained MDP for budget allocation in sequential incentive marketing. InProceedings of the 28th ACM International Conference on Information and Knowledge Management. 971–980

    2019

  35. [42]

    Tete Xiao, Mannat Singh, Eric Mintun, Trevor Darrell, Piotr Dollár, and Ross Girshick. 2021. Early convolutions help transformers see better.Advances in neural information processing systems34 (2021), 30392–30400

    2021

  36. [43]

    Shuli Zhang, Hao Zhou, Jiaqi Zheng, Guibin Jiang, Bing Cheng, Wei Lin, and Guihai Chen. 2025. Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data.arXiv preprint arXiv:2510.19517(2025)

    2025

  37. [45]

    Kui Zhao, Junhao Hua, Ling Yan, Qi Zhang, Huan Xu, and Cheng Yang. 2019. A unified framework for marketing budget allocation. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1820–1830

    2019

  38. [46]

    Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018. Deep reinforcement learning for page-wise recommendations. In Proceedings of the 12th ACM conference on recommender systems. 95–103

    2018

  39. [47]

    Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. 2018. DRN: A deep reinforcement learning framework for news recommendation. InProceedings of the 2018 world wide web conference. 167–176

    2018

  40. [48]

    Hao Zhou, Rongxiao Huang, Shaoming Li, Guibin Jiang, Jiaqi Zheng, Bing Cheng, and Wei Lin. 2024. Decision focused causal learning for direct counterfactual marketing optimization. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 6368–6379

    2024

  41. [49]

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long se- quence time-series forecasting. InProceedings of the AAAI conference on artificial intelligence, Vol. 35. 11106–11115

    2021

  42. [50]

    Barret Zoph. 2022. Designing effective sparse expert models. In2022 IEEE In- ternational Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 1044–1044

    2022

  43. [51]

    Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin

  44. [2017]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  45. [2019]

    InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining

    Reinforcement learning to optimize long-term user engagement in recom- mender systems. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2810–2818. APPENDIX A Selection of Candidate𝜆Sets To enable the policy to perceive and ada...

    2026

  46. [2021]

    InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

    Tuta: Tree-based transformers for generally structured table pre-training. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1780–1790

  47. [2022]

    In International conference on machine learning

    Glam: Efficient scaling of language models with mixture-of-experts. In International conference on machine learning. PMLR, 5547–5569

  48. [2023]

    Recommender systems with generative retrieval.Advances in Neural Information Processing Systems36 (2023), 10299–10315

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.