CTR-Driven Ad Text Generation via Online Feedback Preference Optimization

Bo Zheng; Jiale Chen; Qixiang Gao; Si Chen; Tiezheng Ge; Xubin Li; Yanda Chen; Zihui Ren

REVIEW 5 major objections 5 minor 1 cited by

CTR-Driven Ad Text Generation via Online Feedback Preference Optimization

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper reports that LLM-generated ad titles can beat human-crafted ones on click-through rate when preference pairs are reweighted by CTR gain and by whether the win repeats in a parallel control group.

desk verdict A real industry-scale demonstration with an under-reported statistical basis; the confidence-weighting increment is suggestive, not established. read the letter →

arxiv 2507.20227 v3 pith:WQSZ647U submitted 2025-07-27 cs.IR

Yanda Chen , Zihui Ren , Qixiang Gao , Jiale Chen , Si Chen , Xubin Li , Tiezheng Ge , Bo Zheng This is my paper

classification cs.IR

keywords adtextgenerationclick-throughratedirectpreferenceoptimizationonlineA/BtestingAAgroupretrieval-augmentedstyletransferLLMalignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CTOP, a two-stage method for generating e-commerce ad titles that optimize click-through rate (CTR), and reports that it closes the gap between LLM-generated and human-crafted ad text on the key online business metric. Stage one samples diverse candidate titles by imitating retrieved high-performing exemplars through one-shot in-context learning with chain-of-thought tips. Stage two runs CTR-driven preference optimization: it constructs preference pairs from online A/B/n tests, with each winning candidate preferred over the human-crafted title, and reweights each pair by the size of its CTR gain and by a binary confidence score requiring the same win to replicate in a parallel AA control group. The paper reports a 60.2% item win rate and +4.76% relative CTR improvement over human titles in a held-out A/B test, and +1.11% CTR with +1.02% RPM under full-traffic deployment on a large shopping platform. A sympathetic reading is that the method turns noisy click feedback into a usable training signal and demonstrates, at industrial scale, that CTR itself can be the objective for ad-text generation.

What carries the argument

The load-bearing object is the weighted DPO objective $L_{CTRPO} = -\sum_{i=1}^N \sum_{k=1}^{M_i} w^{(k)}_i L_{DPO}$ (Eq. 7), built on DPO, a loss that trains a policy to prefer winning over losing completions without a separate reward model (Eq. 3). The weight $w^{(k)}_i = gain^{(k)}_i \cdot confidence^{(k)}_i$ combines Eq. 5, the within-item normalized CTR gain scaled by $m=2.5$, and Eq. 6, a binary confidence score that requires the same candidate-over-human ordering to appear in a parallel AA control group. The AA group is the key instrument: it turns short-term, uneven page-view feedback into a reproducibility check on each preference pair. The other stage supplies the preference data: a style transfer model fine-tuned on one-shot in-context examples retrieved by retrieval-augmented generation from a repository of high-quality ad texts, each with chain-of-thought imitation tips. In the paper's design this stage replaces low-quality top-k sampling and keeps candidate win rate near 50% even as the number of candidates per item $K$ grows.

What would settle it

On a fresh two-week A/B test, compute the confidence coefficient on a random half of the experimental traffic and evaluate the CTOP-trained policy on the other half. If the AA-group ordering is dominated by noise, the binary confidence threshold will be uncorrelated with real signal, and the weighted-DPO model should not beat plain DPO on the held-out half; if the confidence weighting is genuinely filtering, the win rate should replicate.

Watch

Extended reading notes

Core claim

The central discovery claimed is that preference optimization for ad text becomes substantially more effective when each training pair is weighted by two quantities computed from the same online experiment. With $y_w$ the winning candidate and $y_l$ the human-crafted title, the gain coefficient is $g^{(k)}_i = \Delta^{(k)}_i / \max_{k'} \Delta^{(k')}_i \times m$ with $m=2.5$, where $\Delta^{(k)}_i = ctr(y^{(k)}_w) - ctr(y^{(k)}_l)$ is normalized by the largest CTR difference within the item. The confidence coefficient is 1 only if $y_w$ beats $y_l$ in both the experimental group and a parallel AA group with identical content and traffic allocation; otherwise it is 0.5. These two coefficients multiply into a per-pair weight $w^{(k)}_i$ applied to the Direct Preference Optimization (DPO) loss, and the paper argues this both amplifies large, stable CTR improvements and suppresses unreliable preferences from low-page-view items. The claimed result is that this reweighting improves item win rate from 56.0% (plain DPO) to 60.2% and relative CTR from +3.48% to +4.76%, and that the deployed system sustains +1.11% CTR and +1.02% RPM under full traffic.

Load-bearing premise

The load-bearing premise is that the parallel AA control group, run with identical content and traffic allocation, produces an ordering of candidate versus human title that is a trustworthy, unbiased replica of the experimental group's ordering; if that control ordering is itself noisy or shares the experimental group's biases, the 0.5 confidence threshold will not remove the noise and the weighting may amplify artifacts rather than signal.

Editorial extensions

If this is right

A deployed system can close the gap between LLM-generated and human-crafted ad text: under full traffic, CTOP titles improved CTR by 1.11% and RPM by 1.02% over the human baseline since late 2024.
Preference-pair quality, not just quantity, matters: weighting by confidence and gain raised item win rate from 56.0% (plain DPO) to 60.2% in the held-out A/B test.
Diverse sampling alone is insufficient: a style-transfer sampler that holds candidate win rate near 50% still needs preference alignment to convert diversity into CTR gains, as shown by Top-1 sampling's -0.6% CTR.
The approach works with implicit, delayed feedback (clicks per page view) rather than explicit human ratings, using the human-crafted title as a per-item baseline.
The gain-and-confidence reweighting can be layered onto an existing DPO training loop without changing the base model or data collection, so it is a drop-in upgrade to standard preference optimization in ad-text pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: the same confidence-weighting trick could transfer to other off-policy preference optimization settings where a parallel control group is available, such as recommendation re-ranking or web content optimization, because it only assumes the control ordering is a noisy replica of the treatment ordering.
Editorial extension: the within-item normalization of gain means the method ignores differences in baseline CTR across items; a cross-item gain normalization might change which pairs dominate training, a testable variation the paper does not explore.
Editorial extension: the reported AA-group confidence threshold is binary at 0.5; a graded confidence based on the statistical significance of the ordering difference might extract more signal from intermediate-page-view items than a hard cutoff.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a serious industry paper with real online A/B evidence, but the statistical support for its two weighting innovations is thinner than the headline numbers suggest. The RAG-based one-shot style transfer for candidate generation is a practical and apparently effective fix for sampling diversity, and the gain/confidence weighting scheme is a sensible way to make DPO robust to noisy CTR feedback. The paper also deserves credit for deploying on a large platform and reporting full-traffic gains.

What's new: the combination of retrieval-augmented style transfer with DPO weighted by CTR gain and an AA-group confidence indicator is not in the cited prior work, and the AA-based confidence mechanism is a reasonable attempt at a replication filter. The online evaluation is large: 1M items for main A/B, 500K preference pairs, real RPM gains. That is real evidence of a working system.

Soft spots, in order of severity. First, there are no confidence intervals or significance tests anywhere. The ablation shows 56.0→58.5→60.2 win rate and +3.48→+3.79→+4.76 CTR, but we are not told whether those increments are stable across traffic splits or random seeds. With 1M items they might well be significant, but the paper doesn't demonstrate it. Second, the stress-test concern about Eq. (6) is legitimate: since preference pairs are built only from Exp-group winners, the first conjunct in the confidence condition is automatic, so confidence reduces to an AA-replication indicator. For a null pair the AA ordering is a coin flip, so the 0.5 floor only halves the weight of noisy pairs; it doesn't remove them. The paper gives no replication-rate analysis (e.g., what fraction of winners are confirmed in AA, and how that compares to chance) and no sensitivity to the 0.5 threshold. So the ablation's confidence increment is suggestive but not established. Third, the gain coefficient's scaling factor m=2.5 is a hand-set hyperparameter with no sensitivity analysis; that is a minor issue but typical of the paper's lack of robustness checks. Fourth, the preference pairs have human-written text as the fixed dispreferred response, which means the model only learns to beat the human baseline, not to distinguish among candidates; that's a selection-bias concern worth a direct response.

Bottom line: the central claim—that CTOP beats DPO and human text in a large-scale online setting—is plausible and probably true, but the paper currently under-reports the statistics that would make it verifiable. A serious referee should push for confidence intervals, a replication-rate analysis for the confidence coefficient, and sensitivity on m and the 0.5 floor. The paper deserves review rather than desk rejection.

Referee Report

5 major / 5 minor

Summary. The paper proposes CTOP, a two-stage framework for CTR-driven ad text generation. In stage one, diverse candidate ad texts are sampled via one-shot in-context learning with retrieval-augmented generation and chain-of-thought imitation tips. In stage two, preference pairs are constructed from online A/B/n test winners versus human-crafted texts, and a DPO-style loss is weighted by a gain coefficient (Eq. 5) and a confidence coefficient derived from a parallel AA control group (Eq. 6). The authors report offline diversity and preference-accuracy analyses, online A/B tests comparing CTOP to baselines (Table 3), ablations of the gain/confidence components (Table 5), and a full-traffic deployment with +1.11% CTR and +1.02% RPM.

Significance. If the reported results hold, the paper would demonstrate a practical, deployed method that closes the online CTR gap between LLM-generated and human-crafted ad text at scale, with a concrete algorithmic recipe (RAG+CoT diversity, weighted preference optimization) that other e-commerce systems could adopt. The paper's strengths are that the online A/B evidence is real and large-scale, the two-stage framework is clearly motivated by the identified challenges, and the deployment metrics (+1.11% CTR, +1.02% RPM) are economically meaningful. However, the paper's central novelty claim—that the confidence weighting in Eq. (6) is what drives the improvement over standard DPO—is not statistically supported as presented, and several details of the online measurement and weighting scheme are left unspecified.

major comments (5)

[Eq. (6), Section 'CTR-Driven Preference Optimization'] The confidence coefficient is not validated as a noise filter. Because preference pairs are constructed only from candidate winners in the experimental group, the condition y_w ≻_exp y_l is automatically true; confidence reduces to an indicator that the winner also beats the human title in the AA group. For a null pair (true CTR difference zero), the AA ordering is a coin flip, so roughly half of such pairs receive confidence=1 and half confidence=0.5. The 0.5 floor therefore halves, but does not remove, the weight of unreliable pairs. The paper neither reports the agreement/replication rate between Exp and AA orderings nor provides a sensitivity analysis of the 0.5 fallback value. Without such evidence, the ablation increments in Table 5 (56.0→58.5→60.2; +3.48→+3.79→+4.76) could arise from noise or from the gain coefficient alone; the claimed benefit of the confidence component over unweighted DPO is not established. This is load-bearing because the confidence weighting is the paper's main novel component.
[Table 3 and Section 'Evaluation Metrics'] The main online results are reported without any uncertainty quantification. The paper states win rates and relative CTR improvements for 1 million items, but provides no confidence intervals, no significance tests, and no description of how CTR is aggregated across items (e.g., pooled clicks over pooled PV, or per-item average). The definition of 'Overall Relative CTR Improvement' leaves open whether items are weighted equally or by PV. Position-bias handling and the exact mechanics of the A/B test (e.g., how titles are displayed, whether the human title is always shown, traffic allocation per title) are also unspecified. Given the 1M-item scale, even small effects may be statistically significant, but the paper does not demonstrate this, and the lack of variance reporting undermines the strength of the claim that CTOP 'significantly' outperforms baselines.
[Eq. (5), Section 'CTR-Driven Preference Optimization'] The gain coefficient is not robustly specified. It normalizes each CTR difference by the maximum difference within the same item and then multiplies by a scaling factor m=2.5. No sensitivity analysis is provided for m, and the normalization makes the weight of every pair in an item depend on the noisiest pair (the one with the maximum observed difference). For low-PV items, the maximum difference is itself an extreme-order statistic, so the normalization likely amplifies noise. The interaction between this normalization and the confidence coefficient is not analyzed. A sensitivity sweep over m (or an alternative normalization) would be needed to show that the reported gains are not an artifact of this specific hand-set hyperparameter.
[Figure 2 and Section 'Challenge I'] The item win rate, defined as the proportion of items with at least one candidate beating the human title, is a multiple-comparison statistic that increases with K even if all candidates are random. The paper acknowledges the candidate win rate decreases with K, but uses the increasing item win rate as evidence that 'introducing diversity contributes to generating more high-CTR ad texts.' This interpretation should be qualified: the item win rate is expected to rise with K under the null hypothesis. Reporting an adjusted win rate (e.g., comparing the best-of-K candidate to a Bonferroni-corrected threshold) or an analogous permutation baseline would make the diversity argument more convincing.
[Section 'Online Results'] The long-term deployment claim rests on a comparison to 'the human-crafted baseline' under 'full platform traffic,' but the paper does not describe the measurement window, the stability of the deployed traffic split, or whether the baseline is the same human-crafted titles used in the A/B/n tests. If the deployed model replaced the human baseline for all items, the reported +1.11% CTR and +1.02% RPM lack any confidence or sensitivity information. This makes it difficult to assess the reliability of the headline business claim.

minor comments (5)

[Figures 2, 4, 5] The captions and axis labels contain a rendering artifact: '/glyph1197um ber of C andidates per Item (K)' should read 'Number of Candidates per Item (K)'. Please ensure the final PDF renders properly.
[Section 'Evaluation Metrics'] The definition of 'Win Rate' in the evaluation is 'the proportion of items where the generated title achieves a higher CTR than the human-crafted title.' This is a pairwise comparison per item; clarify whether ties are counted as losses, and whether the CTR comparison uses the same testing period and traffic allocation as the training data.
[Ablation, Table 5] The ablation table would be more informative with explicit per-configuration sample sizes or confidence intervals. At a minimum, the number of items in each online A/B test should be reported, as it is only stated for the main evaluation (1 million items).
[Section 'Experiment Setup'] The description of the AA group says 'with identical content and traffic allocation.' Please specify whether the AA group uses the same 5% traffic bucket as the experimental group or an independent 5% allocation, and how the AA group is kept orthogonal to the experimental group.
[Related Works and References] Some references contain inconsistent formatting (e.g., 'V . Y .' with odd spacing, 'Y .' in author initials). A pass to harmonize reference formatting would improve presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported CTR gains come from a held-out online A/B test, not from the training-time weighting equations.

full rationale

The derivation chain is empirical and self-contained. CTOP's objective (Eq. 2) is to maximize ctr(y) - ctr(y_hu); the method constructs preference pairs from candidates that won an A/B/n test against human titles, reweights them with gain (Eq. 5) and confidence (Eq. 6), and trains weighted DPO (Eq. 7). The final claims (60.2% win rate, +4.76% CTR; +1.11% CTR/+1.02% RPM at scale) are measured on a separate deployment: "We select 1 million items for evaluation, ensuring that none of them overlap with those in the training set." No equation in the paper reduces those reported numbers to the gain/confidence weights: m=2.5 is a hand-set scaling factor, not fit to the test result, and the confidence coefficient is a second noisy replication draw rather than a definition of the evaluation metric. The redundancy that y_w ≻_exp y_l is automatic for all constructed pairs is a modeling-design issue (it makes confidence effectively an AA-only indicator), not a circularity, because the held-out CTR lift is not computed from that indicator. Self-citations to Chen et al. 2025 and Wei et al. 2022 appear only in related-work context and are not load-bearing for the claimed results. The central claim is therefore independently grounded; concerns about AA-group noise or lack of significance on ablations are correctness risks, not circularity.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central result rests on standard DPO math, a set of domain assumptions about CTR testing on the platform, and a few hand-set hyperparameters (notably m=2.5 and the confidence fallback). No new scientific entities are introduced.

free parameters (5)

m (gain scaling factor) = 2.5
Hand-set in Eq. 5 to amplify small CTR differences; no sensitivity analysis is reported.
DPO beta = Not reported
Preference strength beta in the DPO loss (Eq. 3) is not specified in the implementation details.
Confidence fallback weight = 0.5
Eq. 6 assigns 0.5 when preferences are not replicated in the AA group; the exact value is chosen without analysis.
Number of candidates K = 5 (training, top-5 retrieved exemplars); 10 (analysis)
K controls diversity and win rates; 5 is used for main experiments.
Sampling temperature and top_k for baseline = temperature=1.5, top_k=100
Used for the sampling-based decoding baseline in Challenge I; not part of CTOP.

assumptions (6)

standard math DPO loss with implicit Bradley-Terry reward model (Rafailov et al. 2023)
Eq. 3 is taken as given from the DPO paper; the paper relies on its validity.
domain assumption CTR is a valid proxy for ad text quality and business value
Eqs. 1-2 define the objective as maximizing CTR; clicks are treated as prerequisite for revenue.
domain assumption Human-crafted ad texts are a strong baseline that defines high CTR
Eq. 2 reformulates objective as beating y_hu; all evaluation metrics compare against human text.
domain assumption Online A/B/n with equal-probability display gives unbiased CTR estimates
Used in Challenge I and the main evaluation; position and context biases are not discussed.
domain assumption AA group replicates the experimental group and can validate preference order
Eq. 6 depends on observing the same y_w > y_hu order in both groups; interference or shared noise would break this.
domain assumption CoT imitation tips generated by GPT-4 capture transferable style
The repository and style-transfer training (appendix) rely on GPT-4-generated structural tips; these are assumed to guide faithful style transfer.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CTR-Driven Ad Text Generation via Online Feedback Preference Optimization." pith.science (2026). https://pith.science/paper/WQSZ647U

@misc{pith2026250720227,
  author       = {Pith},
  title        = {Pith review of: CTR-Driven Ad Text Generation via Online Feedback Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WQSZ647U}},
  note         = {Machine review of arXiv:2507.20227}
}

read the original abstract

Advertising text plays a critical role in determining click-through rates (CTR) in online advertising. Large Language Models (LLMs) offer significant efficiency advantages over manual ad text creation. However, LLM-generated ad texts do not guarantee higher CTR performance compared to human-crafted texts, revealing a gap between generation quality and online performance of ad texts. In this work, we propose a novel ad text generation method which optimizes for CTR through preference optimization from online feedback. Our approach adopts an innovative two-stage framework: (1) diverse ad text sampling via one-shot in-context learning, using retrieval-augmented generation (RAG) to provide exemplars with chain-of-thought (CoT) reasoning; (2) CTR-driven preference optimization from online feedback, which weighs preference pairs according to their CTR gains and confidence levels. Through our method, the resulting model enables end-to-end generation of high-CTR ad texts. Extensive experiments have demonstrated the effectiveness of our method in both offline and online metrics. Notably, we have applied our method on a large-scale online shopping platform and achieved significant CTR improvements, showcasing its strong applicability and effectiveness in advertising systems.

Figures

Figures reproduced from arXiv: 2507.20227 by the authors.

**Figure 1.** Challenges in high-CTR ad text generation. (a) Poor diversity and inferior CTR performance of samplingbased decoding strategy. (b) Noisy online feedback due to uneven page view (P V ) distribution across items. (c) We propose an innovative two-stage framework: (1) diverse ad text sampling and (2) CTR-driven preference optimization tailored for these challenges. for online metrics, particularly CTR. CTR has long be… view at source ↗

**Figure 2.** Item and candidate win rates over human-crafted [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Distribution of item page views (P V ) after two weeks of online testing [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Overall framework of our CTR-driven ad text generation method, CTOP. (a) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Win rate (over human-crafted ad texts) comparison [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Comparison cases between human-crafted item titles and our generated titles on the online shopping platform. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Detailed comparisons among human-crafted ad texts, underperforming candidates in diverse ad text sampling, and [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

AI-Generated Content in Cross-Domain Applications: Research Trends, Challenges and Propositions
cs.AI 2025-09 conditional novelty 2.0 of 10

A cross-domain vision paper that surveys AI-generated content and proposes research directions, without introducing new empirical results.

Reference graph

Works this paper leans on

42 extracted references · 29 canonical work pages · cited by 1 Pith paper

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

G.; Guo, Z

Azar, M. G.; Guo, Z. D.; Piot, B.; Munos, R.; Rowland, M.; Valko, M.; and Calandriello, D. 2024. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, 4447--4455. PMLR

2024
[4]

Bartz, K.; Barr, C.; and Aijaz, A. 2008. Natural language generation for sponsored-search advertisements. In Proceedings of the 9th ACM Conference on Electronic Commerce, 1--9

work page 2008
[5]

Y.; Golobokov, K.; Zhu, C.; and Liu, Y

Chai, J.; Pryzant, R.; Dong, V. Y.; Golobokov, K.; Zhu, C.; and Liu, Y. 2022. Fast: Improving controllability for text generation with feedback aware self-training. arXiv preprint arXiv:2210.03167

arXiv 2022
[6]

Chen, X.; Feng, W.; Du, Z.; Wang, W.; Chen, Y.; Wang, H.; Liu, L.; Li, Y.; Zhao, J.; Li, Y.; et al. 2025. CTR-Driven Advertising Image Generation with Multimodal Large Language Models. In Proceedings of the ACM on Web Conference 2025, 2262--2275

work page 2025
[7]

F.; Leike, J.; Brown, T

Christiano, P. F.; Leike, J.; Brown, T. B.; Martic, M.; Legg, S.; and Amodei, D. 2017. Deep Reinforcement Learning from Human Preferences. In Guyon, I.; von Luxburg, U.; Bengio, S.; Wallach, H. M.; Fergus, R.; Vishwanathan, S. V. N.; and Garnett, R., eds., Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Proces...

work page 2017
[8]

Duan, S.; Li, W.; Cai, J.; He, Y.; and Wu, Y. 2021. Query-variant advertisement text generation with association knowledge. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 412--421

work page 2021

Show all 42 references

[9]

Ethayarajh, K.; Xu, W.; Muennighoff, N.; Jurafsky, D.; and Kiela, D. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306

2024 arXiv
[10]

Y.; Gu, M.; Chi, B.; Cao, J.; Yan, Y.; and Liu, Y

Golobokov, K.; Chai, J.; Dong, V. Y.; Gu, M.; Chi, B.; Cao, J.; Yan, Y.; and Liu, Y. 2022. DeepGen: Diverse Search Ad Generation and Real-Time Customization. In Che, W.; and Shutova, E., eds., Proceedings of the The 2022 Conference on Empirical Methods in Natural Language Proc...

2022
[11]

Holtzman, A.; Buys, J.; Du, L.; Forbes, M.; and Choi, Y. 2020. The Curious Case of Neural Text Degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net

2020
[12]

W.; Chang, K.-h.; and Zhang, R

Hughes, J. W.; Chang, K.-h.; and Zhang, R. 2019. Generating better search engine text advertisements with deep reinforcement learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2269--2277

2019
[13]

Ji, J.; Qiu, T.; Chen, B.; Zhang, B.; Lou, H.; Wang, K.; Duan, Y.; He, Z.; Zhou, J.; Zhang, Z.; et al. 2023. Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852

2023 arXiv
[14]

Jin, Y.; Bhatia, A.; Wanvarie, D.; and Le, P. T. V. 2023. Towards improving coherence and diversity of slogan generation. Nat. Lang. Eng., 29(2): 254--286

2023
[15]

Kamigaito, H.; Murakami, S.; Zhang, P.; Takamura, H.; and Okumura, M. 2024. Generating Attractive Ad Text by Facilitating the Reuse of Landing Page Expressions. In Proceedings of the 17th International Natural Language Generation Conference, 597--608

2024
[16]

Kamigaito, H.; Zhang, P.; Takamura, H.; and Okumura, M. 2021. An empirical study of generating texts for search engine advertising. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies...

2021
[17]

S.; Das, G.; A, P.; and Negi, S

Kanungo, Y. S.; Das, G.; A, P.; and Negi, S. 2022. Cobart: controlled, optimized, bidirectional and auto-regressive transformer for ad headline generation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 3127--3136

2022
[18]

Kim, D.; Kim, Y.; Song, W.; Kim, H.; Kim, Y.; Kim, S.; and Park, C. 2024. sdpo: Don't use your data all at once. arXiv preprint arXiv:2403.19270

2024 arXiv
[19]

Li, H.; Huang, Y.; Gong, Y.; Jiao, J.; Zhang, R.; Baldwin, T.; and Duan, N. 2022. CULG: Commercial universal language generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: ...

2022
[20]

J.; and Wang, X

Liu, T.; Qin, Z.; Wu, J.; Shen, J.; Khalman, M.; Joshi, R.; Zhao, Y.; Saleh, M.; Baumgartner, S.; Liu, J.; Liu, P. J.; and Wang, X. 2025. LiPO: Listwise Preference Optimization through Learning-to-Rank. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Proceedings of the 2025 C...

2025
[21]

Mishra, S.; Verma, M.; Zhou, Y.; Thadani, K.; and Wang, W. 2020. Learning to create better ads: Generation and ranking approaches for ad creative refinement. In Proceedings of the 29th ACM international conference on information & knowledge management, 2653--2660

2020
[22]

Mita, M.; Murakami, S.; Kato, A.; and Zhang, P. 2024. Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation. In Ku, L.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V...

2024
[23]

Murakami, S.; Hoshino, S.; and Zhang, P. 2023. Natural language generation for advertising: A survey. arXiv preprint arXiv:2306.12719

2023 arXiv
[24]

Murakami, S.; Zhang, P.; Hoshino, S.; Kamigaito, H.; Takamura, H.; and Okumura, M. 2022. Aspect-based Analysis of Advertising Appeals for Search Engine Advertising. In Loukina, A.; Gangadharaiah, R.; and Min, B., eds., Proceedings of the 2022 Conference of the North American C...

2022
[25]

Murakami, S.; Zhang, P.; Kamigaito, H.; Takamura, H.; and Okumura, M. 2025. AdParaphrase: Paraphrase Dataset for Analyzing Linguistic Features toward Generating Attractive Ad Texts. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Findings of the Association for Computational ...

2025
[26]

L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P

Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P. F.; Leike, J.; and Lowe, R. 2022. Training language mo...

2022
[27]

Pal, A.; Karkhanis, D.; Dooley, S.; Roberts, M.; Naidu, S.; and White, C. 2024. Smaug: Fixing failure modes of preference optimisation with dpo-positive. URL https://arxiv. org/abs/2402.13228

2024 arXiv
[28]

D.; Ermon, S.; and Finn, C

Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Informat...

2023
[29]

J.; Marcheret, E.; Mroueh, Y.; Ross, J.; and Goel, V

Rennie, S. J.; Marcheret, E.; Mroueh, Y.; Ross, J.; and Goel, V. 2017. Self-critical sequence training for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7008--7024

2017
[30]

Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

2017 arXiv
[31]

Shao, H.; Wang, J.; Lin, H.; Zhang, X.; Zhang, A.; Ji, H.; and Abdelzaher, T. 2021. Controllable and diverse text generation in e-commerce. In Proceedings of the Web Conference 2021, 2392--2401

2021
[32]

Thomaidou, S.; Leymonis, K.; and Vazirgiannis, M. 2013. GrammAds: Keyword and ad creative generator for online advertising campaigns. In Digital Enterprise Design and Management 2013: Proceedings of the First International Conference on Digital Enterprise Design and Management...

2013
[33]

Thomaidou, S.; Lourentzou, I.; Katsivelis-Perakis, P.; and Vazirgiannis, M. 2013. Automated snippet generation for online advertising. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management, 1841--1844

2013
[34]

Wang, Y.; Huang, H.; Yan, Y.; and Liu, X. 2019. Quality-Sensitive Training! Social Advertisement Generation by Leveraging User Click Behavior. In Liu, L.; White, R. W.; Mantrach, A.; Silvestri, F.; McAuley, J. J.; Baeza - Yates, R.; and Zia, L., eds., The World Wide Web Confer...

2019
[35]

K.; Ramnath, K.; Chaudhuri, S.; Mehrotra, S.; Mao, X.-B.; Asur, S.; et al

Wang, Z.; Bi, B.; Pentyala, S. K.; Ramnath, K.; Chaudhuri, S.; Mehrotra, S.; Mao, X.-B.; Asur, S.; et al. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216

2024 arXiv
[36]

Wei, P.; Yang, X.; Liu, S.; Wang, L.; and Zheng, B. 2022. CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuning. In Loukina, A.; Gangadharaiah, R.; and Min, B., eds., Proceedings of the 2022 Conference of the North American Ch...

2022
[37]

Wu, J.; Xie, Y.; Yang, Z.; Wu, J.; Gao, J.; Ding, B.; Wang, X.; and He, X. 2024.  -DPO: Direct Preference Optimization with Dynamic  . In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J. M.; and Zhang, C., eds., Advances in Neural Informatio...

2024
[38]

Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Wei, H.; Lin, H.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Lin, J.; Dang, K.; Lu, K.; Bao, K.; Yang, K.; Yu, L.; Li, M.; Xue, M.; Zhang, P.; Zhu, Q.; Men, R.; Lin, R.; Li, ...

2024 arXiv
[39]

Yang, H.; Yuan, J.; Yang, S.; Xu, L.; Yuan, S.; and Zeng, Y. 2024 b . A new creative generation pipeline for click-through rate with stable diffusion model. In Companion Proceedings of the ACM Web Conference 2024, 180--189

2024
[40]

Youngmann, B.; Yom-Tov, E.; Gilad-Bachrach, R.; and Karmon, D. 2020. The automated copywriter: Algorithmic rephrasing of health-related advertisements to improve their performance. In Proceedings of the Web Conference 2020, 1366--1377

2020
[41]

Zhang, C.; Zhou, J.; Zang, X.; Xu, Q.; Yin, L.; He, X.; Liu, L.; Xiong, H.; and Dou, D. 2021. CHASE: Commonsense-enriched advertising on search engine with explicit knowledge. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management

2021
[42]

M.; Stiennon, N.; Wu, J.; Brown, T

Ziegler, D. M.; Stiennon, N.; Wu, J.; Brown, T. B.; Radford, A.; Amodei, D.; Christiano, P.; and Irving, G. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593

2019 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

G.; Guo, Z

Azar, M. G.; Guo, Z. D.; Piot, B.; Munos, R.; Rowland, M.; Valko, M.; and Calandriello, D. 2024. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, 4447--4455. PMLR

2024

[4] [4]

Bartz, K.; Barr, C.; and Aijaz, A. 2008. Natural language generation for sponsored-search advertisements. In Proceedings of the 9th ACM Conference on Electronic Commerce, 1--9

work page 2008

[5] [5]

Y.; Golobokov, K.; Zhu, C.; and Liu, Y

Chai, J.; Pryzant, R.; Dong, V. Y.; Golobokov, K.; Zhu, C.; and Liu, Y. 2022. Fast: Improving controllability for text generation with feedback aware self-training. arXiv preprint arXiv:2210.03167

arXiv 2022

[6] [6]

Chen, X.; Feng, W.; Du, Z.; Wang, W.; Chen, Y.; Wang, H.; Liu, L.; Li, Y.; Zhao, J.; Li, Y.; et al. 2025. CTR-Driven Advertising Image Generation with Multimodal Large Language Models. In Proceedings of the ACM on Web Conference 2025, 2262--2275

work page 2025

[7] [7]

F.; Leike, J.; Brown, T

Christiano, P. F.; Leike, J.; Brown, T. B.; Martic, M.; Legg, S.; and Amodei, D. 2017. Deep Reinforcement Learning from Human Preferences. In Guyon, I.; von Luxburg, U.; Bengio, S.; Wallach, H. M.; Fergus, R.; Vishwanathan, S. V. N.; and Garnett, R., eds., Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Proces...

work page 2017

[8] [8]

Duan, S.; Li, W.; Cai, J.; He, Y.; and Wu, Y. 2021. Query-variant advertisement text generation with association knowledge. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 412--421

work page 2021