Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds

Chuan Yu; Dagui Chen; Fei Pan; Han Li; Jian Xu; Junqi Jin; Jun Wang; Kun Gai; Lvyin Niu; Weinan Zhang

REVIEW 4 major objections 4 minor 25 references

Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes that business traffic allocation can affect organic traffic allocation in blended e-commerce feeds, and that a bidding policy trained on this Leverage effect can substantially increase a product's recommended organic…

desk verdict A plausible and novel Leverage mechanism backed by a real deployment, but the proposed HTLB algorithm's advantage is not yet convincingly demonstrated. read the letter →

arxiv 1908.06698 v1 pith:ED6ILRAW submitted 2019-08-19 stat.ML cs.LG

Dagui Chen , Junqi Jin , Weinan Zhang , Fei Pan , Lvyin Niu , Chuan Yu , Jun Wang , Han Li

show 2 more authors

Jian Xu Kun Gai

This is my paper · ORCID

classification stat.MLcs.LG

keywords onlineadvertisingorganictraffice-commerceproductfeedreinforcementlearningrecommendationsystemsbiddingstrategyLeveragemechanismDDPG

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most e-commerce feeds blend paid and recommended products selected from the same candidate pool, and both platforms train on the same consumer behavior logs. This paper tries to establish that the paid impressions a product wins are not isolated transactions: they feed into the recommendation model's training data and thereby shift the product's future organic, free traffic. It then argues that advertisers can treat this Leverage effect as a controllable resource and learn bidding policies that maximize cumulative organic traffic, not just clicks or purchases. The proposed hybrid training algorithm is validated offline and in a live deployment, where the reported gap in organic traffic growth between optimized and baseline buckets reaches tens of percentage points within days.

What carries the argument

The load-bearing object is the Leverage dynamic model built from two functions: the traffic-win function T, mapping a product's average recommended score to its organic exposure, and the exposure-effect function U, mapping exposure back to the next recommended score; composed, they form a Markov chain whose stable point is the product's organic traffic under fixed conditions. On top of this, the paper defines a Partially Known MDP (PKMDP) in which the state is $[o, x]$, where $o$ is the advertisement-related state, simulable via an advertising emulator $f_{ad}$, and $x$ is the recommendation-related state, unknown but independent of the current bidding action. The HTLB transition expansion exploits that independence to synthesize transitions $\langle[o_j, x_j], a_j^{(m)}, r_{j+1}, [f_{ad}([o_j, x_j], a_j^{(m)}), x_{j+1}]\rangle$ for $M$ exploratory actions, sharing the real $x_{j+1}$, which is what cuts sample complexity.

What would settle it

Run the bidding policy in a live feed while randomly perturbing bid adjustments within a decision window and measure whether recommendation-side exposure or predicted scores move before the next scheduled model update; if same-window changes appear, the action-independence assumption fails, and if no changes appear until the update, it holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that business traffic allocation can affect organic traffic allocation: when a product is advertised, the extra consumer feedback changes the historical data the recommendation platform retrains on, which changes the product's recommended score and hence its organic traffic. The authors first document the phenomenon in real feed data, showing that most advertised products gain organic traffic during an ad campaign and some keep the gain after the campaign ends, including products that escape the recommendation system's cold start. They then formulate the optimization as a Markov decision process whose state decomposes into a model-known advertising state and an action-independent recommendation state, and propose HTLB-DDPG, a hybrid training algorithm that expands experience replay with emulator-simulated bids while reusing the real next recommendation state. Offline learning curves and an online A/B deployment with over a thousand products support the claim that the optimized bidding policy increases leveraged organic traffic, with the reported gap in relative organic traffic increments between treated and control buckets reaching 46.64% to 64.71% at Level 1 by 20180925.

Load-bearing premise

The hybrid training method assumes that the recommendation platform's state one day later does not depend on today's bid adjustment, so one observed next state can be reused for many simulated bids; if the recommender reacts to ad delivery within the same day, the expanded training samples are invalid.

Editorial extensions

If this is right

Advertisers can optimize organic traffic as a direct objective rather than treating it as an unresponsive externality.
Because organic traffic is free, gains from the optimized policy translate into added exposure without additional media cost.
Spending more on ads is not enough: the comparison of minimum and maximum bid ratio policies shows that allocation, not volume, determines organic gains.
The HTLB training scheme is portable: applying it to A2C also improves learning speed and stability, suggesting any model-free RL bidder can exploit the action-independence structure.
If the Leverage mechanism operates in other blended feeds, the same bidding-as-optimization perspective should transfer to search result pages and social feeds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One implication the paper leaves implicit: if paid exposure systematically improves organic scores, the auction itself becomes a driver of recommendation quality, so a platform's choice of which exposures enter the shared training data will partly determine which products get recommended.
A testable extension would be to replace the one-day action-independence assumption with lead-lag analysis: compare recommendation-score changes under randomized bid shocks delivered at different times of day to find the true delay between ad delivery and recommender update.
The reported online gaps suggest a feedback flywheel: early wins in organic traffic feed the recommendation model, compounding later gains; measuring how long the gains persist after the policy stops would separate the flywheel from the direct effect.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, this paper is the first in the cited bidding/rec-ads literature to treat the recommender's training-data feedback loop as something an advertiser can deliberately exploit, and it backs that up with a real randomized A/B deployment on Taobao's Guess What You Like feed. The core observation—ad exposure changes user behavior, which becomes training data for the recommender, which shifts organic traffic—is worth taking seriously. Second, the headline algorithm HTLB-DDPG is not as well supported as the Leverage phenomenon itself. The offline advantage over plain DDPG is within noise, and the online deployment used CEM, not the proposed method.

What is genuinely good: the PKMDP decomposition is sensible. The advertising state is emulatable, the recommendation state is not, and the two-step action-independence assumption, if true, gives a clean Dyna-style hybrid sample expansion. The observational analysis of 1,000 products, showing that organic gains persist after ads stop, is a nice real-data observation. The appendix includes a second online experiment with a different date range, which helps.

The soft spots are real and mostly center on the transition expansion in Eq. 6. Each real transition contributes one x_{j+1} that gets attached to M counterfactual actions. That is only valid if a day's bidding cannot affect the recommendation state within the same day. The paper asserts this in Sec. 5.1 and gives the two-time-step story, but the Sec. 4 analysis is daily levels, not lags. If same-session clicks or rapid model refreshes matter, the hybrid samples are not counterfactuals and HTLB's faster convergence is an artifact of a false model. The offline simulator embeds the fitted exposure-effect function U_i, so it cannot detect this misspecification. Also, Table 2 shows overlapping error bars (HTLB 257,755 ± 11,589 vs. DDPG 228,610 ± 49,138), and the online A/B gap jumps around before stabilizing, with no significance tests or confidence intervals.

These are addressable, not fatal. The central Leverage claim is not defined into existence: the randomized A/B split shows a large, sustained positive gap after deployment, which is strong evidence that ad bidding does influence organic traffic. The HTLB advantage, though, needs better evidence.

Who is this for: anyone working on ad-organic interplay, e-commerce feed design, or industrial RL. It deserves a serious referee. My recommendation: accept with major revisions requiring (i) evidence on the lag between ad delivery and recommendation model updates, (ii) statistical tests or confidence intervals for the online A/B, and (iii) an offline evaluation that does not rely on the same fitted U_i used in training. The authors are at Alibaba; that data should be available to them.

Referee Report

4 major / 4 minor

Summary. This paper studies blended advertising and recommendation feeds in e-commerce. The authors argue that consumer behavior on advertised items becomes training data for the recommendation model, so the advertising platform's traffic allocation influences the recommendation platform's organic traffic allocation, a mechanism they call Leverage. They analyze Taobao 'Guess What You Like' data to document organic-traffic changes before, during, and after advertising, then formulate the advertiser's bidding problem as a Partially Known MDP whose state is split into advertisement-related and recommendation-related components. To reduce sample complexity, they propose Hybrid Training Leverage Bidding (HTLB), which expands each real transition by simulating advertisement-related next states with an advertising emulator while reusing the observed recommendation-related next state. Offline experiments compare HTLB-DDPG with DDPG, A2C, CEM, and manual bids inside an exposure-effect simulator; an online A/B test compares a CEM-based deployed policy with the baseline. The paper claims faster convergence, better stability, and superior traffic increments.

Significance. If the central claims hold, the paper opens a new and practically important axis for bidding optimization in blended feeds: advertisers can strategically acquire organic traffic through the shared behavioral-data loop. The real randomized online A/B test is a genuine strength and provides field evidence that Leverage exists and that bidding policy can shift organic traffic; this is valuable regardless of the RL algorithm used. The state-decomposition and hybrid-sample idea is interesting and, if the action-independence assumption can be validated, could be a useful general technique for partially known MDPs. However, the headline claim of HTLB's superiority is not yet strongly supported: the offline simulator is driven by a fitted exposure-effect function, HTLB's advantage over DDPG lies within overlapping error bars, and the online deployment uses CEM rather than HTLB-DDPG. These gaps are load-bearing for the paper's main algorithmic contribution.

major comments (4)

[§5.1, Eq. (6)] Equation (6) expands each real transition by attaching the same observed recommendation-related next state x_{j+1} to M emulated actions. This is valid only if x_{j+1} is independent of the day's bidding action a_j. The paper asserts in §5.1 that it requires at least two time steps for the bidding action to influence the recommendation-related state, but no lag analysis is reported; §4's observational comparison is at the level of before/during/after advertising averages, not a lagged dependence test. The assumption is especially concerning because the offline time step is one day (§6.1) whereas the online system uses a one-hour decision window (§6.6), so same-window feedback is more plausible in deployment. I ask for direct evidence of the lag structure, for example cross-correlation between exposure and next-period organic scores, and a sensitivity analysis in which x_{t+1} is allowed to depend on a_t or on exposure within the window.
[§6.1, Table 2] Table 2 reports converged episode organic-traffic increments of 257,755±11,589 for HTLB-DDPG and 228,610±49,138 for DDPG. The two are well within each other's error bars, and no significance test is reported, so the superior performance of HTLB over DDPG is not established by these numbers. Moreover, the offline environment embeds the non-parametrically fitted exposure-effect function U_i, so every algorithm, including the manual baseline, is evaluated under a model trained on the same product data. This makes the comparison unable to detect misspecification of U_i. Please add out-of-sample predictive checks for U_i, report paired or standard errors over the five runs, and state explicitly what portion of the HTLB advantage is attributable to the emulator expansion rather than to the fitted model.
[§6.6, Tables 4-5] The online A/B test uses the CEM algorithm, not HTLB-DDPG, and additionally changes the decision window from one day to one hour, clips bid adjustments, and adds auxiliary rewards. Table 5 therefore provides evidence that a Leverage-aware bidding policy can increase organic traffic over the baseline, but it does not validate the HTLB training method or its claimed offline advantage. The abstract and Section 7 say the online deployment demonstrates superior performance of the approach; this overstates what the deployment shows. Please either deploy HTLB online or clearly separate the claims: the online experiment validates the Leverage mechanism and the CEM policy, while HTLB remains validated only through the offline simulator.
[§4.2, Eq. (1)] Assumption 2 is the load-bearing Markov assumption z_{t+1}=U_i(p_t) that the exposure-effect function depends only on the current period's exposure. The paper motivates it with the qualitative observation in Figure 3b but presents no estimate of U_i or its lag structure in the main text; the offline experiments then use a fitted U_i as ground truth. Given that this assumption is the mechanism connecting business traffic to organic traffic, the paper should at least provide an empirical check that a one-lag Markov model predicts held-out organic scores no worse than models with longer lags or with additional state, such as recent ad exposure.

minor comments (4)

[§4.1, Table 1] The text refers to Table 6 but the table caption says Table 1, and the column counts are hard to parse; renumber tables consistently and clarify that the last column counts only the 302 items with a significant af-ad drop.
[§6.6, Table 5] The entry '49..57%' for 20180925, Level 2 should read '49.57%'.
[Figure 7] The axis labels of Figure 7 are duplicated and misaligned; give each panel its own clearly labeled x- and y-axis.
[§4.1] Please define the stable-traffic selection threshold used to choose the 1,000 products; the variance threshold is mentioned but its value is not given.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the Leverage effect is independently supported by observational and online A/B evidence, while the fitted U_i simulator and the action-independence lag are disclosed assumptions rather than definitional reductions.

full rationale

The central claim is not defined into existence. Section 4.1 compares 1,000 real products across be-ad/wh-ad/af-ad stages (Table 6), and Section 6.6 reports a randomized online A/B test with a real bucket split (Table 5); these provide independent evidence for the Leverage mechanism. The offline evaluation (Section 6.1) fits an exposure-effect function U_i from real multi-day data and embeds it in the simulator, and the paper explicitly states 'we cannot get the exposure effect to respond to different delivered results,' so this is a disclosed modeling limitation rather than a hidden prediction; all algorithms are compared inside the same simulator. The HTLB transition expansion (Eq. 6) depends on the stated assumption that recommendation-related state x_{t+1} is action-independent within one time step; if that lag assumption is false, the offline gains would be artifacts, but this is a correctness risk, not a circular reduction to the paper's inputs. The paper also notes the online deployment uses CEM rather than HTLB-DDPG ('Considering the limitation of Taobao's online engine...'), so the specific convergence benefit of HTLB is not confirmed online, which is an incomplete-validation gap, not circularity. Self-citations ([11], [25]) are background bidding formulations and are not load-bearing for the Leverage claim. No equation was found to be equivalent to its input by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The main unstated costs are: the Markov/action-independent decomposition of the recommendation state; the assumption that the fitted exposure-effect simulator is a valid environment for evaluating counterfactual policies; and the shared-training-data premise behind Leverage. These are not standard math; they are domain or ad hoc assumptions that the paper does not independently validate.

free parameters (6)

Per-product exposure-effect function U_i = Non-parametric curves fitted from real multi-day Taobao logs (Sec. 6.1)
Embedded into the offline evaluation flow to simulate how exposure changes the next recommended score; every offline reward depends on this fitted function.
Stable-traffic selection threshold = Not numerically specified, described as 'variance less than a threshold'
Used to choose the 1,000 products in Sec. 4.1 and the 186 products in Sec. 6.1; changing it changes the observational and offline conclusions.
Episode length H = 7 time steps, one day each
Set in Sec. 6.1; directly defines the episodic return in Eq. (2) and the amount of delayed reward visible to the agent.
Discount factor gamma = 0.9
Set in Sec. 6.1; changes the weight of future leveraged traffic in the learning objective.
Reward weights eta_k = 1 for all k
Chosen in Sec. 6.1 to maximize total increment; alternate weights change the objective as noted in Sec. 5.1.
Transition expansion count M = 10
Hyperparameter of HTLB (Algorithm 1 line 10); controls how many simulated transitions are added per real transition and is central to the claimed sample-complexity reduction.

assumptions (5)

domain assumption Assumption 1 (traffic-win function): under stable conditions, organic traffic p_t is a nonlinear, monotonically increasing function T(z_t) of the average recommended score, with a threshold below which no organic traffic is obtained.
Sec. 4.2; no empirical estimate of T is provided; the shape with a threshold and cold-start point is imposed to explain observed cases.
ad hoc to paper Assumption 2 (Markov exposure-effect): the next average recommended score z_{t+1} depends only on the current exposure p_t, so z_{t+1}=U(p_t) and the exposure-effect satisfies the Markov property.
Sec. 4.2 Eq. (1); the Markov property is asserted without empirical validation and is questionable because recommender models are usually trained on multi-day histories.
domain assumption Action-independent recommendation state: the transition P_x(x' | [o,x], a) does not depend on the bidding action a_t; bidding changes only o_{t+1}, with x affected only after at least two time steps.
Sec. 5.1, Figure 4c and Eq. (6); load-bearing for HTLB's transition expansion, where x_{j+1} is reused for M simulated actions with different o' values.
ad hoc to paper Offline validity of fitted U_i: the non-parametrically fitted exposure-effect functions U_i in Sec. 6.1 correctly predict organic traffic responses under counterfactual bidding actions not present in the training data.
Used to build the offline evaluation flow; if the fitted simulators misrepresent real feedback, the offline comparisons in Table 2 do not transfer to the real system.
domain assumption Leverage premise: consumers' behaviors on advertised results are included in the recommendation model's training data, so delivered ads can change recommendation scores.
Sec. 1 and Figure 1; supported only by observational before/during/after comparisons and the online A/B test, not by direct inspection of the recommendation training pipeline.

invented entities (2)

Leverage mechanism independent evidence
purpose: Names the causal channel through which advertising traffic changes organic recommendation scores via shared consumer behavior data.
Online A/B test (Sec. 6.6) provides a falsifiable handle: organic traffic should diverge between treatment and control after the bidding algorithm is enabled.
Traffic-win function T and exposure-effect function U
purpose: Conceptual curves used to explain Leverage dynamics and to construct the offline simulator.
U is fit from real logs but T is only schematic; neither is validated against held-out counterfactual outcomes, so they are internal modeling constructs without independent evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds." pith.science (2026). https://pith.science/paper/ED6ILRAW

@misc{pith2026190806698,
  author       = {Pith},
  title        = {Pith review of: Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ED6ILRAW}},
  note         = {Machine review of arXiv:1908.06698}
}

read the original abstract

Most e-commerce product feeds provide blended results of advertised products and recommended products to consumers. The underlying advertising and recommendation platforms share similar if not exactly the same set of candidate products. Consumers' behaviors on the advertised results constitute part of the recommendation model's training data and therefore can influence the recommended results. We refer to this process as Leverage. Considering this mechanism, we propose a novel perspective that advertisers can strategically bid through the advertising platform to optimize their recommended organic traffic. By analyzing the real-world data, we first explain the principles of Leverage mechanism, i.e., the dynamic models of Leverage. Then we introduce a novel Leverage optimization problem and formulate it with a Markov Decision Process. To deal with the sample complexity challenge in model-free reinforcement learning, we propose a novel Hybrid Training Leverage Bidding (HTLB) algorithm which combines the real-world samples and the emulator-generated samples to boost the learning speed and stability. Our offline experiments as well as the results from the online deployment demonstrate the superior performance of our approach.

Figures

Figures reproduced from arXiv: 1908.06698 by the authors.

**Figure 2.** An overview of Guess What You Like of Taobao App. e displayed advertised products are tagged with “Hot” and surrounded by the recommended results. the high sample complexity, we introduce the HTLB algorithm to augment training samples, which achieves beer convergence and stability. 3 EVALUATION PLATFORM Without loss of generality, we choose one of the largest e-commerce platform in China, Taobao, as the testbed to… view at source ↗

**Figure 3.** e schematic diagram of the Leverage principle. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: e dierence between Leverage optimization [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 7.** Figure 7: Learning curves of algorithms with/without HTLB. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: e relative increment in organic trac obtained [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

**Figure 9.** Figure 9: e explanatory diagram for Table 6 [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

**Figure 10.** Figure 10: e architecture of our online deployment. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

**Figure 11.** Figure 11: e proportion of products with dierent organic trac changes. [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

[1]

Ashish Agarwal, Kartik Hosanagar, and Michael Smith. 2011. Sponsored Search: Do Organic Results help or hurt the Performance and under what conditions? (2011)

work page 2011
[2]

Ashish Agarwal, Kartik Hosanagar, and Michael D Smith. 2015. Do organic results help or hurt sponsored search performance? Information Systems Research 26, 4 (2015), 695–713

work page 2015
[3]

Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, and Defeng Guo. 2017. Real-Time Bidding by Reinforcement Learning in Display Advertising. In 10th WSDM. ACM, 661–670

work page 2017
[4]

Qingpeng Cai, Aris Filos-Ratsikas, Pingzhong Tang, and Yiwei Zhang. 2018. Reinforcement Mechanism Design for e-commerce. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Commi/t_tee, 1339–1348

work page 2018
[5]

Colin Campbell and Lawrence J Marks. 2015. Good native advertising isn/f_it a secret. Business Horizons 58, 6 (2015), 599–606. Conference’17, July 2017, Washington, DC, USA Dagui and Junqi et al

work page 2015
[6]

Cristian Danescu-Niculescu-Mizil, Andrei Z Broder, Evgeniy Gabrilovich, Vanja Josifovski, and Bo Pang. 2010. Competing for users’ a/t_tention: on the inter- play between organic and sponsored search results. In Proceedings of the 19th international conference on World wide web . ACM, 291–300

work page 2010
[7]

David S Evans. 2009. /T_he online advertising industry: Economics, evolution, and privacy. Journal of Economic Perspectives 23, 3 (2009), 37–60

work page 2009
[8]

Victor Gabillon, Mohammad Ghavamzadeh, and Bruno Scherrer. 2013. Approx- imate dynamic programming /f_inally performs well in the game of Tetris. In Advances in neural information processing systems . 1754–1762

work page 2013

Show all 25 references

[9]

Avi Goldfarb and Catherine Tucker. 2011. Online display advertising: Targeting and obtrusiveness. Marketing Science 30, 3 (2011), 389–404

2011
[10]

Yujing Hu, Qing Da, Anxiang Zeng, Yang Yu, and Yinghui Xu. 2018. Reinforce- ment Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application. arXiv preprint arXiv:1803.00710 (2018)

2018 arXiv
[11]

Junqi Jin, Chengru Song, Han Li, Kun Gai, Jun Wang, and Weinan Zhang. 2018. Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Adver- tising. arXiv preprint arXiv:1802.09756 (2018)

2018 arXiv
[12]

Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)

2015 arXiv
[13]

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Tim- othy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asyn- chronous methods for deep reinforcement learning. In ICML. 1928–1937

2016
[14]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529

2015
[15]

Claudia Perlich, Brian Dalessandro, Rod Hook, Ori Stitelman, Troy Raeder, and Foster Provost. 2012. Bid optimizing and inventory scoring in targeted online advertising. In 18th SIGKDD. ACM, 804–812

2012
[16]

David Silver, Guy Lever, Nicolas Heess, /T_homas Degris, Daan Wierstra, and Martin Riedmiller. 2014. Deterministic policy gradient algorithms. In ICML

2014
[17]

1998.Reinforcement learning: An intro- duction

Richard S Su/t_ton and Andrew G Barto. 1998.Reinforcement learning: An intro- duction. Vol. 1. MIT press Cambridge

1998
[18]

Istv´an Szita and Andr´as L¨orincz. 2006. Learning Tetris using the noisy cross- entropy method. Neural computation 18, 12 (2006), 2936–2941

2006
[19]

Yu Wang, Jiayi Liu, Yuxiang Liu, Jun Hao, Yang He, Jinghe Hu, Weipeng Yan, and Mantian Li. 2017. LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions. arXiv preprint arXiv:1708.05565 (2017)

2017 arXiv
[20]

Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, and Kun Gai. 2018. Budget constrained bidding by model-free reinforcement learning in display advertising. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management ...

2018
[21]

Lizhen Xu, Jianqing Chen, and Andrew Whinston. 2012. Eﬀects of the presence of organic listing in search advertising. Information Systems Research 23, 4 (2012), 1284–1302

2012
[22]

Sha Yang and Anindya Ghose. 2010. Analyzing the relationship between organic and sponsored search advertising: Positive, negative, or zero interdependence? Marketing Science 29, 4 (2010), 602–623

2010
[23]

Weinan Zhang, Shuai Yuan, and Jun Wang. 2014. Optimal real-time bidding for display advertising. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining . ACM, 1077–1086

2014
[24]

Ke Zhou, Miriam Redi, Andrew Haines, and Mounia Lalmas. 2016. Predicting pre- click quality for native advertisements. In Proceedings of the 25th International Conference on World Wide Web . International World Wide Web Conferences Steering Commi/t_tee, 299–310

2016
[25]

wh-ad vs. be-ad

Han Zhu, Junqi Jin, Chang Tan, Fei Pan, Yifan Zeng, Han Li, and Kun Gai. 2017. Optimized cost per click in taobao display advertising. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 2191–2200. A MORE EXPLANATION ABO...

2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Ashish Agarwal, Kartik Hosanagar, and Michael Smith. 2011. Sponsored Search: Do Organic Results help or hurt the Performance and under what conditions? (2011)

work page 2011

[2] [2]

Ashish Agarwal, Kartik Hosanagar, and Michael D Smith. 2015. Do organic results help or hurt sponsored search performance? Information Systems Research 26, 4 (2015), 695–713

work page 2015

[3] [3]

Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, and Defeng Guo. 2017. Real-Time Bidding by Reinforcement Learning in Display Advertising. In 10th WSDM. ACM, 661–670

work page 2017

[4] [4]

Qingpeng Cai, Aris Filos-Ratsikas, Pingzhong Tang, and Yiwei Zhang. 2018. Reinforcement Mechanism Design for e-commerce. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Commi/t_tee, 1339–1348

work page 2018

[5] [5]

Colin Campbell and Lawrence J Marks. 2015. Good native advertising isn/f_it a secret. Business Horizons 58, 6 (2015), 599–606. Conference’17, July 2017, Washington, DC, USA Dagui and Junqi et al

work page 2015

[6] [6]

Cristian Danescu-Niculescu-Mizil, Andrei Z Broder, Evgeniy Gabrilovich, Vanja Josifovski, and Bo Pang. 2010. Competing for users’ a/t_tention: on the inter- play between organic and sponsored search results. In Proceedings of the 19th international conference on World wide web . ACM, 291–300

work page 2010

[7] [7]

David S Evans. 2009. /T_he online advertising industry: Economics, evolution, and privacy. Journal of Economic Perspectives 23, 3 (2009), 37–60

work page 2009

[8] [8]

Victor Gabillon, Mohammad Ghavamzadeh, and Bruno Scherrer. 2013. Approx- imate dynamic programming /f_inally performs well in the game of Tetris. In Advances in neural information processing systems . 1754–1762

work page 2013