{"id":"a93d97d6-a866-4309-9120-14398111cb01","arxiv_id":"2501.02961","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A framework that casts repeated personalized action delivery as policy optimization over neural temporal point processes, with a proposed heavy-tailed event-time family.","lead":"This paper proposes a Bayesian decision-theoretic framework for repeatedly delivering personalized actions to users, modeling user behavior as marked temporal point processes. It also introduces a heavy-tailed distribution for event waiting times and sketches a policy-gradient procedure for optimizing delivery policies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The no-unobserved-confounding assumption in Section 2 is load-bearing, yet the paper's footnote 3 admits realistic violations; without a test, Algorithm 5's unbiasedness is unverified.","rationale":"I agree with the reader's weakest_assumption. The no-confounding condition is the first gate in the argument: Eq. (1) and Eq. (15) are only causal (rather than merely associational) if actions are assigned on the basis of logged information alone. Algorithm 5 inherits this requirement because the RNN is trained on observational logs and then used to simulate counterfactual action sequences. The paper itself signals the instability of the premise in footnote 3, and provides no data supporting the 'by construction' claim. I do not see a mathematical error in the TPP construction itself; the issue is an unvalidated premise on which the practical conclusion rests. Because the reader's CONDITIONAL verdict already reflects this gap, my stress-test does not move the verdict.","tokens_in":8745,"tokens_out":4829,"duration_ms":48309,"concrete_test":"Run a synthetic control experiment with a known generative model: a user latent variable U shifts event intensity and also influences a logging policy's action choice, but U is deliberately not included in the recorded log. Train the neural point process on these confounded logs, run Algorithm 5, and compare the achieved utility against the true optimal policy found by brute force in a small action space. Also run the same pipeline under an unconfounded logging policy that depends only on logged features. If the confounded variant does not converge to the true optimum while the unconfounded variant does, the central no-unobserved-confounding premise fails and the method needs an explicit causal correction or a restriction on admissible logging policies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central operational claim is that Algorithm 5's policy-gradient update is an unbiased estimator of expected utility. This depends on the neural point process learned from logs (Eq. 15) being a valid model of the interventional distribution of user events under arbitrary candidate actions. That validity requires conditional ignorability: action assignment must be independent of any unobserved variable that also affects future events, conditional on logged history H. The paper asserts in Section 2 that online interactive systems satisfy this 'by construction,' but footnote 3 immediately concedes that hierarchies of models with different features can cause unobserved confounding. Many deployed systems choose actions using latent user state, model scores, or context that is not fully written to the log. When such a variable exists, Eq. 15 estimates a confounded association rather than a causal effect, and Algorithm 5 maximizes the wrong expected utility. The manuscript offers no synthetic or real experiment that checks this premise. This is not a disagreement with [19,7,4] for its own sake; it is a missing validation of the exact condition under which the proposed optimization is correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Bayesian decision-theoretic framework for optimizing repeated personalized actions to users from logged interaction data. Section 2 formulates the likelihood and posterior over an abstract parameter θ, defines a policy πξ, and derives REINFORCE-style expected-utility maximization. Section 3 specializes observations to marked temporal point processes, with actions triggered by events, and introduces a three-parameter piecewise power-law family for inter-event times (Eqs. 10–11). Section 4 proposes recurrent neural networks to parameterize the event distributions and gives algorithms for likelihood computation (Algorithm 2), event sampling (Algorithms 3–4), and policy gradient optimization (Algorithm 5). The paper contains no experimental evaluation.","tokens_in":8968,"tokens_out":9969,"duration_ms":96451,"significance":"If its correctness conditions hold, the paper is a useful conceptual bridge between Bayesian decision theory and neural temporal point processes for repeated interventions. The probability derivations in Eqs. (1)–(11) and Algorithms 2–3 are internally consistent, and the censoring correction in Algorithm 2 is carefully handled. The contribution is, however, purely formal: no data, experiments, or code are provided, and the central causal assumption (no unobserved confounding) is asserted rather than validated. As a result, the paper's usefulness for the stat.ML audience is not yet demonstrated.","major_comments":[{"comment":"The unbiasedness of the policy-gradient update in Algorithm 5 depends critically on conditional ignorability: actions must be independent of any unobserved variable that also affects future events, conditional on the logged history. The paper asserts this is satisfied 'by construction' for online interactive systems, but footnote 3 immediately concedes that hierarchies of models accessing different features produce unobserved confounding. Since Eq. (15) is estimated from observational logs, any such hidden variable biases both the learned event model and the expected-utility gradient, so Algorithm 5 maximises the wrong objective. The manuscript provides no diagnostic, sensitivity analysis, or experiment to check this premise. I ask the authors to state the exact recorded-variable condition under which ignorability holds and to demonstrate, at least in a synthetic experiment with a hidden confounder, that the proposed estimator behaves as claimed.","section":"Section 2, Eq. (15), Algorithm 5"},{"comment":"The paper motivates the piecewise power-law family as a good model for real systems and as enabling tractable likelihood and sampling, but no real or synthetic data are used to support this modelling claim. The family is presented as a suggested example, yet the operational claims of the paper — that Algorithms 2–5 provide a practical solution — rest on this unvalidated choice. Please add at least one empirical demonstration, such as fitting the family to a standard point-process dataset or a simulation study with known ground truth, and discuss the non-smoothness and non-convexity that the authors themselves note.","section":"Section 3, Eqs. (10)–(11)"},{"comment":"The Bayesian formalism (posterior P(θ|D), Eq. (2)) underpins Algorithm 1, but Algorithms 4–5 instead input a single 'trained RNN R', with no discussion of whether θ is a point estimate or a posterior sample. The paper should clarify the status of Bayesian uncertainty in the proposed implementation; as written, the central 'Bayesian decision theory' claim and the actually presented algorithms are disconnected. Without such clarification, the expected utility optimized by Algorithm 5 is not obviously the Bayesian expected utility defined in Eq. (4).","section":"Section 2 vs Section 4"}],"minor_comments":[{"comment":"The phrase 'user-advertiser interaction in online advertiser' appears to be an editing error; 'online advertiser' should likely be 'online advertising', and the first sentence is grammatically incomplete.","section":"Abstract"},{"comment":"The distribution P(H_k|H_{0:k-1}, a_{1:k-1}) conditions on actions up to k−1, but the described sequence is H0, a1, H1, a2, H2, ..., so H_k is preceded by a_k; the conditioning set should be a_{1:k}, as is already used in Eq. (1) and the predictive distribution.","section":"Section 2, first display"},{"comment":"The variable eB is used both for the previous event and for the newly sampled event, and it is initialized with B=0; renaming it to e_prev or e_{B-1} would avoid confusion.","section":"Algorithm 4"},{"comment":"After defining the mark m as the pair m=(v,x), the text reduces marks to integers {1,...,M}; please clarify the encoding of categorical marks and feature vectors into this finite index set.","section":"Section 3"},{"comment":"Algorithms 1 and 5 leave λ, initialization, and stopping criteria as placeholders; this is acceptable at a high level, but for reproducibility the authors should specify at least one concrete default choice in each case.","section":"Algorithms 1 and 5"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the paper is a reasonably clear conceptual contribution, but it falls well short of the empirical standards expected in a stat.ML venue. The no-unobserved-confounding assumption is not a philosophical side point; it is the exact condition under which Algorithm 5 is unbiased, and the authors' own footnote 3 admits realistic violations. The heavy self-citation pattern (references [5], [11], [12]) in support of controversial claims about causality and propensity scores should be reviewed. I would consider acceptance only after a revision that provides a precise ignorability statement and at least one synthetic or real-data experiment demonstrating the behaviour of the proposed estimator."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this as a position paper with explicit algorithms, not as a new result. The formalism is exactly what the title says: it puts repeated personalized action delivery into the language of Bayesian decision theory, specializes to marked temporal point processes, and then uses REINFORCE for policy optimization. There is no new theorem and no experiment. The novelty is in the assembly, and on that front it is mostly clean and readable.\n\nWhat the paper does well: it spells out the likelihood with a censoring correction (Algorithm 2) carefully, including the no-event contribution. The heavy-tailed family in equations (10)-(11) is a simple piecewise power-law, but it is genuinely useful in this context because both the CDF and its inverse are elementary, so likelihood evaluation and sampling are cheap and exact. Algorithm 5 is a straightforward policy-gradient loop, and the paper is honest that many details — convergence, step size, posterior sampling — are left unspecified. In fact, the algorithms are littered with comments like \"to be specified\" and \"requires some explanation\", which is refreshing even if it undercuts the claim to a ready-to-use method.\n\nThe soft spot is the unobserved-confounding assumption. The paper asserts in Section 2 that online interactive systems satisfy conditional ignorability \"by construction\", and then footnote 3 immediately concedes that hierarchies of models with different features can create unobserved confounding. That is the exact condition on which Algorithm 5's unbiasedness rests. If actions depend on latent user state or on features not written to the log, then the learned point process is a confounded association, not an interventional model, and the policy gradient maximizes the wrong objective. The paper offers no synthetic or real check of this premise. That is not a fatal mathematical flaw, but for an applied position paper it is the load-bearing assumption, and it is waved through too quickly. I also agree with the reader that the heavy-tailed family is not justified on real delay data; the reference to [16] is suggestive but not evidence for this specific parameterization.\n\nAll that said, this paper deserves a serious referee. It addresses an important applied problem, the formalization is coherent, and the explicit algorithms give a referee something concrete to probe. My recommendation: send it to review, and push the authors either to add a small simulation study that varies the confounding strength and checks the policy-gradient bias, or to temper the \"by construction\" claim to a clearly stated assumption with conditions. As it stands, it is a promising framework missing its own validation.","headline":"A clean, honest synthesis of Bayesian decision theory, marked TPPs, and REINFORCE for repeated personalized actions, but the central confounding assumption is asserted rather than tested and there are no experiments or new theorems.","tokens_in":9464,"tokens_out":1561,"would_cite":false,"duration_ms":98161,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Repeated personalized action delivery is a Bayesian decision problem, and neural temporal point processes make it tractable.","keywords":["temporal point processes","neural point processes","Bayesian decision theory","policy gradient","causal inference","personalized action delivery","computational advertising","reinforcement learning"],"falsifier":"Find one deployed interactive system whose logging policy uses state not present in the dataset D, or show in logs that adding a previously unlogged covariate changes the estimated expected utility of a fixed policy; either would refute the claim that unobserved confounding is absent by construction.","tokens_in":8524,"feed_emoji":"🎯","tokens_out":7904,"duration_ms":67862,"temperature":0.7,"pith_summary":"The paper argues that the problem of repeatedly delivering personalized actions to users—whether ads, recommendations, or reminders—is best understood as Bayesian decision theory applied to a marked temporal point process. In this view, each user's history is a time-ordered event stream, and the system's actions are appended to special 'request for action' events. If the conditional distribution of the next event's time delay and mark is modeled by a recurrent neural network, the paper shows that likelihood-based training, forward simulation, and policy-gradient optimization all become concrete algorithms. The payoff is a unified way to optimize both when to intervene and what action to deliver, using only logged data and without requiring the do-calculus or propensity-score adjustment. The authors further claim that online interactive systems whose actions are chosen by a known logged policy satisfy the no-unobserved-confounding assumption by construction, which would make the policy-gradient estimate of expected utility unbiased.","feed_headline":"Repeated personalized actions become a Bayesian decision problem","feed_subtitle":"Neural point processes give tractable likelihoods, sampling, and policy gradients for when and what to deliver.","key_machinery":"The carrying mechanism is the conditional event distribution $P(\\tau, m \\mid \\text{history}, \\theta)$, factorized as $p(\\tau \\mid m, \\theta) Q(m \\mid \\theta)$, with a recurrent network mapping each history to the parameter vector $\\varphi$ of these distributions. The paper proposes a specific three-parameter heavy-tailed delay family (Equation 10) whose CDF and inverse CDF are elementary functions, so likelihood evaluation and inverse-CDF sampling are both cheap. On top of this generative model sits the policy $\\pi_\\xi(a_s \\mid a_{1:s-1}, H_{0:s-1})$, which at each 'request for action' event selects an action; the REINFORCE estimator $\\sum \\nabla_\\xi \\log \\pi_\\xi(f_k \\mid e_{1:k-1}) U(e)$ provides the gradient step for maximizing expected utility.","core_discovery":"The central claim is that the expected utility of a personalized action policy πξ can be written as an expectation under the Bayesian posterior predictive distribution over user event sequences, and that this expectation can be maximized by a REINFORCE-style stochastic gradient ascent using sequences simulated from a neural temporal point process. The paper's formal contribution is to show that the abstract model of Section 2—products of conditional observation distributions and actions—specializes to marked temporal point processes in which each event's probability is the product of a delay distribution and a mark distribution, conditioned on the entire history through an RNN. Within this specialization, Algorithm 2 computes the likelihood of a logged sequence, Algorithm 4 simulates future sequences under any policy, and Algorithm 5 performs the gradient update that optimizes the policy's parameters.","pith_inferences":["A natural extension is to continuous or structured action spaces, such as bid amounts or message text, by reparameterizing the policy distribution while keeping the point-process model unchanged.","The no-unobserved-confounding claim is empirically testable: if production logs contain auxiliary features not used by the logging policy, computing estimated utility with and without those features and finding divergence would indicate hidden confounding.","Stationarity may be the more restrictive assumption in practice, since user behavior drifts; extending the framework to non-stationary processes or online weight adaptation would be needed for long-running systems.","The proposed delay family is a design choice; swapping in a more flexible distribution, such as a normalizing flow or learned mixture, should preserve every algorithm while potentially improving fit."],"forward_implications":["Any repeated personalized intervention—ad display, recommendation, reminder—fits the same formalism, provided the user process is stationary, users do not interfere with each other, and the logging policy is known.","Offline policy evaluation and optimization can be carried out purely by simulating sequences from the trained point-process model, without online experimentation.","The closed-form delay distribution makes likelihood computation and event sampling computationally cheap, keeping both Bayesian posterior updating and policy-gradient iteration practical.","Under the no-unobserved-confounding assumption, propensity-score adjustment and do-calculus are unnecessary for this problem class, simplifying the estimation pipeline.","Timing and content of interventions are optimized jointly within one objective, rather than as separate subproblems."],"supporting_citations":[{"why":"Supplies the Bayesian decision-theoretic foundation that the whole formulation is built on.","marker":"[3]"},{"why":"Provides the SUTVA no-interference assumption that justifies treating users' histories as independent.","marker":"[14]"},{"why":"The REINFORCE rule on which the policy-gradient update in Algorithm 5 is based.","marker":"[20]"},{"why":"The neural temporal point process framework that the model class in Section 4 adopts.","marker":"[17]"},{"why":"Provides the intensity-free learning approach and the conjecture about heavy-tailed delays that motivate the paper's delay distribution family.","marker":"[16]"}],"fun_headline_variants":["Bayesian point process optimizes personalized action delivery","Neural point processes drive when-and-what delivery policies","Repeated actions? Learn the right delivery via Bayesian utility","Point process model turns user actions into decision problem","Optimize repeated personalized actions with neural point processes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that every action in the logs was chosen with probabilities determined only by the recorded history, so no hidden variable influences both the action and the user's later behavior; the authors assert this holds by construction for online interactive systems, but if a hidden variable exists, the policy-gradient estimate is biased.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian point process optimizes personalized action delivery","Neural point processes drive when-and-what delivery policies","Repeated actions? Learn the right delivery via Bayesian utility","Point process model turns user actions into decision problem","Optimize repeated personalized actions with neural point processes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000577,"raw_usage":{"total_tokens":2612,"prompt_tokens":724,"completion_tokens":1888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":340,"completion_tokens_details":{"reasoning_tokens":1813}},"tokens_in":340,"tokens_out":1888,"duration_ms":13589,"temperature":1.0,"reasoning_tokens":1813,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:59:25.463387+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find one deployed interactive system whose logging policy uses state not present in the dataset D, or show in logs that adding a previously unlogged covariate changes the estimated expected utility of a fixed policy; either would refute the claim that unobserved confounding is absent by construction.","supporting_citations":[{"cited_title":"de Finetti","cited_arxiv_id":null,"evidence_quote":"Supplies the Bayesian decision-theoretic foundation that the whole formulation is built on."},{"cited_title":"Comment: Which ifs have causal answers","cited_arxiv_id":null,"evidence_quote":"Provides the SUTVA no-interference assumption that justifies treating users' histories as independent."},{"cited_title":"Simple statistical gradient-following algorithms fo r connectionist reinforcement learning","cited_arxiv_id":null,"evidence_quote":"The REINFORCE rule on which the policy-gradient update in Algorithm 5 is based."}],"review_version":1}