{"id":"7799646a-0a5d-456f-8834-5076f83ad549","arxiv_id":"2507.05175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Bayesian-optimization-based 'strategic querying' method recovers nearly all non-privacy-preserving targeting value from limited, noisy aggregate queries in simulations and on Criteo data.","lead":"This paper develops a querying strategy that lets advertisers run targeted campaigns using only limited, noisy aggregate data from privacy-protected platforms. The method matches the performance of a state-of-the-art, full-data targeting algorithm in tests on a large ad dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 6 omits the inverse-volume normalization required for query averages, so the GP posterior and TAAF are miscalibrated and the reported 97-101% result is not supported by the formulas as written.","rationale":"The reader's weakest assumption concerned generalization of the smooth, low-dimensional GP prior to other datasets. I identified a more fundamental, internal issue: Equation 6 as written is not the covariance of the query averages defined in Equation 2. The formula omits the inverse-volume normalization and therefore gives a covariance that grows with region size instead of shrinking, which miscalibrates the posterior predictive and the acquisition function. This is a concrete, checkable mathematical error that directly affects the central claim because the empirical 97-101% result is obtained with TAAF without size restrictions, relying on the unnormalized kernel. If the normalization is missing, the method as described is not computing a valid GP posterior over the queried averages; if it was corrected in code, the paper's formulas and the simulation story about region-size penalties need revision. The concern is therefore load-bearing for the paper's internal consistency, not just for external generalizability. I do not recommend outright rejection because the empirical result could survive a corrected kernel, and the fix is straightforward; a conditional verdict requiring the normalization to be applied and results re-reported is appropriate.","tokens_in":19306,"tokens_out":9416,"duration_ms":115417,"concrete_test":"Re-derive the covariance of two query averages from Eq. 5: Cov( (1/(t-s)) * integral_s^t f(x)dx, (1/(t'-s')) * integral_{s'}^{t'} f(x')dx' ). The correct expression is Eq. 6 with an additional factor 1/((t-s)(t'-s')). For a single range [0,T] with T >> l, the normalized expression tends to 0 while Eq. 6 diverges. Then re-run the Criteo application in Section 6 with the normalized kernel and compare point estimates and confidence intervals in Table 2; a material change in the 97-101% figures would confirm that the reported result is an artifact of the misspecified covariance. A complementary check is to simulate a GP, generate noisy averages over two regions of different sizes, and compare the posterior predictive from Eqs. 7-8 (using Eq. 6) with the true conditional distribution from the normalized kernel.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 4.2.2 treats the query output in Eq. 2 as a conditional average over a hyperrectangle, and places a GP prior on the latent CATE surface tau(x). For a squared-exponential kernel k(x,x') = alpha exp(-(x-x')^2/l^2), the covariance of two noisy query averages over ranges (s,t) and (s',t') is (1/((t-s)(t'-s'))) * integral of k over the two ranges. Equation 6, however, gives k_FF without the 1/((t-s)(t'-s')) factor; it is the covariance of the integrals, not of the averages. Consequently, for a single 1-D interval [0,T] with T >> l, Eq. 6 implies the prior variance grows like T, whereas the variance of a true average tends to 0 as T grows. The posterior predictive formulas (Eqs. 7-8) and the TAAF acquisition function (Eq. 9) therefore use a misspecified covariance: large regions are systematically over-weighted, and the posterior mean is not the conditional mean of the queried averages. This explains the paper's own observation in Section 5.1 that 'regions that are too large naturally have a high variance' and why size penalties/constraints were needed in the simulation study; those features may be compensating for a kernel error rather than for a real property of the targeting task. The empirical application in Section 6.1 uses TAAF 'with no region size restrictions or penalty,' so the reported 97-101% of Causal Forest performance depends on the unnormalized kernel exactly as written. If the implementation actually normalizes the kernel, then the method run differs from the method described; if it does not, the posterior is not the posterior of the observed averages. Either way, the central methodological derivation in the paper is not internally consistent.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies personalized targeting when an advertiser can only interact with a platform's data through a limited number of noisy aggregate queries, as in third-party privacy-preserving interfaces. The author proposes a Bayesian optimization procedure, called strategic querying, that (i) uses integral GP updating to learn from queries over ranges rather than points, and (ii) introduces a targeting-aware acquisition function (TAAF) that prioritizes regions with high variance and near-zero posterior mean. The method is compared to a uniform-querying benchmark in simulations and on the Criteo uplift dataset, where strategic querying is reported to achieve 97–101% of the targeting value of a Causal Forest baseline across four privacy settings.","tokens_in":19642,"tokens_out":6779,"duration_ms":84079,"significance":"If the method performs as claimed, this is a practically important result: it would show that effective targeting can be retained under restrictive third-party privacy interfaces, and it would quantify how much of the non-private targeting value can be preserved with fewer than 50 noisy aggregate queries. The paper combines existing ideas from Gaussian process regression for binned data, Bayesian optimization, and policy-aware acquisition in a new application, and it evaluates against an external benchmark (Causal Forest) on a large public dataset. The simulation study systematically varies data-generating parameters and privacy levels, which is a useful contribution. However, the correctness of the GP aggregate-query covariance is central to the method, and the current manuscript contains a concrete specification error in that component; the empirical headline therefore needs to be re-established after the correction.","major_comments":[{"comment":"The quantity called the 'kernel for averages over a range' is the covariance of the integrals of the GP over the two ranges, not the covariance of the averages. For query outputs defined as averages in Eq. (2), the covariance should include the factor 1/((t-s)(t'-s')). As written, for a single 1-D interval [0,T] with T >> l, Eq. (6) gives a prior variance that grows linearly in T, whereas the variance of a true average must tend to 0 as T grows. This misspecification inflates the posterior variance of large regions. That is exactly the behavior the paper later describes in Section 5.1 as 'regions that are too large naturally have a high variance' and then tries to counteract with size penalties and constraints. More importantly, Section 6.1 states that the empirical application uses TAAF 'with no region size restrictions or penalty,' so the reported 97–101% result relies on either the unnormalized covariance exactly as written or on an implementation that differs from the formula in the text. The authors should correct Eq. (6), re-run the simulation and empirical analyses with the normalized covariance, and clarify which covariance was actually used.","section":"Section 4.2.2, Eq. (6)"},{"comment":"The query output in Eq. (2) is a conditional average over the empirical distribution of X within the queried hyperrectangle, not an unweighted integral of the latent CATE surface. The GP formulation in Section 4.2.2 treats the query as an integral over a range with uniform weighting. This distinction matters in the Criteo application because f0 and f6 are not uniformly distributed and the 'third variable' is a sum of many covariates, so the conditional average is a density-weighted integral. The paper does not describe how the GP accounts for the covariate density inside a queried region. If the implementation ignores this weighting, the posterior predictive mean and variance are computed for a different object than the query actually returns. The authors should state the exact functional relationship between the query statistic and the GP latent function and, if density weighting is ignored, justify that approximation.","section":"Section 6.1, query model vs. GP prior"},{"comment":"The claim that strategic querying is 'statistically indistinguishable' from Causal Forest is much weaker than the paper's language suggests. The 95% confidence intervals for the ratio to Causal Forest are extremely wide, for example (38%, 151%), (50%, 166%), (58%, 152%), and (49%, 164%) in the four settings. These intervals are consistent with large differences in either direction, so they provide little evidence of equivalence. The phrase '97–101% of Causal Forest performance' should be presented as a point estimate with the associated uncertainty clearly emphasized, and the statistical-indistinguishability claim should be qualified accordingly.","section":"Section 6.2, Table 2"},{"comment":"The empirical analysis collapses 12 covariates into three: f0, f6, and the sum of all remaining variables. This is justified only by the observation that f0 and f6 have variation across the 25th, 50th, and 75th percentiles in Table 1. The choice is consequential because the reported 97–101% result may depend on the fact that the informative heterogeneity is concentrated in exactly these two variables. The paper does not report any robustness check with respect to the variable collapse or a comparison against using a different subset of covariates. Without such evidence, the generality of the empirical conclusion—that targeting is an information-light task—is not established.","section":"Section 6.1, variable collapse"}],"minor_comments":[{"comment":"The differential privacy mechanism is described only through a noise scale that is inversely proportional to the square root of the number of affected users. Please clarify how the constant s relates to a formal DP guarantee (e.g., epsilon and delta) and why this mechanism is appropriate for the range queries used here.","section":"Section 3, Eq. (4)"},{"comment":"The pairwise dominance matrix is difficult to read because the caption does not fully explain the construction of the rows and columns. In particular, the text should define what 'dominated' means in terms of the 95% threshold and should explain why the diagonal entries are zero.","section":"Section 5.1, Figure 9"},{"comment":"The description of GP hyperparameter estimation says MLE starts at step 10 with fixed values before that, and that the next region is chosen randomly among the top five TAAF values. Please report the sensitivity of the empirical results to these implementation choices, because they can affect the number of effective queries and the exploration-exploitation balance.","section":"Section 6.1, hyperparameters"},{"comment":"The TAAF weighting parameter beta is set to 3 - i/100 in the application, but the scale of beta relative to the posterior variance is not discussed. Since the covariance misspecification in Eq. (6) directly affects the magnitude of V, the interpretation of this schedule is unclear. This should be clarified after the normalization issue is resolved.","section":"Section 4.2.3, Eq. (9)"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is plausible and the application is relevant, but the paper currently does not provide code or a precise implementation description. Because the reported 97–101% figure hinges on whether the aggregate covariance is normalized as in Eq. (6) or normalized secretly in the code, I would strongly encourage the editor to request a corrected derivation, a re-run of the results with the corrected covariance, and open implementation details before accepting the headline claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper has a genuinely useful idea and a strong empirical demonstration, but the current formulas do not support the headline claim. The central derivation appears to be wrong: Eq. 6 is the covariance of integrals over ranges, not the covariance of the averages that the query interface actually returns. For a single interval [0,T] with T much larger than the lengthscale, Eq. 6 gives prior variance growing like T, while the variance of a true average goes to zero. So the posterior predictive (Eqs. 7-8) and the TAAF acquisition function are built on the wrong object. This also explains why the simulation study needed size penalties and constraints — those are compensating for a misspecified kernel, not for a real property of the targeting task. And Section 6.1 uses TAAF with no size restriction, so the 97-101% number is either produced by an implementation that silently normalizes the kernel (meaning the paper describes a different method) or by an unnormalized kernel (meaning the posterior is not the posterior of the observed averages). Either way, the methodological core is internally inconsistent.\n\nThat is a serious flaw, but fixable. Add the 1/((t-s)(t'-s')) factor to Eq. 6, re-run the simulations, and re-check the empirical numbers. I would be surprised if all qualitative findings flip — targeting is information-light and the GP smoothing is doing real work — but the percentages could change and the size-penalty story needs to be re-told.\n\nWhat is genuinely new: the combination of Bayesian optimization with integral updates over hyperrectangles, and the targeting-aware acquisition function that focuses queries where the sign of the treatment effect is most uncertain. The simulation design is thorough (144 settings, multiple acquisition functions, pairwise dominance analysis). The Criteo application is a sensible external benchmark with proper IPW evaluation and bootstrap confidence intervals. The paper is also honest about the settings where strategic querying does not help.\n\nThe other soft spot is generalization: the headline result depends on a smooth, low-dimensional treatment-effect surface and a specific covariate collapse. That is a real concern but secondary to the normalization issue.\n\nWho is this for: marketing and applied ML researchers working on privacy-preserving targeting, and anyone designing query interfaces for ads platforms. It deserves a serious referee, because the problem is important and the design pattern is valuable. But I would not accept the current version; I would send it back with a request to fix the kernel, release code, and re-report the results.","headline":"Worth reading and worth fixing: the targeting idea is good and the empirical results are striking, but Eq. 6 is the covariance of integrals, not averages, so the core posterior and the headline numbers rest on a math error as written.","tokens_in":20210,"tokens_out":6234,"would_cite":false,"duration_ms":69731,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62G08"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that strategic querying—an adaptive Bayesian-optimization method that learns from noisy aggregate queries—can design targeting policies that capture 97-101% of the value of a state-of-the-art method with access to all…","keywords":["third-party privacy","targeting","Bayesian optimization","Gaussian processes","differential privacy","heterogeneous treatment effects","aggregate queries","uplift modeling"],"falsifier":"Re-run the Criteo empirical study without collapsing the twelve covariates to three, letting the query space include all available features, and check whether fewer than 50 noisy aggregate queries still achieve near-parity with a full-data Causal Forest; if the lift ratio drops materially below 97%, the central claim does not generalize beyond the paper's coarse query space.","tokens_in":19064,"feed_emoji":"🎯","tokens_out":9656,"duration_ms":99278,"temperature":0.7,"pith_summary":"This paper asks whether advertisers can still design effective targeting policies when platforms stop giving access to raw individual-level data and instead offer only a limited number of noisy aggregate queries. It develops a method named strategic querying, which adaptively chooses which regions of the data to ask about based on what is most useful for the targeting decision, and tests it in simulations and on the large-scale Criteo uplift dataset. The central claim is that strategic querying retains 97-101% of the targeting value of Causal Forest, a state-of-the-art method that sees all individual records, across four privacy settings, while using fewer than 50 noisy averages instead of 600,000 raw data entries. A simple uniform querying benchmark drops to 33% of that value in one setting. The paper concludes that privacy protections and data-driven personalization are not inherently incompatible, because targeting mostly requires the sign of treatment effects for fairly broad segments rather than precise individual-level estimates.","feed_headline":"Noisy aggregate queries hit 97-101% of full-data targeting value","feed_subtitle":"Strategic querying matches Causal Forest on 14M users with under 50 noisy averages instead of 600,000 raw entries.","key_machinery":"The load-bearing mechanism is the pair of Bayesian-optimization extensions the paper introduces. Integral updating extends Gaussian-process regression to observations that are averages over ranges rather than point values, using the closed-form kernel for integrals of a squared-exponential process, so the posterior predictive mean and variance can be evaluated for any candidate query region. The targeting-aware acquisition function, $\\text{TAAF}([\\underline{X},\\bar{X}]) = \\beta V([\\underline{X},\\bar{X}]) - |m([\\underline{X},\\bar{X}])|$, operationalizes the observation that an additional query has the largest expected value when the region's estimated treatment effect is near zero and uncertain; these are the regions where the optimal targeting decision is still in doubt. Size constraints on candidate regions are needed in simulations to prevent the algorithm from spending queries on overly broad or overly granular cells. The combination makes the query sequence adaptive and decision-aware, which is what distinguishes strategic querying from a fixed uniform grid.","core_discovery":"The paper's central discovery is that a targeting policy can be learned from a handful of carefully chosen aggregate queries without meaningfully losing value relative to full-data machine learning. Treating the unobserved individual-level treatment-effect surface as a Gaussian process, the method uses integral updating to form posteriors over averages of arbitrary query regions, and a targeting-aware acquisition function to select the next region: it queries where the posterior mean is close to zero and the posterior variance is high, because those are the regions whose treatment or control assignment could change. In the Criteo application, with twelve covariates collapsed to three and two query budgets (27 and 64) crossed with two differential-privacy noise scales, strategic querying achieves 97-101% of Causal Forest's inverse-propensity-weighted lift and is statistically indistinguishable from it, while uniform querying falls to 33% in the most restrictive setting. The empirical result is presented as evidence that the method works exactly where a non-adaptive benchmark fails: when the granularity of treatment effects is unknown and privacy noise is substantial.","pith_inferences":["The 97-101% result is demonstrated after collapsing twelve covariates to three; if the informative heterogeneity lives in the discarded individual variation of the other nine variables, the required query count would likely grow, and testing the method with the full covariate space on Criteo would show how much of the result depends on that collapse.","The same recipe—Bayesian optimization over aggregate queries with a decision-aware acquisition function—could be applied to adjacent marketing tasks such as pricing, segmentation, or best-arm selection by replacing the zero-crossing criterion with the relevant policy objective.","The paper's comparison implies a broader information-theoretic conjecture: a small number of noisy sign queries can recover a piecewise-constant targeting policy whose value is close to the full-data optimum when segments are broad, which could be tested directly by varying the number of segments in a synthetic policy and measuring the query count needed for parity.","Platforms could build the integral-updating Gaussian-process machinery into their query interfaces and let advertisers plug in custom acquisition functions, which would turn privacy-preserving clean rooms from passive reporting tools into active decision-support systems."],"forward_implications":["Advertisers can build effective targeting policies in privacy-restricted platforms without access to raw individual-level data.","The method's advantage over uniform querying is largest when the query budget is moderate and differential-privacy noise is nontrivial; uniform querying only suffices when the marketer knows the granularity of treatment effects and noise is low.","Targeting is an information-light task: recovering the sign of treatment effects for broad segments, rather than precise individual-level estimates, is enough for near-oracle policy value.","In simulations, the best variant of strategic querying is not dominated by any other method in any of the 144 settings, suggesting that the targeting-aware acquisition function with region-size penalties and constraints is the reliable configuration."],"supporting_citations":[{"why":"It supplies the closed-form Gaussian-process kernel for averages over ranges, which makes integral updating possible.","marker":"Smith et al. (2018)"},{"why":"It defines Causal Forest, the full-data state-of-the-art targeting method used as the empirical benchmark.","marker":"Athey et al. (2019)"},{"why":"It provides the Criteo uplift dataset of roughly 14 million users used in the empirical application.","marker":"Diemert et al. (2018)"},{"why":"It introduces Bayesian optimization, the adaptive-sampling framework that strategic querying builds on.","marker":"Kushner (1964)"},{"why":"It gives the Gaussian-process posterior predictive equations used for updating and prediction from noisy observations.","marker":"Williams and Rasmussen (2006)"},{"why":"It derives the expected value of querying a treatment effect and motivates the zero-mean, high-variance acquisition target.","marker":"Shchetkina and Berman (2024)"},{"why":"It defines differential privacy, the noise mechanism protecting the aggregate query outputs.","marker":"Dwork (2006)"},{"why":"It supports the inverse-propensity-weighting evaluation of targeting policies used to compare methods without winner's curse.","marker":"Simester et al. (2020)"}],"fun_headline_variants":["Strategic querying hits 97-101% of full-data targeting value","Careful queries match Causal Forest under privacy noise","Bayesian optimization adapts to aggregate private data","Few targeted queries beat uniform sampling for uplift","Integral posterior updating enables private targeting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline result depends on the treatment-effect surface being smooth enough and low-dimensional enough that a Gaussian-process prior over only three collapsed covariates can capture it from fewer than 50 noisy region averages.","fun_headline_variants_meta":{"raw":{"variants":["Strategic querying hits 97-101% of full-data targeting value","Careful queries match Causal Forest under privacy noise","Bayesian optimization adapts to aggregate private data","Few targeted queries beat uniform sampling for uplift","Integral posterior updating enables private targeting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000287,"raw_usage":{"total_tokens":1734,"prompt_tokens":1042,"completion_tokens":692,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":618}},"tokens_in":658,"tokens_out":692,"duration_ms":7181,"temperature":1.0,"reasoning_tokens":618,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:28:59.342397+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Criteo empirical study without collapsing the twelve covariates to three, letting the query space include all available features, and check whether fewer than 50 noisy aggregate queries still achieve near-parity with a full-data Causal Forest; if the lift ratio drops materially below 97%, the central claim does not generalize beyond the paper's coarse query space.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the Criteo uplift dataset of roughly 14 million users used in the empirical application."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It introduces Bayesian optimization, the adaptive-sampling framework that strategic querying builds on."},{"cited_title":"and Berman, R","cited_arxiv_id":null,"evidence_quote":"It derives the expected value of querying a treatment effect and motivates the zero-mean, high-variance acquisition target."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It defines differential privacy, the noise mechanism protecting the aggregate query outputs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supports the inverse-propensity-weighting evaluation of targeting policies used to compare methods without winner's curse."}],"review_version":1}