{"id":"95e5412b-e7e4-43df-accc-055441eacc6f","arxiv_id":"2506.23855","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper presents a differentially private methodology and a public synthetic dataset of Topics API traces that match real re-identification risk within one standard deviation on two attacks.","lead":"Google researchers created a synthetic, privacy-protected dataset of Topics API outputs based on statistics from over 100 million real Chrome users. The goal is to let outside researchers study the API's re-identification risk without access to proprietary browsing data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline re-identification match is validated only on the same four-week window used to estimate the statistics; the stationarity and single-transition assumptions that make it portable are the load-bearing, under-tested conditions.","rationale":"The paper's core contribution is a plausible DP post-processing pipeline: private two-week statistics, a differentiable mixture model, and an empirical re-identification comparison. The DP reasoning (Gaussian mechanism, sensitivity proofs, post-processing) is standard, and I do not see an internal privacy flaw. The strongest claim is narrower than the title: it says the released synthetic data reproduces re-identification risk under two specific attacks. The reported figure supports that for the studied 4-week period. The load-bearing condition is that the two weeks of training statistics can be extended to the full four weeks and beyond; this is exactly the stationarity assumption the authors flag in Section 7 and test only via Pearson correlations over the same four adjacent weeks. A correlation above 99.8% on marginal and adjacent-pair statistics is reassuring but does not by itself certify the joint distribution over 4-week traces, especially since q*_across is estimated from a single transition rather than from all transitions the model enforces. Thus the concern is not that the experiments are wrong; it is that the general claim has a narrower evidentiary base than stated. Since the reader already made the verdict CONDITIONAL on addressable details including this assumption, my read does not change the verdict.","tokens_in":16860,"tokens_out":14857,"duration_ms":173996,"concrete_test":"Collect a later, disjoint 4-week Topics API log from the same measurement pipeline (e.g., February 2025), compute the same DP statistics from its first two weeks, train the model, and compare the Hamming and asymmetric-Hamming re-identification risk of the generated traces against the real later 4-week traces using the Section 6.1 protocol. If either attack's synthetic mean moves outside 1 standard deviation of the later real mean, the reported match is specific to the original window and the stationarity assumption fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim (Section 6.1.3) is that synthetic traces match real re-identification risk within 1 standard deviation on the Hamming and asymmetric-Hamming attacks. The model is trained on DP statistics computed from only two weeks, D=(S0,S1) (Section 4.2). In particular, q*_across is estimated from a single observed transition (week 0 to week 1) and then imposed on every consecutive week pair of the generated r-week traces, while q*_single and q*_within are averaged over the two training weeks and imposed on all weeks. The only evidence that this extrapolation is valid is Appendix C, which reports Pearson correlations >= 99.8% among the four adjacent weeks of one December 2024-January 2025 window. That is a single, short, same-population check; it does not establish stationarity of higher-order joint structure or of user-level temporal persistence. Because the synthetic model is a mixture with conditional independence across weeks given the latent type, it can match all fitted marginals and adjacent-pair correlations while misrepresenting the joint distribution that drives trace-level re-identification. Section 7 itself flags the stationarity assumption as 'crucial for the validity of our results.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for releasing differentially private synthetic Topics API output traces. It computes a set of DP statistics (single-topic frequencies, within-week topic-pair frequencies, and across-week transition frequencies) from two weeks of real Chrome logs, fits a mixture-of-types generative model to those statistics via gradient-based optimization, and samples synthetic user traces from the fitted model. The central empirical claim is that re-identification risk measured on 4-week synthetic traces is within one standard deviation of the risk on real traces for two attacks from prior work (Hamming and Asymmetric Hamming). The paper also reports that an unconstrained statistic, the number of distinct topics over 4 weeks, has 89% Pearson correlation between real and synthetic data.","tokens_in":17104,"tokens_out":8413,"duration_ms":94694,"significance":"If the central claim holds, the paper provides a valuable community resource: a large-scale, publicly released synthetic dataset of Topics API outputs with explicit DP guarantees, enabling external researchers to replicate and audit prior re-identification studies that previously required proprietary data. Strengths include the open-source code and data release, a clean DP pipeline with stated epsilon/delta and sensitivity analysis, and two independent validation checks (the re-identification attack match and the unconstrained distinct-topics statistic) that go beyond reporting the training objective. The principal weakness is that the stationarity assumption, which the authors themselves flag as crucial, is validated only over four adjacent weeks of a single data collection window, and the independent validation is limited to two specific attacks and one unconstrained statistic.","major_comments":[{"comment":"The stationarity assumption is load-bearing: q*_single and q*_within are averaged over two training weeks, and q*_across is estimated from the single observed transition, then imposed on every week and every consecutive pair in generated r-week traces (Section 4.1). Section 7 states this assumption is 'crucial for the validity of our results.' The only direct evidence is Appendix C, which reports Pearson correlations >=99.8% among four adjacent weeks within one Dec 2024-Jan 2025 window. Those correlations concern the marginal/pairwise statistics, not the user-level joint distribution over traces, and the window is the same period from which the re-identification comparison is drawn. Please either add a validation on a temporally non-overlapping period (or with a longer horizon) or explicitly scope all portability claims to the measured four-week window and describe the released dataset accordingly.","section":"Section 4.1 / Appendix C / Section 7"},{"comment":"The central empirical claim is stated as 'within 1 standard deviation' but the text does not report the numeric means, standard deviations, or the number of users/trials for each attack. With only visual error bars over 10 trials, the reader cannot judge whether the agreement is meaningful or dominated by sampling noise; for the real data the 10 trials resample the same four-week dataset, so the error bars do not capture temporal variation. Please report exact re-identification rates and standard deviations for both attacks, and consider a formal comparison (e.g., confidence intervals for the difference or an equivalence test) rather than a one-standard-deviation overlap heuristic.","section":"Section 6.1.3 / Figure 6"},{"comment":"The description of the DP budget split is inconsistent: it states that 'the count vectors in f11 and f22 received 25% of this budget while the counts f22 received 50%,' which must be a typo for f12 in the second clause. The Gaussian noise scale (sigma) is not reported, so the realized noise level cannot be audited from the paper even though epsilon, delta, and sensitivity are given. Please correct the split and report the per-statistic noise scale or the exact mechanism configuration (e.g., the Plume pipeline parameters) used in the release.","section":"Section 4.2"}],"minor_comments":[{"comment":"The heading 'Differentially privacy' should be 'Differential privacy.'","section":"Section 4.2"},{"comment":"The sentence 'The proof of Section 5.1 essentially uses the inclusion-exclusion principle...' should reference Appendix A and be phrased as a proposition or claim rather than as a citation to a proof.","section":"Section 5.1"},{"comment":"Clarify what the 10 independent trials for the real data represent (resampling the 10,240 attack users versus resampling the 10-million-user population) and confirm that the synthetic trials include re-training the model.","section":"Section 6.1.3"},{"comment":"The 89% Pearson correlation is between the two probability-mass functions shown; please state the binning and sample sizes so the reader can interpret the correlation.","section":"Figure 5"},{"comment":"The statement 'There are 469^2 = 219,961 such statistics' is correct but would be clearer with the multiplication formatted consistently as 469^2.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The open data and code release is a genuine strength and will be useful to the community. My main concern is the narrow validation window for the stationarity assumption, which the authors themselves label as crucial; the end-to-end re-identification match partially mitigates this, but only for two attacks on one time window. The paper would be acceptable if the claims are scoped to the measured window or if a non-overlapping validation is added. The DP accounting typo and the missing noise scale should also be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid, useful paper. It gives the research community the first large-scale differentially private synthetic dataset of Topics API outputs, built from real Chrome logs of hundreds of millions of users, and it shows that two standard re-identification attacks behave about the same on the synthetic traces as on real data. The methodology is not revolutionary—it is the standard recipe of computing DP statistics, then fitting a parametric model by post-processing—but the application, the scale, and the actual data release make it a valuable contribution.\n\nWhat works well. The statistics design is thoughtful: single-topic frequencies, within-week pair co-occurrences, and across-week transitions capture the marginal and temporal structure that drives trace re-identification. The slot-and-type model with T=500 types is expressive enough to fit these statistics, and the sensitivity analysis for the Gaussian mechanism is standard and correct. The two independent validations carry real weight: the re-identification risk under Hamming and asymmetric-Hamming attacks is within one standard deviation of the real-data risk across ten trials, and the unconstrained 'number of distinct topics over four weeks' distribution has 89% Pearson correlation with the real data. These are not just convergence checks on the training objective; they are downstream quantities that the model was not explicitly fit to. The authors also deserve credit for shipping the dataset on Kaggle and for flagging the stationarity assumption as 'crucial' in the limitations section.\n\nSoft spots. They are addressable but should be fixed. The privacy reporting is sloppy: the text says f11 and f22 each got 25% of the budget and then says f22 got 50%, which has to be a typo for f12, and the actual noise scale sigma is never reported. Without sigma, a reader cannot independently check the utility-noise tradeoff or the DP composition. The stationarity assumption is validated only on four adjacent weeks (December 2024 to January 2025) from one population, and the re-identification experiment uses the same four-week window, two weeks of which were used to estimate the training statistics. So the extrapolation to longer horizons or new populations is genuinely untested, exactly as the authors concede. Finally, there are no simple baselines—say, a model that ignores across-week transitions—so it is hard to tell how much of the re-id match comes from the temporal structure rather than just the marginal distributions. That omission is minor, but it would strengthen the paper.\n\nRecommendation. I would send this to peer review. The dataset and methodology will be cited and built upon by researchers studying the Topics API and by anyone working on practical DP synthetic data. The fixes I want are mostly reporting and one additional validation, not a change to the core approach.","headline":"Useful, credible DP synthetic data release for Topics API traces; the re-id risk match holds up, but stationarity and DP reporting are the soft spots.","tokens_in":17659,"tokens_out":4242,"would_cite":true,"duration_ms":48061,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper releases a differentially private synthetic version of Topics API outputs whose re-identification risk under two attacks matches real data within one standard deviation.","keywords":["differentially private synthetic data","Topics API","re-identification risk","Privacy Sandbox","Gaussian mechanism","trace generation","stationarity assumption","privacy-preserving ads"],"falsifier":"Collect real Topics API outputs from a period not adjacent to the training weeks, for instance several months later or after the topic taxonomy changes, and rerun the Hamming and Asymmetric Hamming attacks on traces generated from the released model. If the re-identification rate on real data moves outside one standard deviation of the synthetic rate, or if the single-week and transition statistics correlate with the released ones below the 99.8% level reported in the paper's own validation, the stationarity assumption, and with it the claim that the released traces faithfully represent the API's re-identification risk, fails.","tokens_in":16642,"feed_emoji":"🛡️","tokens_out":10489,"duration_ms":90704,"temperature":0.7,"pith_summary":"The paper aims to establish that a synthetic dataset of Topics API outputs, built from only differentially private aggregate statistics, can be realistic enough for outside researchers to study the API's privacy properties. The Topics API, a component of the Privacy Sandbox initiative, emits a weekly topic per user per site, and prior measurements of how easily such outputs re-identify users disagree by orders of magnitude depending on whether the analysis used hundreds of millions of real users or a few thousand. The proposed pipeline computes private statistics of topic popularity, topic co-occurrence, and week-to-week topic transitions from real browsing data; fits a parameterized distribution over multi-week traces to those statistics; samples synthetic traces; and verifies that the re-identification risk of two published attacks on the synthetic data matches the risk on real data within one standard deviation. If the claim holds, researchers without access to proprietary browsing logs can run realistic re-identification analyses and audit privacy claims about the API.","feed_headline":"Synthetic Topics API data matches real re-identification risk","feed_subtitle":"A differentially private release from 100M+ users lets any researcher rerun Topics API privacy studies at scale.","key_machinery":"The machinery is a two-stage design: private statistics first, then a trainable generative model. From two weeks of real data the authors compute three families of statistics, $q^*_{\\mathrm{single}}(o)$ (the chance a random user has topic $o$ in a random week), $q^*_{\\mathrm{within}}(o_1,o_2)$ (the chance two distinct topics co-occur in a week), and $q^*_{\\mathrm{across}}(o_1,o_2)$ (the chance topic $o_1$ in one week is followed by topic $o_2$ in the next), noised with the Gaussian mechanism calibrated by the $\\ell^2$ sensitivity of the counts. The generative model is a uniform mixture of $T=500$ types; each type holds one slot per week per topic position ($r \\times k$ slots, with $k=5$), each slot is a softmax distribution over the 469-topic taxonomy, and a sampled trace takes the set of unique topics drawn from each week's slots, padded to size $k$. By the inclusion-exclusion principle all three target statistics are differentiable functions of the slot logits, so gradient descent with the Adam optimizer can drive the model's statistics toward the private targets, and because everything downstream is post-processing of differentially private counts, the fitted model and the sampled traces inherit the privacy guarantee.","core_discovery":"The paper's central claim is that its release pipeline produces a differentially private dataset that closely matches the re-identification risk properties of the real Topics API data. On a comparison of 10 million real user traces against 10 million synthetic traces over four weeks, both the Hamming and the Asymmetric Hamming attacks re-identify users at a rate below 1%, and on both attacks the synthetic-data risk falls within one standard deviation of the mean risk on the real data. The fitted model also reproduces statistics that were not part of its training objective, such as the distribution of the total number of distinct topics a user sees over four weeks, which matches the real distribution with 89% Pearson correlation. The authors present the result as the first public large-scale dataset of Topics API outputs derived from the real browsing data of hundreds of millions of users, released under an $(\\varepsilon,\\delta)$-differential-privacy guarantee with $\\varepsilon = \\log(3)$ and $\\delta = 10^{-15}$.","pith_inferences":["If the claim holds, the dataset likely supports studies beyond re-identification, such as cross-site topic overlap or ad-tech reach, but only for properties that depend on the same single-topic, pair-within-week, and pair-across-week marginals the model was fitted to; higher-order structure across more than two weeks is not guaranteed.","A testable extension the paper does not run is generating traces of lengths beyond four weeks and checking that re-identification risk still tracks real data, since the stationarity validation only covers four adjacent weeks.","The privacy budget is reported at a single setting; how much the re-identification match degrades at tighter budgets is not measured, so dataset users cannot tell from the release alone how sensitive the fit is to the noise scale.","A user of the released data should treat weeks 3 and 4 of the four-week traces as extrapolations, because the model is fitted to statistics from the first two weeks and their fidelity rests entirely on the stationarity assumption."],"forward_implications":["External researchers can now run re-identification studies of the Topics API on a public dataset comparable in scale to industry-internal ones, without access to private browsing logs.","The statistics, the fitted model, and the sampled traces all carry a provable differential-privacy guarantee because each is derived by post-processing of the private counts.","Both attacks studied stay below a 1% re-identification rate on real and synthetic data alike, with the synthetic rate inside one standard deviation of the real rate.","The same fixed parameter set can generate traces of any length, because the stationarity assumption lets the model impose the same statistics on every week rather than learning week-specific ones.","The methodology is presented as generalizable to other Privacy-Preserving Ads APIs, as long as analogous single-week and transition statistics can be defined."],"supporting_citations":[{"why":"Supplies the random-user re-identification model and the two attacks (Hamming and Asymmetric Hamming) whose risk the synthetic data must match.","marker":"[9]"},{"why":"Provides the adaptive-projection approach of fitting a model to differentially private statistics that this paper adapts.","marker":"[6]"},{"why":"Gives the analytical Gaussian mechanism calibration used in Proposition 4 to set the noise level.","marker":"[7]"},{"why":"Describes the production differential privacy pipeline used to compute the noised statistics.","marker":"[3]"},{"why":"The Topics API specification whose output distribution the model is built to reproduce.","marker":"[21]"},{"why":"An industry re-identification analysis on proprietary large-scale data that the authors aim to make replicable.","marker":"[17]"},{"why":"A prior academic re-identification study on a dataset of about two thousand users, the small-scale baseline this release aims to supersede.","marker":"[8]"},{"why":"The public dataset release that embodies the paper's contribution.","marker":"[11]"}],"fun_headline_variants":["First public DP synthetic Topics API dataset matches real risk","DP synthetic Topics API data: real re-identification risk, 100M users","Public DP synthetic Topics API data for accurate privacy research","DP synthetic Topics API data reproduces real re-identification risk"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the statistics describing which topics appear, co-occur, and transition from week to week stay essentially unchanged over time, so numbers measured on two weeks of real data can generate traces of any length that still match the real joint distribution; if topic interests drift over longer horizons, the synthetic data stops resembling reality.","fun_headline_variants_meta":{"raw":{"variants":["First public DP synthetic Topics API dataset matches real risk","DP synthetic Topics API data: real re-identification risk, 100M users","Public DP synthetic Topics API data for accurate privacy research","DP synthetic Topics API data reproduces real re-identification risk"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3371,"prompt_tokens":1037,"completion_tokens":2334,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":2263}},"tokens_in":653,"tokens_out":2334,"duration_ms":15815,"temperature":1.0,"reasoning_tokens":2263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:30:46.275514+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect real Topics API outputs from a period not adjacent to the training weeks, for instance several months later or after the topic taxonomy changes, and rerun the Hamming and Asymmetric Hamming attacks on traces generated from the released model. If the re-identification rate on real data moves outside one standard deviation of the synthetic rate, or if the single-week and transition statistics correlate with the released ones below the 99.8% level reported in the paper's own validation, the stationarity assumption, and with it the claim that the released traces faithfully represent the API's re-identification risk, fails.","supporting_citations":[{"cited_title":"Measuring re-identification risk","cited_arxiv_id":null,"evidence_quote":"Supplies the random-user re-identification model and the two attacks (Hamming and Asymmetric Hamming) whose risk the synthetic data must match."},{"cited_title":"Differentially private query release through adaptive projection","cited_arxiv_id":null,"evidence_quote":"Provides the adaptive-projection approach of fitting a model to differentially private statistics that this paper adapts."},{"cited_title":"Improving the gaussian mechanism for differential privacy: Analytical cali- bration and optimal denoising","cited_arxiv_id":null,"evidence_quote":"Gives the analytical Gaussian mechanism calibration used in Proposition 4 to set the noise level."},{"cited_title":"The Topics API","cited_arxiv_id":null,"evidence_quote":"The Topics API specification whose output distribution the model is built to reproduce."},{"cited_title":"Measures of Cross-Site Re-Identification Risk: an Analysis of the Topics API Pro- posal","cited_arxiv_id":null,"evidence_quote":"An industry re-identification analysis on proprietary large-scale data that the authors aim to make replicable."},{"cited_title":"A Public and Reproducible Assessment of the Topics API on Real Data","cited_arxiv_id":null,"evidence_quote":"A prior academic re-identification study on a dataset of about two thousand users, the small-scale baseline this release aims to supersede."},{"cited_title":"Data release of the DP synthetic Topics API outputs","cited_arxiv_id":null,"evidence_quote":"The public dataset release that embodies the paper's contribution."}],"review_version":1}