{"id":"89e19134-3a23-4276-8658-6dfa5c33e9c6","arxiv_id":"2605.24173","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Infilling extraction on diffusion language models extracts up to three times more verbatim sequences than prefix methods and achieves higher recall on redacted emails than autoregressive models.","lead":"Diffusion language models can have their training data extracted more effectively by using arbitrary masks to infill missing tokens rather than only prefix prompts. This matters because it shows higher privacy leakage risks in these bidirectional models than previously measured, relevant for anyone deploying or regulating AI systems.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Whether infilling modes with arbitrary masks and redacted-PII corpora model realistic adversary capabilities remains the key untested assumption.","rationale":"The reader’s weakest_assumption is precisely the load-bearing one; the full manuscript would need to demonstrate that the chosen modes are not merely the most permissive but are also the ones an adversary could actually mount. No other internal inconsistency is visible from the abstract-level description.","tokens_in":1781,"tokens_out":320,"duration_ms":28059,"concrete_test":"Re-run the redacted-email extraction (Table X, redacted-PII corpus) restricting the adversary to only prefix-conditioned and suffix-conditioned masks; if the DLM recall advantage over the AR baseline falls below 1.2×, the claim that bidirectional access opens inaccessible channels no longer holds under realistic API constraints.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline result (edge masks yield up to 3× more verbatim recovery; DLMs outperform scale-matched AR models on redacted emails) rests on the claim that the five extraction modes, three pipelines, and three corpora instantiate a realistic attack surface. If a deployed DLM only exposes prefix/suffix or fixed-length infilling (common API constraints), or if the redaction pattern in the adversary’s corpus does not match the training distribution, the bidirectional advantage and the redacted-email recall gap could shrink or vanish. The abstract provides no quantitative evidence that the chosen mask geometries are attainable by an adversary who lacks white-box mask control or exact knowledge of redaction locations.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that diffusion language models (DLMs) allow substantially higher training-data extraction than previously measured because they support infilling at arbitrary positions. It introduces an infilling-extraction protocol parameterized by binary masks, shows on LLaDA-8B and Dream-7B that edge-conditioned masks recover up to three times more verbatim sequences than prefix-conditioned masks, and reports that a realistic adversary supplied with redacted-PII training data can extract redacted email addresses at higher recall from DLMs than from scale-matched autoregressive models. The results are obtained across five extraction modes, three training pipelines, and three corpora.","tokens_in":1911,"tokens_out":455,"duration_ms":37972,"significance":"If the empirical findings hold, the work is significant because it demonstrates that standard prefix-only probing systematically underestimates memorization risk in bidirectional models and that mask geometry is a first-order determinant of extractability. The redacted-PII experiment directly addresses a practical privacy threat. The multi-model, multi-pipeline design and explicit comparison to autoregressive baselines are strengths that make the central claim falsifiable.","major_comments":[{"comment":"Abstract: the assertion that the five extraction modes and three corpora 'realistically model' an adversary's capabilities is load-bearing for both the 3× verbatim claim and the redacted-email recall gap, yet the manuscript provides no evidence that arbitrary mask control (as opposed to prefix/suffix or fixed-length infilling) is attainable under typical deployment constraints.","section":"Abstract"},{"comment":"Abstract: the quantitative statements ('up to three times more', 'higher recall') are presented without accompanying tables, confidence intervals, or statistical tests in the summary; because these numbers are the primary support for the claim that 'mask geometry governs extractability,' their robustness cannot be assessed from the provided information.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract introduces LLaDA-8B and Dream-7B only in the final sentence; moving the model names earlier would improve readability.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed review and constructive comments on our work. The two major comments focus on the framing of the adversary model in the abstract and the presentation of quantitative claims. We address each point below with proposed revisions to improve clarity and precision without altering the core empirical contributions.","responses":[{"response":"We agree that the phrasing 'realistically model' in the abstract is imprecise and that the manuscript does not demonstrate the availability of arbitrary mask control in typical closed deployments. The intent was to show that prefix-only extraction underestimates risk for bidirectional models when infilling is possible. In revision we will replace the phrase with 'model an adversary with infilling access' and add a short discussion paragraph on deployment contexts (open-weight models, fine-tuned checkpoints, or APIs exposing infilling) where such access could occur. This qualifies the claim without changing the experimental results.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the assertion that the five extraction modes and three corpora 'realistically model' an adversary's capabilities is load-bearing for both the 3× verbatim claim and the redacted-email recall gap, yet the manuscript provides no evidence that arbitrary mask control (as opposed to prefix/suffix or fixed-length infilling) is attainable under typical deployment constraints."},{"response":"The abstract is a high-level summary; the supporting tables, per-model breakdowns, and multi-run results appear in Sections 4–5. We acknowledge that the abstract alone does not convey robustness. We will revise it to add a brief qualifier directing readers to the experimental sections and will ensure the main text reports confidence intervals or statistical tests for the key comparisons if they are not already present. The multi-model, multi-corpus design already provides some robustness evidence.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the quantitative statements ('up to three times more', 'higher recall') are presented without accompanying tables, confidence intervals, or statistical tests in the summary; because these numbers are the primary support for the claim that 'mask geometry governs extractability,' their robustness cannot be assessed from the provided information."}],"tokens_in":1416,"tokens_out":464,"duration_ms":34191,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that diffusion language models leak more training data when attacks can use arbitrary infilling masks instead of just prefixes, with edge masks recovering up to three times as many verbatim sequences and better recall on redacted emails than scale-matched autoregressive models.\n\nThe new piece is the infilling extraction protocol that uses binary masks to match the bidirectional bias of DLMs. It subsumes prefix probing and they apply it across five modes, three pipelines, and three corpora on LLaDA-8B and Dream-7B. That setup lets them measure how mask position affects extractability and show that a follow-up finetuning stage does not erase the prior memorization. The comparative results on redacted PII are the most direct addition to the literature.\n\nThe experiments look like straightforward empirical work with multiple conditions, which is the right way to test the claim. The finding that tunable decoding parameters change extraction rates is also useful to note.\n\nThe soft spot is whether the chosen masks and redacted corpora actually model what an adversary can do in practice. If real APIs restrict infilling to fixed patterns or prefix/suffix only, or if the redaction locations in the adversary's data do not match the training distribution, the reported threefold gap and the redacted-email advantage could shrink. The abstract states the numbers clearly but the strength of the realism argument will depend on the methods details and any controls for data overlap.\n\nThis is for people working on memorization, privacy, and diffusion-based generation. Readers who want to extend extraction attacks beyond autoregressive models will find the protocol and the mask-geometry results worth seeing.\n\nIt deserves a serious referee because the core method is new for this model type and the measurements are direct. I would send it to peer review with requests for more on attack realism and statistical details.","headline":"Infilling extraction shows DLMs leak more via mask geometry than prefix attacks, but the realistic-adversary claim needs checking against API limits.","tokens_in":2376,"tokens_out":444,"would_cite":false,"duration_ms":39778,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Diffusion language models leak up to three times more verbatim training sequences when extraction uses infilling masks instead of prefixes.","keywords":["diffusion language models","training data extraction","infilling extraction","memorization","mask geometry","redacted PII","bidirectional access","privacy risks"],"falsifier":"Run the same set of edge-conditioned and prefix masks on a diffusion language model and a scale-matched autoregressive model trained on identical redacted data; if recall on redacted email addresses is not higher for the diffusion model, the central claim on bidirectional advantage fails.","tokens_in":2668,"feed_emoji":"🔍","tokens_out":715,"duration_ms":27099,"temperature":0.7,"pith_summary":"Diffusion language models can fill in masked tokens at any position rather than only continuing from a prefix. The paper introduces infilling extraction as a protocol that applies arbitrary binary masks to test memorization under the models' bidirectional bias. Edge-conditioned masks recover up to three times as many verbatim sequences as prefix masks across multiple models and corpora. The same bidirectional probes also recover more redacted email addresses than scale-matched autoregressive models do. These results show that standard prefix-based tests miss substantial extraction channels available in diffusion language models.","feed_headline":"Infilling masks extract three times more data from diffusion models","feed_subtitle":"Edge masks and bidirectional probes reveal extraction channels that prefix tests miss in these models.","key_machinery":"Infilling extraction: a data-extraction protocol that applies an arbitrary binary mask to diffusion language models, allowing token recovery at any positions rather than only from a left-to-right prefix.","core_discovery":"Infilling extraction, parameterized by arbitrary binary masks, shows that mask geometry governs extractability in diffusion language models: edge-conditioned masks extract up to three times more verbatim sequences than prefix-conditioned ones. Bidirectional access opens channels inaccessible in autoregressive models, enabling a realistic adversary with redacted training data to achieve higher recall on extracting email addresses from DLMs than from scale-matched autoregressive models. Tunable decoding parameters affect extraction rates while a subsequent supervised finetuning stage leaves prior memorization intact.","pith_inferences":["Evaluation protocols for memorization in non-autoregressive models will need to test multiple mask geometries to be considered complete.","Applications that handle sensitive data may face different privacy trade-offs when switching from autoregressive to diffusion language models.","Defenses developed for prefix-based extraction may leave residual channels open when applied to diffusion language models.","The same mask-based probing approach could be applied to other bidirectional or non-autoregressive architectures to check for similar extractability patterns."],"forward_implications":["Mask geometry, not merely model scale, determines how much training data can be recovered from diffusion language models.","Prefix-only tests systematically underestimate memorization risk in diffusion language models.","Adversaries holding partial or redacted training data can extract more from diffusion language models than from autoregressive counterparts.","Tuning of decoding parameters during extraction measurably changes success rates.","A supervised finetuning stage after initial training does not remove the memorization that enables extraction."],"fun_headline_variants":["Edge masks extract three times more than prefix masks in DLMs","DLMs extract more redacted emails than AR models","Mask geometry governs extractability in diffusion LMs","Finetuning leaves memorization intact in DLMs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The chosen extraction modes, mask geometries, and corpora accurately reflect the access and capabilities a realistic adversary would have when targeting deployed diffusion language models.","fun_headline_variants_meta":{"raw":{"variants":["Edge masks extract three times more than prefix masks in DLMs","DLMs extract more redacted emails than AR models","Mask geometry governs extractability in diffusion LMs","Finetuning leaves memorization intact in DLMs"]},"model":"grok-4.3","cost_usd":0.007112,"raw_usage":{"total_tokens":3305,"prompt_tokens":705,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":71124500,"prompt_tokens_details":{"text_tokens":705,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2538,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":705,"tokens_out":62,"duration_ms":28221,"temperature":1.0,"reasoning_tokens":2538,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T15:53:28.862909+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Run the same set of edge-conditioned and prefix masks on a diffusion language model and a scale-matched autoregressive model trained on identical redacted data; if recall on redacted email addresses is not higher for the diffusion model, the central claim on bidirectional advantage fails.","supporting_citations":[],"review_version":1}