Training Diffusion Language Models for Black-Box Optimization

Can Chen; Christopher Pal; Haolun Wu; Jiayao Gu; Xue Liu; Ye Yuan; Zipeng Sun

REVIEW 3 major objections 2 minor 1 cited by

Diffusion language models, adapted with delimiter tokens and a two-stage SFT-then-RL recipe, reach state-of-the-art offline black-box optimization from limited design–label data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 22:51 UTC pith:HG6YFJNK

load-bearing objection We only have the DiBO abstract; the supplied full text is SegFly, so the SOTA and domain-gap claims cannot be checked. the 3 major comments →

arxiv 2603.17919 v3 pith:HG6YFJNK submitted 2026-03-18 cs.CE

Training Diffusion Language Models for Black-Box Optimization

Zipeng Sun , Can Chen , Ye Yuan , Haolun Wu , Jiayao Gu , Christopher Pal , Xue Liu This is my paper

classification cs.CE

keywords offline black-box optimizationdiffusion language modelsDesign-Benchbidirectional generationsupervised fine-tuningreinforcement learningdelimiter tokensdesign optimization

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Offline black-box optimization tries to find better designs from a fixed set of labeled examples when new experiments are costly, as in robotics and DNA design. Recent work casts these tasks as natural-language prompts for autoregressive language models, but left-to-right generation poorly captures the strong bidirectional dependencies that design problems often have. This paper adapts diffusion language models, which model tokens bidirectionally, to the same offline setting. To close the gap between natural-text pretraining and BBO’s mixed signals (prompts, designs, labels), the authors build a unified prompt–response corpus with delimiter tokens that mark field boundaries, then post-train in two stages: masked-response supervised fine-tuning followed by reinforcement learning that rewards label improvements. The method is reported to achieve state-of-the-art results on Design-Bench under small-data settings while needing only 1.5 H100 GPU hours for discrete tasks.

Core claim

Adapting a diffusion language model to offline black-box optimization—via a unified prompt–response corpus with delimiter tokens and a two-stage pipeline of masked-response supervised fine-tuning plus reinforcement learning on label-improvement rewards—yields state-of-the-art designs on Design-Bench in the small-data regime, with highly efficient training.

What carries the argument

The two-stage post-training framework on a unified delimiter-token corpus: stage one is supervised fine-tuning by masked-response prediction; stage two is reinforcement learning whose reward is defined by label improvement, aligning bidirectional generation with high-label designs.

Load-bearing premise

Delimiter tokens plus the two-stage SFT-then-RL recipe are enough to close the domain gap between natural-text pretrained diffusion models and BBO’s heterogeneous prompts, designs, and labels, so that bidirectional generation beats strong autoregressive LLM baselines on the same offline data.

What would settle it

On Design-Bench small-data splits, run a controlled comparison with the same data budget and evaluation protocol: diffusion LLM with the full delimiter-plus-two-stage recipe versus an autoregressive LLM baseline and ablations that remove delimiters or the RL stage; if the full method does not improve the quality of the highest-label generated designs, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Bidirectional token modeling can capture design dependencies that left-to-right autoregressive generation misses in offline BBO.
A unified prompt–response format with explicit field delimiters can transfer natural-text diffusion LLMs into heterogeneous design domains.
Label-improvement rewards give a usable RL signal when only offline design–label pairs are available.
Discrete offline BBO tasks can be adapted with modest compute (about 1.5 H100 GPU hours in the reported setting).
Design-Bench small-data leaderboards may shift once diffusion-LLM baselines are standard.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same delimiter-and-two-stage recipe may transfer to other structured generation settings where context, candidate, and score fields are heterogeneous.
If the main gain is bidirectionality, continuous design spaces may need different tokenization or noise schedules than the discrete tasks highlighted for training efficiency.
Offline BBO methods that currently rely on autoregressive prompting may need re-benchmarking against diffusion LLM variants before claims of superiority can be settled.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

We only have the DiBO abstract; the supplied full text is SegFly, so the SOTA and domain-gap claims cannot be checked.

read the letter

The punchline first: the document we were given for 2603.17919 is the wrong paper. Metadata and abstract describe DiBO (diffusion LLMs for offline black-box optimization). The full manuscript block is SegFly (aerial RGB-T segmentation, 2603.17920). So this is an abstract-only read of DiBO, not a full review.

From the abstract alone, the move is clear and reasonable. Offline BBO with scarce labels is a real bottleneck. Prior work already put autoregressive LLMs on Design-Bench-style tasks via natural-language prompts. DiBO’s claimed novelty is switching to diffusion LLMs for bidirectional design structure, then closing the text-to-BBO domain gap with a unified prompt–response corpus, field delimiter tokens, masked-response SFT, and RL on label-improvement rewards. The efficiency claim (1.5 H100 hours on discrete tasks) and code link are the parts that would make this useful if they hold.

What we cannot do is verify any of the load-bearing pieces. No Design-Bench tables, no ablations on delimiters vs. plain formatting, no comparison protocol against the AR-LLM baselines, no training curves, no continuous-task numbers. The weakest assumption in the abstract—that delimiters plus two-stage post-training actually make bidirectional generation beat strong AR baselines under the same offline data—is exactly the claim that needs those experiments. The stress-test is right: that evidence is absent from the supplied text, not hidden in a soft section.

On its face the problem framing and pipeline are coherent enough that a complete paper with solid Design-Bench results would deserve referee time. As delivered here, there is nothing to engage with beyond a methods sketch. I would not bring this to reading group until the real PDF is in hand, and I would not cite from the abstract alone. If the correct manuscript shows up with the claimed ablations and small-data SOTA, re-read it then; until then treat the claims as unverified.

Referee Report

3 major / 2 minor

Summary. The submission (arXiv:2603.17919) claims that diffusion language models can be adapted to offline black-box optimization by (i) building a unified prompt–response corpus with delimiter tokens to mark field boundaries among prompts, designs, and labels, and (ii) a two-stage post-training pipeline: masked-response supervised fine-tuning followed by reinforcement learning with rewards defined by label improvements. The abstract asserts that this closes the domain gap between natural-text pretraining and heterogeneous BBO signals, that bidirectional generation better captures design dependencies than left-to-right autoregressive LLMs, and that the method reaches state-of-the-art on Design-Bench under small-data regimes with only 1.5 H100 GPU hours for discrete tasks. Code is said to be available at a public repository.

Significance. If the empirical claims hold, the work would be a useful contribution to offline BBO and to the emerging use of LLMs as design generators: bidirectional diffusion LMs are a natural fit for structured designs with non-causal dependencies (e.g., DNA, robotics), and a lightweight domain-adaptation recipe plus efficient training would matter for practitioners with limited labels and compute. The abstract also promises open code, which would aid reproducibility. However, significance cannot be assessed from the materials actually supplied for review, because the full manuscript text provided is an unrelated paper (SegFly, aerial RGB-T segmentation), not DiBO.

major comments (3)

Manuscript identity mismatch: the abstract and paper_id refer to DiBO (Training Diffusion Language Models for Black-Box Optimization, arXiv:2603.17919, cs.CE), but the full manuscript body supplied for review is SegFly (a 2D–3D–2D aerial RGB-T semantic segmentation paper, arXiv:2603.17920, cs.CV). No Design-Bench tables, ablations, training curves, algorithm boxes, or DiBO method sections appear in the provided full text. The central claims (SOTA under small data, 1.5 H100 hours, necessity of bidirectional modeling, and efficacy of delimiter tokens + two-stage SFT/RL) therefore cannot be checked against any experimental or methodological evidence in the document under review.
Load-bearing premise unverifiable: the abstract’s key technical bridge—that delimiter tokens plus a unified prompt–response corpus and masked-response SFT then label-improvement RL close the domain gap so diffusion LLMs outperform strong autoregressive LLM baselines—is asserted without any supporting sections, equations, or tables in the supplied manuscript. Without those materials, it is impossible to assess internal consistency, baselines, data splits, oracle vs. offline evaluation, or whether gains are attributable to bidirectionality rather than training recipe or compute.
Efficiency and SOTA claims cannot be audited: the abstract’s quantitative claims (Design-Bench SOTA in small-data settings; 1.5 H100 GPU hours for discrete tasks) require tables, compute accounting, and comparison protocols that are absent from the provided full text. A referee cannot confirm or refute them from the materials given.

minor comments (2)

Abstract-only material that can be noted for a future correct submission: define the offline BBO objective and Design-Bench protocol early; specify the diffusion LLM backbone, masking/noise schedule, delimiter vocabulary, and exact RL reward (label improvement relative to what baseline); and report ablations isolating bidirectionality vs. SFT/RL and delimiter tokens.
When the correct DiBO manuscript is provided, ensure figure/table numbering and section cross-references match the Design-Bench results and the 1.5 H100-hour claim, and that code links resolve to the DiBO repository rather than unrelated projects.

Circularity Check

0 steps flagged

No circular derivation: DiBO is an empirical training pipeline; available text shows no self-definitional reduction or fitted-as-prediction claim.

full rationale

Only the DiBO abstract is available for arXiv:2603.17919; the CACHEABLE full manuscript is an unrelated SegFly paper, so no DiBO equations, ablations, or self-citation chains can be walked. From the abstract alone, the method is a standard domain-adaptation recipe (unified prompt–response corpus, delimiter tokens, masked-response SFT, then RL with label-improvement rewards) evaluated for SOTA on Design-Bench. That is an empirical methods claim, not a first-principles derivation that could collapse to its inputs by construction. Label-improvement rewards define the BBO objective itself and are not a circular “prediction” of a fitted constant unless evaluation reuses the same labels without holdout—which cannot be verified or quoted from the supplied text. No uniqueness theorem, ansatz smuggled via self-citation, or renaming of a known result appears. Per hard rules, without a quotable reduction of a claimed prediction to its inputs, the honest finding is no significant circularity (score 0, empty steps).

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

Review is abstract-only for 2603.17919; the supplied full manuscript is SegFly (different paper). Ledger entries are therefore the load-bearing premises and constructs named in the abstract, not a full methods audit.

free parameters (3)

RL reward definition (label improvement)
How much a design's label must improve, and relative to which baseline, is a design choice that shapes the second-stage objective; exact form not specified in the abstract.
Masking schedule / noise schedule for masked-response SFT
Diffusion LLM fine-tuning depends on which response tokens are masked and at what rates; abstract names masked-response prediction without values.
Delimiter token vocabulary and field layout
Special tokens that mark prompt/design/label boundaries are introduced ad hoc for domain adaptation; their exact set is a free modeling choice.

axioms (4)

domain assumption Design problems in the target domains have strong bidirectional dependencies that left-to-right generation fails to capture well.
Stated as the motivation for replacing autoregressive LLMs with diffusion LLMs; load-bearing for why the switch should help.
domain assumption An offline dataset of designs and labels is sufficient to discover improved designs without online queries.
Defines the offline BBO setting the method optimizes for.
ad hoc to paper Delimiter tokens plus a unified prompt–response corpus adequately bridge the domain gap between natural-text pretraining and heterogeneous BBO signals.
Core adaptation claim of the paper; not a standard mathematical fact.
ad hoc to paper Reinforcement learning with rewards defined by label improvements aligns generation toward high-label designs better than SFT alone.
Justifies the second training stage; empirical necessity not verifiable from abstract alone.

invented entities (2)

Unified prompt–response corpus with field delimiter tokens for BBO no independent evidence
purpose: Format heterogeneous prompts, designs, and labels so a text-pretrained diffusion LLM can be domain-adapted.
Construct introduced by the paper to close the domain gap; no independent existence outside this pipeline.
Two-stage DiBO post-training pipeline (masked-response SFT then label-improvement RL) no independent evidence
purpose: Align diffusion LLM generation with high-label designs for offline BBO.
Named training framework that is the paper's main methodological product.

pith-pipeline@v1.1.0-grok45 · 7893 in / 3084 out tokens · 31036 ms · 2026-07-13T22:51:47.476033+00:00 · methodology

0 comments

read the original abstract

We study offline black-box optimization (BBO), aiming to discover improved designs from an offline dataset of designs and labels, a problem common in robotics and DNA with limited labeled samples. While recent work applies autoregressive LLMs to BBO by formatting tasks as natural-language prompts, their left-to-right design generation struggles to capture the strong bidirectional dependencies inherent in design problems. To address this, we propose adapting diffusion LLMs to offline BBO to leverage their bidirectional modeling capabilities. However, a domain gap exists between the natural text pre-training of diffusion LLMs and the heterogeneous signals in BBO (prompts, designs, and labels). To bridge this gap, we construct a unified prompt--response corpus and introduce delimiter tokens to explicitly mark field boundaries for domain adaptation. We further propose a two-stage post-training framework to align the diffusion LLM generation with high-label designs. The first stage performs supervised fine-tuning on the unified dataset via masked-response prediction, and the second stage adopts reinforcement learning with rewards defined by label improvements. Our method achieves state-of-the-art results on Design-Bench under small-data settings with highly efficient training, requiring only $1.5$ H100 GPU hours for discrete tasks. Code for our work is available here: https://github.com/zpointS/DiBO.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

One Sentence, One Drama: Personalized Short-Form Drama Generation via Multi-Agent Systems
cs.CV 2026-05 unverdicted novelty 5.0

A hierarchical multi-agent framework converts a single sentence into a short drama using debate-based scripting, 3D-grounded first frames for spatial consistency, and multi-stage reviewer loops.