{"id":"53e9df32-ff6c-4bb4-aa96-0a8ac3b2c371","arxiv_id":"2512.22999","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"JADAI jointly trains a design policy, a history encoder, and a diffusion-based posterior estimator end-to-end, using per-step posterior-loss reductions as a proxy for information gain, and reaches state-of-the-art performance on adaptive design benchmarks.","lead":"JADAI trains a single system that both chooses what experiments to run next and estimates unknown parameters from the data collected so far, using diffusion models for the estimates. If it works, scientists who run adaptive experiments can get accurate posterior beliefs at every step without separate design and inference code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The diffusion/flow utility proxy ℓ_{t-1}−ℓ_t is the load-bearing step: it is not shown to track expected information gain, and the paper explicitly replaces the Barber–Agakov log-ratio with a score-matching loss difference without a supporting argument.","rationale":"The reader’s weakest assumption is exactly the load-bearing concern: the per-step posterior-loss difference is a heuristic proxy for EIG when the posterior is an implicit generative model. My reading of the manuscript confirms this is the least secure step in the central argument. The formal derivation in Eq. (14) is valid only for models with tractable log-densities; the extension to diffusion/flow-matching losses in Eq. (15) is a non sequitur, and the paper itself states that the Barber–Agakov interpretation no longer holds and appeals to experiments. The empirical results are strong and the detach/telescoping gradient mechanism is internally coherent, but they do not establish that the optimized quantity tracks information gain. I considered whether a more severe internal inconsistency existed — e.g., that the utility collapses to −ℓ_T and therefore ignores intermediate losses — but the stop-gradient baselines intentionally preserve per-step gradient contributions while canceling the forward value, so that mechanism is sound. Thus the proxy misalignment remains the single most load-bearing concern. A direct calibration test against true incremental EIG on a tractable model would settle whether the concern lands. Given the reader already assigned CONDITIONAL with this concern, I recommend no change to the verdict.","tokens_in":49113,"tokens_out":8215,"duration_ms":85338,"concrete_test":"Take a tractable model with a known posterior, e.g., a 2D Gaussian likelihood with conjugate updates. Train a diffusion posterior estimator to convergence on rollouts from a fixed policy. Then, for many sampled θ and rollout histories h_{t-1}, compute Δ_t = ℓ_{t-1} − ℓ_t for the trained score network and compare it to the true incremental EIG, ΔEIG_t = E_{x_t|h_{t-1}}[KL(p(θ|h_{t-1},x_t) ‖ p(θ|h_{t-1}))], evaluated by quadrature. Report the Spearman correlation and sign-agreement rate over all t. If the correlation is not significantly positive — or worse, is negative — the proxy is not a valid surrogate and the JADAI objective is not maximizing expected information gain, independent of any end-task metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical hinge is Eq. (14), which is exact only for a normalized density estimator where ℓ_t = −log qψ(θ|h_t): there the TEIG lower bound telescopes into per-step log-density differences. For diffusion and flow-matching estimators, ℓ_t is instead a weighted score-matching or flow-matching regression loss (Eqs. 27 and 34). The difference ℓ_{t-1}−ℓ_t is not a log-density ratio, not a bound on the incremental EIG, and not even an unbiased estimator of the score difference shown in Eq. (15). Section 3.2 concedes that the Barber–Agakov interpretation 'is no longer the same' and appeals to experiments. This is the precise place where the central claim — that JADAI maximizes information gain by reducing posterior loss at every step — loses its formal footing. If the diffusion model is miscalibrated or the loss surface rewards 'easier' histories rather than more informative ones, the policy can decrease ℓ_t without increasing EIG. The detach/telescoping mechanism itself is internally consistent and clever; the unproven part is the semantic identification of score-matching loss differences with information gain. No theoretical guarantee, calibration check, or monotonicity result is provided, so the empirical sPCE/SSIM results carry the entire weight of this identification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces JADAI, a framework that jointly amortizes Bayesian adaptive design and posterior inference by training a design policy, a history summary network, and a generative posterior estimator end-to-end. The training objective is the generic utility u_T = Σ_{t=0}^T (ℓ*_{t-1} − ℓ_t), where ℓ_t is a per-step posterior loss and ℓ* denotes a detached copy. For a normalized density estimator (e.g., a normalizing flow), this utility is a telescoping form of the Barber–Agakov lower bound on total expected information gain (Eqs. 12–14). For diffusion and flow-matching estimators, ℓ_t is instead a weighted score-matching or flow-matching regression loss (Eqs. 27 and 34), and the paper explicitly states that the Barber–Agakov interpretation no longer applies and appeals to experimental performance. Experiments on Location Finding, CES, and MNIST Image Discovery report competitive or superior sPCE and SSIM/NRMSE relative to recent baselines, with ablations in Appendix B.","tokens_in":49463,"tokens_out":7487,"duration_ms":77243,"significance":"If the empirical findings hold, JADAI is a practically valuable contribution: it extends amortized adaptive design to high-dimensional, multimodal posteriors without reinforcement-learning value networks, and it provides posterior estimates at every acquisition step. The normalized-density case gives a clean connection to TEIG, and the experiments are substantial, including random-policy controls that support the claim that the performance gap in Image Discovery comes primarily from the learned policy (Section 5.3, Figure 6). The paper is also transparent about limitations, including the heuristic status of the implicit-model objective (Section 3.2) and the need for differentiable simulators (Section 6). The main weakness is theoretical: for the diffusion/flow setting that powers the flagship high-dimensional results, the objective is not shown to track or bound expected information gain, so the paper's formal claim is not established for its primary regime.","major_comments":[{"comment":"The central claim that JADAI maximizes information gain rests on treating ℓ_{t-1}−ℓ_t as incremental information. For a normalized density estimator, Eq. (14) is an identity and the Barber–Agakov bound is clean. For diffusion models (Eq. 27) and flow matching (Eq. 34), ℓ_t is a regression loss against noise/velocity targets; ℓ_{t-1}−ℓ_t is not a log-density ratio and is not shown to bound, estimate, or track incremental EIG. The manuscript concedes this in Section 3.2 and appeals to experiments. This is load-bearing because the policy is trained through this difference. A policy can reduce ℓ_t by steering to histories that are easy for the current posterior estimator without increasing true information gain unless additional calibration or monotonicity holds. I ask for either (a) a formal statement of conditions under which ℓ_{t-1}−ℓ_t is a valid surrogate for incremental information, or","section":"Section 3.2, Eqs. (16), (27), (34)"},{"comment":"For the high-dimensional Image Discovery benchmark, the central evaluation is SSIM/NRMSE against the ground-truth image, not an information-theoretic measure of posterior quality. The random-policy controls in Figure 6 are informative, but SSIM is a perceptual reconstruction metric and can be high even if the posterior is miscalibrated. To support the claim that the learned policy improves inference rather than only reconstruction, please report a posterior calibration or coverage diagnostic at each measurement step, for example simulation-based calibration or conditional coverage on a validation set.","section":"Section 5.3, Table 2, Figure 4"}],"minor_comments":[{"comment":"The notation suggests a score with respect to θ_τ, but in the diffusion derivation the score is with respect to the latent z_τ; please align notation with Eqs. (23)–(24).","section":"Section 3.2, Eq. (15)"},{"comment":"The heading 'Constant Elasticity of Substitutions' should be 'Constant Elasticity of Substitution' for consistency with the CES benchmark name.","section":"Section 5.2 / Appendix C.3"},{"comment":"Entries marked '−' are not defined in the caption; please specify whether they indicate 'not reported' or 'not run'.","section":"Table 1"},{"comment":"The corner plot axis labels are difficult to read at the current resolution; please ensure they are legible in the final version.","section":"Section 5.1, Figure 2"},{"comment":"The requirement of a differentiable simulator is stated clearly, but it should also be mentioned in the abstract or introduction, since it materially constrains the scope of the method.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The main gap is not a correctness error in the mechanics but a mismatch between the formal TEIG framework and the actual objective for the flagship implicit-model experiments. A calibration analysis and carefully qualified claims would be sufficient for reconsideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely useful: it trains a policy, a history encoder, and a diffusion/flow-matching posterior estimator end-to-end for adaptive design, using the detach-and-telescope trick to turn per-step posterior losses into a training signal that also shapes the policy. That is a clean, practical idea, and the experiments back it up: the method beats or matches strong baselines on Location Finding and CES, and on MNIST Image Discovery it is clearly ahead of CoDiff. The ablation study is honest — nested BPTT is unstable, truncated BPTT hurts, fixed rollout length helps terminal but not intermediate horizons. That kind of reporting earns credit.\n\nThe soft spot is exactly where the stress test lands. For a normalized density estimator, Eq. (14) is a real telescoping of the Barber–Agakov bound, so the objective has a clean information-theoretic reading. For diffusion and flow matching, ℓ_t is a regression loss, and the difference ℓ_{t-1} − ℓ_t is not a log-density ratio, not a bound on incremental EIG, and not an unbiased estimator of the score difference in Eq. (15). The paper says so itself in Section 3.2 — 'the Barber–Agakov interpretation is no longer the same' — and then appeals to experiments. That is an honest concession, but it leaves the central claim resting entirely on empirical performance. If a diffusion model is miscalibrated, the policy could reduce score-matching loss along easier trajectories without gaining information. No calibration check or monotonicity argument is offered. This is a real gap, but I would not call it fatal: the objective is a plausible surrogate, the sPCE and SSIM evaluations are external to the training loss, and the results are strong enough to be interesting even without a formal guarantee. The differentiability requirement is also acknowledged, and it is a genuine limitation for black-box simulators.\n\nThe citation pattern looks fine. The paper builds on the relevant DAD/iDAD/ALINE line and cites it properly. Some self-citation appears in the diffusion tutorial reference, but it is not load-bearing.\n\nWho is this for? People working on amortized Bayesian experimental design or simulation-based inference with diffusion posteriors. It deserves a serious referee: the empirical contribution is substantial, the method is clearly described, and the missing theoretical link is the thing a reviewer should push on. I would engage with it and would want the authors to either provide a formal argument for the proxy or add calibration diagnostics showing that the diffusion posterior tracks the true posterior along policy rollouts.","headline":"A well-engineered joint policy+inference framework with strong empirical results, whose main theoretical hinge for diffusion models is an unproven proxy — worth serious refereeing, not a desk reject.","tokens_in":49924,"tokens_out":983,"would_cite":true,"duration_ms":12229,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"JADAI shows that adaptive experimental design and Bayesian posterior inference can be trained together, end to end, around a single objective that rewards reducing posterior error at every step of an experiment.","keywords":["Bayesian experimental design","adaptive design","amortized inference","simulation-based inference","diffusion models","flow matching","posterior estimation","design policy"],"falsifier":"In a small Gaussian adaptive-design problem where the true expected information gain is computable in closed form, train JADAI with a deliberately miscalibrated posterior estimator so that posterior loss decreases while the posterior is inaccurate, then compare the resulting policy's estimated information gain or final inference accuracy against a policy trained on the true objective; a measurable gap would show the surrogate is not tracking information gain.","tokens_in":49025,"feed_emoji":"🧪","tokens_out":8012,"duration_ms":73661,"temperature":0.7,"pith_summary":"Bayesian adaptive design (choosing each next measurement to maximize information) and posterior inference (estimating parameters from the measurements) have usually been solved as separate problems: a design policy proposes experiments, and a slow inference method is run afterwards. This paper tries to establish that the two can be amortized jointly: one policy network chooses each next design, one history network summarizes the measurements so far, and one diffusion- or flow-matching network estimates the posterior at every step, all trained end-to-end on a scalar utility that sums per-step reductions in posterior error. The claim is that maximizing this utility, even though it is not an exact expected-information-gain objective when the posterior is an implicit generative model, yields policies competitive with or better than existing approaches on standard adaptive-design benchmarks, including a high-dimensional multimodal image-discovery task. If correct, this turns sequential experiment design and inference into a single fast, fully amortized step rather than a two-stage loop.","feed_headline":"Joint training tops separate design-and-inference pipelines","feed_subtitle":"A single training objective teaches networks what to measure next and how to revise beliefs, beating standard benchmarks.","key_machinery":"The load-bearing object is the incremental posterior-loss utility u_T(θ, h_{0:T}) = Σ_{t=0}^T (ℓ*_{t-1} - ℓ_t), with ℓ_{-1}=0 and ℓ*_t = detach(ℓ_t), where ℓ_t is the per-step posterior estimation loss conditioned on the history summary. For a normalized density model this sum telescopes exactly and reproduces the lower bound on total expected information gain; for implicit models it is used as a proxy. A second mechanism is the detached-history design pass: the policy receives a summary with gradients stopped, so the history network is trained only through the posterior losses while the policy is trained through the observations its designs produce; this removes nested backpropagation throu","core_discovery":"The paper's central discovery is an objective identity: for a normalized-density posterior estimator, the variational lower bound on total expected information gain telescopes into a sum over experiment steps of per-step differences in posterior loss, log q(θ|h_t) - log q(θ|h_{t-1}). JADAI builds on this by using the same sum, u_T = Σ_{t=0}^T (ℓ*_{t-1} - ℓ_t), as a generic training utility even when the posterior estimator is an implicit diffusion or flow-matching model whose log-density is unavailable, with the previous loss value detached. Although the variational bound interpretation no longer applies, the objective still drives rollouts along which posterior loss decreases at every step.","pith_inferences":["Beyond the paper: if the per-step loss proxy is as reliable as the experiments suggest, the same utility should extend to other implicit posterior families, such as consistency models or adversarial generators, without re-deriving an information-theoretic objective.","Beyond the paper: because the utility is defined entirely through posterior losses, JADAI could be coupled with gradient-free design optimization or policy-gradient estimators to handle black-box simulators; the paper only tests differentiable simulators.","Beyond the paper: the paper notes but does not implement sharing a transformer backbone across policy and history networks; doing so could improve sample efficiency on harder design spaces.","Beyond the paper: a testable consequence is that deliberately miscalibrating the posterior estimator, while keeping the posterior loss artificially low, should degrade the learned designs; monitoring this gap would validate the surrogate."],"forward_implications":["At any point in an experiment, JADAI provides an approximate posterior over parameters, not only after the full horizon, enabling active data-acquisition workflows.","A policy trained to minimize posterior losses over a longer horizon remains competitive at shorter horizons, so longer training horizons can improve early decisions.","Diffusion- and flow-matching-based posterior estimators can be inserted into the same design objective, making high-dimensional multimodal posteriors tractable in adaptive design.","When the design policy is replaced by a random policy, performance falls to baseline levels, indicating that most of the gain comes from the learned policy rather than the posterior network.","Test-time rollouts take milliseconds on low-dimensional tasks and are roughly an order of magnitude faster than prior methods on the high-dimensional benchmark."],"fun_headline_variants":["Telescoping loss powers joint adaptive design and diffusion inference","One objective telescopes to train design and inference together","Diffusion posteriors enable a joint loss for adaptive design","Stepwise posterior loss differences make design and inference joint"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim rests on the detached per-step posterior-loss difference (ℓ_{t-1} - ℓ_t) being a faithful proxy for expected information gain when the posterior is an implicit diffusion or flow-matching model; if that proxy misaligns with true information gain, the policy can be optimal for the surrogate while poor for the actual experiment.","fun_headline_variants_meta":{"raw":{"variants":["Telescoping loss powers joint adaptive design and diffusion inference","One objective telescopes to train design and inference together","Diffusion posteriors enable a joint loss for adaptive design","Stepwise posterior loss differences make design and inference joint"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000662,"raw_usage":{"total_tokens":2798,"prompt_tokens":617,"completion_tokens":2181,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":361,"completion_tokens_details":{"reasoning_tokens":2129}},"tokens_in":361,"tokens_out":2181,"duration_ms":15364,"temperature":1.0,"reasoning_tokens":2129,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T13:43:14.637004+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In a small Gaussian adaptive-design problem where the true expected information gain is computable in closed form, train JADAI with a deliberately miscalibrated posterior estimator so that posterior loss decreases while the posterior is inaccurate, then compare the resulting policy's estimated information gain or final inference accuracy against a policy trained on the true objective; a measurable gap would show the surrogate is not tracking information gain.","supporting_citations":[],"review_version":1}