{"id":"e378357e-7919-4257-a655-eedcfd240d57","arxiv_id":"2607.16927","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"An offline-trained policy network adaptively selects the next factorial experiment, and a Gibbs sampler then identifies active factors with credible intervals, beating static designs mainly when the run budget is below the identifiable limit.","lead":"DABS trains a neural network in advance to decide which experiments to run next for finding which factors matter, then uses a Gibbs sampler to report which factors are likely active. It could help engineers and drug developers screen many candidate variables with far fewer experimental runs than a one-shot design.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The deployment Gibbs decoder never updates interaction indicators and Eq. 36 ignores strong heredity, so the reported interaction posterior probabilities and Case Study 3 results are not produced by a valid posterior sampler.","rationale":"I agree with the reader's rejection of the paper, but I identify a different load-bearing flaw. The reader's weakest_assumption is that the Student-t marginal in Eq. 10 is derived only for a main-effects-only model and is applied to the full interaction model without proof. Inspecting Eq. 37, however, the Student-t form does survive for the diagonal prior beta|z,sigma^2 ~ N(0,sigma^2 diag(tau_z^2)): conditioned on z, the model is y = X beta + epsilon with beta having covariance sigma^2 D_z, so y|sigma^2,z ~ N(0, sigma^2 (I + X D_z X^T)) and integrating sigma^2 yields exactly a multivariate Student-t. The missing derivation is a presentation gap, not a mathematical break. The true break is in the Gibbs sampler. Algorithm 2 samples only main-effect indicators, even though the model and the case study require interaction indicators. Eq. 36's full conditional ignores the heredity constraint, so it cannot be the correct conditional posterior under the stated model. Since all reported interaction recovery and the credible-interval claims depend on this sampler, the central posterior-inference claim fails as written. The configuration-space reduction claim in Section 3.1 is also false for pi_int<1, further indicating that the paper does not actually implement its stated model. The concrete test above would settle the sampler issue directly. The reader's verdict of REJECT remains appropriate; my concern is a more precise reason for it, hence 'partial' agreement on the weakest_assumption.","tokens_in":24343,"tokens_out":8505,"duration_ms":90973,"concrete_test":"Take the p=4, pi_int=1 configuration of Case Study 3, generate one history h_T from the model, and enumerate the exact posterior over all admissible configurations (main-effect patterns with z_ij = z_i z_j) using the closed-form Student-t marginal of Eq. 37. Compare the exact P(z_ij=1 | h_T) with the estimate produced by Algorithm 2 on the same h_T. Then repeat with pi_int=0.75, where exact enumeration must treat z_ij as free variables and Algorithm 2 has no update step for them. If the Gibbs estimates differ beyond Monte Carlo error, the decoder is not sampling from the stated model and the interaction-recovery results in Section 5.4 are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DABS yields posterior probabilities of factor activity and credible intervals via Gibbs inference. Algorithm 2 (Appendix F) updates only main-effect indicators z_k, k=1..p (line 12), and Appendix G's Step B gives Eq. 36 for P(z_k=1 | z_-k, h_T), where z_-k is 'all activity indicators except z_k.' But Section 3.1 defines z to include interaction indicators z_ij, and Algorithm 2 never updates them. Under strong heredity with pi_int<1, z_ij are free Bernoulli(pi_int z_i z_j) variables and need their own Gibbs step. With pi_int=1, z_ij = z_i z_j is a deterministic function of the main indicators, so changing z_k in Eq. 36 necessarily changes the dependent z_ij; yet Eq. 36 evaluates p(h_T | z_k=1, z_-k) and p(h_T | z_k=0, z_-k) with the same fixed z_-k. In either case, Eq. 36 is not the correct full conditional for the strong-heredity model: it assigns positive posterior weight to configurations with an interaction active while a parent main effect is inactive, configurations that have prior probability zero under the model. Thus the sampler does not leave the strong-heredity posterior invariant; it effectively samples a different, heredity-free model. The interaction inclusion probabilities used in Section 5.4 and Appendix H.3 therefore cannot be posterior estimates under the stated model. This aligns with the false claim in Section 3.1 that heredity reduces the configuration space to at most {0,1}^p for all pi_int in (0,1] — true only at pi_int=1. The Student-t marginal issue raised by the reader is secondary: for a diagonal normal prior beta|z,sigma^2 ~ N(0,sigma^2 diag(tau_z^2)), the Student-t form in Eq. 37 does survive the extension to interactions. The load-bearing gap is the sampler, not the marginal.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Deep Adaptive Bayesian Screening (DABS), an amortized policy-based method for sequential binary factorial screening under a spike-and-slab prior with strong heredity. The policy is trained offline by maximizing a prior-contrastive lower bound on the mutual information between the history and the active-set indicators, using a claimed closed-form Student-t marginal to integrate out effect sizes and noise. At deployment, a Gibbs sampler is used to recover posterior activity probabilities and credible intervals. The method is evaluated on three simulation benchmarks calibrated to published studies, with a 'wedge prediction' that adaptivity helps only below the identifiability boundary T=p.","tokens_in":24750,"tokens_out":9190,"duration_ms":98779,"significance":"The paper addresses a genuine gap in amortized Bayesian experimental design: discrete factorial screening with posterior uncertainty quantification. It has notable strengths: a clean model specification, pseudocode for training and deployment, a falsifiable wedge prediction, extensive ablations, and a stated intent to release code. If the closed-form marginal and Gibbs decoder were valid for the full model with interactions, the contribution would be useful. However, the analytic marginal is derived for a simpler main-effects-only model, the claimed complexity reduction under heredity is incorrect for pi_int<1, and the deployment Gibbs sampler is not a valid posterior sampler for the stated model. The reported interaction posteriors and credible intervals are therefore not supported. Because all benchmarks are simulated from the paper's own spike-and-slab model, the experiments also provide no external validation of the modeling assumptions.","major_comments":[{"comment":"Appendix C derives p(h_T|z) for a main-effects-only model: y|beta_z,sigma^2 ~ N(X_z beta_z, sigma^2 I), beta_z|sigma^2,z ~ N(0, sigma^2 tau_z^2 I_{p_z}). The full model in Section 3.1 has main effects, two-factor interactions, spike-and-slab with distinct tau_active and tau_inactive, and heredity. Conditional on z, the correct marginal is y|z,sigma^2 ~ N(0, sigma^2(I + X diag(tau_z^2) X^T)) with a vector tau_z^2, not the X_z X_z^T form. Eq. (10) claims the Appendix C form applies to the full model, but no derivation is given. Appendix G's Eq. (37) uses the diag form but calls it a 'tight approximation' of the exact Section 3.1 prior, which is inconsistent because the Section 3.1 prior is exactly the diag form. Since this marginal defines the sPCE training objective and the Gibbs full conditionals, the policy and decoder may be optimizing and evaluating the wrong likelihood.","section":"Section 3.2 / Eq. (10) / Appendix C"},{"comment":"The claim that strong heredity reduces the admissible activity-pattern space to at most {0,1}^p is false for pi_int in (0,1). Under z_ij|z_i,z_j ~ Bernoulli(pi_int z_i z_j), a main-effect pattern with m active factors admits 2^{m(m-1)/2} possible interaction patterns. For p=15 and three active factors there are 2^3 possible interaction patterns, not 1. The space is at most {0,1}^p only in the deterministic limit pi_int=1. This matters because the tractability of contrastive sampling is argued from this reduction: drawing contrastive samples only from {0,1}^p omits interaction indicators, so the p(h_T|z_i) in Eq. (4) is not the marginal of the stated model.","section":"Section 3.1, 'Under heredity ... reduces to at most {0,1}^p'"},{"comment":"Algorithm 2 updates only main-effect indicators z_k (line 12); interaction indicators z_ij are never updated. Eq. (36) defines P(z_k=1|z_-k,h_T) using p(h_T|z_k=1,z_-k) and p(h_T|z_k=0,z_-k) with the same fixed z_-k. Under strong heredity, z_ij are either free Bernoulli variables (pi_int<1) or deterministic functions z_i z_j (pi_int=1). In neither case is Eq. (36) the correct full conditional. With pi_int=1, toggling z_k changes the dependent z_ij, so the two likelihood terms must use different induced patterns. With pi_int<1, z_ij require their own Gibbs update, which is absent. The sampler therefore assigns positive posterior weight to heredity-forbidden configurations and never explores interaction states. Consequently, the interaction inclusion probabilities and credible intervals reported in Section 5.4 and Appendix H.3 are not posterior estimates under the stated model.","section":"Algorithm 2 / Appendix G, Eqs. (36)-(37)"},{"comment":"The model places a Beta(a,b) hyperprior on rho, but the Gibbs sampler in Algorithm 2 and Eq. (36) treats rho as a fixed constant. No Gibbs step for rho is provided. Posterior activity probabilities are therefore computed conditional on an arbitrary rho value rather than marginalizing over the Beta prior. This is a mismatch with the model and affects the reported posterior inclusion probabilities.","section":"Section 3.1 / Algorithm 2: handling of rho"}],"minor_comments":[{"comment":"The proof concludes that 'any maximizer' of xi^T Sigma xi over [-1,1]^p lies in {-1,+1}^p, but the coordinate-wise argument only establishes that there exists a maximizer at a vertex. For example, f(xi)=xi_1^2 has non-vertex maximizers. The statement should be weakened to existence of a vertex maximizer.","section":"Appendix A, Proposition 1 proof"},{"comment":"Eq. (11) defines the design via sign(ell_t + g_t), while Algorithm 1 writes xi_t <- GumbelSoftmax(ell_t, tau). These are different operations; the pseudocode should clarify the straight-through relaxation used in the forward/backward pass.","section":"Section 3.3 / Algorithm 1"},{"comment":"The abstract describes a 'single-pass Gibbs decoder', but Algorithm 2 runs G Gibbs iterations after burn-in. This is not a single pass; consider rephrasing to 'a closed-form Gibbs sampler'.","section":"Abstract"},{"comment":"The text says the scale matrix is 'determined by ... pi_int', but p(h_T|z) conditional on z does not depend on pi_int; pi_int appears in the prior over z, not in the marginal likelihood. This should be clarified.","section":"Eq. (10)"},{"comment":"The statement 'Code, model checkpoints and evaluation scripts are available from the author' lacks a URL or repository identifier, making the reproducibility claim unverifiable.","section":"Section 5.1 / Code availability"}],"recommendation":"reject","confidential_remarks":"The paper has multiple load-bearing technical errors: the analytic marginal is derived for a simpler model, the heredity dimensionality claim is false for pi_int<1, and the Gibbs sampler is not a valid posterior sampler for the stated model. The interaction results in Section 5.4 are therefore unsupported. The experimental evaluation is also entirely simulated from the paper's own model, so it cannot validate the modeling assumptions. A substantially revised version with a corrected marginal, a valid Gibbs sampler that updates interaction indicators and rho, and external or at least mismatched-model benchmarks could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core idea is real: training an amortized policy to maximize a screening-specific mutual information I(z; h_T), with effect sizes and noise marginalized out, is a sensible and non-obvious combination of existing tools (sPCE, Gumbel-Softmax, spike-and-slab). The paper is clearly written, the ablations are extensive, and the wedge prediction — adaptivity helps only when the budget T is below the identifiability boundary — is elegant and tested along complementary axes. Credit where it's due: this is not a restatement of DAD or iDAD, and the authors know the relevant literature.\n\nNow the soft spots, in order of severity. The Gibbs sampler is load-bearing and broken as written. Algorithm 2 updates only main-effect indicators z_k; it never updates interaction indicators z_ij. Under strong heredity with pi_int < 1, z_ij is a free Bernoulli conditioned on the parents and needs its own Gibbs step. With pi_int = 1, z_ij = z_i z_j is deterministic, so toggling z_k necessarily changes z_ij for its two parents, but Eq. 36 evaluates the marginal likelihood with z_−k held fixed. Either way the sampler assigns positive posterior probability to configurations that have prior probability zero under the stated model (an interaction active while a parent is inactive). That's not a small implementation detail — it means the reported interaction posterior probabilities in Section 5.4 and Appendix H.3 are not posterior estimates under the model. The same confusion underlies the false claim in Section 3.1 that heredity reduces the configuration space to at most 2^p for all pi_int in (0,1]. That is only true at pi_int = 1; for pi_int < 1, every active parent pair adds a free Bernoulli.\n\nThe student-t marginal issue the reader raised is, I think, secondary. The derivation in Appendix C is for a main-effects-only model, but with a diagonal normal prior on beta|z it extends directly to the full model with interactions, giving the X diag(tau_z^2) X^T + I scale matrix in Eq. 37. The paper should have shown that derivation instead of hand-waving, but it's not a fatal flaw. What is fatal is the sampler.\n\nAlso, all benchmark simulations draw from the paper's own spike-and-slab model, so the empirical validation doesn't independently confirm the advantage over static designs. That's a common limitation in this literature, but it matters more here because the central mechanism is already suspect.\n\nMy take: the training recipe has promise and could be salvaged with a corrected sampler — either add Gibbs steps for interaction indicators or restrict to pi_int = 1 and re-derive the full conditionals properly — and then re-run the experiments. As it stands, the paper's inferential claims don't hold. An editor should still send it to peer review rather than desk-reject: the novelty is real and the flaws are specific and fixable, and a good referee can point the authors to exactly what needs to change. But I wouldn't cite the current version, and I'd bring it to a reading group only to dissect why the sampler is invalid.","headline":"A genuinely new adaptive screening method with a clever training objective, but the deployment Gibbs sampler as written is not sampling the stated heredity-constrained posterior — so all interaction posterior probabilities and Case Study 3 results are in question.","tokens_in":25360,"tokens_out":2544,"would_cite":false,"duration_ms":31508,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62K05","62F15","62K15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Deep Adaptive Bayesian Screening (DABS) claims that an offline-trained neural policy can sequentially choose binary factorial experiments that maximize information about which factors are active, with effect sizes and noise integrated out a","keywords":["adaptive experimental design","factorial screening","spike-and-slab prior","strong heredity","amortized Bayesian optimal experimental design","Prior Contrastive Estimation","Gibbs sampling","active-set recovery"],"falsifier":"Take a small problem with interactions and heredity (e.g., p=4 with one active interaction), generate random histories from the true generative model, and compare the closed-form Student-t p(h_T | z) against a numerically integrated or long-run MCMC estimate of the same marginal. A systematic mismatch would mean the sPCE training objective and the Gibbs full conditionals evaluate the wrong likelihood. Additionally, test the wedge prediction directly: for T just above p, the DABS-versus-static F1 gap should be statistically indistinguishable from zero; a persistent gap in that regime would cont","tokens_in":24178,"feed_emoji":"🧪","tokens_out":3809,"duration_ms":43340,"temperature":0.7,"pith_summary":"The paper introduces DABS, a method for adaptive factorial screening that trains a policy network entirely offline so that, at deployment, each new experiment is selected in a single forward pass without per-step optimization. The key claim is that focusing the design objective on the active-set indicators — with effect sizes and noise variance marginalized out analytically — yields a screening-specific learning signal that outperforms both static designs and greedy adaptive baselines, but only when the experimental budget is smaller than the number of candidate factors. Below this identifiability boundary, adaptivity lets the policy steer subsequent experiments toward promising factors; above it, the paper predicts and observes that a well-chosen static design matches the adaptive policy. At deployment, a Gibbs sampler exploiting the same closed-form Student-t marginals recovers posterior probabilities of factor activity and credible intervals on effect sizes from a single pass through the data. If correct, DABS provides an amortized, ready-to-use screening pipeline for settings where experiments are expensive and budgets are tight.","feed_headline":"Adaptive screening beats static designs when runs are tight","feed_subtitle":"An offline-trained policy picks each next experiment in one pass, then a Gibbs sampler recovers active factors with credible intervals.","key_machinery":"The load-bearing object is the closed-form multivariate Student-t marginal p(h_T | z) obtained by analytically integrating effect sizes and noise variance out of the conjugate spike-and-slab linear-Gaussian model. This single formula serves three roles: it defines the screening-specific sPCE training objective (contrastive samples are drawn only over the discrete activity indicators z), it makes the policy network's input the exactly sufficient statistics of the history, and it supplies the full conditional distributions of the deployment Gibbs sampler. The policy itself is a two-layer MLP mapping sufficient statistics to per-factor logits, with a straight-through Gumbel-Softmax relaxation t","core_discovery":"The paper's central claim is that sequential adaptive screening can be amortized by training a policy network offline to maximize a contrastive lower bound on the mutual information between the experimental history and the discrete active-set indicators, rather than on the full parameter vector. Because the spike-and-slab prior and Gaussian likelihood are conjugate, the continuous effect sizes and noise variance are integrated out analytically, leaving a closed-form multivariate Student-t marginal for the history given the active set. This makes the training objective tractable and lets the policy be learned with standard gradient methods on discrete {−1,+1} designs. At deployment, the same","pith_inferences":["A practical corollary of the wedge prediction is a decision rule for practitioners: if the experimental budget is at or above the number of candidate factors, static designs are likely sufficient, and adaptive screening earns its extra training cost only in the under-identified regime.","The sufficient-statistics encoder's input dimension grows as O(p^4), which sets a practical ceiling around p≈26 in the paper's experiments; scaling to much larger factor sets would require a more compact representation of the design history.","Because the Student-t marginal rests on conjugacy, extending DABS to non-Gaussian responses or non-conjugate priors would require a likelihood-free contrastive estimator or auxiliary-variable augmentation; the paper identifies this as a limitation, so a natural next step is testing such extensions on binary or count responses.","The paper's ablation suggests that targeting I(z; h_T) rather than I(θ; h_T) is decisive; a similar recasting of other amortized design frameworks toward discrete support recovery could let them benefit from the same analytic marginalization whenever conjugacy holds."],"forward_implications":["Under the wedge prediction, adaptive screening is most valuable when the experimental budget is below the identifiability boundary; once T is at or above the number of candidate factors, a well-chosen static design achieves the same expected information gain.","DABS matches or outperforms the strongest non-adaptive Bayesian baseline across three published screening benchmarks, with large gains at tight budgets and no significant loss above the boundary.","The trained policy adds essentially no per-step optimization cost at deployment: each design is a single forward pass, and the Gibbs decoder returns activity probabilities and effect-size credible intervals in one pass through the data.","Targeting the screening-specific mutual information I(z; h_T) rather than total parameter information improves factor-recovery F1; the paper's ablation shows a measurable drop when the total-information objective is used.","The logit-L2 penalty is load-bearing: removing it collapses the policy into a nearly constant design and reduces main-effect F1 from 0.303 to 0.080.","The method assumes strong heredity for interactions; the paper reports that a moderate heredity mismatch at evaluation causes only a small F1 drop, suggesting some robustness to this assumption."],"fun_headline_variants":["Amortized adaptive screening beats static on tight budgets","Deep Bayesian policy screens adaptively to beat static designs","Adaptive screening beats static under limited runs","Offline policy network, Gibbs inference: fast adaptive screening","Policy network learns to pick experiments for adaptive screening"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole training objective and the Gibbs decoder rely on the closed-form Student-t marginal p(h_T | z) being exact, but that formula is derived only for a main-effects-only linear model with a common slab variance; the paper applies it without proof to the full model with interactions, heredity, and a second slab, so if that marginal is inexact under the full model, both the learned policy and the posterior probabilities are aimed at the wrong likelihood.","fun_headline_variants_meta":{"raw":{"variants":["Amortized adaptive screening beats static on tight budgets","Deep Bayesian policy screens adaptively to beat static designs","Adaptive screening beats static under limited runs","Offline policy network, Gibbs inference: fast adaptive screening","Policy network learns to pick experiments for adaptive screening"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001798,"raw_usage":{"total_tokens":6859,"prompt_tokens":626,"completion_tokens":6233,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":370,"completion_tokens_details":{"reasoning_tokens":6159}},"tokens_in":370,"tokens_out":6233,"duration_ms":51671,"temperature":1.0,"reasoning_tokens":6159,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T19:32:10.925260+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small problem with interactions and heredity (e.g., p=4 with one active interaction), generate random histories from the true generative model, and compare the closed-form Student-t p(h_T | z) against a numerically integrated or long-run MCMC estimate of the same marginal. A systematic mismatch would mean the sPCE training objective and the Gibbs full conditionals evaluate the wrong likelihood. Additionally, test the wedge prediction directly: for T just above p, the DABS-versus-static F1 gap should be statistically indistinguishable from zero; a persistent gap in that regime would cont","supporting_citations":[],"review_version":1}