{"id":"37570e47-db5b-4fc7-a094-a0a5d3f6cf24","arxiv_id":"2511.17812","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Proposes importance-weighted non-IID joint sampling for flow matching, using a trained residual velocity to estimate per-sample weights along the sampling trajectory.","lead":"A method to draw multiple samples from a flow-matching generative model jointly so they cover diverse modes, then compute importance weights to fix the bias this introduces. Useful for estimating expectations of image classifiers or risk metrics with very few samples.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unbiasedness rests on Corollary 1's closed-form score identity for v+r_phi, which is exact only for an ideal rectified flow; no error bound or large-scale r_phi training is provided, so the weight integral is an untested approximation.","rationale":"The reader's weakest assumption identifies precisely the fragile point: Corollary 1's closed-form score substitution is exact only for ideal rectified flows. My read confirms this is the central load-bearing approximation, and the paper's own Limitations section admits that r_phi is not trained for large models. The theoretical derivation is internally consistent conditional on this approximation; the concern is the absence of an error bound and the lack of empirical validation at scale. Since the reader's verdict is already CONDITIONAL and this concern is what motivates that condition, no verdict adjustment is needed.","tokens_in":19699,"tokens_out":10138,"duration_ms":97472,"concrete_test":"On the Gaussian-mixture benchmark, obtain a numerically accurate p'_1 by pooling many joint draws and using a high-order density estimator (or an analytic density if the joint ODE map is inverted). Then compare three quantities for the same joint samples: (i) log w from Corollary 1 (closed-form scores), (ii) log w from Theorem 2 with a separately learned score s''(x,t), and (iii) exact log(p1/p'_1). If (i) deviates from (iii) while (ii) tracks (iii), the rectified-flow score identity is the load-bearing approximation and an error bound is needed; if (i) and (ii) both match (iii), the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Corollary 1 (Sec. 3.2), which replaces both scores s and s'' in Theorem 2 with the rectified-flow identity s(x,t)=(t v(x,t)-x)/(1-t) (Eq. 10; Supp. B). For v+r_phi, this identity is exact only if v+r_phi is an ideal rectified flow, i.e., the velocity equals the conditional expectation E[X1-X0 | Xt=x] under independent X0~p0 and X1~p'_1. Eq. (23) is the rectified-flow objective, so in the infinite-capacity, infinite-data limit Corollary 1 is valid. But in practice v is a frozen finite-capacity network and r_phi is a small network trained only via Eq. (23); there is no guarantee that the learned v+r_phi has the correct intermediate-time score, and the paper gives no error bound linking the Eq. (23) loss to error in the weight integral. In the only experiment where importance weights are evaluated (Gaussian mixture, Table 2), the 'ground-truth' weights are themselves produced by LLDE (Sec. F), not an exact p'_1. For Stable Diffusion 3.5 and FLUX.1-Fill, no r_phi is trained at all—the Limitations section states this explicitly—so the headline claim of unbiased importance-weighted expectation estimation for real flow models is untested. This is an unquantified approximation gap, not an internal inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a framework for non-IID, jointly drawn samples from flow-matching models, with two components: (i) score-based regularization of diversity velocities to keep samples on-manifold, and (ii) importance weighting of the resulting non-IID samples by learning a residual velocity field whose marginal matches the joint sampler, then evolving log-importance weights along sample trajectories. The method is developed for rectified flows, where the score is obtained from the velocity in closed form. Theorems 1 and 2 derive the weight dynamics from the continuity equation, and Corollary 1 specializes to rectified flows. Experiments on a Gaussian mixture evaluate diversity, quality, importance-weight accuracy, and expectation estimation, while Stable Diffusion 3.5 and FLUX.1 experiments evaluate only diversity/quality via representation error. The paper emphasizes that residual velocity training for large models is left to future work.","tokens_in":20038,"tokens_out":2146,"duration_ms":25233,"significance":"If the theoretical claims held in practice, the paper would offer a useful contribution: a principled way to debias jointly sampled, diversity-enforced flow outputs, which is relevant to risk-sensitive estimation and coverage-oriented sampling. The derivations from the continuity equation are a strength, and the trajectory-based estimator (Theorem 2 / Corollary 1) is well motivated and avoids the out-of-distribution issue of the fixed-position variant. The toy experiments show that the proposed residual-flow weight estimator outperforms density baselines. However, the load-bearing unbiasedness claim rests on the assumption that the learned velocity plus residual is an exact rectified flow, and this is neither established theoretically nor tested at scale; the large-model experiments do not train the residual velocity at all. The result is a promising method with an unquantified approximation gap rather than a fully established unbiased estimator.","major_comments":[{"comment":"Corollary 1 replaces the two scores in Theorem 2 with the rectified-flow identity s(x,t)=(tv(x,t)-x)/(1-t) for both v and v+r_phi. This identity is exact only when the velocity is the true conditional expectation E[X1-X0 | Xt=x]. In practice, v is a finite-capacity pretrained network and r_phi is a small network trained only with the endpoint-marginal objective (23). No error bound, monotonicity, or regularity condition links the (23) loss to error in the weight integral. The unbiasedness claim in Eq. (3) and Table 3 therefore holds only under an idealization that is not quantified; without such a bound, the reported improvements on the toy example cannot be extrapolated.","section":"Sec. 3.2, Corollary 1 and Eq. (23)"},{"comment":"For Stable Diffusion 3.5 and FLUX.1-Fill, no residual velocity r_phi is trained; the paper states this explicitly. Tables 4 and 5 therefore evaluate only the score-regularized diversity sampler, not the importance-weighting component. The central claim of 'accurate importance-weight estimates and debiased expectation estimates' is thus untested for real flow models. This is not a presentation issue but a gap between the stated contribution (first importance-weighting method for non-IID flow samples) and the experiments supporting it.","section":"Sec. 4.3, 4.4 and Limitations"},{"comment":"The 'ground-truth' importance weights in Table 2 are themselves obtained by Local-Likelihood Density Estimation (LLDE) from pooled non-IID samples, not from the exact marginal density p'_1 of the coupled sampler. This means the evaluation target is an estimate, and any error in the LLDE density appears as error in the reported SE/ranking metrics. The conclusions about superiority of the trajectory-based estimator are plausible, but the claim would be stronger if the toy setup allowed an exact or quasi-exact weight computation (e.g., by enumerating the joint sampler's symmetry or using a much larger reference set with a known density), or if sensitivity to the LLDE bandwidth choice were reported.","section":"Sec. 4.2 and Sec. F"},{"comment":"The fixed-position variant (Eq. 25) is argued to be problematic because inputs (X_1, t) for small t are out-of-distribution for r_phi. This is an intuitive explanation, but the paper does not quantify how often such points occur or provide a concrete diagnostic (e.g., norm of r_phi on those inputs). Since the trajectory-based estimator is recommended largely on this basis, a direct comparison with controlled removal of low-t inputs or a visualization of r_phi's behavior would strengthen the argument.","section":"Sec. 3.3, Eq. (25)"},{"comment":"The soft regularization strength alpha(t)=sqrt(1-t) is introduced and used in experiments, but the text does not specify how alpha(t) interacts with the normalization of g_reg or with the global diversity strength lambda in Eq. (14). The soft variant appears to give consistently worse RMSE than hard in Table 1, yet remains included as a recommended choice. Clarifying the tuning of alpha(t) and its sensitivity would make the contribution easier to reproduce.","section":"Sec. 3.1, Eq. (18)"}],"minor_comments":[{"comment":"The phrase 'first approach for importance weighting of non-IID flow samples' is a strong novelty claim; the paper should state what exactly distinguishes it from existing importance-sampling methods for deterministic transforms, and ideally give a brief citation to classical change-of-variable weighting approaches.","section":"Abstract and Sec. 1"},{"comment":"Equation (10) is stated as an approximation without a formal statement of when it is exact. Since it is used later in the main theorems, a short remark on the ideal-rectified-flow setting and finite-capacity effects would be useful.","section":"Sec. 2.1, Eq. (10)"},{"comment":"The phrase '8-D near-planar distribution' is confusing because the distribution is a 10-component Gaussian mixture in the first two dimensions with tiny variance in the remaining six; the term 'planar' may mislead readers into expecting an exactly low-dimensional manifold. Suggest rephrasing.","section":"Sec. 4.1.1"},{"comment":"The notation Ours and Ours† is not explicitly defined in the caption. The text says † denotes the fixed-position variant, but the caption should state this directly.","section":"Table 2"},{"comment":"The typo in prompt T4 ('a releastic cat') is likely intentional to test robustness to typos, but the paper does not say so. Please clarify.","section":"Sec. 4.3"},{"comment":"The LLDE construction is described in detail but the resulting density is still only an estimate. A short discussion of the bias-variance trade-off of LLDE in this 10-dimensional setting (e.g., number of samples per kNN ball) would help the reader judge the reliability of the ground-truth weights.","section":"Sec. F"},{"comment":"Definition 3 uses dXt = (v+r_phi) dt, but the paper elsewhere uses X_0 ~ p_0 and time in [0,1]. It would be clearer to explicitly state that the same time interval and base distribution are used.","section":"Sec. 3.2, Definition 3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a sound core derivation and a well-posed toy evaluation, but the headline unbiased-estimation claim is not supported at scale and the approximation gap in Corollary 1 is unquantified. I do not view this as rejection-worthy; the authors could strengthen the paper by adding an error analysis or by clearly repositioning the contribution as an idealized framework with toy validation. The lack of r_phi training for the large models is acknowledged in the limitations, but it should be reflected more prominently in the claims and in the abstract. I would recommend major revision, with the main requirement being either a quantitative error bound for Corollary 1 or an explicit and prominent restriction of the unbiasedness claim to the setting where the residual flow is trained."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. The importance-weighting scheme for non-IID flow samples — learning a residual velocity r_phi that reproduces the marginal of the coupled sampler, then integrating a log-weight ODE along the trajectory — is genuinely new relative to Particle Guidance and DiverseFlow. And the unbiased-estimation claim is only demonstrated on a Gaussian mixture toy; on Stable Diffusion 3.5 and FLUX the residual flow is never trained, so the central corollary sits on an unquantified approximation.\n\nWhat's done well. Theorems 1 and 2 follow cleanly from the continuity equation and the supplement proofs check out. The toy experiments are well-aimed: trajectory-based weighting beats fixed-position weighting and the density baselines by a wide margin on weight accuracy, and the expectation-estimation result (JS divergence vs a true reference from 1M IID samples) shows weighting recovering and slightly beating IID at a fixed budget. The score-based diversity regularization is a modest but sensible fix to off-manifold drift, and the quality metrics support it. The paper is honest about its scope — the limitations section says large-scale r_phi training is future work.\n\nThe soft spots. The main one is what the stress-test flagged: Corollary 1 assumes v+r_phi is an ideal rectified flow so the closed-form score identity holds. That's exact in the infinite-capacity limit; in practice v is frozen and finite-capacity, r_phi is small and trained only to match the endpoint marginal (Eq. 23), and no error bound links the training loss to error in the weight integral. The intermediate-time score of v+r_phi is never checked. This is an approximation gap, not an internal inconsistency — the trajectory integral would give the exact log-ratio if the scores were right — but it means the headline claim of unbiased expectation estimation for real flow models is untested. The abstract overreaches by claiming 'accurate importance weight and expectation estimation' without that experiment. I'd also push back mildly on the reader's 'circularity' label: training r_phi on samples from the non-IID marginal is just density estimation of the proposal, standard and fine. The weakness is the unquantified score approximation and the limited experimental scope, not a loop.\n\nTwo smaller notes. The SE and ranking numbers in Table 2 are computed against 'ground-truth' weights that are themselves LLDE estimates of the non-IID marginal — estimator vs estimator; the expectation table avoids that problem by using a true reference, so the main toy claim stands. And the trajectory log-weight integral with a divergence term is structurally close to annealed importance sampling and the Neural ODE change-of-variables formula; a couple of citations there would help place the novelty.\n\nVerdict: worth a serious referee. I'd send it to review, with the r_phi-at-scale experiment as the key requested change. If that lands, the paper is solid.","headline":"First importance-weighting scheme for non-IID flow samples; the idea is new and the toy results hold up, but the unbiased-estimation claim is untested on real models because the residual flow is never trained at scale.","tokens_in":20504,"tokens_out":9359,"would_cite":true,"duration_ms":84335,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that jointly drawing diverse samples from a flow-matching model need not bias expectation estimates: if one learns a residual velocity field that reproduces the non-IID sampler's marginal distribution, importance weights ca","keywords":["flow matching","non-IID sampling","importance weighting","score-based regularization","rectified flow","expectation estimation","diversity sampling","Monte Carlo variance reduction"],"falsifier":"On the Gaussian-mixture test bed, check the score identity directly: draw (X0, X1) pairs, condition on intermediate Xt, compare the empirical conditional mean E[X1 − X0 | Xt = x] to the formula (t v − x)/(1−t); a mismatch shows Corollary 1's dynamics does not compute log(p/p''). At scale: train a residual velocity for a large conditional model, estimate E[f(X)] by trajectory-weighted non-IID samples and by a far larger IID ground truth; they should agree within Monte Carlo error while equal-weight diverse samples remain biased.","tokens_in":19545,"feed_emoji":"⚖️","tokens_out":9704,"duration_ms":77051,"temperature":0.7,"pith_summary":"This paper argues that the usual Monte Carlo practice of drawing independent samples from a flow-matching model is needlessly wasteful for computing expectations: under a fixed sampling budget, independent draws often cluster on one mode and miss rare but high-impact regions. It proposes instead to draw n samples jointly, with a diversity velocity that pushes trajectories apart, and then to correct the induced bias by assigning each sample an importance weight, the ratio of the true density to the sampler's marginal density. The obstacle is that the marginal density of the joint sampler is unknown, since the joint draw happens once; the paper's central step is to learn a lightweight residual velocity field such that the perturbed flow has the same endpoint marginal as the joint sampler, and to derive ODEs (Theorems 1-2, Corollary 1) that evolve the log density ratio along each trajectory. For rectified flows—flows whose paths are straight interpolations between noise and data—the score (gradient of log density) enters in closed form, so no extra score network is needed. If the claim holds, a practitioner can get both diversity and unbiased expectation estimates from the same batch of samples, at a cost the experiments suggest can beat IID sampling.","feed_headline":"Reweighting diverse samples makes flow-model estimates unbiased","feed_subtitle":"A learned residual velocity recovers the sampler's marginal, making diversity-enforced draws unbiased.","key_machinery":"The load-bearing objects are the residual velocity r_phi and the trajectory-based weight dynamics. r_phi is trained so the ODE Ẋ = v + r_phi reproduces the joint sampler's marginal, letting p/p'' stand in for the true weight p/p'. Theorem 2 gives the ODE for log w along a real trajectory; for rectified flows the score identity s(x,t) = (t v(x,t) − x)/(1−t) turns all score terms into closed-form expressions in r_phi, u and v (Corollary 1), so no learned score network is needed. A second mechanism, score-based regularization, projects each diversity gradient against the score direction, damping or zeroing moves that reduce log-density—keeping spread while staying on-manifold.","core_discovery":"On its own terms, the discovery is that importance weights for non-IID flow samples are computable even though the joint sampler's marginal is never observed: a lightweight residual velocity r_phi is trained so that the ODE v + r_phi reproduces that marginal, and the log ratio log p/p'' is evolved along each real trajectory (Theorem 2), which for rectified flows becomes closed-form in r_phi, u, and v alone (Corollary 1). Integrating along the path gives per-sample weights making the weighted average unbiased for the true expectation. The same score function regularizes the diversity velocity, damping pushes that would leave the manifold. On a Gaussian mixture the weights beat density baselin","pith_inferences":["Editor's inference, not the paper's claim: if the residual-velocity construction transfers to large models—the paper leaves this untested—the same weighting scheme would give a practical route to risk-sensitive expectations (safety-critical tail events) for production-scale conditional generation, where rare-but-high-impact outputs currently dominate estimator variance.","Editor's inference: the trajectory-integration design principle—evolve the weight along the actual sampling path rather than evaluating density ratios at fixed endpoints—plausibly generalizes beyond rectified flows to any sampler whose marginal can be represented by a learnable ODE, such as diffusion models, at the price of learning a score function for the perturbed flow.","Editor's inference: a direct testable extension is to train r_phi at scale on one conditional model and compare weighted non-IID expectation estimates against a much larger IID ground truth; the paper explicitly leaves large-scale training of r_phi to future work, so this remains open."],"forward_implications":["If v + r_phi approximates the joint sampler's marginal well, the weighted estimator (1/n) Σ w_i f(X_i) is unbiased for E[f(X)], so diversity-enhanced batches can be used for estimation, not just qualitative coverage.","Trajectory-based integration of the weight ODE avoids feeding r_phi out-of-distribution inputs (final samples evaluated at early times), which the paper shows empirically yields better weight estimates than fixed-position integration.","The closed-form rectified-flow score identity means the whole weighting scheme runs with only the existing velocity evaluations plus a small network r_phi, so unbiased non-IID estimation costs little beyond the diversity sampling itself.","Score-based regularization improves sample quality (log-density, RMSE to mode) while keeping joint mode coverage essentially unchanged across all tested diversity objectives, shrinking the diversity-quality trade-off.","On text-to-image and inpainting models, score-regularized diverse batches achieve lower representation error than IID batches of the same size, meaning fewer wasted samples under a fixed budget."],"fun_headline_variants":["Flow matching with reweighted joint samples: unbiased estimates","Score-regularized diversity enables unbiased flow estimates","Unbiased expectations from diverse flow samples via learned weights","Joint sampling with score regularization and unbiased reweighting"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that v + r_phi is close enough to an ideal rectified flow that the closed-form score identity holds at every intermediate time along the path—an assumption validated only on a synthetic mixture, since the paper does not train the residual velocity for the large image models.","fun_headline_variants_meta":{"raw":{"variants":["Flow matching with reweighted joint samples: unbiased estimates","Score-regularized diversity enables unbiased flow estimates","Unbiased expectations from diverse flow samples via learned weights","Joint sampling with score regularization and unbiased reweighting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1070,"prompt_tokens":705,"completion_tokens":365,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":302}},"tokens_in":449,"tokens_out":365,"duration_ms":14719,"temperature":1.0,"reasoning_tokens":302,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:53:13.787936+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the Gaussian-mixture test bed, check the score identity directly: draw (X0, X1) pairs, condition on intermediate Xt, compare the empirical conditional mean E[X1 − X0 | Xt = x] to the formula (t v − x)/(1−t); a mismatch shows Corollary 1's dynamics does not compute log(p/p''). At scale: train a residual velocity for a large conditional model, estimate E[f(X)] by trajectory-weighted non-IID samples and by a far larger IID ground truth; they should agree within Monte Carlo error while equal-weight diverse samples remain biased.","supporting_citations":[],"review_version":1}