{"id":"d2a5aaa9-6cdc-4c03-a89d-02c4d523b5b9","arxiv_id":"2603.20455","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Time-reversed BSDEs produce adapted adjoints that yield more stable, lower-variance gradients for SOC fine-tuning of diffusion models than non-adapted adjoint matching.","lead":"The paper gives a time-reversed BSDE solver that produces filtration-adapted adjoints for gradient estimation when fine-tuning diffusion models by stochastic optimal control. Adapted adjoints reduce variance relative to the non-adapted adjoints used by adjoint matching, with better stability shown on linear, pendulum, and 1-D toy diffusion tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified beyond the reader's already-flagged PDE regularity assumption.","rationale":"The central claim rests on two pillars that both hold under the paper's stated hypotheses: (i) the adapted adjoint is the conditional expectation of the pathwise adjoint (Sec. III-D), which immediately implies variance reduction by projection onto Ft, and (ii) the time-reversed BSDE (15) together with the iterative regression (16) recovers a function φ whose evaluation supplies that adapted process. The only place where the second pillar can fail is the classical solvability of PDE (8), which the reader already flagged. Because that assumption is conventional, the mathematics is non-circular, and the three toy experiments (with public code) are consistent with the theory, no stronger load-bearing concern is present. The CONDITIONAL verdict and HIGH confidence therefore remain appropriate; the concrete test above simply verifies the claim in the regime where the PDE solution is known exactly.","tokens_in":13748,"tokens_out":557,"duration_ms":6442,"concrete_test":"Re-run the linear-quadratic experiment of Sec. V-A with the exact Riccati solution G(t) as the ground-truth φ and with the same neural-network architecture used for TR-BSDE; if the reported MSE gap versus PNAA remains of the same order across the ε range, the adapted-gradient claim is empirically confirmed under a setting where PDE regularity is known a priori.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest_assumption correctly isolates the load-bearing condition: Sec. IV-A assumes the PDE (8) admits a sufficiently regular classical solution so that (Yt, Zt) = (φ(t, Xt), g⊤ ∂x φ) solves the BSDE and the regression (16) recovers φ. That assumption is standard in the BSDE literature the paper cites and is not contradicted by any internal step. The conditional-expectation identity Yt = E[non-adapted | Ft] in Sec. III-D is derived under the usual linear-BSDE representation and holds whenever the non-adapted process is square-integrable; the numerical claims (lower MSE in Fig. 1a, tighter concentration in Fig. 1b-c, better histograms for larger β in Fig. 2) are consistent with that identity and with the public code. No additional hidden inconsistency or circularity appears in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper formulates fine-tuning of diffusion models as a KL-regularized stochastic optimal control problem and studies gradient estimation for the resulting objective. It contrasts the non-adapted (pathwise) adjoint used by adjoint matching with the adapted adjoint process arising from the stochastic maximum principle BSDE. Under a Markov feedback assumption the authors derive a time-reversed BSDE representation of the adapted adjoint, solve it by alternating forward simulation of the reversed process with regression onto a neural network for the PDE solution φ, and use the resulting adapted gradients to update both the control law and the initial distribution. Theoretical comparison shows that the adapted adjoint is the conditional expectation of the non-adapted adjoint (hence lower variance). Three toy experiments (linear-quadratic MSE, inverted-pendulum initial-distribution optimization, and 1-D bimodal diffusion fine-tuning) indicate improved gradient stability and competitive or superior performance relative to projected non-adapted adjoints and adjoint matching.","tokens_in":13990,"tokens_out":1202,"duration_ms":11252,"significance":"If the claims hold, the work supplies a principled, filtration-adapted alternative to the currently popular adjoint-matching estimator for SOC-based diffusion fine-tuning. The conditional-expectation identity of Sec. III-D is standard but cleanly applied, the time-reversal algorithm is explicit (Algorithms 1–2), and the public code together with the linear-quadratic MSE curves (Fig. 1a, min–max over 10 runs) and the qualitative fine-tuning histograms (Fig. 2) make the numerical claims reproducible. The contribution is incremental relative to the authors’ earlier linear time-reversal papers, yet it is the first systematic demonstration that the adapted adjoint yields measurably more stable gradients on nonlinear diffusion fine-tuning tasks. For the math.OC / generative-modeling interface this is a useful methodological advance, provided the regularity and scalability caveats are acknowledged.","major_comments":[{"comment":"Sec. IV-A, PDE (8) and representation (10): the entire numerical pipeline (Algorithm 1 and the claim that the learned φ supplies accurate adapted gradients) rests on the assumption that (8) admits a sufficiently regular classical solution. The paper only cites the general existence theory of Yong–Zhou and does not supply quantitative conditions (growth, Lipschitz constants, non-degeneracy of G) under which the iterative regression (16) is guaranteed to recover that solution. A short statement of the precise regularity hypotheses needed for the representation and for the regression error to control the gradient error would make the central claim load-bearing rather than formal.","section":null},{"comment":"Sec. V-C and Fig. 2: the fine-tuning comparison is performed only on a one-dimensional bimodal Gaussian mixture with a quadratic tilt. While the histograms show a clear advantage for larger β, the experiment does not stress the high-dimensional regime in which diffusion models are typically used, nor does it report quantitative metrics (e.g., KL or Wasserstein distance to the target tilted law, variance of the gradient estimator across seeds). Without such metrics or a higher-dimensional example, the claim of “competitive performance” and “improved gradient stability” remains only partially substantiated.","section":null},{"comment":"Sec. IV, assumption (7): the derivation of the time-reversed BSDE and the subsequent algorithm require the control to be Markov feedback Ut = k(t,Xt). The original SOC problem (3) allows general adapted controls. The paper never discusses whether the restriction is without loss of optimality for the diffusion fine-tuning objective, nor how the method would extend if a non-Markov control were needed. This gap should be acknowledged or closed.","section":null}],"minor_comments":[{"comment":"Throughout: several typographical inconsistencies (“time-reveresal”, “filteration”, “defiend”, “G ˆateaux”) should be corrected.","section":null},{"comment":"Sec. III-B: the non-adapted adjoint is introduced under the standing assumption g(t,x)=g(t); the subsequent comparison with the adapted process inherits this restriction. A brief remark on the state-dependent diffusion case would clarify the scope.","section":null},{"comment":"Fig. 1b–c: the heat-map scale and the omitted outlying points of PNAA are not quantified; adding a short caption note on the number of discarded points would improve readability.","section":null},{"comment":"Algorithm 2, line 5: the update of (μ,Q) is performed only every five outer iterations; the rationale for this schedule is not stated.","section":null},{"comment":"References: the recent literature on SOC fine-tuning of diffusion models is cited, but a short comparison with path-integral / importance-sampling estimators (beyond the high-variance remark) would situate the contribution more clearly.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a natural nonlinear extension of the authors’ two prior CDC/ACC papers on time-reversed BSDEs. The novelty relative to those works is real but modest; the journal should decide whether the application to diffusion fine-tuning and the accompanying numerical evidence are sufficient for the venue. The public code is a clear plus. I see no integrity issues."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that they take the time-reversed BSDE construction from their own linear-quadratic papers and make it work for the nonlinear Hamiltonian that appears in diffusion fine-tuning. The payoff is an Ft-adapted adjoint that is literally the conditional expectation of the pathwise (non-adapted) adjoint used by adjoint matching. That identity is standard SMP/BSDE theory, cleanly derived in Sec. III-D, and it immediately explains why their gradients should have lower variance.\n\nWhat they do well: the comparison of adapted vs non-adapted adjoints is transparent, Algorithms 1–4 are explicit, the linear-quadratic MSE curves include min–max ranges over 10 runs, and the 1-D diffusion fine-tuning histograms show a clear advantage once the tilt β gets larger. Code is public. Self-citations are to the prior technical pieces that the method rests on; that is fine.\n\nSoft spots are real but proportional. The whole pipeline assumes the PDE that represents the BSDE has a classical solution so that the regression can recover φ; they state the assumption without quantitative conditions, which is standard in the literature they cite but still load-bearing. Everything is low-dimensional (2-D linear, pendulum, 1-D diffusion). Score estimation error is acknowledged but not stress-tested. None of these break the central claim on the toys they actually run.\n\nThis is for people already working on iterative diffusion optimization or path-integral control who care about gradient variance. A serious referee should see it; the math is inspectable and the experiments are reproducible. I would send it out.","headline":"Clean nonlinear extension of their own time-reversed BSDE solver that gives a properly adapted adjoint and visibly lower-variance gradients on the toys they test; useful for the SOC-diffusion niche, not a field-changer.","tokens_in":14595,"tokens_out":435,"would_cite":true,"duration_ms":4766,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93E20","60H10","49N90"],"pacs":[],"model":"grok-4.5","headline":"A time-reversed BSDE gives filtration-adapted adjoints that stabilize gradient estimates for fine-tuning diffusion models.","keywords":["diffusion models","stochastic optimal control","backward SDEs","time reversal","adjoint matching","gradient estimation","fine-tuning"],"falsifier":"On any of the three toy problems, replace the learned φ by the true PDE solution (available analytically for the linear case) and check whether the MSE of the time-reversed estimator remains strictly lower than that of the projected non-adapted adjoint; if the gap disappears or reverses, the claimed variance reduction fails.","tokens_in":14638,"feed_emoji":"🔄","tokens_out":881,"duration_ms":9317,"temperature":0.7,"pith_summary":"Fine-tuning a pre-trained diffusion model so that it samples from a reward-tilted distribution can be cast as a stochastic optimal control problem. Current iterative methods, including adjoint matching, obtain gradients from a pathwise adjoint that is not adapted to the forward filtration and can therefore be high-variance or unstable. This paper shows that the correct adapted adjoint is the solution of a backward stochastic differential equation, and that a time-reversed formulation of that equation lets one simulate the adjoint forward in time after a simple score-matching step. The resulting adapted process is exactly the conditional expectation of the non-adapted adjoint, which automatically reduces variance. On linear and nonlinear toy problems the adapted estimator achieves lower mean-squared error and more reliable optimization; on a one-dimensional fine-tuning task it matches or outperforms adjoint matching, especially at larger tilt strengths.","feed_headline":"Time-reversed BSDEs stabilize diffusion fine-tuning gradients","feed_subtitle":"An adapted adjoint, learned by forward simulation, cuts variance versus pathwise matching","key_machinery":"Time-reversed BSDE: after estimating the score of the controlled diffusion, the original terminal-value BSDE is rewritten as a forward SDE for the pair (Ỹ, Z̃) that starts from the terminal cost gradient; the Markov map φ that recovers the adapted adjoint is then obtained by iterative least-squares regression on the simulated trajectories.","core_discovery":"The adapted adjoint process that supplies unbiased, low-variance gradients for the SOC fine-tuning objective is the conditional expectation of the classical pathwise adjoint, and this process can be recovered by solving a time-reversed BSDE whose solution is learned by regression against forward-simulated trajectories.","pith_inferences":["The same time-reversal-plus-regression pattern could be used to solve other terminal-value BSDEs that arise in continuous-time reinforcement learning or Schrödinger-bridge problems, not only diffusion fine-tuning.","If the score estimate is systematically biased, the learned φ will still be consistent on the support of the reversed trajectories, suggesting a natural robustness that could be quantified by measuring score error versus gradient MSE.","Scaling the method beyond one- and two-dimensional toys will hinge on whether the neural regression for φ remains accurate in the high-dimensional regions visited by the reversed process."],"forward_implications":["Gradient estimates used inside iterative diffusion optimization can be replaced by the adapted process obtained from the time-reversed BSDE, reducing variance without changing the outer optimization loop.","The same construction supplies the Gâteaux derivatives of both the control law and the initial distribution, so joint fine-tuning of policy and start measure becomes a single regression problem.","Because the adapted adjoint is the conditional expectation of the pathwise adjoint, any further Monte-Carlo averaging of the non-adapted process is unnecessary once φ is learned.","The method extends immediately to any control-affine diffusion whose score can be estimated by implicit score matching."],"fun_headline_variants":["Time-reversed BSDEs give adapted adjoints for stable diffusion gradients","Adapted BSDE adjoints reduce variance in diffusion model fine-tuning","Reverse-time BSDEs yield filtration-adapted gradients for SOC fine-tuning","Time-reversed BSDE estimator stabilizes pathwise adjoint gradients","Conditional pathwise adjoints via reverse BSDE for diffusion fine-tuning"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The paper assumes that the nonlinear PDE linked to the BSDE always admits a sufficiently smooth classical solution so that the regression step can recover the true adapted adjoint.","fun_headline_variants_meta":{"raw":{"variants":["Time-reversed BSDEs give adapted adjoints for stable diffusion gradients","Adapted BSDE adjoints reduce variance in diffusion model fine-tuning","Reverse-time BSDEs yield filtration-adapted gradients for SOC fine-tuning","Time-reversed BSDE estimator stabilizes pathwise adjoint gradients","Conditional pathwise adjoints via reverse BSDE for diffusion fine-tuning"]},"model":"grok-4.5","effort":"low","cost_usd":0.006452,"raw_usage":{"total_tokens":1606,"prompt_tokens":700,"num_sources_used":0,"completion_tokens":95,"cost_in_usd_ticks":64520000,"prompt_tokens_details":{"text_tokens":700,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":811,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":700,"tokens_out":95,"duration_ms":7598,"temperature":1.0,"reasoning_tokens":811,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T21:31:09.289197+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On any of the three toy problems, replace the learned φ by the true PDE solution (available analytically for the linear case) and check whether the MSE of the time-reversed estimator remains strictly lower than that of the projected non-adapted adjoint; if the gap disappears or reverses, the claimed variance reduction fails.","supporting_citations":[],"review_version":1}