{"id":"8a605ef8-468f-4a05-85a9-a5b0c8a7625f","arxiv_id":"2505.18239","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A unified backward-filtering and forward-guiding scheme provides weighted posterior samples for latent processes on trees and DAGs with intractable transition densities.","lead":"This paper introduces a general inference framework, BFFG, for smoothing latent stochastic processes on trees and directed acyclic graphs when data are observed only at leaves. It works by a backward filtering step that builds potentials from observations, then a forward guided simulation under a change of measure, yielding weighted posterior samples.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Continuous-time exactness unverified: in the SDE example Ag/g is unbounded, so Proposition A.3's good-function condition fails and the weight estimator may be biased.","rationale":"The reader identified the user-supplied good function g as the weakest assumption, and noted that Proposition A.3 conditions are not verified in the numerical examples. My stress-test sharpens this into a specific, testable failure mode: in the continuous-time SDE example of Section 9.1, the auxiliary linear-Gaussian g is log-quadratic while the true process has bounded tanh drift and different diffusion, so Ag/g grows quadratically. This makes the sufficient condition of Proposition A.3 fail on the nose. The paper provides no alternative proof of martingality, and the numerical experiments do not check weight unbiasedness. If the exponential local martingale has expectation strictly less than 1, the guided measure is defective and the weighted-sample output is biased, contradicting the paper's central claim. The concern is concrete and can be settled by comparing the Monte Carlo average of the full weight to an independent likelihood estimate. Because the paper's mathematical framework may still be correct with a verified good g, the appropriate verdict remains conditional: the central claim is plausible but not yet established for the presented continuous-time examples. I therefore do not recommend changing the reader's verdict.","tokens_in":27140,"tokens_out":15012,"duration_ms":160145,"concrete_test":"For a small version of the SDE example (e.g., a tree with 3 leaves), simulate R = 10^5 independent guided paths under P° for a fixed parameter vector θ, computing the full unnormalized weight Ψ(X°;θ) = g_r(x_r) * prod_e exp(∫_0^{τ_e} (Ag/g)(u, X°_u) du). Compare its empirical mean to an independent, high-accuracy estimate of h_r(x_r;θ) obtained by deterministic numerical solution of the Kolmogorov backward equation (9) on the same tree. If the two estimates differ by more than 2-3 Monte Carlo standard errors, the exponential change of measure is not a true martingale, and the central exactness claim fails for this example.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of exact weighted sampling in the continuous-time case depends on the exponential change of measure Z^g being a true martingale. Section 4.2 and Proposition A.3 give sufficient conditions for g to be 'good' (boundedness of g and Ag/g). In the SDE example (Section 9.1), the guided process is built from a log-quadratic g (Section 7.1) and the true process has bounded tanh drift with constant diffusion. Then Ag/g = (L - L~)g/g, which includes a quadratic term of the form (F(u)-H(u)x)'(a - a~)(F(u)-H(u)x)/2. Since the diffusion coefficients a and a~ differ, this term grows quadratically in x, making Ag/g unbounded on R^2. Proposition A.3 therefore does not apply. The paper supplies no alternative verification that Z^g is a true martingale or that the weight product has the correct expectation. If Z^g is only a local martingale with E[Z^g_τ] < 1, then P° is not the intended probability measure and the estimator g_r(x_r) * prod(exp(∫ Ag/g du)) is biased for h_r(x_r), so the weighted samples do not target P⋆. This is not a missing cosmetic check; it is an unverified condition that underpins the exactness theorem in the continuous-time setting.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces Backward Filtering Forward Guiding (BFFG), a framework for smoothing in discrete- and continuous-time stochastic processes on trees and DAGs. The backward step computes likelihood-informed potentials g from the leaves toward the root, typically using a tractable approximate dynamics; the forward step simulates a guided process under an exponential change of measure built from g. The central claim is that this yields weighted samples from the posterior distribution over latent paths, with weights given in Theorem 4.2 for discrete edges and Proposition 4.4 for continuous edges. The authors provide closed-form backward filtering, simulation, and weight formulas for several families: nonlinear Gaussian kernels, finite-state Markov chains, branching particles, Gamma increment processes, SDEs with linear Gaussian guiding, continuous-time Markov chains, and a Wright-Fisher diffusion approximation. A DAG extension is given in Section 8, and numerical experiments on tree-structured diffusions and a Kunita flow are reported.","tokens_in":27495,"tokens_out":14413,"duration_ms":124855,"significance":"If the exactness conditions are met, this is a useful unification: it connects message-passing on graphical models with guided-diffusion importance sampling and makes the backward/forward computations explicit for several tractable families. The paper is also honest about the role of g as a user-supplied approximation that is corrected by weights, and it benefits from the authors' earlier work on guided diffusions. The discrete-time results are self-contained and appear correct, and the availability of code is a concrete strength. However, the continuous-time exactness claim rests on an unverified martingale property that appears to fail for the main numerical example, and the DAG extension contains a normalization error and relies on conditional distributions that are never constructed. These issues affect the central claims and require substantial revision.","major_comments":[{"comment":"In the continuous-time setting, exactness in Proposition 4.4 requires that the exponential local martingale Z^g be a true martingale. Proposition A.3 gives sufficient conditions: bounded g and bounded Ag/g, or bounded g and Ag with positive infimum. In the SDE example of Section 9.1 the potential is log-quadratic as in Section 7.1, and using the auxiliary linear Gaussian process, Remark 5 gives (L-L~)g/g = sum_i (b_i-b~_i)r_i + (1/2) sum_{ij}(a_{ij}-a~_{ij})(partial_j r_i + r_i r_j) with r_i = F_i - H_i x. Since the diffusion coefficients a and a~ are different constant matrices and b~ is linear while b is bounded, this integrand is unbounded on R^2; hence g is not in B(E) and Ag/g is not in B(E). The paper supplies no alternative verification, such as a Novikov-type condition or a direct density computation, that E[Z^g_T]=1. Without this, P^o is only a subprobability and the weight estimator g_r(x_r) exp(integral Ag/g du) is not shown to be unbiased for h_r(x_r), so the weighted samples are not shown to target P*. This is a load-bearing gap in the main continuous-time numerical demonstration.","section":"Section 8 / Proposition 8.2 / Appendix B"},{"comment":"The DAG extension is built on a KL projection onto product form, but the normalization in the proof of Proposition 8.2 is incorrect. The stationarity equation integrates to lambda_j c_j = integral pi(x)g(x) dx, not to lambda_j c_j = 1; the proof silently assumes integral pi g = 1. Consequently the minimizer formula g_i(x_i)=c_i^{-1} E_pi[g(X)|X_i=x_i] is only valid under this unstated normalization, and with the choice c_i=1 made in the text it generally violates the stated constraint when integral pi g is not 1. Moreover, the conditional distributions pi(x_-u | x_u) required in Eqs. (40)-(41) are never constructed or specified in any example, and no guidance is given for choosing the auxiliary pi. Until this is repaired, Theorem 8.1's claim that Theorem 4.2 applies to arbitrary DAGs is not established. The DAG version of Eq. (15) also writes g_0(x_0) in the denominator, although Figure 2 explicitly has two roots; the denominator should be a product over all roots.","section":"Section 8, Eqs. (40)-(41), Appendix B"}],"minor_comments":[{"comment":"There is a typo: \"continuoos-time\" should be \"continuous-time\".","section":"Section 1.1"},{"comment":"The phrase \"we call an edge... discrete if it is assumed that onethe probabilistic evolution\" contains a textual error; \"onethe\" should be removed or corrected.","section":"Section 3.1, Definition 3.1"},{"comment":"The sentence \"just like in message passing algoritheorems in general\" appears to contain the typo \"algoritheorems\".","section":"Section 6.1"},{"comment":"The notation in the paragraph defining the auxiliary process is inconsistent: the model in Eq. (42) uses parameters (theta_0, theta_1, sigma_0, sigma_1), but B is written with theta_1 and theta_2 and sigma~ with sigma_1 and sigma_2. Please clarify which parameters are used.","section":"Section 9.1"},{"comment":"The caption contains the typo \"valews\" for \"values\".","section":"Figure 4 caption"},{"comment":"The text says Z^g_u is a \"mean-one F_u-local martingale\"; a local martingale is not automatically mean-one, and for a nonnegative local martingale the property E[Z^g_u]=1 is equivalent to being a true martingale. This should be stated as an assumption, not as a consequence of Corollary A.2.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the continuous-time part of the paper is in large part a repackaging of the authors' earlier guided-diffusion results (Mider et al. 2021), and the new tree/DAG message-passing wrapper is the main claimed novelty. That is acceptable for a methodology paper, but the revision should make the boundary explicit. The numerical validation is entirely self-contained; adding a small-tree benchmark against an exact smoother would help establish that the method is not only internally consistent. My recommendation of major revision is driven by the unverified martingale property and the DAG normalization error, both of which affect claims that are central to the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Frank,\n\nThe BFFG paper is a serious piece of work. What is genuinely new: a single template covering discrete and continuous edges on trees and DAGs, with exact importance weights expressed through the good-function framework, plus a DAG weight formula (Theorem 8.1) I have not seen elsewhere. The paper also credibly recovers Ju et al. (2021) and Stoltz et al. (2021) as special cases. The theoretical core, Theorem 4.2 and its DAG variant, is derived cleanly; the telescoping in the density ratio is neat. The authors are also honest that the backward filtering step is problem-specific and not automatic. The numerical examples are self-contained but not benchmarked.\n\nThe main soft spot is exactly where the stress-test lands. In the continuous-time SDE example, the guided process is built from a log-quadratic g, and the forward and auxiliary diffusions differ (even ignoring the apparent typos with undefined theta2 and sigma2). Then Ag/g contains a quadratic term in the state, so it is unbounded. Proposition A.3 therefore cannot be applied, and the paper provides no other verification that Z^g is a true martingale. Since the weight in Proposition 4.4 is the exponential change of measure, a merely local martingale makes the weighted samples biased for the smoothing posterior. This is not a cosmetic check; it underpins the exactness claim for all continuous-edge examples. The discrete cases do not have this problem, and the DAG extension inherits it only when continuous edges are present.\n\nThe DAG section is more of a recipe than a fully worked algorithm: the pullback kernel in (41) requires conditionals pi(x_{-u} | x_u), which are introduced abstractly and never constructed for a concrete example. That is a real gap, though not a fatal one. The paper would also be stronger with at least one baseline comparison in the numerics, and the undefined theta2 and sigma2 notation should be fixed.\n\nBottom line: the paper deserves a serious referee. The formalism is useful, the discrete-side theory is solid, and the DAG weight formula is a contribution. But the continuous-time exactness needs either verified good-function conditions in the examples or an explicit statement that the method is approximate in that regime. I would send it out, with that as the main referee request.","headline":"Useful unification of guided inference on trees and DAGs, but the continuous-time exactness claim rests on an unverified martingale condition.","tokens_in":28016,"tokens_out":3799,"would_cite":false,"duration_ms":31744,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60J05","60J25","60J27","60J60","62M05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Exact weighted smoothing on trees and DAGs is possible without transition densities, via a backward information filter followed by a forward guided process.","keywords":["backward information filter","guided process","smoothing","Doob h-transform","exponential change of measure","directed acyclic graph","branching diffusion","particle filtering"],"falsifier":"On a linear-Gaussian tree, where forward-filtering backward-sampling gives exact smoothing expectations, run BFFG with a deliberately misspecified $g$ and increase the number of guided samples; if the weighted average of a bounded test function does not converge to the FFBS expectation, the condition $P^\\star \\ll P^\\circ$ or the weight identity in Theorem 4.2 fails. On a DAG with a collider, compute the weight in (38) numerically: if the denominator $\\prod_{u\\in\\mathrm{pa}(s)} g_{u,s}(x_u)$ vanishes on a set of positive prior probability, the Radon-Nikodym derivative is infinite and the exactness claim breaks down.","tokens_in":26936,"feed_emoji":"🌳","tokens_out":9249,"duration_ms":69269,"temperature":0.7,"pith_summary":"This paper introduces Backward Filtering Forward Guiding (BFFG), a smoothing method for latent stochastic processes on trees and directed acyclic graphs where only leaf-node values are observed. The central claim is that by combining a backward information filter that approximates the likelihood-informed potentials with a forward guided process built from those potentials, one obtains weighted samples that are exactly from the smoothing posterior. Exactness hinges on an importance-weight identity, Theorem 4.2, which corrects for the gap between the true h-transform and the tractable approximation g, so the method works even when transition densities are intractable or unavailable. If correct, BFFG extends smoothing to branching diffusions, continuous-time Markov chains, interacting particle systems, and other structured dynamics, and it admits integration with MCMC, particle filters, and probabilistic programming.","feed_headline":"Two-pass algorithm gives exact smoothing on trees and DAGs","feed_subtitle":"Backward filtering builds approximate potentials; forward guiding corrects them with exact importance weights.","key_machinery":"The engine is Doob's h-transform run twice. The backward information filter computes, exactly where possible, functions $h_t(x)$ equal to the density of the leaf observations in the subtree below $t$ given $X_t=x$; when this is intractable, a user-selected family $g$ is substituted so that every operation — pullback under a Markov kernel, fusion at a branching vertex, and initialisation at a leaf — closes in closed form. On continuous edges the guided process is defined by an exponential change of measure with density $Z^g_u = \\frac{g_u(X_u)}{g_0(x_s)}\\exp\\!\\left(-\\int_0^u \\frac{Ag}{g}\\,d\\tau\\right)$, and a function $g$ is called 'good' when $Z^g$ is a true martingale. The guided process then evolves with kernels $\\kappa^\\circ(x,dy) \\propto g_s(y)\\,\\kappa(x,dy)$ on discrete edges and the corresponding generator on continuous edges, and the Radon-Nikodym weight $\\exp\\!\\left(\\int_0^{\\tau_e} \\frac{Ag}{g}\\,du\\right)$ is precisely the correction that appears in Theorem 4.2.","core_discovery":"The central claim is that the smoothing distribution on a tree or DAG can be represented exactly by a weighted sample from a guided process that preserves the forward model's dependency structure. Theorem 4.2 gives the identity $E^\\star f(X_S) = \\frac{g_r(x_r)}{h_r(x_r)}\\, E^\\circ\\!\\left[ f(X_S)\\, \\prod_{s\\in S\\cup V} w_{\\mathrm{pa}(s),s}(X_{\\mathrm{pa}(s)}) \\right]$, where $P^\\star$ is the conditional law given leaf observations, $P^\\circ$ is the law of the guided process, and the edge weights $w$ are ratios of kernel pullbacks under $g$ and the true $h$. The same identity extends to DAGs (Theorem 8.1) after redefining the denominator of the weights to run over all parents of each vertex. A direct corollary is that the guided weight forms a positive unbiased estimator of the likelihood $h_r(x_r)$, making BFFG usable inside SMC, MCMC, and likelihood-based parameter estimation even when no transition density is available in closed form.","pith_inferences":["The practical fidelity of BFFG is set by the choice of $g$: if the true $h$ is far from the tractable family, the weights in Theorem 4.2 gain variance and the scheme degrades toward naive importance sampling, so methods that adaptively refine $g$ from the empirical weight distribution would make the approach self-tuning.","On DAGs the product-normalisation in the weights of Theorem 8.1 keeps the guided process Markovian, but its quality depends on the KL projection in Proposition 8.2; a testable extension is whether a learned non-product approximation of $g$ at colliders beats the product form.","The compositional property proved in the companion category-theory paper suggests that BFFG is a natural compiler primitive: any probabilistic program written as a composition of Markov kernels could be automatically conditioned by rewriting each kernel in its guided form, leaving the backward information filter as the only inference-specific component.","The butterfly-wing example positions BFFG as a candidate engine for phylogenetic trait-evolution models on high-dimensional shape spaces; a head-to-head comparison against particle marginal Metropolis-Hastings on the same phylogeny would quantify when the non-resampling likelihood estimator wins."],"forward_implications":["Any model whose forward process can be simulated, and for which a tractable approximation to the h-transform can be found, becomes amenable to smoothing; transition densities no longer need to be known.","The weight formula yields a positive unbiased estimate of the likelihood, so parameter estimation can proceed by plugging BFFG into existing SMC or MCMC samplers, including gradient-based schemes where resampling non-differentiability is avoided.","Because the guided process inherits the forward conditional-independence structure, the same program that simulates the unconditional process can be transformed to simulate the guided one, opening a route to automatic conditioning inside probabilistic programming languages.","The DAG extension (Theorem 8.1) makes smoothing possible on graphs with multiple parents, at the cost of an additional product-normalisation in the weights.","The framework unifies prior algorithms: the discrete SMC computations of Ju et al. and the diffusion-tree backward filtering of Stoltz et al. appear as special cases."],"supporting_citations":[{"why":"Supplies the 'good function' martingale condition that defines the guided process on continuous edges via exponential change of measure.","marker":"Palmowski and Rolski (2002)"},{"why":"Provides the martingale theorem used in the appendix to turn the h-transform into a mean-one local martingale for the guided process.","marker":"Ethier and Kurtz (1986)"},{"why":"Source of the backward-information-filter recursions that BFFG adapts to trees and continuous edges.","marker":"Briers et al. (2010)"},{"why":"Gives the closed-form Gaussian-potential backward filtering for linear SDEs that Theorem 7.1 relies on for continuous-edge pullbacks.","marker":"Mider et al. (2021)"},{"why":"Introduced guided proposals for partially observed diffusions, the lineage from which the SDE guided-process construction and its weight formula descend.","marker":"Schauer et al. (2017)"}],"fun_headline_variants":["Two-pass algorithm yields exact smoothing on trees and DAGs","Backward filtering + forward guiding = exact smoothing on trees","Exact posterior paths on DAGs with intractable transition densities","Smoothing without transition densities: two-pass exact inference","Guided simulation gives exact posterior over latent paths"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For every edge the user must supply a tractable function $g$ that approximates the true backward potential $h$ well enough, can be pulled back and fused in closed form, and yields a guided process that can be simulated and weighed; the paper states that this backward-filtering step is problem specific and not automatic.","fun_headline_variants_meta":{"raw":{"variants":["Two-pass algorithm yields exact smoothing on trees and DAGs","Backward filtering + forward guiding = exact smoothing on trees","Exact posterior paths on DAGs with intractable transition densities","Smoothing without transition densities: two-pass exact inference","Guided simulation gives exact posterior over latent paths"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001059,"raw_usage":{"total_tokens":4442,"prompt_tokens":944,"completion_tokens":3498,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":3416}},"tokens_in":560,"tokens_out":3498,"duration_ms":18506,"temperature":1.0,"reasoning_tokens":3416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:38:06.111129+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a linear-Gaussian tree, where forward-filtering backward-sampling gives exact smoothing expectations, run BFFG with a deliberately misspecified $g$ and increase the number of guided samples; if the weighted average of a bounded test function does not converge to the FFBS expectation, the condition $P^\\star \\ll P^\\circ$ or the weight identity in Theorem 4.2 fails. On a DAG with a collider, compute the weight in (38) numerically: if the denominator $\\prod_{u\\in\\mathrm{pa}(s)} g_{u,s}(x_u)$ vanishes on a set of positive prior probability, the Radon-Nikodym derivative is infinite and the exactness claim breaks down.","supporting_citations":[{"cited_title":"and Rolski, T","cited_arxiv_id":null,"evidence_quote":"Supplies the 'good function' martingale condition that defines the guided process on continuous edges via exponential change of measure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the martingale theorem used in the appendix to turn the h-transform into a mean-one local martingale for the guided process."},{"cited_title":"and Maskell, S","cited_arxiv_id":null,"evidence_quote":"Source of the backward-information-filter recursions that BFFG adapts to trees and continuous edges."},{"cited_title":"and Van der Meulen, F","cited_arxiv_id":null,"evidence_quote":"Gives the closed-form Gaussian-potential backward filtering for linear SDEs that Theorem 7.1 relies on for continuous-edge pullbacks."},{"cited_title":"and van Zanten , H","cited_arxiv_id":null,"evidence_quote":"Introduced guided proposals for partially observed diffusions, the lineage from which the SDE guided-process construction and its weight formula descend."}],"review_version":1}