Pith. sign in

REVIEW 3 major objections 4 minor 2 references

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pre-trained GFlowNets can be composed at inference time to sample weighted-sum rewards exactly, with no retraining.

desk verdict Reaching-probability policy mixing is new and the linear exactness proof is correct under exact flows, but the paper's Remark (ii) about estimated Z_i is wrong and the real-world β≠1 case is not covered by the theory. read the letter →

arxiv 2602.21565 v3 pith:GXI3ZKRA submitted 2026-02-25 cs.LG

Seokwon Yoon , Youngbin Choi , Seunghyuk Cho , Seungbeom Lee , MoonJeong Park , Dongwoo Kim This is my paper
classification cs.LG
keywords GFlowNetsmulti-objectivegenerationmodelcompositionreachingprobabilitytraining-freeinferencemolecularscalarizationlogicaloperators
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a training-free way to combine several already-trained GFlowNets into one sampler for a multi-objective reward. The key move is to mix each model's action distribution in proportion to how likely that model is to reach the current state — its reaching probability u_i(s)=F_i(s)/Z_i. For linear scalarization (weighted sums of rewards), the paper proves the mixed policy exactly induces the target distribution, something no previous composition method guarantees without extra training. For non-linear combinations (logical operators like conjunction and subtraction, and sharpened rewards with β≠1), exactness generally fails, but the gap is characterized by a distortion factor and shown empirically to be small precisely where the target distribution has mass. A reader would care because it turns a set of single-objective generative models into a multi-objective toolkit at essentially zero adaptation cost.

What carries the argument

The mixing policy of Eq. (4): at each state, weight each base model's forward transition by its reaching probability u_i(s)=F_i(s)/Z_i, combine through the composition operator G, and renormalize locally. The reaching probability is the central object — it converts a state's flow into the probability that the model would actually visit it, and it is what makes the weighted mixture of policies induce the mixture of terminal distributions. For scalarization the normalization collapses to u_M(s), making the mixture a convex combination of base policies and yielding the exactness proof.

What would settle it

Train two GFlowNets on a small grid using trajectory balance only, so no explicit state flow F_i(s) is learned; estimate u_i(s) from samples or a surrogate, run the mixing rule for a two-objective weighted sum, and compare the empirical terminal distribution to the exact target Σω_iR_i/Z. If the L1 gap does not vanish as samples grow, the exactness claim fails when reaching probabilities are approximate — confirming that flow-parameterizing training is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is Proposition 4.1: given k GFlowNets with terminating distributions p_i(x) ∝ R_i(x), the mixing policy p_{M,F}(s'|s) ∝ Σ_i ω_i Z_i u_i(s) p_{i,F}(s'|s) yields an induced distribution p_M(x) ∝ Σ_i ω_i R_i(x) — the exact weighted-sum target. The same framework handles non-linear operators through the same formula with a general composition function G, where the ratio δ(x)=u_M(x)/N_M(x) quantifies the deviation from the target. Experiments on a 2D grid and on fragment- and atom-based molecular generation show that the approach matches or beats preference-conditioned and classifier-guided baselines that require retraining, while keeping inference cost low and preserving molecu

Load-bearing premise

Everything rests on the per-state reaching probabilities u_i(s)=F_i(s)/Z_i being accurate for every state the mixed sampler can visit; if a learned state flow is wrong where the mixed policy goes — which can be off-support for an individual base model — the claimed induced distribution is not the target, and for non-linear operators there is no error bound to lean on.

Editorial extensions

If this is right

  • Any new scalarization weight vector can be sampled immediately from a fixed set of pre-trained models, without retraining or fine-tuning, enabling rapid exploration of reward trade-offs.
  • With flow-parameterizing training objectives (flow matching, detailed balance, sub-trajectory balance), the mixed policy realizes p ∝ Σ ω_i R_i exactly, so multi-objective sampling costs only k forward passes per step.
  • For non-linear operators (harmonic mean, contrast, and β≠1 scalarization), the induced distribution's L1 distance to the target factors as the weighted distortion |δ(x)−1/Z|·G(p_1,…,p_k), and experiments show distortion stays near constant in high-density regions.
  • Logical compositions such as conjunction and subtraction can be sampled without training an auxiliary classifier, and the method preserves near-perfect validity of generated molecules, which classifier guidance degrades.
  • For logical operators, inference is roughly 40–70× faster than classifier-guided composition because the mix requires computation only at the current state rather than over all successor states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The proof's structure suggests the mixing identity is not GFlowNet-specific: any collection of stochastic constructors that can report per-state occupancy probabilities should satisfy the same exactness for scalarization, so the technique could port to other sequential samplers (e.g., diffusion or MCMC) with analogous occupancy estimates.
  • The distortion-factor analysis offers a practical diagnostic for non-linear operators: monitor δ(x) on live samples; if it stays flat near the high-density region (as on the grid), the sampler quality is reliable even where no L1 guarantee exists.
  • A testable extension: for β≠1 scalarization, the paper mixes reward-sharpened models; one could train base models at several temperatures β and mix across them to modulate the diversity-versus-peak-reward trade-off without retraining.
  • Since exactness relies only on reaching probabilities, the method could be combined with learned state-flow approximations that are actively updated on the mixed policy's support, potentially tightening the approximation for non-linear operators over time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper proposes a training-free procedure for composing pre-trained GFlowNets by mixing their forward policies at inference time. For each component model i and state s, the mixing policy uses the reaching probability u_i(s) (equivalently normalized state flow F_i(s)/Z_i) as the weight in a composition operator G. The paper proves (Proposition 4.1, Appendix A.1) that for linear scalarization with β=1, the induced terminal distribution is exactly proportional to Σ_i ω_i R_i(x), assuming exact base distributions and exact reaching probabilities. For nonlinear composition operators (β≠1, harmonic mean, contrast), the paper introduces a distortion factor δ(x)=u_M(x)/N_M(x) and derives an exact L1 decomposition in terms of δ. Experiments on a 32×32 grid and on fragment- and atom-based molecule generation compare the method against preference-conditioned GFlowNets (MOGFN, HN-GFN) and classifier-guided compositional sculpting, reporting comparable or better sample quality and substantially lower inference cost for logical operators.

Significance. The core idea—flow-weighted mixing of forward policies to realize a weighted-sum reward target without retraining—is elegant and practically useful. The algebra of Proposition 4.1 is sound under the stated idealization, and the synthetic grid experiments validate the distortion-factor decomposition. The paper also demonstrates a clear practical advantage: for logical operators it is 40–70× faster at inference than classifier guidance, while avoiding auxiliary classifier training. If the practical caveats concerning estimated flows and partition functions are resolved, this would be a valuable unified framework for multi-objective GFlowNet composition. The paper is honest in separating the exact linear case from the approximate nonlinear case, though the nonlinear analysis is empirical rather than bounded.

major comments (3)
  1. [Appendix A.1, Remark (ii); Eq. (11)–(14)] The remark that using estimated partition functions \hat Z_i only rescales the target is false as stated. If \hat Z_i replaces Z_i in Eq. (11), the induced terminal distribution becomes Σ_i ω_i(\hat Z_i/Z_i) R_i(x) / (Σ_j ω_j \hat Z_j), which equals the claimed target only when all ratios \hat Z_i/Z_i are equal. However, the exactness result does not actually require Z_i: in Eq. (14) the factors cancel, yielding pM,F(s'|s) ∝ Σ_i ω_i F_i(s) p_i,F(s'|s). The paper should state this cancellation explicitly and correct Remark (ii); as written, the algorithm description in Section 4.1 (computing u_i(s)=F_i(s)/Z_i) invites the erroneous use of \hat Z_i and leaves the reader with a false robustness claim.
  2. [Section 4.1, Reaching probability estimation; Appendix A.2] The practical algorithm requires learned state flows F_i(s) at every state encountered by the mixed policy. A pre-trained GFlowNet's flow estimator is trained on trajectories from its own policy; the mixed policy can visit states that are off the training support (or have very low probability) for one or more components, where F_i(s) is an unconstrained extrapolation. The theorem assumes exact reaching probabilities, and no sensitivity analysis, error bound, or diagnostic is provided for the realistic case of inaccurate off-support flows. This is especially relevant for the molecule experiments, where the state space is large. The authors should at least state this limitation explicitly and ideally add an experiment that perturbs F_i(s) or compares against an on-support-only variant.
  3. [Tables 3 and 4] The empirical comparisons are reported as single numbers without standard deviations, repeated seeds, or significance tests. Several headline differences are tiny (e.g., SEH-QED fragment average reward 0.772 vs 0.773; GAP-QED QM9 0.787 vs 0.788), so the claims that the method 'matches' or 'outperforms' baselines are not statistically supported as presented. The authors should run multiple seeds and report means/standard deviations (or equivalent uncertainty quantification) for the main comparisons. This is particularly important because the conclusion rests partly on the claim of competitive performance against trained baselines.
minor comments (4)
  1. [Abstract; Section 4.2] The paper says it 'quantifies' the approximation quality for nonlinear operators through a distortion factor, but Section 4.2 only defines δ(x) and gives an exact L1 re-expression; it provides no bound or quantitative characterization. Consider rephrasing to 'analyzes' or 'characterizes'.
  2. [Eq. (13)–(14) and Section 5.3] The proof introduces v_i and u_M(s) = Σ_i v_i u_i(s), but the policy in Eq. (14) can be written directly as Σ_i ω_i F_i(s) p_i,F(s'|s) / Σ_j ω_j F_j(s). Making this simplification explicit would clarify why partition functions are not needed and would connect more directly to the algorithm.
  3. [Fig. 3 and Fig. A6] The IQR-based outlier definition is arbitrary and is not tied to the L1 decomposition in Eq. (9). Reporting the actual contribution of outlier states to the total L1 error would be more informative than a qualitative outlier designation.
  4. [Section B.2.1, Eq. (25)–(26)] The reward-sharpened mixing formula for β≠1 is introduced as a heuristic extension, but this is not stated explicitly. To avoid confusion with the exact β=1 result, please add a sentence clarifying that for β≠1 the same distortion-factor analysis applies and no exactness is claimed.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: Proposition 4.1 is derived, not assumed; the main caveat is an unsupported claim about estimated partition functions in Appendix A.1 Remark (ii).

full rationale

The central derivation (Proposition 4.1, Appendix A.1) is mathematically self-contained. The mixing policy Eq. (4) is defined from base forward policies and reaching probabilities, and the proof uses Lemmas A.1-A.3 to show the induced terminal distribution equals p_M(x) = sum_i v_i p_i(x), with v_i = omega_i Z_i / sum_j omega_j Z_j. Since p_i = R_i/Z_i, this equals the target sum omega_i R_i normalized by sum omega_j Z_j. The target is not assumed inside the policy; the policy weights are fixed by the base models' partition functions, and no parameter is fitted to the composed target. The L1 experiments compare the induced distribution against externally defined rewards, so the evaluation is not circular. No load-bearing self-citation or uniqueness theorem is invoked. The one substantive caveat is Appendix A.1 Remark (ii): 'If estimated partition functions \hat Z_i are used in place of Z_i, the sampler targets the scalarized reward up to a global rescaling, which does not affect the induced distribution.' Substituting \hat Z_i yields p_M(x) proportional to sum_i omega_i (\hat Z_i/Z_i) R_i(x), which equals the intended target only if all \hat Z_i/Z_i are equal; independent estimation errors can distort the target. This is a correctness/robustness gap in the practical instantiation, not a circularity, because the theorem's statement assumes exact Z_i and the induced distribution is not defined as the target by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The method inherits standard GFlowNet assumptions (exact base training, DAG structure, flow consistency) and adds two practical requirements: flow-parameterizing training objectives and accurate per-state flows under the mixed policy. No new entities are postulated; the distortion factor is a descriptive diagnostic, not an invented mechanism. The hand-chosen β and evaluation thresholds are the main parameters affecting reported results.

free parameters (2)
  • temperature β for reward sharpening in molecule experiments = β=32 (QM9), 64 (fragment scalarization), 32 (fragment logical operators)
    Hand-chosen per task following prior work; the exactness theorem only covers β=1, so the real-world scalarization results rely on the distortion factor being benign at β=32/64 without a bound.
  • 8-bin reward thresholds for logical-operator evaluation = SEH 0.5, SA 0.6, QED 0.25 (fragment); GAP 0.85, SA 0.4, QED 0.4 (QM9)
    Hand-chosen per task and per reward; the target-bin percentages in Tables 4 and A3-A6 depend directly on these thresholds, and no principled selection criterion is given.
assumptions (5)
  • domain assumption The pre-trained GFlowNets are exactly trained, i.e. p_i(x) = R_i(x)/Z_i on the shared state DAG
    The exactness theorem (Prop. 4.1) assumes this; in practice SubTB converges approximately, and the residual L1=0.003 on the grid reflects this.
  • domain assumption Learned state flows F_i(s) and partition functions Z_i are available and accurate for all states reached by the mixed policy
    Section 4.1 and Appendix A.2: u_i(s) = F_i(s)/Z_i is used at every inference step; TB-trained models are excluded; off-support flow errors would break the claimed target.
  • domain assumption The k base GFlowNets share the same state graph (S, A), so the local normalization N_M(s) in Eq. (5) is well-defined across all components
    Stated in Appendix A.1 Remark (iii); composing models trained on different state spaces is out of scope.
  • standard math Reaching-probability recursion u_i(s) = Σ_{s*} u_i(s*) p_{i,F}(s|s*) holds for the DAG state graph
    Standard GFlowNet result (Eq. 2), needed by Lemma A.2.
  • standard math Non-negative composition function G and finite target normalization Z_M
    The target p*_M ∝ G(...) requires 0 < Z_M < ∞ (Section 4.2, Eq. 20).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation." pith.science (2026). https://pith.science/paper/GXI3ZKRA

@misc{pith2026260221565,
  author       = {Pith},
  title        = {Pith review of: Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GXI3ZKRA}},
  note         = {Machine review of arXiv:2602.21565}
}
read the original abstract

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest as real-world applications often involve multiple, conflicting objectives. However, existing approaches require joint training for each combination of objectives, meaning that any change in the objective set necessitates retraining from scratch. We propose a framework that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without fine-tuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex nonlinear operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization, and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule generation tasks demonstrate that our approach achieves performance comparable to baselines.

Figures

Figures reproduced from arXiv: 2602.21565 by the authors.

Figure 1
Figure 1. Qualitative result of scalarization on a 2D grid domain. We visualize the density on each grid of the true distribution (left), MOGFN (middle), and ours (right). MOGFN (Jain et al., 2023b) and HN-GFN (Zhu et al., 2023). For logical operators, we compare against the classifier guid￾ance approach of Garipov et al. (2023). For both settings, we also include a simple ensemble baseline that mixes forward policies without… view at source ↗
Figure 2
Figure 2. Qualitative result of logical operations on a 2D grid do￾main. We visualize the density on each grid of the true distribution (left), classifier guidance (middle), and ours (right). the qualitative results. Our mixing policy more accurately recovers the multi-modal structure of the true distribution, whereas MOGFN produces slightly blurred modes. Logical operators [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Distortion factor δ(x) = uM(x)/NM(x) vs. unnormalized target density G(p1(x), p2(x)) on the 2D grid domain. Red points indicate outliers, defined as states where δ(x) falls outside [Q1 − 1.5 · IQR, Q3 + 1.5 · IQR] (gray dotted lines), where Q1, Q3 are the 25th/75th percentiles and IQR = Q3 − Q1. The green dashed line marks 1/ZM with ZM = P x G(p1(x), p2(x)). Additional results are in Fig. A6. mixing policy. Both suc… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Density visualization of pSEH, pQED, and the com￾posed distribution pM induced by our mixing policy via scalar￾ization on fragment-based molecule generation. Each distribu￾tion is estimated from 5,000 samples. We vary the weights as (a) pM ∝ (0.3 · RSEH + 0.7 · RQED) 3…
Figure 5
Figure 5. Figure 5: Distributions pM induced by classifier-guided and our mixing policy for fragment-based molecule generation. Each distribution is estimated from 5,000 samples. Base distributions pSEH and pSA are shown alongside the composed distribution. (a) Classifier-guided mixing po…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

2 extracted references · 1 linked inside Pith

  1. [2020]

    proxy trained on QM9, normalized using percentile-based scaling and clipped to[0,2] ;SAis the synthetic accessibility score (Ertl & Schuffenhauer, 2009), computed using RDKit (Landrum, 2010) and transformed as (10−SA raw)/9 to map to [0,1] ; andQEDis the quantitative estimate of drug-likeness (Bickerton et al., 2012), computed using RDKit (Landrum, 2010),...

    2009

  2. [4848]

    Jo, J., Lee, S., and Hwang, S

    PMLR, 2020. Jo, J., Lee, S., and Hwang, S. J. Score-based generative modeling of graphs via the system of stochastic differen- tial equations. InInternational conference on machine learning, pp. 10362–10383. PMLR, 2022. Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. InInternational Conference on Learning Representations, 2015. Koh, H...

    arXiv 2020

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.