Relative Trajectory Balance is equivalent to Trust-PCL

Doina Precup; Padideh Nouri; Tristan Deleu; Yoshua Bengio

REVIEW 2 major objections 4 minor 1 cited by

Relative Trajectory Balance is equivalent to Trust-PCL

T0 review · 2 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The RTB objective is exactly the Trust-PCL objective, up to temperature squared.

desk verdict A clean algebraic equivalence between RTB and Trust-PCL that mostly holds, paired with an empirical section that overreaches a bit. read the letter →

arxiv 2509.01632 v1 pith:YFRL6Z2I submitted 2025-09-01 cs.LG

Tristan Deleu , Padideh Nouri , Yoshua Bengio , Doina Precup This is my paper

classification cs.LG

keywords RelativeTrajectoryBalanceTrust-PCLKL-regularizedreinforcementlearningGFlowNetsmaximum-entropyRLgenerativemodelfine-tuningoff-policyrewarddesign

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that Relative Trajectory Balance (RTB), an objective proposed for fine-tuning sequential generative models in the GFlowNet literature, is not a genuinely new method: it is mathematically the same as Trust-PCL, an existing off-policy reinforcement-learning algorithm with KL regularization. The claimed equivalence is exact: on the MDP considered in the paper, with a reward function whose sum along any trajectory equals the negative energy of the terminal state, the Trust-PCL loss equals alpha-squared times the RTB loss, under the correspondence that maps RTB's learned normalizer to Trust-PCL's soft state value and RTB's flow probability to the policy. If this is right, any success of RTB should be understood as a success of KL-regularized RL, and in particular the failure of KL-regularized RL reported in the RTB paper is attributed to a reward design that changed the target distribution and to the choice of a weaker on-policy algorithm. The paper also shows that, with the corrected reward, off-policy REINFORCE with KL regularization recovers the tilted target distribution on the RTB paper's illustrative 2D task, matching RTB.

What carries the argument

The load-bearing object is the residual identity between the two losses. RTB's residual (6) is the log of [exp(-E(s_T)/alpha) times the product of prior transition probabilities, divided by Z_psi times the product of learned transition probabilities]; Trust-PCL's residual (7) is -V_soft(s0) plus the sum of rewards plus alpha times the log-ratio of prior to policy. Under reward condition (3), the cumulative reward telescopes to -E(s_T), and with the dictionary V_soft(s0)=alpha log Z_psi and pi_phi=P_phi, the second residual is alpha times the first. Because both losses are half the expected squared residual, the whole identity follows. A secondary mechanism is Appendix C's gradient identity:

What would settle it

Pick a two-step MDP and a fixed policy and normalizer; compute the RTB residual (6) and the Trust-PCL residual (7) with a terminal reward exp(-E(s_T)) instead of -E(s_T). The ratio between the squared losses will not be alpha^2, and the optimal policies will differ, demonstrating that the proposition's reward condition is active. Alternatively, train Trust-PCL and RTB with identical off-policy data on a task satisfying (3): if the loss curves and samples do not match up to the alpha^2 factor, the equivalence is false in practice.

Watch

Extended reading notes

Core claim

Proposition 1 states that L_T-PCL(phi,psi) = alpha^2 L_RTB(phi,psi) whenever the reward satisfies Eq. (3) — the sum of rewards along a trajectory equals minus the energy of its terminal state — and with V_soft(s0)=alpha log Z_psi and pi_phi=P_phi. The RTB residual is a log-ratio of prior to learned flow probabilities minus E(s_T)/alpha; the Trust-PCL residual is the negative soft value at the start plus cumulative reward plus alpha times the log-ratio of prior to policy. The reward condition makes the cumulative reward telescope into -E(s_T), turning the Trust-PCL residual into exactly alpha times the RTB residual. Unlike earlier GFlowNet MaxEnt-RL correspondences, no reward correction with

Load-bearing premise

The equivalence holds only when the MDP's reward is chosen so that the total reward along every trajectory equals the negative energy of the terminal state; if the reward is instead placed as exp(-energy) at the final step, the target distribution tilts differently and the two losses no longer match.

Editorial extensions

If this is right

RTB inherits the theoretical guarantees, off-policy trust-region machinery, and design space of Trust-PCL; any improvement to Trust-PCL is a potential improvement to RTB.
The equivalence gives a direct dictionary between GFlowNet quantities (Z_psi, P_phi) and RL quantities (soft value, policy), so results can be translated across fields without reward corrections.
The negative result for KL-regularized RL in the RTB paper is not inherent: correcting the reward to satisfy (3) and using off-policy data with self-normalized importance sampling lets REINFORCE with KL regularization recover the target distribution on the illustrative task.
On-policy RTB and REINFORCE with KL regularization have gradients that agree up to a constant and differ only in the control variate, with Z_psi serving as a global baseline.
Future empirical comparisons of RTB against RL baselines need to fix the same target distribution through reward functions satisfying (3); otherwise they are comparing different tilted targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the equivalence holds by a residual identity, it is likely to survive under function-approximation classes where both residuals are representable; the hard part is estimating Z_psi or V_soft accurately.
The global-baseline interpretation of Z_psi points to a continuum of baselines: replacing the learned normalizer with batch estimates or learned state-dependent baselines interpolates between RTB and REINFORCE-style estimators.
If a task's reward does not satisfy the telescoping condition — for example, with process rewards or per-step penalties — the equivalence will not hold directly, and RTB's target under that reward should be re-derived before claiming KL-regularized RL equivalence.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth knowing: Proposition 1 is correct. Under the reward condition sum r = -E(s_T) and the mapping V_soft(s0)=alpha log Z_psi, pi_phi=P_phi, the Trust-PCL residual becomes alpha times the RTB residual, so the losses differ by alpha^2. That is a solid, two-line algebra identity, and it is stated honestly in the body — the title's categorical "is equivalent" is scoped by the explicit MDP assumptions in Section 2.1. As far as I can tell there is no circularity: they use the RTB and Trust-PCL definitions as given, no fitted constants, no recycled predictions.

What is new: the explicit RTB–Trust-PCL pairing. Earlier work connected GFlowNets to MaxEnt-RL in general, but nobody had named this specific pair. The proof in Appendix A of the tilted marginal is standard but well done. So as a conceptual unification, this is useful and publishable.

Where it is soft: the empirical section. The claim that REINFORCE with KL regularization matches RTB on the 2D example is not a like-for-like comparison. Algorithm 2 changes three things at once: the reward is corrected to satisfy (3), the squared KL penalty becomes a linear KL term inside the reward, and the data becomes off-policy with SNIS. Fixing the reward is legitimate and points to a real issue in the RTB paper, but the headline "KL-regularized RL methods achieve comparable performance" is therefore about a modified algorithm, not the original REINFORCE-with-KL that Venkatraman et al. ran. Figure 1 has no error bars or quantitative distance metric, so the empirical support is qualitative. Appendix C's gradient equivalence is fine but again conditional on the on-policy setting and a particular baseline.

The title perhaps overstates: "equivalent" is true only on MDPs whose reward telescopes per (3). That is a meaningful but restricted class. Minor point, since the body is precise.

Overall: the central result holds up, the empirical overreach is identifiable and not load-bearing. The paper is worth a serious referee — the algebra is correct, the literature linking is honest, and the clarification should be on record.

Referee Report

2 major / 4 minor

Summary. The paper claims that the Relative Trajectory Balance (RTB) loss, defined by residual (6), is proportional to the off-policy KL-regularized RL objective of Trust-PCL, defined by residual (7), on the finite-horizon MDP introduced in Section 2.1 under the reward condition (3). Proposition 1 states L_T-PCL(φ,ψ) = α^2 L_RTB(φ,ψ) with the identifications V_soft(s0) = α log Z_ψ and π_φ = P_φ. The proof is a direct substitution: with Σ r = -E(s_T) and V_soft(s0) = α log Z_ψ, the T-PCL residual becomes α times the RTB residual. The paper then revisits an illustrative experiment from the RTB paper, arguing that the reported failure of KL-regularized RL was due to a reward misspecification (Appendix B) and showing that a corrected off-policy REINFORCE with KL regularization matches RTB. Appendix C additionally shows that, on-policy, the RTB gradient equals a REINFORCE-with-KL gradient up to a baseline choice.

Significance. The algebraic identity in Proposition 1 is clean, explicit, and does not rely on fitted constants or hidden assumptions: the reward condition (3) is stated as a hypothesis, and the scoping of the MDP is clear. If the identification with Trust-PCL is correct, the result meaningfully situates RTB in the established KL-regularized RL landscape and strengthens the GFlowNet/MaxEnt-RL correspondence. The paper also helpfully provides code and a self-contained proof in Appendix A. The secondary empirical claim is less well supported: it rests on a single qualitative figure with no error bars or distribution-distance metric, and the algorithm comparison in Appendix B changes several components at once.

major comments (2)

[Section 3, Eq. (7); Proposition 1] The residual in Eq. (7) is presented as 'the Trust-PCL objective', but the original Trust-PCL [Nachum et al., 2018] is a path-consistency objective with KL regularization against the previous policy and with value residuals summed over all states in a trajectory, not only the initial state. Here the residual uses V_soft(s0) only and a fixed prior π_prior. Please provide the exact equation from Nachum et al. that reduces to Eq. (7) under the assumptions of Section 2.1, or reframe the claim as an equivalence to a PCL-type / KL-regularized off-policy RL objective. This is load-bearing because the title and abstract attribute the equivalence specifically to Trust-PCL.
[Section 3.1, Figure 1; Appendix B] The claim that corrected REINFORCE with KL regularization 'recovers the target distribution as accurately as RTB' is supported only by a qualitative plot with no error bars, number of seeds, or distribution-distance metric (e.g., MMD or Sinkhorn divergence). In addition, Algorithm 2 differs from Algorithm 1 in several ways (off-policy sampling with SNIS, moving the log-ratio into the reward, and using a different regularizer), so the attribution of the improvement to the reward correction is not isolated. This does not affect the algebra of Proposition 1, but it is central to the paper's 'reinterpreting the empirical success of RTB' contribution and should be quantified or softened.

minor comments (4)

[Abstract and title] The title says 'equivalent', while Proposition 1 gives proportionality up to the constant α^2. The body is precise, but consider softening the title or explicitly noting the constant in the abstract.
[Appendix B, Algorithm 2] The self-normalized importance weights are defined as w_n ∝ (π_φ(τ_n)/π_b(τ_n))^{1/T}. Please justify the 1/T exponent; a standard SNIS weight would be the full likelihood ratio. If the exponent is intentional (e.g., variance control), a brief explanation would help.
[Appendix C, Eqs. (25)-(29)] The gradient equivalence assumes the gradient is not taken through the sampling distribution ('ignoring differentiation through the policy over which we take the expectation'). State explicitly that this is the stop-gradient variant of RTB, since the true gradient of the RTB loss includes a term from the expectation's dependence on φ.
[Throughout] Minor typographical issues: the author name 'Y oshua' in the title block and the superscript formatting in Eq. (1) ('P ⊤_φ') should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RTB–Trust-PCL equivalence is a direct algebraic identity from explicit definitions and hypotheses.

full rationale

The paper's central claim (Proposition 1) is a substitution identity, not a prediction recycled as evidence. With the explicitly stated correspondences V_soft(s0)=α log Z_ψ and π_ϕ=P_ϕ, and with the MDP reward satisfying the telescope condition (3), the Trust-PCL residual (7) becomes α times the RTB residual (6) by direct algebra, giving L_T-PCL=α^2 L_RTB. No parameter is fitted to data and then called a prediction; no external benchmark is invoked to establish the equivalence; no uniqueness theorem from the authors' prior work is used to force the result. The reward condition (3) is an explicit hypothesis of the MDP, and Appendix B explicitly analyzes the consequences when it is violated, so the equivalence is not covertly built into the definitions. The empirical revisit is an independent experiment, not evidence for the theorem. Self-citations such as Deleu et al. (2024) are pointers to standard GFlowNet/reward settings, not load-bearing circular inferences. Accordingly, no circular step meets the bar of quoting a specific reduction to the paper's own inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central theorem is a pure algebraic identity and introduces no fitted constants, no ad hoc parameters, and no new physical or algorithmic entities. The only assumptions are the standard reward-telescoping condition, the cited definitions of Trust-PCL and RTB, and the known tilted-marginal result. Temperature alpha and partition estimate Z_psi are part of the problem formulation or the optimization variable, not free parameters fitted to data.

assumptions (4)

domain assumption Finite-horizon MDP reward satisfies sum_t r(s_t,s_{t+1}) = -E(s_T) for every trajectory (Eq. 3).
Invoked in Section 2.1 to ensure the optimal KL-regularized policy has the tilted marginal in Eq. (5), and used in Proposition 1 and Appendix B to identify Trust-PCL's residual with RTB's residual.
domain assumption Trust-PCL is defined by the residual in Eq. (7) with soft value function V_soft and anchor pi_prior.
The paper takes this definition from Nachum et al. (2018) and does not re-derive it; if the published Trust-PCL objective has additional terms or corrections, the identification in Proposition 1 would need revision.
domain assumption RTB residual in Eq. (6) and its off-policy averaging define the RTB loss, and its minimizer samples the tilted distribution.
RTB's properties are cited to Venkatraman et al. (2024); the paper relies on this to reinterpret RTB's success.
standard math The optimal KL-regularized policy has marginal pi_prior^T(s_T) exp(-E(s_T)/alpha).
Proved in Appendix A and also cited to Nachum et al. (2018) and Korbak et al. (2022); used to argue that KL-regularized RL targets the same distribution as RTB.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Relative Trajectory Balance is equivalent to Trust-PCL." pith.science (2026). https://pith.science/paper/YFRL6Z2I

@misc{pith2026250901632,
  author       = {Pith},
  title        = {Pith review of: Relative Trajectory Balance is equivalent to Trust-PCL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFRL6Z2I}},
  note         = {Machine review of arXiv:2509.01632}
}

read the original abstract

Recent progress in generative modeling has highlighted the importance of Reinforcement Learning (RL) for fine-tuning, with KL-regularized methods in particular proving to be highly effective for both autoregressive and diffusion models. Complementing this line of work, the Relative Trajectory Balance (RTB) objective was recently introduced in the context of Generative Flow Networks (GFlowNets) to serve the same role of improving fine-tuning in sequential generative models. Building on prior work linking GFlowNets and maximum-entropy RL, we establish in this paper an equivalence between RTB and Trust-PCL, an off-policy RL method with KL regularization. This equivalence situates RTB within the broader theoretical landscape of KL-regularized RL, and clarifies its relationship to earlier methods. Leveraging this insight, we revisit an illustrative example from the RTB paper and show that KL-regularized RL methods achieve comparable performance, offering an alternative perspective to what was previously reported.

Figures

Figures reproduced from arXiv: 2509.01632 by the authors.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference
cs.LG 2026-05 unverdicted novelty 7.0 of 10

FAV aligns few-step generative models by amortizing SVGD updates from reward-tilted sampling into generator parameters via fixed-point regression, requiring only sample access, and shows outperformance on robotics tas...

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Abramson, J

J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3 . Nature, 2024

work page 2024
[2]

Bengio, M

E. Bengio, M. Jain, M. Korablyov, D. Precup, and Y. Bengio. Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation . Advances in Neural Information Processing Systems (NeurIPS), 2021

work page 2021
[3]

Bengio, S

Y. Bengio, S. Lahlou, T. Deleu, E. J. Hu, M. Tiwari, and E. Bengio. GFlowNet Foundations . Journal of Machine Learning Research (JMLR), 2023

work page 2023
[4]

Black, M

K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine. Training Diffusion Models with Reinforcement Learning . International Conference on Learning Representations (ICLR), 2024

work page 2024
[5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 2020

work page 2020
[6]

Deleu, A

T. Deleu, A. G \'o is, C. Emezue, M. Rankawat, S. Lacoste-Julien, S. Bauer, and Y. Bengio. Bayesian Structure Learning with Generative Flow Networks . Conference on Uncertainty in Artificial Intelligence (UAI), 2022

work page 2022
[7]

Deleu, P

T. Deleu, P. Nouri, N. Malkin, D. Precup, and Y. Bengio. Discrete Probabilistic Inference as Control in Multi-path Environments . Conference on Uncertainty in Artificial Intelligence (UAI), 2024

work page 2024
[8]

Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee. DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models . Advances in Neural Information Processing Systems (NeurIPS), 2023

work page 2023

Show all 30 references

[9]

J. Ho, A. Jain, and P. Abbeel. Denoising Diffusion Probabilistic Models . Advances in Neural Information Processing Systems (NeurIPS), 2020

2020
[10]

M. Jain, T. Deleu, J. Hartford, C.-H. Liu, A. Hernandez-Garcia, and Y. Bengio. GFlowNets for AI-Driven Scientific Discovery . Digital Discovery, 2023

2023
[11]

Jaques, S

N. Jaques, S. Gu, D. Bahdanau, J. M. Hern \'a ndez-Lobato, R. E. Turner, and D. Eck. Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control . International Conference on Machine Learning (ICML), 2017

2017
[12]

Jiralerspong, E

M. Jiralerspong, E. Derman, D. Vucetic, N. Malkin, B. Sun, T. Zhang, P.-L. Bacon, and G. Gidel. Robust Reinforcement Learning for Discrete Compositional Generation via General Soft Operators . arXiv Preprint, 2025

2025
[13]

Korbak, E

T. Korbak, E. Perez, and C. L. Buckley. RL with KL penalties is better viewed as Bayesian inference . Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022

2022
[14]

Malkin, M

N. Malkin, M. Jain, E. Bengio, C. Sun, and Y. Bengio. Trajectory Balance: Improved Credit Assignment in GFlowNets . Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[15]

Malkin, S

N. Malkin, S. Lahlou, T. Deleu, X. Ji, E. Hu, K. Everett, D. Zhang, and Y. Bengio. GFlowNets and variational inference. International Conference on Learning Representations (ICLR), 2023

2023
[16]

Mohammadpour, E

S. Mohammadpour, E. Bengio, E. Frejinger, and P.-L. Bacon. Maximum entropy GFlowNets with soft Q-learning . International Conference on Artificial Intelligence and Statistics (AISTATS), 2024

2024
[17]

Nachum, M

O. Nachum, M. Norouzi, K. Xu, and D. Schuurmans. Trust-PCL: An Off-Policy Trust Region Method for Continuous Control . International Conference on Learning Representations (ICLR), 2018

2018
[18]

Ouyang, J

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[19]

Precup, R

D. Precup, R. S. Sutton, and S. Singh. Eligibility traces for off-policy policy evaluation. International Conference on Machine Learning (ICML), 2000

2000
[20]

Rombach, A

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-Resolution Image Synthesis with Latent Diffusion Models . IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

2022
[21]

Schulman, F

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms . arXiv Preprint, 2017

2017
[22]

Sendera, M

M. Sendera, M. Kim, S. Mittal, P. Lemos, L. Scimeca, J. Rector-Brooks, A. Adam, Y. Bengio, and N. Malkin. On diffusion models for amortized inference: Benchmarking and improving stochastic control and sampling . Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[23]

Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models . arXiv Preprint, 2024

2024
[24]

Tiapkin, N

D. Tiapkin, N. Morozov, A. Naumov, and D. Vetrov. Generative Flow Networks as Entropy-Regularized RL . International Conference on Artificial Intelligence and Statistics (AISTATS), 2024

2024
[25]

Uehara, Y

M. Uehara, Y. Zhao, T. Biancalani, and S. Levine. Understanding reinforcement learning-based fine-tuning of diffusion models: A tutorial and review. arXiv Preprint, 2024

2024
[26]

Uesato, N

J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins. Solving Math Word Problems with Process-based and Outcome-based Feedback . arXiv Preprint, 2022

2022
[27]

Venkatraman, M

S. Venkatraman, M. Jain, L. Scimeca, M. Kim, M. Sendera, M. Hasan, L. Rowe, S. Mittal, P. Lemos, E. Bengio, A. Adam, J. Rector-Brooks, Y. Bengio, G. Berseth, and N. Malkin. Amortizing intractable inference in diffusion models for vision, language, and control. Advances in Neur...

2024
[28]

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models . Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[29]

R. J. Williams. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning . Machine learning, 1992

1992
[30]

Zhang, H

D. Zhang, H. Dai, N. Malkin, A. Courville, Y. Bengio, and L. Pan. Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets . Advances in Neural Information Processing Systems (NeurIPS), 2023

2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Abramson, J

J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3 . Nature, 2024

work page 2024

[2] [2]

Bengio, M

E. Bengio, M. Jain, M. Korablyov, D. Precup, and Y. Bengio. Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation . Advances in Neural Information Processing Systems (NeurIPS), 2021

work page 2021

[3] [3]

Bengio, S

Y. Bengio, S. Lahlou, T. Deleu, E. J. Hu, M. Tiwari, and E. Bengio. GFlowNet Foundations . Journal of Machine Learning Research (JMLR), 2023

work page 2023

[4] [4]

Black, M

K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine. Training Diffusion Models with Reinforcement Learning . International Conference on Learning Representations (ICLR), 2024

work page 2024

[5] [5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 2020

work page 2020

[6] [6]

Deleu, A

T. Deleu, A. G \'o is, C. Emezue, M. Rankawat, S. Lacoste-Julien, S. Bauer, and Y. Bengio. Bayesian Structure Learning with Generative Flow Networks . Conference on Uncertainty in Artificial Intelligence (UAI), 2022

work page 2022

[7] [7]

Deleu, P

T. Deleu, P. Nouri, N. Malkin, D. Precup, and Y. Bengio. Discrete Probabilistic Inference as Control in Multi-path Environments . Conference on Uncertainty in Artificial Intelligence (UAI), 2024

work page 2024

[8] [8]

Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee. DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models . Advances in Neural Information Processing Systems (NeurIPS), 2023

work page 2023