{"id":"094d84d4-8fd4-4443-93cb-32933b862197","arxiv_id":"2608.08644","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding a learned path-dependent latent state to GFlowNet policies strictly increases their expressive power and improves convergence on discrete compositional sampling benchmarks.","lead":"The paper shows that GFlowNet samplers, which build discrete objects step by step with a Markovian policy, are limited by state aliasing. It adds a learned latent memory variable so the policy can depend on the full past trajectory, and proves the training losses still work.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 3.1's proof conflates policy log-odds with terminal-distribution span; the claimed expressivity gap is not established by the given argument.","rationale":"The reader's conditional verdict is appropriate: the central lifting and balance theorem (Proposition 4.5) appears mathematically sound, and the deterministic-latent limitation is explicitly scoped in Section D, where the stochastic extension is acknowledged as open. The most load-bearing unaddressed issue is the proof of Proposition 3.1, which supports the paper's expressivity motivation. Since this is a fixable proof gap and does not affect the core algorithm's correctness, I would keep the verdict at CONDITIONAL rather than moving to ACCEPT or REJECT.","tokens_in":35479,"tokens_out":32976,"duration_ms":355305,"concrete_test":"Instantiate LINES with M = 1, N = 3, d = 1, and ψ(p_i) = i. Enumerate the terminal distributions r(k) for a fine grid of k, record the numerical rank of the matrix of centered vectors, and compare with the proof's claimed value rank Ψ = 1. If the numerical rank is 1, the proof's interpretation may be salvageable; if it is greater than 1, then dim span(WM) is not rank Ψ and Proposition 3.1 requires a different proof, e.g., bounding the manifold dimension of the image instead of its linear span.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The expressivity contribution rests on Proposition 3.1, but its proof in Appendix A.1 is not valid as stated. WM is defined as equivalence classes of terminal distributions r over Q, yet the proof analyzes only the policy log-odds o = Ψk (Eq. 13) and concludes dim WM = rank Ψ. The map from o to r is nonlinear: with M = 1, α_i = sigmoid(o_i) and r_i = (1 − α_i) ∏_{j<i} α_j. The linear span of the image of a d-dimensional linear subspace under this map can exceed d; for example, with N = 2, d = 1, and ψ(p_i) = i, the Markovian set has dim span 2, not 1. Thus the inequality 'dim span(WM) ≤ dim span(WP)' is not proven by the text. This does not invalidate Proposition 4.5 (the balance-loss correctness seems sound), but it undermines a headline theoretical claim that path-dependent linear models strictly expand the learnable distribution family. The proof also contains index typos, e.g., the c_i definition in Eq. (14).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to lift the pointed DAG of a GFlowNet by attaching a deterministic, learnable latent dynamical system W_t = W_{t-1} + phi(s_t, W_{t-1}), so that the forward policy can condition on the whole trajectory through W_t. It argues that Markovian policies suffer from state aliasing and limited expressivity, illustrates this with linear LINES and GNN-based graph examples, and proves (Propositions 4.5-4.7) that trajectory balance, subtrajectory balance, and contrastive balance remain valid on the lifted DAG. It also proves (Proposition 4.8) that a Markovian backward policy forces the optimal forward policy to be Markovian. Experiments on set generation, sequence design, grid worlds, preference learning, bit sequences, lazy random walks, and Ising models report improved goodness-of-fit and faster convergence relative to Markovian baselines with comparable parameter counts.","tokens_in":35673,"tokens_out":21662,"duration_ms":209677,"significance":"If the main theoretical claims held as stated, the paper would make a useful contribution: a principled recipe for recurrent/path-dependent GFlowNet policies with the same training objectives as standard GFlowNets, plus a formal demonstration that lifting can strictly increase the representable family in some settings. The proof strategy for Propositions 4.5-4.7 is sound: the lifted graph is a finite pointed DAG for fixed deterministic phi, and the standard balance-condition argument applies. The experimental section is broad, code is promised in the supplement, and the ablations (rotation matrix, link function, RNN baselines) support the claimed mechanism. The main reservation is that one of the headline expressivity results (Proposition 3.1) is not established by its proof, and the appendix proof of Proposition 4.8 contains a nontrivial omission. The central balance-loss extension is nonetheless credible and worth publishing after revision.","major_comments":[{"comment":"The proof equates the expressive power of the Markovian sampler with the dimension of the set of achievable policy log-odds, writing o = Psi k (Eq. 13) and concluding dim W_M = rank Psi. This is not valid: the map from log-odds o to the terminal distribution r is nonlinear, since for M = 1, r_i = (1 - sigma(o_i)) prod_{j < i} sigma(o_j). For N = 2, d = 1 and psi(p_i) = i, the three achievable normalized terminal vectors r(0) = (1/2, 1/4, 1/4), r(-ln 2) = (2/3, 4/15, 1/15), and r(ln 2) = (1/3, 2/15, 8/15) are linearly independent (determinant -1/60), while rank Psi = 1. Hence the linear span of the achievable terminal distributions can have dimension 3 even though the log-odds lie on a line; the claimed inequality dim span(W_M) <= dim span(W_P) and the conclusion dim W_M <= d are not established. The proposition should be reformulated (e.g., in terms of the intrinsic dimension of the achievable distribution manifold) and proved for that statement.","section":"Section 3.1, Proposition 3.1; Appendix A.1"},{"comment":"In the marginalization step, the proof uses Eq. (16) to replace products of pF with products of pB-tilde, but Eq. (16) contains the factor R(s_T) on the right-hand side. When computing pS(s_l | xi*) as a ratio of sums over continuations tau: s_l -> X and tau': s_{l-1} -> X, the factors R(x_tau) and R(x_tau') appear inside the sums and do not cancel with the displayed pB-tilde factors. They are dropped in the manuscript's derivation. The conclusion is recoverable: define A(s_l) = sum_{tau: s_l -> X} R(x_tau) pB-tilde(tau | s_l) and B(s_{l-1}) = sum_{tau': s_{l-1} -> X} R(x_tau') pB-tilde(tau' | s_{l-1}); then the conditional is pB-tilde(s_{l-1} | s_l) A(s_l) / B(s_{l-1}), which still depends only on s_{l-1} and s_l. The proof should be rewritten accordingly, and the notation pB(xi* | s_T) for an incomplete prefix xi* should be clarified or replaced.","section":"Appendix A.4, proof of Proposition 4.8"}],"minor_comments":[{"comment":"The first inclusion is written R_P(S) subset of R_M(S); it should be R_M(S) subset of R_P(S). The same reversal appears in the sentence beginning 'Consequently, R_P(S) subset of R_M(S)'.","section":"Appendix A.2, proof of Proposition 3.2"},{"comment":"The sentence 'Corollaries 4.6, 4.7, B.1 follow directly from Proposition 3.2' should refer to Proposition 4.5, not Proposition 3.2.","section":"Appendix A.3"},{"comment":"The definition c_i = sum_{0 <= j <= i} psi(p_i) uses the wrong index; it should be psi(p_j).","section":"Appendix A.1, Eq. (14)"},{"comment":"In the line 'P_{tau: s_o* -> s*} pF(tau | s*) = 1', the sum should be over the backward policy pB(tau | s*), not pF.","section":"Appendix A.3, proof of Proposition 4.5"},{"comment":"The text says K is in {16, 32}, but Table 1 reports K = 16 and K = 24; the cited cardinalities 10^14 and 10^18 correspond to K = 16 and K = 32, respectively. Please align the text and table.","section":"Section 5, Table 1 and surrounding text"},{"comment":"The main text and Section D appear to claim that a stochastic latent dynamical system can be developed, while the Conclusions state that the feasibility of a stochastic dynamics remains open; please clarify which parts of Section D are formal results and which are preliminary.","section":"Section D and Conclusions"}],"recommendation":"major_revision","confidential_remarks":"The balance-loss extension (Propositions 4.5-4.7) is the strongest part of the paper and appears sound. The main concern is Proposition 3.1: as written, the statement about the linear span of achievable terminal distributions is not proved and appears false under the natural reading of 'span'; the authors should replace it with a correct dimensional statement or clearly restrict the claim. The A.4 proof of Proposition 4.8 also needs correction, though the underlying statement is plausibly true. The stochastic extension in Section D is only a sketch and should not be advertised as a completed theory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Colleague],\n\nHere's my read of arXiv:2608.08644.\n\nThe one thing to know: this paper has a solid core and a shaky headline. The core is the lifted pointed DAG construction (Definition 4.1) and the proof that trajectory balance, subtrajectory balance, and contrastive balance remain valid on it (Propositions 4.5–4.7). The proof is correct: the lifted DAG is still a pointed DAG, and with deterministic latent dynamics the number of trajectories per terminal state is finite, so the standard flow-matching argument goes through. That's a real contribution.\n\nWhat's also good: the empirical work is broad and honest. The authors test on set generation, sequence design, grid world, Ising, lazy random walk, plus ablations on the rotation matrix, link function, and a comparison with LSTM/GRU. The path-dependent SRWM consistently beats the Markovian baseline. The code is in the supplement, though a public repository would be better.\n\nNow the soft spots. The expressivity claims are the weak link. Proposition 3.1 claims that path-dependent linear policies strictly expand the set of learnable distributions, but the proof in Appendix A.1 is not valid. It equates the dimension of the set of achievable terminal distributions with the rank of the state embedding matrix, ignoring that the map from policy log-odds to terminal distributions is softmax-based and nonlinear. A one-dimensional subspace of log-odds can easily produce terminal distributions whose linear span is two-dimensional (the N=2, d=1 example from the stress test shows this). So the inequality dim span(WM) ≤ dim span(WP) is not proven. Proposition 3.2 has a reversed set inclusion in its proof (RP ⊆ RM instead of RM ⊆ RP), which is probably a typo but should be fixed. And Proposition 4.8's proof drops R(x) factors when marginalizing; it's not convincing as written.\n\nNone of these soft spots break the central correctness result. But the paper's own framing leans on these expressivity propositions, so they deserve serious repair. As it stands, the expressivity gain is better treated as an empirical observation than a proven theorem.\n\nWho should read this: GFlowNet researchers working on structured discrete spaces where state aliasing is a practical issue. It might be a useful drop-in improvement. It won't reshape the field, but it's a legitimate step forward.\n\nMy recommendation: send it to peer review. The main method and its theoretical underpinning are sound enough to warrant referee time, and the authors should be pushed to fix or downgrade the expressivity claims before publication.\n\nBest,\n[You]","headline":"Solid central contribution (balance losses on a lifted DAG for path-dependent GFlowNets) undermined by a flawed expressivity proof in Proposition 3.1 that should be fixed or downgraded before peer review.","tokens_in":36266,"tokens_out":5062,"would_cite":true,"duration_ms":49570,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deterministic latent memory added to a GFlowNet's state space lets the policy depend on the whole past trajectory, and the paper proves the standard balance losses still yield the correct terminal distribution.","keywords":["GFlowNets","path-dependent policies","latent dynamical systems","discrete amortized sampling","trajectory balance","state aliasing","self-referential weight matrix","compositional object sampling"],"falsifier":"Train the LINES environment from Section 3 with the same recurrent policy but a stochastic latent update $W_t = W_{t-1} + \\phi(s_t, W_{t-1}) + \\varepsilon_t$ and minimize the TB loss; if the terminal marginal's distance to $R$ does not approach the deterministic version's level while all else is held equal, the finiteness assumption behind Proposition 4.5 is the reason.","tokens_in":35231,"feed_emoji":"🧠","tokens_out":7895,"duration_ms":77741,"temperature":0.7,"pith_summary":"GFlowNets learn to sample compositional discrete objects---graphs, sequences, subsets---in proportion to an unnormalized target by training a Markovian policy that builds objects step by step. This paper argues that the Markovian restriction is the bottleneck: identical states reached by different histories are aliased, so the policy cannot tell them apart, and credit from the final reward propagates poorly. The proposed fix is to lift the state space with a learnable latent dynamical system $W_t = W_{t-1} + \\phi(s_t, W_{t-1})$, making the policy depend on the whole trajectory while remaining trainable with the same trajectory-balance, subtrajectory-balance, and contrastive-balance losses. The paper proves these lifted losses still make the terminal marginal proportional to the target, and reports on benchmark discrete-sampling tasks that the recurrent policy often converges faster and fits the target better than a Markovian counterpart.","feed_headline":"Latent memory lets GFlowNet samplers beat Markovian limits","feed_subtitle":"Recurrent policies trained with the same balance losses sample closer to target and explore better.","key_machinery":"The central object is the lifted pointed DAG $S^\\star = S \\times \\Omega$ with deterministic latent dynamics $W_t = W_{t-1} + \\phi(s_t, W_{t-1})$: it expands the state graph so that each distinct history receives its own latent context, while keeping the lifted space finite. The paper implements $\\phi$ with a modified self-referential weight matrix (a fast-weight update rule that rotates the weight matrix by a fixed ergodic rotation and self-adjusts its update strength), so the memory both encodes the past and actively reduces state aliasing. The subtrajectory balance, trajectory balance, and contrastive balance conditions are then evaluated on this lifted graph, and the balance identity is what transfers the correctness guarantee from the Markovian setting to the path-dependent one.","core_discovery":"On a lifted pointed DAG $S^\\star = S \\times \\Omega$, where each state carries a latent memory updated by a deterministic, state-conditional rule, the central result (Proposition 4.5) is that if the subtrajectory balance condition $F(s^\\star_i)p_F(s^\\star_{i+1:j} \\mid s^\\star_i) = p_B(s^\\star_{i:j-1} \\mid s^\\star_j)F(s^\\star_j)$ holds for every slice of every trajectory, then the marginal of the forward policy over terminal states is proportional to the flow $F$, and therefore to the target $R$ on terminal objects. Trajectory balance and contrastive balance follow as corollaries. Because the latent update is deterministic, each discrete path determines a unique latent sequence, keeping the number of paths through each lifted state finite, which is what carries the proof. The same lifting is shown to strictly enlarge the representable class of distributions for linear policies and for GNN-based policies, and a two-terminal toy graph exhibits a uniform Markovian output where the path-dependent model can fit any target. A companion result shows that a Markovian backward policy forces the optimal forward policy to be Markovian as well, so the lift buys expressivity only when the backward model is also allowed to be path-dependent.","pith_inferences":["A natural testable extension is stochastic latent dynamics: the paper sketches a measure-theoretic formulation but leaves the learning guarantees open, so an open problem is whether any SubTB-type condition can be proven when each history branches over infinitely many latent continuations.","The expressivity results suggest that symmetry-rich generation tasks, where GNN-based GFlowNets are known to collapse to uniform or near-uniform outputs, are prime candidates for this memory-based lifting.","Because the lifted policy is effectively a recurrent policy, techniques from partially observable MDPs and recurrent reinforcement learning---belief-state compression, memory regularization, action abstraction---could be imported directly into discrete amortized sampling.","The ablation study singles out the rotation matrix as the component that most accelerates convergence, which suggests a cheap improvement test for other recurrent samplers: swap the learned memory update for a rotation-based fast-weight rule before scaling the network."],"forward_implications":["Existing GFlowNet training pipelines can swap a Markovian policy head for a recurrent policy and keep the same balance-loss guarantees that the terminal marginal is proportional to the target.","Path-dependent parameterizations can represent distributions that Markovian parameterizations of the same architecture cannot, so state aliasing is an expressivity ceiling, not just an optimization difficulty.","A Markovian backward policy drives the optimal lifted forward policy back to being Markovian; to benefit from path-dependence, the backward model must also be path-dependent, which the paper's experimental designs do.","The vector field $\\phi$ and the policy are learned jointly from a single loss, so the method adds memory without introducing a separate training objective.","Distributed and streaming amortized inference schemes transfer to the lifted setting, as shown in the supplement, because the balance conditions remain valid there."],"supporting_citations":[{"why":"Defines GFlowNets and the balance/flow conditions on pointed DAGs that the lifted construction extends.","marker":"(Bengio et al., 2023)"},{"why":"Introduces the trajectory balance condition and loss that Corollary 4.6 carries over to the lifted graph.","marker":"(Malkin et al., 2022)"},{"why":"States the subtrajectory balance condition that Proposition 4.5 proves sufficient on the lifted DAG.","marker":"(Madan et al., 2022)"},{"why":"Provides the contrastive balance condition that Corollary 4.7 adapts to the path-dependent setting.","marker":"(Zhang et al., 2023b)"},{"why":"Supplies the expressivity analysis of GFlowNet policy networks that the paper's linear and GNN lifting results (Propositions 3.1 and 3.2) improve upon.","marker":"(Silva et al., 2025a)"},{"why":"Supplies the self-referential weight matrix update used to implement the learned latent dynamics.","marker":"(Irie et al., 2022)"}],"fun_headline_variants":["Path-dependent policies widen GFlowNet sampling reach","Latent memory unlocks path-dependent discrete samplers","Recurrent latent states boost GFlowNet beyond Markov","Lifting with latent dynamics improves GFlowNet convergence","Path-aware policies outperform Markovian discrete samplers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the latent update is deterministic, $W_t = W_{t-1} + \\phi(s_t, W_{t-1})$, so each discrete path determines a unique latent trajectory and the lifted graph stays finite; a stochastic latent update breaks that finiteness and the paper leaves that case open.","fun_headline_variants_meta":{"raw":{"variants":["Path-dependent policies widen GFlowNet sampling reach","Latent memory unlocks path-dependent discrete samplers","Recurrent latent states boost GFlowNet beyond Markov","Lifting with latent dynamics improves GFlowNet convergence","Path-aware policies outperform Markovian discrete samplers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1912,"prompt_tokens":978,"completion_tokens":934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":859}},"tokens_in":594,"tokens_out":934,"duration_ms":10200,"temperature":1.0,"reasoning_tokens":859,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:29:28.044736+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the LINES environment from Section 3 with the same recurrent policy but a stochastic latent update $W_t = W_{t-1} + \\phi(s_t, W_{t-1}) + \\varepsilon_t$ and minimize the TB loss; if the terminal marginal's distance to $R$ does not approach the deterministic version's level while all else is held equal, the finiteness assumption behind Proposition 4.5 is the reason.","supporting_citations":[],"review_version":1}