{"id":"2e778b24-34f4-4e5f-8533-b1c020c261e4","arxiv_id":"2412.06636","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Free-Gate composes control primitives by minimizing variational free energy, and its per-step weight optimization is provably convex and optimal.","lead":"This paper introduces Free-Gate, a method that composes simple control primitives by minimizing variational free energy, with a proof that each step of the resulting optimization is convex even for nonconvex costs and nonlinear dynamics. The method is demonstrated on a robot navigation task where no single primitive can reach the goal, but the composed policy can.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The optimality proof requires state-dependent gating weights, but Problem 1 states them as a fixed vector sequence; as written, Algorithm 1 may solve a different problem.","rationale":"The reader's weakest_assumption is model mismatch: the optimality guarantee is relative to the supplied transition density and generative model. That is a legitimate limitation for deployment, but it does not attack the mathematical claim as stated; every model-based optimal control result has this caveat. The more load-bearing issue is internal to the proof of Property 1. The proof's backward induction requires minimizing the per-step objective pointwise in x_{k-1}, which is justified only if w_k can depend on x_{k-1}. Problem 1's notation and the constraint set suggest w_k is a fixed vector, while Algorithm 1 and the experimental protocol compute w_k for the currently observed state. If w_k is constant across states, then replacing an expectation over x_{k-1} with the pointwise minimizer is not valid, and Algorithm 1 is not guaranteed to minimize Problem 1. If w_k is state-dependent, then the formal statement of Problem 1 is missing that dependence. This is an addressable formalization issue, not a sign that the convexity or dynamic-programming ideas are wrong; the per-step convexity proof itself appears correct. I would keep the reader's CONDITIONAL verdict, with the condition being a corrected statement of the decision variables and a verification of the pointwise minimization step. My disagreement with the reader is therefore only about which concern is most load-bearing, not about the overall assessment of the paper.","tokens_in":13517,"tokens_out":10915,"duration_ms":117553,"concrete_test":"Construct an N=1 instance with two equally likely states a and b, two linearly independent primitives, and costs such that the pointwise optima w*(a) and w*(b) are different. Compute the cost of the state-dependent policy returned by Algorithm 1 and the cost of the best single weight vector minimizing the averaged objective. If the averaged cost of the state-dependent policy is strictly smaller, Algorithm 1 solves a different problem from Problem 1 as written. Equivalently, re-derive Property 1 with w_k(x_{k-1}) made explicit; if the proof goes through only under that reading, then Problem 1 must be revised to state the dependence on x_{k-1}.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Property 1's proof depends on the claim in Appendix A that, at the final step, 'the expectation in the cost functional does not depend on w_N,' so the outer expectation E_{p(x_{N-1})}[...] can be minimized pointwise. That pointwise minimization is valid only if w_N is a function of x_{N-1}. Problem 1, however, defines the decision variables as {w_k}_{1:N} with w_k in the simplex and no state argument, and Algorithm 1's output is a policy built from a weight vector chosen for each observed state while running. If w_k is meant to be a single vector common to all states, the correct per-step optimization is min_{w_k} E_{p(x_{k-1})}[DKL(p(x_k,u_k|x_{k-1})||q(...)) + E_{p(...)}[bar_c]], which does not decouple into pointwise problems; minimizing (2) separately for each observed x_{k-1} is then not globally optimal. The experiments and the running text (e.g., Fig. 2(d), where weights evolve along a trajectory) indicate the intended meaning is state-feedback weights, but then Problem 1 and Algorithm 1 must explicitly set w_k = w_k(x_{k-1}). This is a formal gap in the central optimality theorem, not merely a robustness limitation.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Free-Gate, a mechanism for composing control primitives by minimizing a variational free-energy functional. The composition problem is cast as a finite-horizon optimal control problem (Problem 1) whose decision variables are the primitive weights, and Algorithm 1 solves it by backward recursion. The paper proves a dynamic-programming-style optimality property (Property 1) and a per-step convexity property (Property 2), then validates the approach on a Robotarium navigation task with four directional primitives that individually cannot reach the goal. The appendix contains self-contained proofs based on the KL chain rule and an explicit Hessian computation.","tokens_in":13789,"tokens_out":6737,"duration_ms":75627,"significance":"If the formalization is corrected, Free-Gate offers a principled normative account of policy composition: the per-step weight optimization is convex even for nonconvex costs and nonlinear, stochastic, non-stationary dynamics, which is a useful step beyond stationary mixture-of-experts and ad hoc soft-max gating. The paper supplies self-contained proofs, a documented code repository, and both simulation and hardware demonstrations. However, the central optimality theorem currently has a formal gap concerning state dependence of the weights, the global convexity claim in the abstract is stronger than what is proved, and the experiments use a heuristic cost-to-go rather than the exact recursion in Algorithm 1. These issues are fixable within the scope of the manuscript, but they need to be addressed before the claims can be accepted as stated.","major_comments":[{"comment":"The decision variables in Problem 1 are defined as a sequence of vectors {w_k}_{1:N} with no dependence on the state, yet the proof of Property 1 minimizes the final-stage objective pointwise in x_{N-1}, writing the optimal cost as E_{p(x_{N-1})}[l*_N(X_{N-1})] with l*_N obtained from a pointwise minimization. This step is valid only if w_N is allowed to be a function of x_{N-1}. If w_N is instead a single vector common to all states, then min_{w_N} E[...] is not equal to E[min_{w_N} ...] in general, so Algorithm 1 solves a different problem from the one stated. The running text and Fig. 2(d) indicate that state-feedback weights are intended; Problem 1 and Algorithm 1 should be rewritten so that w_k = w_k(x_{k-1}) with an explicit measurability condition, and Property 1 should be stated as a dynamic programming result for feedback policies.","section":"§3.1 (Problem 1) and Appendix A"},{"comment":"The abstract claims that the finite-horizon composition problem 'remains convex,' but the proof establishes only that, for each fixed x_{k-1}, the per-time problem (2) is convex in w_k. If Problem 1 is interpreted with state-feedback weights, the global problem is an infinite-dimensional optimization and no convexity argument is given for it; if it is interpreted with a single vector per time step, the pointwise Hessian computation does not establish convexity of the expectation over states. The convexity claim should be restricted to the per-step, per-state optimization problem (2), and the abstract and contributions should be reworded accordingly.","section":"Abstract, §3.2 (Property 2)"},{"comment":"The experiments replace the exact cost-to-go l*_{k+1} computed by Algorithm 1 with a heuristic approximation, described as 'the cost obtained by applying twice the same input.' Therefore the in-silico and hardware demonstrations do not validate the optimality guarantee of Algorithm 1; they validate a variant with an approximate planning value. The paper should either implement the full backward recursion on the discretized state space or explicitly state that the experiments test an approximate version and discuss the implications for the optimality claims.","section":"§4, Validation paragraph"},{"comment":"The 'transcendence' claim in Remark 3 does not follow from strict convexity. Strict convexity gives J_k(w) < Σ_i w_i J_k(e_i), but a weighted average of primitive costs can be larger than the minimum primitive cost. For example, with two primitives satisfying J_k(e_1)=10 and J_k(e_2)=0 and w=(0.5,0.5), strict convexity only implies J_k(w)<5, not J_k(w)<0. The remark should be corrected or removed unless an additional condition, such as equal primitive costs, is imposed.","section":"§3.2, Remark 3"}],"minor_comments":[{"comment":"The proof header reads 'Proof of Proposition 1,' but the statement being proved is Property 1 in Section 3.2; please correct the cross-reference.","section":"Appendix A"},{"comment":"Algorithm 1 writes l*_{N+1}(X_N) ← 0 and later uses l*_{k+1}(X_k), but it does not specify how these value functions are represented or computed over a continuous state space. The discretization used in the experiments should be described either as part of the algorithm or as a numerical approximation.","section":"Algorithm 1"},{"comment":"The text says 'CVX [7] is used to solve the optimization problem,' but reference [7] is the CVXPY paper; please align the name with the cited software.","section":"§4, implementation details"},{"comment":"The weight trajectories in Fig. 2(d) are not clearly labeled; adding a legend or axis labels identifying each primitive would help the reader connect the plot to the four cardinal-direction primitives.","section":"Fig. 2(d)"},{"comment":"Assumption 3 requires each primitive to have full support over U, while the primitives are introduced as Gaussian densities on R^2. After restricting or discretizing to the 7×7 action grid this is likely satisfied, but the paper should state explicitly how the Gaussian support is reconciled with the compact action space.","section":"Assumption 3 and §4"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle is the formal state-dependence gap in Problem 1 and the proof of Property 1; this is fixable by reformulating the problem over feedback policies. The global convexity claim and Remark 3 are overstatements that should be softened or corrected. The experiments are a useful demonstration but do not exercise the exact Algorithm 1, so the authors should be explicit about that limitation in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Rossi, Garrabé, and Russo propose Free-Gate: compose a set of black-box control primitives by solving a free-energy minimization at each time step, with the primitives' weights as decision variables. The genuinely new piece is the per-step convexity result: under the stated assumptions the Hessian is a scaled outer product, so the weight optimization is convex (strictly convex when the primitives are linearly independent). The backward-recursion optimality argument is also sound once you read the weights as state-feedback functions. That combination — a normative composition mechanism plus tractable per-step optimization — is a real step beyond the same authors' earlier free-energy inverse optimal control and data-composition work, and the paper ships code and a Robotarium demo. The core idea should be useful to the active-inference and modular-control community.\n\nThe soft spots are real but mostly fixable. The biggest one is a formal gap in Property 1. Problem 1 defines the {w_k} as a fixed vector sequence with no state argument, but the proof of the optimality decomposition minimizes pointwise in x_{N-1}; that only makes sense if w_k = w_k(x_{k-1}). The running text and the experiments (weights recomputed from the observed state) clearly intend state-dependent weights, so this is a notation and statement-of-problem issue rather than a dead end — but as written, Algorithm 1 solves a different problem from the one stated. The Appendix even contains a garbled sentence: \"the expectation in the cost functional does not depend on w_N,\" which is not true for fixed weights. Fix Problem 1 and the proof goes through.\n\nSecond, the abstract claims the finite-horizon composition problem \"remains convex.\" What is actually proven is per-step convexity, not joint convexity of the multi-stage problem. That overstatement should be toned down.\n\nThird, the validation is suggestive, not thorough: a heuristic cost-to-go replaces the full recursion, there are no baselines (fixed-weight mixtures, individual primitives, or a standard MPC), and five runs without error bars is thin. The hardware demo is nice but not a stress test.\n\nThe citation pattern is honest; the paper builds directly on the authors' prior free-energy framework and says so. Net: a genuine contribution with a patchable formal gap. I would engage with it; the right verdict is conditional acceptance with a request to fix the state-dependence issue and sharpen the experiments. Send to review.","headline":"Free-Gate gives a clean per-step convex formulation for composing control primitives, but the central optimality proof quietly assumes state-dependent weights that Problem 1 never states.","tokens_in":14307,"tokens_out":4307,"would_cite":true,"duration_ms":45718,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C25","93E20"],"pacs":[],"model":"deepseek-v4-flash","headline":"Free-Gate provably computes the optimal composition of control primitives via free energy gating, with a per-step convex problem even in nonlinear stochastic environments.","keywords":["Free Energy Minimization","Policy Composition","Autonomous Systems","Decision Making","Optimal Control","convex optimization","variational free energy","robot navigation"],"falsifier":"Run Algorithm 1 on a small discrete MDP with nonconvex costs, then brute-force Problem 1 by exhaustive search over a fine discretization of the weight simplex; any mismatch between the algorithm's output and the brute-force optimum beyond numerical tolerance would refute the claimed optimality.","tokens_in":13321,"feed_emoji":"🤖","tokens_out":13346,"duration_ms":122936,"temperature":0.7,"pith_summary":"Free-Gate is a computational rule for combining a library of simple control policies—'primitives'—into a single policy that can perform a task. The paper's central claim is that the optimal weighting of these primitives is the solution to a finite-horizon optimal control problem whose cost is variational free energy, and that this problem can be solved exactly. It proves that the per-time-step weight optimization is convex even when the environment is nonlinear and stochastic and the task cost is nonconvex, so the optimal composition can be computed by a backward recursion. The practical payoff is that a robot with only basic motor primitives (move left, right, up, down) can reach a goal and avoid obstacles, which no primitive could do individually.","feed_headline":"Free-Gate provably finds the optimal blend of simple motor policies","feed_subtitle":"By minimizing variational free energy, Free-Gate blends simple actions into a provably optimal policy.","key_machinery":"The central machinery is the chain rule for the KL divergence, which splits the free energy functional of Problem 1 into a sum of per-step terms and enables the backward dynamic-programming recursion of Algorithm 1. At each step the decision variable is the weight vector on the probability simplex, and the per-step objective is the variational free energy of the joint state–action distribution, with the optimal cost-to-go folded into the expected loss. Convexity is proven by computing the Hessian of this objective: it is an outer product of the vector of primitives scaled by a nonnegative fraction, hence positive semidefinite, and it is positive definite when the primitives are linearly independent.","core_discovery":"The paper establishes that the problem of optimally composing control primitives via free energy gating—minimizing $D_{\\mathrm{KL}}(p_{0:N}\\|q_{0:N})$ plus expected state and action costs, subject to the policy being a convex combination of primitives with weights on the probability simplex—can be solved exactly by Algorithm 1. The algorithm runs a backward recursion in which each step solves a convex optimization over the current weight vector, with the optimal value of the next step's problem embedded as a cost-to-go. Property 1 proves that this recursion returns the global optimum of the finite-horizon problem, and Property 2 proves that each per-step problem is convex in the weights, strictly convex when the primitives are linearly independent. From strict convexity and an interior optimum, the paper derives a transcendence result: the composed policy strictly outperforms every individual primitive, so tasks that are infeasible with the primitives alone become feasible.","pith_inferences":["The experiments use a heuristic one-step lookahead for the cost-to-go, not the exact backward recursion, so the demonstrated performance is for the approximate version; whether the full recursion improves it in practice is not tested in the paper.","The full-support assumption (every primitive is stochastic everywhere) is stronger than typical deterministic controllers; relaxing it, perhaps by Gaussian smoothing, would make the method applicable to deterministic primitives while preserving the convexity proof.","Since the optimality statement is model-relative, Free-Gate would benefit from a learning or adaptation layer for the transition density and generative model; the paper does not analyze robustness to model error.","The free-energy gating rule could be transplanted into deep mixture-of-experts architectures as a principled alternative to learned gating; whether it scales and yields better expert selection is an open question."],"forward_implications":["The optimal composition of primitives can be computed exactly by solving a sequence of per-time convex problems, so standard convex solvers can be used online.","Free-Gate gives a normative justification for gating in control: the weights are the solution of a well-defined optimization, not an arbitrary softmax heuristic or a learned policy.","Because the per-step cost includes the optimal cost-to-go, the composition is plan-aware: even a myopic per-step solver incorporates future consequences.","The transcendence property guarantees that a combination of simple primitives strictly outperforms each primitive alone, so individually infeasible tasks become feasible through composition.","The formulation holds for nonlinear, non-stationary, stochastic environments and nonconvex costs, without stationarity or discounting assumptions, so it applies broadly in planning and control."],"supporting_citations":[{"why":"Supplies the chain-rule-for-KL proof technique used to derive the backward recursion (Property 1).","marker":"[13]"},{"why":"Provides the free-energy cost functional and the obstacle-avoidance cost structure used in the navigation experiments.","marker":"[14]"},{"why":"Gives the convexity argument for the constraint set that the paper follows to prove Property 2.","marker":"[15]"},{"why":"Supplies the convex solver used to run each per-step optimization in the experiments.","marker":"[7]"},{"why":"Provides the public hardware and simulation testbed on which the rover navigation experiments are run.","marker":"[59]"}],"fun_headline_variants":["Free-Gate: provably optimal blend of simple policies","Free energy gating provably composes simple primitives","Provably optimal mixing of simple control primitives","Free-Gate unlocks tasks infeasible with individual primitives"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The optimality guarantee holds only for the exact transition density and the fixed generative model supplied to the algorithm; if the real environment differs from that model in any way, the composed policy is optimal only for the model, and the paper provides no adaptation or robustness bound for model error.","fun_headline_variants_meta":{"raw":{"variants":["Free-Gate: provably optimal blend of simple policies","Free energy gating provably composes simple primitives","Provably optimal mixing of simple control primitives","Free-Gate unlocks tasks infeasible with individual primitives"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001123,"raw_usage":{"total_tokens":4642,"prompt_tokens":884,"completion_tokens":3758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":3692}},"tokens_in":500,"tokens_out":3758,"duration_ms":23915,"temperature":1.0,"reasoning_tokens":3692,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:28:05.233016+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 1 on a small discrete MDP with nonconvex costs, then brute-force Problem 1 by exhaustive search over a fine discretization of the weight simplex; any mismatch between the algorithm's output and the brute-force optimum beyond numerical tolerance would refute the claimed optimality.","supporting_citations":[{"cited_title":"Automatica 137, 110121 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the chain-rule-for-KL proof technique used to derive the backward recursion (Property 1)."},{"cited_title":"Automatica173, 112015 (2025)","cited_arxiv_id":null,"evidence_quote":"Provides the free-energy cost functional and the obstacle-avoidance cost structure used in the navigation experiments."},{"cited_title":"IEEE Control Systems Letters 7, 2557–2562 (2023)","cited_arxiv_id":null,"evidence_quote":"Gives the convexity argument for the constraint set that the paper follows to prove Property 2."},{"cited_title":"Journal of Machine Learning Research 17(83), 1–5 (2016)","cited_arxiv_id":null,"evidence_quote":"Supplies the convex solver used to run each per-step optimization in the experiments."},{"cited_title":"IEEE Control Systems Magazine 40(1), 26–44 (2020)","cited_arxiv_id":null,"evidence_quote":"Provides the public hardware and simulation testbed on which the rover navigation experiments are run."}],"review_version":1}