{"id":"05ed009c-ee11-4ba5-a78f-9714f3addd80","arxiv_id":"2511.04054","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Necessary and sufficient conditions are claimed for concurrent execution of learned value functions in a min-norm control framework, plus a discounted-value-function correction; the sufficiency proof has serious gaps.","lead":"This paper states conditions for when multiple tasks encoded by learned value functions can be executed simultaneously by a min-norm optimization controller. It also adds a correction so the controller can handle value functions trained with a discount factor.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 3 is false as stated: smooth value functions can be flat and oscillatory at a shared zero, so opposing gradients occur arbitrarily close to the goal; the Taylor-linearization step in Eqs. (6)-(9) cannot be justified.","rationale":"The reader's weakest_assumption correctly identifies the Taylor-linearization step as the load-bearing weakness. The proof's Eq. (7) ignores higher-order terms around a point where all derivatives can vanish; this is not a harmless approximation but the crux of the theorem. I would only note that the reader's illustrative function e^{-1/x^2}(2+sin(1/x)) has derivative of one sign sufficiently close to 0; the e^{-1/r}(2+sin(1/r^2)) construction above provides the needed sign oscillations. The discounted-sigma extension appears correct, and Lemma 1 and Proposition 1 are broadly plausible, but they cannot rescue the necessary-and-sufficient claim. Since Proposition 3 is false under the stated smoothness assumptions, the paper's central claim is unsupported and the existing REJECT verdict should stand.","tokens_in":10457,"tokens_out":23279,"duration_ms":192417,"concrete_test":"Verify the counterexample analytically: define J1,J2 as above for f=0,g=1, choose beta>0, and set q_i=beta J_i + (J_i')^2/4. Check that J1,J2 are smooth nonnegative, solve the discounted HJB, and that along r_n=1/s_n with s_n=sqrt(pi/4+2 pi n), for all large n, J1'(r_n)<0 and J2'(r_n)>0, so the gradients are nonzero and opposite. If this reproduces, Proposition 3 is falsified; if it fails, identify the missing hypothesis (e.g., analyticity or a positive-definite Hessian at the goal) that the proof would need to add.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central sufficiency claim rests on Proposition 3 (Section IV.B). Its proof linearizes each J_i about a common zero and asserts that, because first-order terms dominate for small delta, gradient sums cannot oppose. That is not a consequence of smoothness. Set n=m=1, f=0, g=1, and for r=|x| define J1(r)=e^{-1/r}(2+sin(1/r^2)), J2(r)=e^{-1/r}(2+sin(1/r^2+pi/2)), J_i(0)=0. These are C^infty, nonnegative, zero only at the origin. For s=1/r -> infinity, J1'(r) has sign -cos(s^2) and J2'(r) has sign sin(s^2). There are arbitrarily large s with cos(s^2) and sin(s^2) both positive, so at the corresponding x the two gradients are nonzero and opposite. Definition 4 is then violated, so no neighborhood F of the common goal is concurrently controllable. If one insists on the form in Eq. (2), take any beta>0 and set q_i=beta J_i + (J_i')^2/4; the discounted HJB is then verified, so these are valid value functions. The algebra in Eqs. (8)-(9) is also a non-sequitur: it only gives <sum_P grad J_i - k sum_Q grad J_i, x-a>=0, which does not prevent the two gradient sums from being opposite.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the concurrent execution of multiple learned value functions using the pointwise min-norm controller from prior work [7], [8]. It introduces definitions of concurrent executability and concurrent controllability, proves necessary conditions (Propositions 1-2), and states a sufficiency result (Proposition 3) claiming that if learned value functions share a common zero at an equilibrium and the system is overactuated (g full row rank), then there is a neighborhood of that state in which the tasks are concurrently controllable and hence executable. It also proposes a modified slack/right-hand-side expression σ̅_i in the min-norm QP to handle discount factors. The paper includes simulations with a planar single-integrator robot illustrating the propositions.","tokens_in":10765,"tokens_out":11908,"duration_ms":106720,"significance":"If the main characterization were correct, it would fill a real gap in the optimization-based multi-task RL literature by telling practitioners when learned value functions can be combined. The discounted-sigma extension (Section IV.C) is a modest but sound contribution: substituting the optimal input into the HJB equation verifies the proposed expression. However, the central sufficiency theorem is false as stated, and the proposed definition of concurrent controllability does not actually imply concurrent executability even at the level of linear inequalities. These are load-bearing errors, not presentation issues. The paper therefore does not deliver the advertised necessary-and-sufficient characterization.","major_comments":[{"comment":"Concurrent controllability does not imply concurrent executability. Let n=m=2 and consider three gradient vectors v1=(2,0), v2=(-1,1), v3=(-1,-2). No subset sum is a negative multiple of another disjoint subset sum, so Definition 4 holds. But 0 lies in the convex hull of {v1,v2,v3}, so by Farkas' lemma there is no u with v_i · u < 0 for all i. Thus the tasks are not concurrently executable in the sense of Definition 3, contradicting Remark 2. This invalidates the foundational link on which the sufficiency result is built.","section":"Section III, Definition 4 and Remark 2"},{"comment":"The proof discards higher-order Taylor terms without any error bound. Smoothness alone does not make the first-order terms dominate near a common zero. For a scalar system f=0, g=1, define J1(r)=e^{-1/r}(2+sin(1/r^2)) and J2(r)=e^{-1/r}(2+sin(1/r^2+π/2)), with J_i(0)=0. These are C^∞, nonnegative, zero only at r=0, and satisfy the HJB equation (2) with q_i=βJ_i+(J_i')^2/4. Yet their derivatives change sign and take opposite signs at points arbitrarily close to 0, so no neighborhood of the common goal is concurrently controllable. Proposition 3 is false under the stated assumptions; it requires an additional hypothesis such as analyticity or a uniform positive-definite Hessian at the goal.","section":"Section IV.B, Proposition 3 proof (Eqs. (6)-(9))"},{"comment":"Even the local algebra in the proof is not a valid inference. Equation (9) only gives ⟨Σ_{i∈P}∇J_i - k Σ_{i∈Q}∇J_i, x-a⟩ = 0 for one particular vector x-a. This does not prevent the two gradient sums from being opposite vectors. The proof needs a uniform angle bound over the neighborhood, which is exactly what the Taylor argument fails to provide.","section":"Section IV.B, Eq. (8)-(9)"},{"comment":"The proof asserts that L_\\bar f ̅V(x)=0 if and only if J_1=···=J_N=0 and x is an equilibrium state. The first implication is fine under the strict-progress assumption, but the \"x is an equilibrium state\" part is not established: if the common zero set of the J_i is not a singleton, a trajectory could converge to and move within that set without x being an equilibrium. The proof cites LaSalle's theorem and appeals to an implicit argument in [16], but the claimed characterization of the invariant set is not a standard consequence unless the zero set is isolated or additional conditions are imposed.","section":"Section IV.A, Lemma 1"}],"minor_comments":[{"comment":"Typos: \"F or\" in the statement of Proposition 1 and \"dyanimics\" in the statement of Proposition 3 should be corrected.","section":"Section IV.A / IV.B"},{"comment":"The expression uses q(x) without a task index; it should be q_i(x). Also, the sentence \"When R(x)=I in (3)\" should refer to (2), since (3) is the controller and does not contain R.","section":"Section IV.C, Eq. (10)"},{"comment":"The definition does not specify whether P and Q are required to be disjoint. Clarify this, and note the implicit nonempty/nonzero conditions, to avoid ambiguity in the main condition.","section":"Section III, Definition 4"},{"comment":"The reference to \"the proof in [16] implicitly proves that the invariant set is nonempty\" is too vague. A direct argument using compactness and the omega-limit set would be more appropriate for a formal proof.","section":"Section IV.A, Lemma 1 proof"}],"recommendation":"reject","confidential_remarks":"The discounted-sigma extension (Section IV.C) is correct and could form part of a smaller paper. However, the central claim of necessary and sufficient conditions fails: Definition 4 does not imply Definition 3, and Proposition 3 is false under the stated smoothness assumptions. These are not fixable by local editing; the theorem would need substantially stronger assumptions and a reworking of the definitions, which changes the paper's contribution. Hence I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline you should know: the paper's new discounted-sigma expression is correct and genuinely useful, but the central sufficiency theorem (Proposition 3) is false as stated, and the proof's linearization step fails. The stress-test counterexample is valid; I checked it against the assumptions in Section II and it satisfies the discounted HJB.\n\nWhat's actually new: Definitions 3 and 4 (concurrent executability and controllability) are natural generalizations of independence/orthogonality from [5], [6], [8], and they are a real step forward for talking about when learned value functions can be combined in a min-norm QP. Section IV.C's derivation of \\bar{\\sigma}_i is clean and correct. The discounted correction term involving \\beta \\tilde{J}_i makes the controller recover the optimal input for discounted value functions, which is a practical improvement over [7].\n\nWhere it falls apart: Proposition 3 claims that if smooth value functions share a common zero equilibrium point and g has full row rank, then there is a neighborhood of that point where the tasks are concurrently controllable. The proof drops higher-order Taylor terms without any error estimate, and even the first-order algebra is not enough: from \\langle v, d\\rangle = k\\langle w, d\\rangle you cannot conclude v and w don't point in opposite directions. More importantly, the claim itself is false under the stated smoothness assumptions. The counterexample with J_1,J_2 flat and oscillatory at the origin is exactly the right construction: both functions are C^\\infty, nonnegative, zero only at 0, and their derivatives have opposite signs at points arbitrarily close to 0, so Definition 4 is violated in every neighborhood. These functions do come from a discounted HJB with q = \\beta J + (J')^2/4. So the \"necessary and sufficient\" package in the abstract isn't supported. To salvage the theorem you'd need something like analyticity or a uniform positive-definite Hessian at the goal, and the paper has neither.\n\nOther soft spots are smaller. Lemma 1 invokes LaSalle without fully checking the compactness/invariance details, and Proposition 2's proof is incoherent in a couple of places. Those could be fixed, but the Proposition 3 gap is load-bearing.\n\nWho gets value: someone working on multi-task RL/control who wants the definitions and the discounted-sigma fix will get something out of it, and anyone who might rely on this kind of local argument should see the counterexample. But the central theorem doesn't hold, so it's not a paper to build on.\n\nRecommendation: Send it to review if an editor wants a careful check, but I would not accept it without a major revision that adds assumptions and a real proof. If the authors can fix Proposition 3, the core idea has merit.","headline":"The discounted-sigma extension is solid, but Proposition 3 – the central sufficiency result – is false as stated; the Taylor-linearization step cannot be justified, and the stress-test counterexample holds up.","tokens_in":11268,"tokens_out":5552,"would_cite":false,"duration_ms":47569,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves necessary and sufficient conditions for concurrently executing multiple RL-learned tasks with a min-norm controller, and extends the controller to discounted value functions.","keywords":["multi-task control","reinforcement learning","value functions","min-norm controller","concurrent execution","control Lyapunov functions","optimization-based control","redundant robots"],"falsifier":"Take a planar single-integrator robot and two smooth nonnegative value functions that both vanish only at the origin, but whose gradients at points x_n toward the origin alternate between alignment and opposition, for example V_i(x)=exp(-1/|x|^2)(2+sin(1/|x|)) with different phase offsets. Run the min-norm controller from points arbitrarily close to the origin. If the trajectory cannot make progress on both tasks in any ball around the origin, Proposition 3's existence claim fails, showing the sufficiency part needs extra regularity.","tokens_in":10298,"feed_emoji":"🤖","tokens_out":4889,"duration_ms":43699,"temperature":0.7,"pith_summary":"The paper asks when a robot trained on several separate reinforcement-learning tasks can execute them all at once through a single optimization-based controller. It defines two precise properties—concurrent controllability and concurrent executability—and then gives necessary conditions (a shared goal equilibrium and a reachability condition on sublevel sets) and a sufficient condition (near a shared goal, with a fully actuated system, the tasks are guaranteed to be jointly executable). It also modifies the controller's constraint right-hand side to handle value functions trained with a discount factor, which is standard in RL but was not covered by the original framework. If correct, these results let engineers check learned task sets ahead of time for compatibility and combine them without retraining.","feed_headline":"Near a shared goal, learned tasks become jointly executable","feed_subtitle":"A new necessary-and-sufficient test tells when RL-trained value functions can share one min-norm controller.","key_machinery":"The central object is the pointwise min-norm controller: a quadratic program that at each state chooses the smallest input satisfying one inequality per task, each requiring the Lie derivative of that task's value function to decrease fast enough. The learned value functions act as candidate control Lyapunov functions—certificates that progress can be made—and their gradients enter the constraints. The argument also relies on a modified right-hand-side expression (the sigma-bar formula) that changes the prescribed decrease rate to match the Hamilton-Jacobi-Bellman equation, including when the value function was learned with a discount factor. The new task-relationship definitions, concurrent","core_discovery":"The paper's central claim is that a set of value functions learned by reinforcement learning can be checked, using only their values and gradients, for whether they can be executed concurrently by an optimization-based min-norm controller. The authors define concurrent controllability as the absence of two disjoint subsets of task gradients being exact negative scalings of each other, and concurrent executability as the existence of an input that decreases every unfinished task's value. They prove necessary conditions: a common goal equilibrium must exist and be reachable without leaving the sublevel sets, and they prove a sufficient condition: whenever all value functions share a goal equil","pith_inferences":["The sufficient condition is local: it guarantees a neighborhood around the shared goal, not a global region; extending it to global guarantees would require showing the concurrently controllable region covers the whole operating set.","The sigma-bar correction is not tied to the specific min-norm formulation and could be applied to any constraint-based controller that uses value functions as control Lyapunov functions.","Enforcing a common goal and a positive-definite Hessian at that goal during RL training would make the local sufficient condition rigorous and could enlarge the guaranteed region substantially.","The necessary conditions suggest a practical diagnostic: task value functions intended to be combined should be trained with overlapping goals, otherwise the quadratic program will eventually deadlock at states where gradients oppose."],"forward_implications":["If tasks share a goal and the robot is sufficiently overactuated, there is a guaranteed region around the goal where all tasks can be progressed simultaneously.","If no state exists where all value functions vanish, no compact region supports concurrent execution, so such task sets can be flagged as incompatible before deployment.","If a task's sublevel sets trap the dynamics, concurrent execution fails; this gives a design check for reward shaping and task training.","The discounted-value modification makes the framework compatible with standard RL discount factors, so policies trained with gamma < 1 can be plugged into the controller without losing optimal-input recovery.","The conditions are stated in terms of Lie derivatives and gradients, so they can be computed automatically from neural-network value functions via backpropagation."],"fun_headline_variants":["Gradient test reveals when learned tasks can run together","When value functions share a goal, min-norm control can join them","A precise check for when RL tasks can execute concurrently","No disjoint gradient flips? Then tasks can share a controller","Common goal plus reachable sublevels: tasks run together"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The proof of the sufficient condition assumes that arbitrarily close to a common goal, the linear (first-order) terms of each value function's Taylor expansion determine the sign of its decrease, so no two gradient sums can point in exactly opposite directions; this can fail for smooth value functions that are very flat at the goal.","fun_headline_variants_meta":{"raw":{"variants":["Gradient test reveals when learned tasks can run together","When value functions share a goal, min-norm control can join them","A precise check for when RL tasks can execute concurrently","No disjoint gradient flips? Then tasks can share a controller","Common goal plus reachable sublevels: tasks run together"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":986,"prompt_tokens":625,"completion_tokens":361,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":369,"completion_tokens_details":{"reasoning_tokens":278}},"tokens_in":369,"tokens_out":361,"duration_ms":3946,"temperature":1.0,"reasoning_tokens":278,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T23:48:37.962509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a planar single-integrator robot and two smooth nonnegative value functions that both vanish only at the origin, but whose gradients at points x_n toward the origin alternate between alignment and opposition, for example V_i(x)=exp(-1/|x|^2)(2+sin(1/|x|)) with different phase offsets. Run the min-norm controller from points arbitrarily close to the origin. If the trajectory cannot make progress on both tasks in any ball around the origin, Proposition 3's existence claim fails, showing the sufficiency part needs extra regularity.","supporting_citations":[],"review_version":1}