{"id":"f6598e35-fff8-4ce8-ad7a-c64a78a46c8b","arxiv_id":"2603.09221","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A symplectic LQR layer inserted as an adapter into pretrained LLMs yields large gains on MATH-500, AMC and AIME by solving a latent optimal-control problem at inference time.","lead":"The paper embeds finite-horizon LQR planning as a differentiable Test-Time Control layer inside LLM forward passes, so the model plans latent trajectories before predicting the next token. If the gains hold, this gives a hardware-efficient architectural route to System-2-style reasoning that sits between pure memory models and external RL or search.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Gains may reflect multi-step latent capacity rather than optimality of the LQR solve itself.","rationale":"The paper’s solver math (symplectic iteration, KKT dual LQR, fused kernels) is standard and carefully derived; empirical gains over listed memory adapters are real and ablations (time-heterogeneity, PLN horizon sampling, test-time T scaling, latent probing) give positive support for the design. The reader correctly flags the modeling leap (short-horizon LQR as a model of contest-math reasoning) as the soft spot and correctly keeps the verdict CONDITIONAL given small AIME N and missing error bars. My concern sharpens that leap: even granting expressivity, the experiments do not yet show that *solving* the LQR (vs using the same multi-step linear structure non-optimally) is what produces the gains. That is the single most load-bearing untested condition for the architectural claim “optimal control as component.” Because the reader already conditioned on a closely related assumption and the technical content is solid, I do not move the verdict; a negative result on the proposed ablation would justify moving toward REJECT or a stronger CONDITIONAL, while a positive result would strengthen the paper toward ACCEPT.","tokens_in":38383,"tokens_out":753,"duration_ms":31559,"concrete_test":"Train a control-ablated twin on the identical Llama-3-Instruct-7B + OpenThoughts2/curated-800K recipe and interleaving: keep Sec. 4 contextualization of {(A_t,B_t,Q_t,R_t)} unchanged, but replace the symplectic/Riccati output u*_1 with (i) open-loop T-step rollout under zero actions then linear decode of the terminal state, or (ii) a learned dense W of size matching the free parameters of the value matrices, mapping h0→u. Evaluate MATH-500 Acc and AMC Pass@8 at the same T_test. If the ablated model recovers ≥90% of TTC-Net’s absolute gain over Full Finetuning, the optimality/planning component is not necessary for the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (that embedding finite-horizon optimal control as an architectural component is what drives reasoning gains beyond memory/TTT adapters) requires that the benefit come from solving the LQR for a value-optimal first action u*_1 (Prop. 3.1 / Thm. 3.3), not merely from inserting a context-conditioned multi-step linear dynamical system of similar depth and parameter budget. Table 2 and the adapter baselines (MLP, Attention, RetNet, Mamba, GDN, MesaNet) do not isolate this: none reuses the Sec. 4 contextualization that synthesizes {(A_t,B_t,Q_t,R_t)} from h0 and then replaces only the optimal solve with a non-optimal map of matched structure (open-loop zero/random-action rollout of the same linear dynamics, fixed non-Riccati K, or a dense readout from h0 with comparable free parameters). Without that isolation, the MATH-500 / AMC / AIME lifts remain consistent with ordinary residual capacity addition, so the “planning before prediction / value function” interpretation is not yet load-bearing for the headline results. The reader’s expressivity concern is related but distinct; even if short-horizon LQR is expressive enough to fit, the optimality of the solve may still be incidental.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that modern LLMs are limited by associative-memory architectures (System-1 style) and that reasoning can be internalized as finite-horizon optimal control. It introduces the Test-Time Control (TTC) layer, which, given a context-encoded latent state h0, synthesizes time-heterogeneous linear dynamics and quadratic costs, solves the resulting LQR via a symplectic-iteration algorithm (Theorem 3.3 / Algorithm 3) implemented as a fused CUDA kernel, and emits the first-step optimal action u*_1 as the next-token representation. Differentiability is obtained by a dual LQR derived from the KKT system (Theorem 3.2). TTC layers are inserted as adapters (every 8 blocks) into pretrained transformers, yielding hybrid TTC-Nets. Empirically, the method improves Sudoku board/cell accuracy over Transformer/Mamba/GDN/Samba baselines and, after SFT on OpenThoughts2 + curated data, raises MATH-500 accuracy by up to 27.8 points over the Llama-3-7B base (and outperforms matched memory adapters) while producing 2–3\times Pass@8 gains on AMC/AIME; test-time horizon scaling further improves accuracy.","tokens_in":38826,"tokens_out":1434,"duration_ms":21066,"significance":"If the central claim holds—that embedding a value-function / LQR planner as a native architectural component, rather than external RL or pure test-time memorization, is what drives the observed reasoning gains—the work supplies a concrete, hardware-efficient primitive for System-2 computation inside LLMs. The symplectic reformulation, KKT gradient derivation, structured diagonal/low-rank parameterization, and fused CUDA kernels are carefully executed and constitute a genuine systems contribution that makes the idea practical. Consistent gains across Sudoku, MATH-500, AMC, AIME, a second backbone (Qwen2.5-Math), and non-math LM benchmarks, together with progressive latent probing and CoT-alignment analyses in the appendix, give the result real weight. The paper therefore has the potential to influence both architectural design and the broader conversation on how to internalize planning, world models, and nested RL objectives.","major_comments":[{"comment":"The headline interpretation—that gains arise specifically from solving the LQR for a value-optimal first action u*_1 (Prop. 3.1 / Thm. 3.3) rather than from the mere insertion of a context-conditioned multi-step linear dynamical system of comparable depth and parameter count—is not isolated by the experiments. Table 2 and the adapter suite (MLP, Attention, RetNet, Mamba, GDN, MesaNet) compare against other memory mechanisms but never reuse the Sec. 4 contextualization that produces {(A_t,B_t,Q_t,R_t)} from h0 and then replace only the optimal solve by a non-optimal map of matched structure (open-loop zero/random-action rollout of the same dynamics, a fixed non-Riccati gain, or a dense readout from h0 with the same free-parameter budget). Without that control, the MATH-500 / AMC / AIME lifts remain consistent with ordinary residual capacity addition; the “planning before prediction / nest","section":"§5.2, Table 2"},{"comment":"The modeling premise that a short-horizon (T_train ~ PLN mean 8, max 32; T_test = 8–16) linear-quadratic regulator whose dynamics and costs are synthesized from a single context vector is a sufficiently faithful model of multi-step contest-math and Sudoku reasoning (Eqs. 5–7 and the power-series contextualization in Sec. 4) remains a strong and largely untested assumption. While Fig. 5 shows beneficial test-time horizon scaling and App. D.6 linear probes / CoT-alignment scores indicate progressive refinement, these do not rule out that the benefit collapses to ordinary multi-step capacity once the true latent dynamics become strongly nonlinear or the relevant horizon far exceeds the trained range. A clearer statement of this limitation, or an experiment that deliberately degrades the optimality of the solve while preserving the dynamics, would strengthen the paper.","section":"§3.1, Eqs. (5)–(7); §4 contextualization"}],"minor_comments":[{"comment":"Abstract and introduction report “+27.8% on MATH-500”; the absolute point gain from the 25.00 base is 27.8, but the wording can be read as a relative percentage. Clarify “points” versus “percent relative improvement.”","section":"Abstract"},{"comment":"Figure 3 throughput is plotted on a log scale and reports “over 10× higher throughput”; the caption and surrounding text should also state the absolute TFLOPs/s numbers and the precise hardware (H200) so that the comparison is reproducible without reading the appendix.","section":"Fig. 3"},{"comment":"The multi-head and basis-sharing design (r=16, d=16 per head) is described only briefly; a short paragraph or table listing the exact parameter count of a TTC adapter versus the memory adapters used in Table 2 would make the “comparable capacity” claim fully transparent.","section":"§4 Multi-Head Structure"},{"comment":"In the Sudoku multi-step protocol (Alg. 6) the model is allowed to fill only the most confident cell each iteration; it would be useful to report whether single-step board accuracy already correlates strongly with multi-step success, or whether the iterative procedure is essential for the reported gains.","section":"§5.1, Alg. 6"},{"comment":"A few minor notational inconsistencies appear (e.g., h0 versus h_init, Q_0 = 0 stated in different places). A single consistent convention throughout Sec. 3 and Appendix A would improve readability.","section":"§3 and App. A"}],"recommendation":"major_revision","confidential_remarks":"The isolation experiment requested in the first major comment is the single most important addition; if the authors can show that a non-optimal rollout of the identical contextualized dynamics loses most of the gain, the paper becomes substantially stronger and I would then be comfortable with minor_revision or accept. The symplectic solver and CUDA work are solid and should be retained regardless. Scope is appropriate for a top ML venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The real news is a drop-in residual adapter that runs finite-horizon LQR over latent states at test time, with a fused symplectic CUDA solver that is actually fast enough for LLMs. They put it into Llama-3-7B and Qwen2.5-Math-7B and get clear lifts on MATH-500, AMC, and AIME over full SFT and over several memory adapters (attention, Mamba, GDN, MesaNet, etc.). The solver math (symplectic iteration, KKT dual LQR for gradients, diagonal structure, kernel fusion) is carefully done and matches classical LQR; the throughput plots look honest.\n\nWhat is new is the combination: contextualized multi-head LQR parameters synthesized from the residual stream, mixed-horizon training, and the claim that this is planning-before-prediction rather than more test-time memory. The Sudoku results and the test-time horizon scaling curve are useful supporting evidence. Citations to Amos et al. and the TTT literature are appropriate; they do not over-claim priority on differentiable LQR itself.\n\nThe soft spot is causal isolation, not the algebra. The stress-test note is right: the baselines do not keep the same context-conditioned linear dynamics and only swap the optimal first action for a non-optimal map of matched structure. So we cannot yet say the Riccati/symplectic solve is load-bearing versus multi-step residual capacity. The modeling assumption that short-horizon linear-quadratic dynamics capture contest math is also a leap, but the paper is open about it and the ablations (time-heterogeneity, horizon sampling, interleaving) are sensible. No error bars and limited data release keep the absolute numbers provisional.\n\nThis is for people who care about architectural test-time compute and hardware-aware sequence models. The formal work and the empirical pattern are strong enough that a serious editor should send it to referees. I would read it carefully and would cite the solver and the adapter design; I would not yet treat the 'value function / planning' interpretation as settled.","headline":"Solid hardware-efficient LQR adapter with real gains on math; the 'planning' story is plausible but not yet isolated from capacity.","tokens_in":39432,"tokens_out":515,"would_cite":true,"duration_ms":5978,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Reasoning is formulated as finite-horizon optimal control inside the model, so language models plan before they predict.","keywords":["test-time control","linear-quadratic regulator","state-space models","LLM reasoning","value function","symplectic iteration","hybrid architecture","planning before prediction"],"falsifier":"If, on the same fine-tuning recipe and parameter budget, replacing the TTC layer with a capacity-matched memory adapter (or a non-planning MLP) erases the reported MATH-500 and Pass@8 gains, or if lengthening the planning horizon past the training distribution systematically degrades rather than improves accuracy, the architectural planning benefit is falsified.","tokens_in":39301,"feed_emoji":"🎯","tokens_out":871,"duration_ms":7442,"temperature":0.7,"pith_summary":"Modern language models mostly work by retrieving and matching stored context, which is good for fluent next-token prediction but weak for deliberate multi-step problem solving. This paper argues that the missing piece is not more memory or more external training, but an architectural layer that plans. It inserts a Test-Time Control (TTC) layer that treats the next-token decision as the first action of a short-horizon Linear-Quadratic Regulator (LQR) problem solved over latent states at inference time. Dynamics and costs are synthesized from the current context, a value function is computed on the fly, and the optimal first action becomes the next representation. A hardware-efficient symplectic solver, fused into CUDA kernels, keeps the cost low enough to run as an adapter inside pretrained models. On Sudoku and contest math, the resulting hybrid networks outperform pure memory adapters and full fine-tuning, with large gains on MATH-500, AMC, and AIME. The claim is that embedding optimal control as a native architectural component lets models reason over future trajectories without treating planning as an external procedure.","feed_headline":"LLMs plan before they predict with an LQR control layer","feed_subtitle":"Adapters that solve short-horizon optimal control lift MATH-500 and contest scores without external search.","key_machinery":"The Test-Time Control (TTC) layer: a differentiable finite-horizon LQR solver that takes a context-encoded initial state, produces time-heterogeneous dynamics and costs, computes the optimal first-step control via symplectic iteration, and returns that action as the next-token representation, with gradients obtained by solving a related dual LQR.","core_discovery":"The paper establishes that reasoning can be internalized as finite-horizon LQR planning over latent states: a TTC layer synthesizes context-dependent linear dynamics and quadratic costs, solves for the optimal first action at test time, and thereby encodes a value function that lets the network plan before it predicts. When these layers are interleaved with attention as adapters in pretrained LLMs, mathematical reasoning accuracy rises substantially relative to memory-only baselines.","pith_inferences":["If short-horizon LQR is already useful, richer but still tractable controllers (time-varying or mildly nonlinear) may further close the gap on harder multi-step domains.","The same adapter pattern could be tried on non-math sequential tasks that reward foresight, such as long-horizon code or multi-turn tool use, to test whether the benefit is domain-specific.","Latent trajectories that align with explicit chain-of-thought suggest the control layer may be compressing token-level reasoning into a compact state-space plan."],"forward_implications":["Planning becomes a native forward-pass operation rather than an external RL or search loop, so longer test-time horizons can be dialed up without changing model weights.","Hybrid memory-plus-control blocks can be dropped into existing transformers as zero-init adapters and still unlock measurable contest-math gains.","Hardware-efficient symplectic LQR solvers make value-function layers practical at the scale of modern LLMs.","Test-time compute can be spent on deeper latent planning instead of only longer token-level search."],"fun_headline_variants":["TTC layers embed finite-horizon LQR so LLMs plan before predicting","Latent LQR control adapters raise MATH-500 and AIME Pass@8","State-space optimal control internalizes reasoning in pretrained LLMs","Test-time LQR solvers as adapters replace external search for math","Hardware-efficient LQR layers encode value functions for LLM planning"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"The claim rests on the premise that a short-horizon linear-quadratic model whose parameters are read from a single context vector is rich enough to stand in for the multi-step reasoning needed by contest math and Sudoku.","fun_headline_variants_meta":{"raw":{"variants":["TTC layers embed finite-horizon LQR so LLMs plan before predicting","Latent LQR control adapters raise MATH-500 and AIME Pass@8","State-space optimal control internalizes reasoning in pretrained LLMs","Test-time LQR solvers as adapters replace external search for math","Hardware-efficient LQR layers encode value functions for LLM planning"]},"model":"grok-4.5","effort":"low","cost_usd":0.00353,"raw_usage":{"total_tokens":1091,"prompt_tokens":754,"num_sources_used":0,"completion_tokens":96,"cost_in_usd_ticks":35300000,"prompt_tokens_details":{"text_tokens":754,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":241,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":754,"tokens_out":96,"duration_ms":2753,"temperature":1.0,"reasoning_tokens":241,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-15T12:07:55.396964+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"If, on the same fine-tuning recipe and parameter budget, replacing the TTC layer with a capacity-matched memory adapter (or a non-planning MLP) erases the reported MATH-500 and Pass@8 gains, or if lengthening the planning horizon past the training distribution systematically degrades rather than improves accuracy, the architectural planning benefit is falsified.","supporting_citations":[],"review_version":1}