{"id":"1662f9c7-443a-4b96-8a20-d7f52b2d6de1","arxiv_id":"2611.14210","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Tree-Ising casts LLM weight-space repair as exactly decomposable row-wise Ising problems over sign-flip and exponent-shift moves, matching or beating an LLM agent's edit at a fraction of the cost.","lead":"This paper replaces an expensive LLM agent that repairs looping language models (a $312, 90-million-token run) with a $0.90 GPU optimization over the model's own BF16 weights, formulated as independent row-wise Ising (QUBO) problems. A generalist should read it because it turns an expensive, non-repeatable weight-repair procedure into a cheap, deterministic optimization that also transfers to a second model without any agent.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader's verdict is CONDITIONAL with moderate confidence, and the weakest_assumption is that the linear surrogate may mispredict. The paper acknowledges this limitation directly in Section 8: 'Our objective functions as a first-order surrogate derived from the statistics of the unedited model: it inherently ignores how an edit in one layer alters the inputs to subsequent layers, and the dose serves merely as a linear proxy for the desired behavioral shift.' The empirical results on two models (0.8B and 2B) suggest the surrogate works in practice, and the transfer experiment without a reference edit provides independent support. The reader's other concerns—dose-matching, code release, single-seed evaluations—are also acknowledged by the authors. The paper is transparent about its limitations and does not overclaim. No additional load-bearing concern beyond what the reader identified and the authors acknowledged. Therefore, the verdict should remain UNCHANGED.","tokens_in":29664,"tokens_out":599,"duration_ms":8810,"concrete_test":"Independently re-run the TREE-ISING recipe on Qwen3.5-0.8B using the released code (when available) with a strictly dose-matched comparison (1.0× the agent's dose) on MATH-500@4. If the accuracy drops below the agent's 51.7%, the headline claim would weaken. If it remains above, the concern about dose-matching is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper is an honestly reported, technically coherent contribution. The central claim—that the repair objective decomposes exactly into independent row-wise Ising problems and that solving those problems produces an edit that beats the LLM agent's edit on five math benchmarks—is supported by the evidence presented. The authors explicitly acknowledge the key limitations: the comparison with the agent is not strictly dose-matched (Section 3.3, [b121]), the recipe was designed retrospectively using the agent's logs (Section 4, [b44]), the layer selection may reflect shared architecture rather than independent validation (Section 3.2, [b111]), and the surrogate objective ignores cross-layer interactions (Section 8, [b322]). These are stated as limitations, not hidden. The transfer to Qwen3.5-2B without a reference edit (Section 5) provides independent support. The code is not yet released, but the authors commit to releasing it. The single-seed evaluations and base-model baseline quoted rather than re-run are acknowledged limitations. The reader's weakest_assumption—that the linear surrogate may mispredict—is a valid concern, but the authors acknowledge it directly and the empirical results on two models suggest it works in practice. No additional load-bearing concern beyond what the reader identified and the authors acknowledged.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper proposes TREE-ISING, which replaces the LLM weight-programming agent of Takehi et al. [55] with a sparse discrete optimization over the model's stored BF16 weights for the same repair task: eliminating the reasoning loops of Qwen3.5-0.8B. A forward-only objective (Equation 5) combines a dose term that weakens a measured loop direction and strengthens a concluding direction, a collateral term that preserves layer outputs on productive tokens, and a sparsity penalty. Admissible edits are sign flips and exponent shifts within a state-tree representation of BF16 weight groups, and the paper derives an exact row-wise decomposition of the objective into independent Ising problems (Equations 6 and 7) for fixed statistics, retained columns, and one preselected move per weight. A GPU solves the 2,048 row problems by simulated annealing or simulated bifurcation. On Qwen3.5-0.8B the edit scores 54.6% on MATH-500@4 versus 51.7% for the agent edit (base model 7.9% as reported), beats the agent on all five math benchmarks with Bonferroni-corrected significance on MATH-500 and GSM8K, and modifies fewer weights and bits at an estimated $0.90 and four forward passes; the paper explicitly states that this comparison is not dose-matched (Section 3.3) and that the recipe was designed retrospectively from the agent's logs (Section 4.1). On Qwen3.5-2B, where the forward scan selects the layers and no reference edit is used, greedy GSM8K accuracy rises from 21.6% to 69.4%.","tokens_in":29978,"tokens_out":28738,"duration_ms":177035,"significance":"If the results hold, weight-space repair reduces to a cheap, reproducible, solver-agnostic discrete optimization: no gradients, no LLM tokens, and an exactness statement rather than a heuristic end-to-end pipeline. The central algebraic claim checks out: for fixed statistics and preselected moves, Equation (5) separates exactly per output row, and the QUBO-to-Ising conversion (Equations 6-7) is correct. The paper deserves credit for unusually candid disclosure: the dose-matching caveat ([b121]), the retrospective design ([b44]), the architecture confound ([b111]), the development-set contamination with a held-out-400 check (Section 4.2), single-seed evaluations ([b321]), and the surrogate's neglect of cross-layer interactions (Section 8) are all stated in the text. The empirical package is well-executed: paired tests, Bonferroni correction across twelve comparisons, an SA/SB solver-equivalence check, a continuous baseline (Section 4.5), and a QAOA demonstration with job identifiers and raw counts, honestly scoped as not evidencing quantum advantage.","major_comments":[{"comment":"The 0.8B headline comparison is calibrated against the baseline it is used to beat, and the abstract-level control is missing. The target dose is 1.6 times the agent edit's dose along the same direction u (Section 3.3), and the paper states that no strictly dose-matched (1.0x) edit was evaluated (Section 4.5). Because Table 6 shows that screening accuracy moves by several points when the dose changes from 1.2x to 2.0x, the +2.9-point MATH-500@4 advantage and the 5-of-5 benchmark sweep cannot be attributed to the method rather than to the more aggressive behavioral target. The disclosure in [b121] is exemplary, but the abstract and Table 1 still present the comparison as a head-to-head win. The fix is cheap and within the paper's scope: run the identical recipe at a 1.0x dose, and ideally at a matched achieved dose, on the full five benchmarks and report the numbers beside the 1.6x configuration; the screen already suggests a lower dose is competitive (57.8% at 1.2x in Table 6). If that experiment is not run, the 0.8B claims should be rescoped to state explicitly that the comparison was made at a deliberately more aggressive target, with the unbiased comparison presented as the Section 5 transfer.","section":"Section 3.3, Table 3, Section 4.5","authors_acknowledge":"The comparison with the agent is therefore not strictly dose-matched.","authors_acknowledge_p":0.76},{"comment":"The claim that the Qwen3.5-2B objective is derived from the base model alone overstates the independence of the dose rule. The value tau=6.5 sits between the two effective values (6.7 and 6.1) produced by the 1.6x agent-derived rule on the 0.8B layers, so the scalar itself inherits information from the agent edit; only the layer selection and the forward statistics are reference-free. This matters because Section 5 is the paper's unbiased test of the method, and Section 3.3 presents the tau=6.5 rule as if it were constructed without the agent. The mitigating evidence is in hand: the practice-screen dose response in Figure 4 (right) is monotone from 0x to 9x, and the 9x/three-layer variant reaches 71.8% without statistical separation from 69.4%. Please state explicitly that tau is inherited from the 0.8B calibration, present the Figure 4 screen sweep as the support for robustness to tau, and replace the base-model-alone phrasing with a precise statement of which quantities are and are not base-model-derived.","section":"Section 3.3, Section 5, Figure 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The layer-scan contribution is validated only on two models of the same architecture, and the scan rule was fixed after inspecting the 0.8B scores. The rule of the two highest-scoring layers excluding the final layer selects full-attention layers 15 and 19 on both models ([b111], [b316]); the agreement may therefore reflect the shared hybrid layout rather than the scan's causal validity, as the paper itself notes in Section 8. Nevertheless, contribution item 2 in Section 1 and the abstract present the scan as automatically identifying the layers to edit without requiring an agent. I ask that this be reworded to say that the scan identified the same layers as the agent on two same-family models, consistent with a full-attention copy mechanism, and that the acknowledged test on a model without this layout be stated as the criterion the claim awaits. The gradient-attribution comparison in Section 3.2 is a useful negative control and should be retained.","section":"Section 3.2, Figure 3, Section 8","authors_acknowledge":"The agreement between the two models may therefore reflect their shared architectural design rather than providing independent evidence that the scan identifies the optimal layers.","authors_acknowledge_p":0.92}],"minor_comments":[{"comment":"The abstract contains TREE-ISINGachieves and the running header contains TREE-ISINGBit-level, both lacking a space; moreover, the title uses Tree-Ising while the body uses TREE-ISING. Please unify the name and fix the spacing.","section":"Abstract, Section 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The state-tree total for the agent edit (720,586) is not transparently the sum of the reported sign flips (19,299), state changes (146,981), shared-exponent, and mantissa counts. Please state the exact counting rule for shared-node writes so the columns can be reconciled.","section":"Table 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The base-model MATH-500 figure (7.9%) and the 93% cut rate are quoted from the blog post [55] and not re-run; since the authors do re-run the Qwen3.5-2B base model, a single harness re-run of the 0.8B base would remove dependence on an unpublished source.","section":"Section 8, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paired p-values are conditional on a single generation seed, which the paper discloses; please report the seed and state this conditioning in the statistics footnote of Table 3.","section":"Section 4.2, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The caption requires the reader to distinguish the 119-CZ diagnostic counts from the 107-CZ p=1 circuit; please move the diagnostic clarification into the first sentence of the caption.","section":"Figure 5 caption","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The 110-versus-4 optimum-rate framing against uniform sampling is weak, as the authors themselves note; consider adding the random-angle control they propose or removing the raw comparison from the main text.","section":"Section 6, Section B.3","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The paper does something genuinely useful: it takes the weight-space programming idea from Takehi et al.—an LLM agent that reads weights and writes a fix—and replaces the agent with a sparse discrete optimization over the model's own BF16 weights. The core technical contribution is clean: given forward statistics from the base model, the repair objective decomposes exactly into independent row-wise Ising problems (Equation 6), each solvable by GPU annealing or simulated bifurcation. The result is a repair that costs $0.90 and 4 forward passes instead of $312 and 50 passes, and it beats the agent edit on five math benchmarks. The transfer to Qwen3.5-2B without a reference edit is the strongest evidence that the method generalizes.\n\nThe paper is honestly reported. The authors state their limitations directly: the 0.8B headline comparison is not dose-matched (the target dose is 1.6× the agent's dose), the recipe was designed after analyzing the agent's logs, the layer selection may reflect shared architecture rather than independent validation, and the surrogate objective ignores cross-layer interactions. They also note that the code is not yet released, that evaluations used a single seed, and that the base-model baseline is quoted rather than re-run. These are real soft spots, but they are acknowledged, not hidden.\n\nThe main unresolved question is whether the accuracy edge survives a strictly dose-matched comparison. The authors say no such edit was executed. That's a gap, but it's a narrow one: the 2B transfer experiment uses a dose rule derived from the base model alone, and it works. The paper's central claim—that the repair can be formulated as a cheap discrete optimization—does not depend on beating the agent by a specific margin; it depends on the formulation itself, which is sound.\n\nWho this is for: anyone working on model editing, weight-space programming, or discrete optimization for neural networks. The paper deserves a serious referee. The soft spots are addressable and the authors have been transparent about them. I would accept peer review and ask for a dose-matched comparison and code release before publication.","headline":"Honest, well-scoped paper that turns an expensive LLM-agent repair into a cheap discrete optimization problem; the central claim is plausible but the headline comparison is not dose-matched, and the code isn't out yet.","tokens_in":30567,"tokens_out":534,"duration_ms":9507},"pith_extraction":{"msc":["68T07","68T20","90C27"],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"This paper claims that repairing a looping language model can be reduced to solving independent Ising problems over the model's own BF16 weights, producing a more accurate edit than a frontier LLM agent while modifying fewer weights and…","keywords":["model editing","weight-space programming","Ising model","QUBO","simulated annealing","simulated bifurcation","reasoning loops","state trees"],"falsifier":"Apply the same recipe to a model from a different family (e.g., Llama 3.2-1B) without hybrid attention blocks, using the same layer-selection rule and dose calibration; if the edit fails to improve accuracy or degrades general capabilities, the method's reliance on architecture-specific patterns (full-attention layers driving loops) would be exposed.","tokens_in":31068,"feed_emoji":"🧩","tokens_out":1624,"duration_ms":26706,"temperature":0.7,"pith_summary":"The paper tries to establish that the repair of reasoning loops in small language models, previously done by an expensive LLM agent reading and writing weights, can be reformulated as a sparse discrete optimization problem solvable in minutes. The authors define an objective over forward statistics that weakens a loop direction and preserves productive outputs, then show this objective decomposes exactly into independent quadratic unconstrained binary optimization (QUBO) problems, one per output row of the MLP projection matrices. Solving these with GPU simulated annealing or simulated bifurcation produces edits that outperform the agent's edit on five math benchmarks, using fewer weight changes and no LLM tokens. If true, this would mean weight-space repair is a cheap, reproducible optimization task rather than a costly agent-driven process, potentially making model debugging accessible without frontier models.","feed_headline":"Repairing language models with 2,048 independent Ising problems","feed_subtitle":"A sparse discrete formulation beats an LLM agent at editing weights, costing $0.90 and modifying fewer bits.","key_machinery":"The central machinery is the decomposition of the repair objective into independent row-wise Ising models (Equation 6). For each output row of an MLP projection matrix, the objective combines a dose term (weakening the loop direction), a collateral term (preserving outputs on productive tokens), and a sparsity penalty. With fixed statistics and preselected candidate moves (sign flips and exponent shifts within the BF16 state tree), the problem separates exactly into one dense QUBO per row, with linear fields $h_a$ and quadratic couplings $J_{ab}$ derived from the forward statistics. This decomposition allows all rows to be solved simultaneously on a GPU via simulated annealing or simulated bifurcation.","core_discovery":"The central claim is that the repair of a language model's reasoning loops can be cast as a sparse discrete optimization over the model's BF16 weights, using only forward statistics from the base model, and that this optimization decomposes exactly into independent Ising problems per output row. The authors demonstrate that on Qwen3.5-0.8B, solving these problems with simulated annealing or simulated bifurcation yields an edit that achieves 54.6% on MATH-500, outperforming the agent edit's 51.7% under the same evaluation protocol, while modifying 13% fewer weights, reducing BF16 bit flips by a factor of 2.2, and costing $0.90 instead of $312. The same recipe transfers to Qwen3.5-2B without any reference edit, raising GSM8K accuracy from 21.6% to 69.4%. The authors argue that the performance gains stem from the mathematical formulation itself, not from the specific solver, and that the edit is fundamentally smaller and more native to the stored format than continuous alternatives.","pith_inferences":["If the formulation generalizes to other failure modes (e.g., refusal, hallucination, safety bypass), then model repair could become a standard optimization subroutine rather than a bespoke agent-driven process, potentially enabling automated debugging loops that run in minutes.","The fact that a dense closed-form solution of the same objective matches accuracy but modifies 62 times more weights suggests that sparsity is not essential for accuracy but is critical for practical editing (compact patches, no mantissa rewrites). This implies the field should focus on sparse discrete formulations rather than continuous rank-one updates.","The dose-calibration rule (τ=6.5) and layer-selection scan (exclude final layer, pick two highest) were established on one model and transferred successfully, but the paper acknowledges the selected layers are full-attention blocks; if this pattern is architecture-dependent, transfer to models without hybrid attention may require recalibration.","The quantum-hardware proof of concept is limited to 8 variables and does not show advantage, but the paper outlines conditional-block extensions (Equation 13) that could make larger quantum solvers relevant if hardware quality improves; the speedup bound (Equation 14) shows solver time is not the bottleneck in the current recipe."],"forward_implications":["Weight-space repair of reasoning loops can be performed without an LLM agent, using only forward passes and GPU optimization, reducing cost from hundreds of dollars to under one dollar per edit.","The repair modifies far fewer weights and bits than agent-based or continuous edits, producing a patch that is exact in BF16 and leaves shared exponents and mantissas untouched, enabling compact storage and minimal checkpoint changes.","The recipe transfers to a different model (Qwen3.5-2B) with no reference edit, using only the base model's forward statistics to select layers and calibrate dose, achieving a 47.8-point gain on GSM8K.","The mathematical formulation is solver-agnostic: simulated annealing and simulated bifurcation produce statistically indistinguishable repairs, and the same Ising problems could be targeted by quantum annealers or QAOA.","The approach reduces the fraction of answers cut at the token cap across all benchmarks, indicating the edit primarily helps the model terminate reasoning rather than improving correctness per se."],"supporting_citations":[{"why":"Supplies the baseline agent edit, the task definition (repairing Qwen3.5-0.8B loops), the public logs used to design the recipe, and the base model accuracy (7.9% on MATH-500).","marker":"[55]"},{"why":"Provides the state-tree representation of neural networks that defines the admissible moves (sign flips and exponent shifts within shared-exponent groups) and the bit-level accounting.","marker":"[3]"},{"why":"Supplies the simulated bifurcation solver used as an alternative to simulated annealing, demonstrating solver-agnostic performance.","marker":"[17, 18]"},{"why":"Provides the simulated annealing algorithm used as the default solver for the row QUBOs.","marker":"[33]"},{"why":"Provides the ROME-like rank-one update structure used as a continuous baseline in Section 4.5.","marker":"[41]"}],"fun_headline_variants":["Ising edits fix a language model for $0.90 in GPU time","Tree-Ising beats an LLM agent at weight repair with 4 forward passes","Repair language model weights as 2,048 independent Ising problems","Sparse BF16 flips lift GSM8K from 21.6% to 69.4%","Bit-level Ising repair beats an agent edit and flips 2.2x fewer bits"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The objective is a first-order linear surrogate that ignores how edits in one layer alter the inputs to subsequent layers, and the dose serves as a linear proxy for behavioral change; if this surrogate mispredicts beyond the practice tokens, the accuracy gains would not transfer to new tasks or models.","fun_headline_variants_meta":{"raw":{"variants":["Ising edits fix a language model for $0.90 in GPU time","Tree-Ising beats an LLM agent at weight repair with 4 forward passes","Repair language model weights as 2,048 independent Ising problems","Sparse BF16 flips lift GSM8K from 21.6% to 69.4%","Bit-level Ising repair beats an agent edit and flips 2.2x fewer bits"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000365,"raw_usage":{"cost":0.000364728,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000364728,"upstream_inference_prompt_cost":3.913e-05,"upstream_inference_completions_cost":0.000325598},"total_tokens":3039,"prompt_tokens":1250,"completion_tokens":1789,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":1025,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":1689}},"tokens_in":1250,"tokens_out":1789,"cost_source":"openrouter_usage","duration_ms":14748,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":1689,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-11T08:30:50.203229+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"falsifier":"Apply the same recipe to a model from a different family (e.g., Llama 3.2-1B) without hybrid attention blocks, using the same layer-selection rule and dose calibration; if the edit fails to improve accuracy or degrades general capabilities, the method's reliance on architecture-specific patterns (full-attention layers driving loops) would be exposed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the baseline agent edit, the task definition (repairing Qwen3.5-0.8B loops), the public logs used to design the recipe, and the base model accuracy (7.9% on MATH-500)."},{"cited_title":"Every Weight Is a Path of Shared Discrete Decisions","cited_arxiv_id":"2610.31931","evidence_quote":"Provides the state-tree representation of neural networks that defines the admissible moves (sign flips and exponent shifts within shared-exponent groups) and the bit-level accounting."},{"cited_title":"Kirkpatrick, C","cited_arxiv_id":null,"evidence_quote":"Provides the simulated annealing algorithm used as the default solver for the row QUBOs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ROME-like rank-one update structure used as a continuous baseline in Section 4.5."}],"review_version":1}