{"id":"06aa5416-ccf8-41d2-a7d5-f47f4ae328cc","arxiv_id":"2611.24578","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":9,"one_line_summary":"A training-free method that lets a vision-language model internalize harness improvements into its weights through recursive self-editing, outperforming fine-tuning on ten benchmarks.","lead":"BaseRSI lets a vision-language model improve its own weights by first finding better ways to prompt itself, then writing those improvements directly into its neurons without any training. The method outperforms standard fine-tuning and prompting on ten visual reasoning benchmarks, suggesting a path toward models that can self-improve without external supervision.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified","rationale":"The paper presents a well-motivated method with clear theoretical foundations, extensive experiments, and thorough ablations. The claims are supported by the evidence. The method is novel and the results are significant. The paper is well-written and the code is provided. No critical red flags were identified.","tokens_in":68843,"tokens_out":197,"duration_ms":4656,"concrete_test":"No significant objection identified. The paper's claims are well-supported by the evidence provided.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. The paper presents a well-motivated method with clear theoretical foundations, extensive experiments, and thorough ablations. The claims are supported by the evidence. The method is novel and the results are significant. The paper is well-written and the code is provided. No critical red flags were identified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes BaseRSI, a recursive self-improvement framework for vision-language models that internalizes harness gains into model weights via training-free edits. The method uses ReGap to compute per-block activation gaps between teacher (model with harness) and student (model without), and ReKey to write these gaps into MLP key-value neurons via ridge regression and low-rank approximation. Experiments on ten visual reasoning benchmarks show consistent improvements over baselines.","tokens_in":69225,"tokens_out":2411,"duration_ms":15627,"significance":"Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 3 of its 4 major objections did not survive that check.","major_comments":[],"minor_comments":[{"quote":"Fine-tuning on the same development questions (SFT, GRPO) barely moves these averages, while the harness that the outer loop verifies lifts them, so the gain lies in what the harness learns rather than in fitting these questions.","comment":"The main results (Table 1) show that BaseRSI outperforms all baselines on every benchmark. However, the comparison with fine-tuning methods (SFT, GRPO) may not be entirely fair: these methods are trained on the development questions until convergence, but the paper does not specify whether they use the same token budget or reasoning effort during inference as BaseRSI. The caption states 'all methods but the two closed-source reference models use Qwen3.8-27B with the same reasoning effort, token budget, and evaluator', but the fine-tuning methods change the weights, so the comparison of inference cost is not directly relevant. More importantly, the fine-tuning baselines are trained on the same development questions that BaseRSI uses for its acceptance test and fitting, so they have access to the same data. The fact that they barely improve suggests that the gain comes from the harness internalization paradigm, but the paper should discuss whether the fine-tuning methods could benefit from the same harness entries (e.g., by training on the teacher's responses).","section":"§5, Table 1, main results","consequence":null,"audit_status":"editorial_preference","objection_id":"O4","quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":null,"comment":"The notation ω_β(z) = z σ(β z) is defined but the sigmoid function σ is not explicitly defined. It should be stated that σ is the logistic sigmoid.","section":"§4.2, Eq. (6)","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":null,"comment":"The ridge regression in Eq. (8) uses a mean gap b_l and a projection matrix P_l. The definition of P_l as 'removes the mean direction of the inputs' is clear, but the text does not specify how the ridge weight λ is chosen. Appendix A.4 states that λ is chosen on a held-out fold, but this detail should be mentioned in the main text for completeness.","section":"§4.3, Eq. (8)","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":null,"comment":"The table caption says 'mean_{±std} over three runs'. The standard deviations are reported, but the number of runs (3) is small for reliable variance estimation. The paper should note this limitation, especially for benchmarks with high variance (e.g., BabyVision std 2.4 for BaseRSI).","section":"§5, Table 1","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":null,"comment":"Figure 4 panels are described in the caption but the actual figures are not visible in the manuscript text. The captions are detailed enough to understand the results, but the paper should ensure that the figures are legible in the final version.","section":"§5, Fig. 4","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"Rescale each readout to unit variance and its value by the inverse factor","comment":"Algorithm 3 (ReKey, one block) states 'Rescale each readout to unit variance and its value by the inverse factor'. This step is important for numerical stability but is not explained in the main text. A brief justification would be helpful.","section":"Appendix A.4, Algorithm 3","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"The $p$-value $p(\\cdot)$ of Eq.~\\eqref{eq:judge} is the one-sided sign-flip $p$-value of the macro mean of the listed benchmark effects, obtained by flipping the signs of the image clusters inside each benchmark.","comment":"The acceptance test uses a one-sided sign-flip p-value. The description 'obtained by flipping the signs of the image clusters inside each benchmark' is clear, but the paper should specify how many flips are used and whether the test is exact or approximate.","section":"§4.1, Eq. (3)","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"Null edits on top of the view edit","comment":"Table 2 (ablations) reports accuracy for many variants. The row 'random readouts' and 'random values' are described as null edits. The paper should clarify whether these null edits are applied on top of the view edit (as stated in the caption) or as standalone edits.","section":"§5, Table 2","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":null,"comment":"Figure 5 panels (c)–(f) show mechanism checks. The caption mentions 'Inside vs. output' and 'Exactness of the gap', but the axes and units are not described in the caption. The paper should ensure that all figure elements are self-contained.","section":"§5, Fig. 5","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":false,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"The key of a block is the ridge solution over three groups of positions on original requests","comment":"The key fitting in Eq. (9) uses three groups of positions. The paper should clarify whether the key is fitted once per block or once per entry (across all blocks). The algorithm description suggests one key per block, but the text is ambiguous.","section":"§4.3, Eq. (9)","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null},{"quote":"LBS-27B is a released checkpoint fine-tuned from the same base model and answers with its own crop loop.","comment":"The baseline 'LBS-27B' is described as a released checkpoint fine-tuned from the same base model. The paper should cite the source of this checkpoint and specify whether it was fine-tuned on the same development questions or on different data.","section":"§5, Table 1","consequence":null,"audit_status":null,"objection_id":null,"quote_matched":true,"required_revision":null,"authors_acknowledge":null,"authors_acknowledge_p":null}],"open_questions":null,"recommendation":"uncertain"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper presents a method for a vision-language model to improve itself by writing what its own harness (prompts, tools, image views) adds into its weights, using forward passes only. The core idea is to treat the activation difference that a harness entry makes at each transformer block as a target, and to fit a low-rank linear readout of each block's MLP input to that gap via ridge regression, then write the result into a few key-value neurons. The method is evaluated on ten visual reasoning benchmarks (five in-distribution, five held-out) using Qwen3.8-27B, and it consistently outperforms prompting, visual zoom-in, harness evolution, and fine-tuning. The gains transfer to other base models (Gemma-4-31B, Qwen3.6-27B, Qwen3.5-9B). The paper is thorough: it provides mathematical propositions with proofs in the appendix, extensive ablations, statistical tests, and released code. The main claims are supported by the evidence. The soft spots are minor. The proofs assume Lipschitz constants and bounded pairs that are not explicitly verified, but the empirical results are consistent with the theory. The rank-64 readout is justified empirically (Table 12) but not theoretically guaranteed. The acceptance test assumes that gains on validation questions generalize to held-out benchmarks, which is a standard assumption and is tested. The paper is well-written and the experiments are carefully designed. This paper deserves a serious referee. It is a solid contribution to the field of recursive self-improvement and weight editing. I recommend engaging with the work.","headline":"BaseRSI: Recursive Self-Improvement of a Base Vision-Language Model via Training-Free Harness Internalization","tokens_in":69439,"tokens_out":379,"duration_ms":5554},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"BaseRSI: a VLM improves its own weights without training","keywords":["recursive self-improvement","harness internalization","weight editing","training-free","vision-language model","teacher-student gap","key-value neurons","visual reasoning"],"falsifier":"If one could find a harness entry whose per-block activation gap has significant non-linear or high-rank components that are not captured by a rank-64 linear readout, and for which the acceptance test passes but the edit fails to generalize to held-out benchmarks, that would falsify the central claim. Concretely, one could design a harness entry that requires a complex transformation of the MLP input (e.g., a non-linear function of multiple input dimensions) and test whether BaseRSI's edit still reproduces the harness's effect and generalizes.","tokens_in":143315,"feed_emoji":"🧠","tokens_out":3852,"duration_ms":23120,"temperature":0.2,"pith_summary":"The paper proposes BaseRSI, a method for a vision-language model to improve its own weights without training, by first evolving a 'harness' (prompts/tools) that helps it answer questions, then writing the effect of that harness into its own weights via a training-free edit. The key idea is that the harness's effect on each transformer block's activations (the 'gap') can be measured and then regressed into a few key-value neurons per block, composing edits from different harness entries. The paper claims this consistently improves the base model on all tested benchmarks, outperforming prompting, fine-tuning, and other harness-evolution methods, and that the edited model exceeds its own harness in the request while answering original requests alone.","feed_headline":"BaseRSI: a VLM improves its own weights without training","feed_subtitle":"The model writes what its prompts and tools add into its weights, outperforming fine-tuning and prompting on every benchmark.","key_machinery":"The central mechanism is the teacher-student gap (ReGap): for each transformer block, the difference in the block's output when the harness is present vs. absent, computed on the same response states. This gap is the exact descent direction that would make the student match the teacher at that block. ReKey then writes this gap into the model's weights by fitting a low-rank linear readout of the MLP input via ridge regression, storing the result in a few key-value neurons per block (r=64). The site of the edit (which weights to change) is determined by measuring which part of the divergence from the teacher is removed by patching the linear-attention layers vs. the MLP. The strength (step size) is set by a line search on held-out responses. The outer loop evolves the harness by proposing candidate entries from the model's own failures and accepting those whose gain generalizes across benchmarks (leave-one-benchmark-out significance test).","core_discovery":"The paper claims that a base vision-language model can recursively improve its own weights by internalizing the gains from its harness (prompts, tools, image views) through a training-free mechanism called ReGap and ReKey. ReGap measures the per-block activation difference that the harness makes (the teacher-student gap) and treats it as a high-dimensional signal. ReKey writes this signal into a few key-value neurons per block via ridge regression, with the site and strength of each edit measured on the model's own responses. The edits from different harness entries compose in one checkpoint without refitting. The paper reports that this method consistently improves the base model on ten visual reasoning benchmarks (five in-distribution, five held-out), outperforming prompting, visual zoom-in, harness evolution, and fine-tuning, and that the edited checkpoint exceeds its own harness in the request while answering original requests on standard inference engines.","headline_check":{"rule":"report_a_claim_v1","sent_back":false,"unattributed":{}},"pith_inferences":["The paper's approach suggests a general paradigm for self-improvement that could extend beyond vision-language models to any transformer-based architecture, as long as a 'harness' (augmented input) can be defined and its per-block activation gap measured.","The reliance on a low-rank (r=64) linear readout of the MLP input to approximate the teacher-student gap implies that the harness's effect on each block is largely linear in the MLP input; if this assumption fails for more complex harness entries, the method might require higher rank or non-linear readouts.","The acceptance test (leave-one-benchmark-out significance) is a practical way to ensure generalization, but it depends on the representativeness of the validation split; for deployment in truly novel domains, the test might need to be adapted or supplemented with additional validation data.","The paper's results show the edited model sometimes exceeds the teacher (harness in the request) on out-of-scope questions, which the authors attribute to the student learning with privileged information (the key that reads its own state). This suggests that internalization can provide benefits beyond simply copying the harness's behavior, potentially leading to better generalization."],"forward_implications":["If BaseRSI works as claimed, a vision-language model can improve its own weights on its own responses without any training (backward passes, gradient updates, or external supervision), using only forward passes and linear solves.","The method would allow a base model to consolidate gains from multiple harness entries (e.g., a finer image view and a counting procedure) into a single checkpoint, where the edits compose without refitting and each can be removed individually.","The recursive loop (outer: evolve harness, inner: internalize into weights) could in principle continue indefinitely, with the model proposing new entries from its improved state and internalizing them in subsequent rounds.","Because the edits are written into a small fraction of weights (0.23% of the language model) and leave the rest bit-identical, the method is interpretable and reversible, and the edited model can be deployed on standard inference engines without the harness in the request."],"supporting_citations":[{"why":"Defines the concept of a harness (prompts, tools, context) around a frozen model and the paradigm of Recursive Harness Self-Improvement, which BaseRSI extends by internalizing the harness into weights.","marker":"[rhi]"},{"why":"Provides the theoretical framework (DAgger, follow-the-leader) for the inner-loop rounds on the student's own responses, which BaseRSI uses to handle the covariate shift of imitation learning.","marker":"[ross2011reduction]"},{"why":"Introduces the idea of locating and editing factual associations in transformer MLPs, which BaseRSI adapts by writing the teacher-student gap into key-value neurons via regression.","marker":"[meng2022locating]"},{"why":"Extends weight editing to mass-editing memories in transformers, providing a baseline for the weight-editing component that BaseRSI improves upon by measuring site and strength.","marker":"[meng2022mass]"},{"why":"Introduces the learning using privileged information (LUPI) paradigm, which BaseRSI invokes to explain why the edited student can exceed the teacher on out-of-scope questions.","marker":"[vapnik2009new]"},{"why":"Describes Meta-Harness, a harness-evolution method that BaseRSI compares against and that represents the state of the art in evolving prompts/tools around frozen weights.","marker":"[lee2026meta]"},{"why":"Describes Harness-Zero, a harness-distillation method that supervises weights at the output, which BaseRSI contrasts with its per-block supervision inside the model.","marker":"[ye2026harness]"},{"why":"Shows the implicit dynamics of in-context learning as equivalent to gradient steps, providing a theoretical basis for treating the teacher-student gap as an implicit weight update.","marker":"[dherin2025learning]"},{"why":"Establishes the equivalence of context and parameter updates in transformer blocks, supporting BaseRSI's approach of converting the harness's effect into a weight edit.","marker":"[goldwaser2025equivalence]"},{"why":"Introduces the idea of transmuting prompts into weights, which BaseRSI generalizes to harness entries (prompts, tools, image views) and applies recursively.","marker":"[mazzawi2025transmuting]"}],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that the teacher-student activation gap at each block can be faithfully approximated by a low-rank (r=64) linear readout of the MLP input, and that the acceptance test (leave-one-benchmark-out significance) ensures generalization to held-out benchmarks.","cache_creation_input_tokens":0},"created_at":"2026-11-19T07:43:43.918631+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00","input_kind":"tex","input_chars":195116,"input_total_chars":195116,"passes_missing":[],"published_standing":"established","published_confidence":"moderate","objection_audit":{"version":"pith-objection-audit@1","counts":{"raised":4,"confirmed":0,"questions":0,"withdrawn":3,"moved_to_minor":1,"math_error_not_certified":0,"withdrawn_reviewer_error":3,"typesetting_minor_dropped":0,"withdrawn_already_addressed":0,"referee_assessment_unpublished":1},"open_questions":[],"standing":{"basis":["first_pass_verdict"],"standing":"established","confidence":"moderate"},"standing_rule":"1. No standing is published (not_assessed) when the review read only the abstract, when the referee pass or the objection check did not run, or when the first pass gave no verdict. 2. The first pass's verdict sets the start: ACCEPT is established, CONDITIONAL is established_narrower, REJECT is not_supported. 3. Each objection that survived the check against the paper's text caps the standing: one that breaks the central claim caps it at not_supported, one that narrows it at established_narrower, one that weakens it at supported_with_gaps. The standing is the lowest of the start and the caps. 4. A first-pass REJECT stands only if a surviving objection breaks the central claim. Without one, the standing is the lowest cap of the surviving objections, or not_assessed when none survived. 5. The confidence is the lowest of the first pass's confidence and the confidence of each objection that set the standing, and is no higher than moderate while a question is open or part of the paper's text was not supplied. 6. If any objection was withdrawn because Pith's reviewer misread the paper, the confidence is no higher than moderate, whatever the first pass said: a review that misread the paper once may have misread it elsewhere.","input_kind":"tex","nothing_to_audit":false,"bar":{"enabled":false}},"run_record":{"models":["deepseek/deepseek-v4-flash"],"passes":{"reader":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.015736,"tokens_in":147364,"tokens_out":43707,"duration_ms":152532,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56","message_sha256":"8aec307e5be9bf70f34df2022c344ce1609f81fe1dd3da13449ef04f633940cb","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"referee":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.006738,"tokens_in":69225,"tokens_out":2411,"duration_ms":15627,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"a852d99aea0ddfd900e1483f76e20777aae3afde9c016d5b8d6e5db01064c728","message_sha256":"c6de5a8d75211d844db4f1b92a48b95b89d57f7bf9288e34807814ca028e62d2","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"skeptic":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.006301,"tokens_in":68843,"tokens_out":197,"duration_ms":4656,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"8f88c0f61488b1e764c317998cddf35fffc33a1f20a3b1be54330d7843386fc7","message_sha256":"c3bfc425c0d6e392091e98ddf4726664f02a43044235d0499d3932faf7710032","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"circularity":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.010492,"tokens_in":68966,"tokens_out":3399,"duration_ms":24429,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"e9946e39d67b76cc60476379bc32abdc44665b5f0c438659796547bb1041a01f","message_sha256":"03e758eed0cf6a5a03a38df3935c93c8bc5cbb57ed98b8aa7db40f5aeea30511","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"desk_editor":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.006388,"tokens_in":69439,"tokens_out":379,"duration_ms":5554,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"e7211f6a3d2734c1025ce7e7e6d48f78b0944d0a32d5021f81be63b27721420a","message_sha256":"27a69907d658d17b31481cd1d5026ed0c83597bb1b9f9d1129d0dfecee5c48e7","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"objection_audit":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.015656,"tokens_in":70014,"tokens_out":21728,"duration_ms":112414,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"6c7a6883d5517cf9e4945d83947200a36dcd152b8b56c32baf7500d8b3841d0a","message_sha256":"3a077043f51c80e3e1d4e54327c2fc3aa200504594f91b8f0f2868a4fafabd59","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"pith_extraction":{"model":"deepseek/deepseek-v4-flash","effort":"medium","cost_usd":0.008618,"tokens_in":143315,"tokens_out":3852,"duration_ms":23120,"temperature":0.2,"input_sha256":"7e3ca86e40402f1e32cc86a77cb1867fcc7b5e02f5145458c9a8ffc405f4857a","prompt_sha256":"e4f56bf568e7854d0a1d2efb6ec96f8697d5132737cdb31cb7fc64acbd923974","message_sha256":"3cb7f89d1c968be81f5613c3703320416df7180f8e36c7af7eca3f11c859e48e","cache_read_input_tokens":0,"cache_creation_input_tokens":0}},"version":"pith-run-record/v1","prompt_set_sha256":"d60f54226f003b7308ab6f0f9f2951c09c726f00cc66fc05af3456f212247e93","configured_pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium"},"basis":{"kind":"full_text","source":"recorded","chars_given":195116,"chars_total":195116},"receipt":{"review_sha256":"2977841aeae544201f377e7a13a127b144ae100533e08f0ca8b98a20e5c03cd1","rule":"pith-review-sha256/v1","hashed_at":"2026-11-19T07:47:02.644980+00:00","fields":["id","arxiv_id","paper_version","created_at","pipeline_version","model_set","run_record","input_kind","input_chars","input_total_chars","passes_missing","verdict","confidence","novelty_score","correctness_risk","formal_verification","parameter_count","discovery_kind","one_line_summary","lead","skeptic_critique","referee_report","author_rebuttal","desk_editor","pith_extraction","objection_audit","published_standing","published_confidence"],"matches_current":true},"falsifier":"If one could find a harness entry whose per-block activation gap has significant non-linear or high-rank components that are not captured by a rank-64 linear readout, and for which the acceptance test passes but the edit fails to generalize to held-out benchmarks, that would falsify the central claim. Concretely, one could design a harness entry that requires a complex transformation of the MLP input (e.g., a non-linear function of multiple input dimensions) and test whether BaseRSI's edit still reproduces the harness's effect and generalizes.","supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":"tex"},"url":"https://pith.science/accuracy"},"review_version":1}