{"id":"09ab60c7-2b1d-4cbb-860d-9ad84f8fcf95","arxiv_id":"2606.31092","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FORA uses function-space projectors from activation covariances to enable capability-preserving fine-tuning of LLMs, outperforming weight-space methods on preservation tasks.","lead":"The paper presents FORA, a fine-tuning method that protects model capabilities by projecting updates orthogonal to activation directions derived from calibration data rather than weight singular vectors. Smart generalists might read it because preserving capabilities during adaptation is key to building reliable, multi-task AI systems without constant retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Calibration inputs may fail to elicit the activation subspace actually used by the preserved capability","rationale":"The reader's weakest_assumption directly names the load-bearing premise required for the 'more faithful' argument. Empirical gains are reported, but without evidence that Q targets the correct subspace the advantage cannot be confidently attributed to function-space rather than incidental regularization; this keeps the verdict at CONDITIONAL.","tokens_in":1828,"tokens_out":334,"duration_ms":18161,"concrete_test":"On the translation-preservation setting, collect activations on a held-out set of translation inputs drawn from the same distribution as the preservation evaluation; compute the fraction of variance in those activations explained by the top-k directions of the calibration-derived Q. If this overlap is low (e.g., <60% for k equal to the rank used in the paper), the subspaces do not match and the function-space interpretation weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that function-space protection via P_Q is more faithful than weight-space methods rests on the premise that the top principal directions Q of the input-activation covariance, estimated from label-free calibration inputs, span the directions the preserved capability (translation or math) actually relies upon. If the calibration distribution does not cover the syntactic, semantic, or numerical patterns encountered when the capability is exercised during new-task training or evaluation, then P_Q = I - QQ^T will leave relevant directions unblocked while unnecessarily constraining others. The reported ablation shows gains from activation-derived over weight-derived projectors, yet provides no direct check that the chosen Q aligns with the capability's usage distribution.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes FORA (Function-space Orthogonal Residual Adaptation) for capability-preserving fine-tuning of LLMs. It estimates per-layer principal directions Q of input-activation covariance from label-free calibration inputs to form a right projector P_Q = I - QQ^T, pairs it with a left projector P_U from the weight SVD, and constructs the update as ΔW = P_U M P_Q + U_2 D_δ V_2^T (extendable to PEFT via low-rank factors). This bars the high-capacity branch from reading capability-relevant function directions while allowing controlled plasticity. Experiments on Qwen3-1.7B across COGS/GSM8K learning while preserving translation and translation while preserving math report consistent preservation gains over weight-space projection and regularization, with an ablation attributing the benefit to activation-derived rather than weight-derived directions.","tokens_in":1970,"tokens_out":640,"duration_ms":17597,"significance":"If the results hold under stronger validation of the subspace assumption, the shift from weight-space proxies to function-space characterization of capabilities would be a substantive contribution to continual learning and multi-task adaptation. The controlled ablation isolating projection source, the code release, and the parameter-free derivation of projectors from data (modulo calibration choice) are positive features supporting reproducibility and falsifiability.","major_comments":[{"comment":"The central claim that function-space protection via P_Q is more faithful than weight-space methods depends on the premise that the top principal directions Q of the input-activation covariance, estimated from the chosen label-free calibration inputs, span the activation subspace actually used by the preserved capability. The ablation demonstrates gains from activation-derived over weight-derived projectors, yet provides no direct verification (e.g., overlap metrics between Q and activations elicited during capability exercise on held-out translation or math examples) that the calibration distribution covers the relevant syntactic, semantic, or numerical patterns.","section":"Method description and ablation study"},{"comment":"Table or figure reporting the three settings (COGS/GSM8K preservation and math-preservation) shows consistent improvements, but the manuscript does not report statistical significance, variance across seeds, or the magnitude of the 'small new-task trade-off' in the math-preservation case relative to baselines; without these, the robustness of the cross-setting claim cannot be fully assessed.","section":"Experiments section"}],"minor_comments":[{"comment":"Notation for the combined projector and the narrow spectral channel (U_2 D_δ V_2^T) is introduced without an explicit equation number or diagram clarifying how the two branches interact during the forward pass.","section":"Method"},{"comment":"The abstract states that the advantage 'comes not from projection itself, but from projecting onto capability-derived rather than weight-derived directions,' but the corresponding ablation description would benefit from an explicit statement of the weight-derived baseline construction to allow direct comparison.","section":"Abstract and ablation"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"Thank you for the detailed review. We address each of the major comments below and will revise the manuscript accordingly to incorporate the suggested enhancements.","responses":[{"response":"We appreciate this observation. The ablation study provides indirect support by showing superior performance when using activation-derived projectors compared to weight-derived ones. However, we agree that direct verification through overlap metrics on held-out examples would more rigorously validate that the calibration inputs capture the relevant activation subspaces. In the revised manuscript, we will add these overlap analyses for the preserved capabilities in each experimental setting.","revision_made":"yes","referee_comment":"[Method description and ablation study] The central claim that function-space protection via P_Q is more faithful than weight-space methods depends on the premise that the top principal directions Q of the input-activation covariance, estimated from the chosen label-free calibration inputs, span the activation subspace actually used by the preserved capability. The ablation demonstrates gains from activation-derived over weight-derived projectors, yet provides no direct verification (e.g., overlap metrics between Q and activations elicited during capability exercise on held-out translation or math examples) that the calibration distribution covers the relevant syntactic, semantic, or numerical patterns."},{"response":"We thank the referee for highlighting this. The reported results are averages, but to better assess robustness, we will include standard deviations across multiple random seeds and conduct statistical significance tests (e.g., paired t-tests) in the updated tables and figures. Furthermore, we will explicitly report the magnitude of the new-task trade-off in the math-preservation setting by comparing the performance drops relative to the baselines.","revision_made":"yes","referee_comment":"[Experiments section] Table or figure reporting the three settings (COGS/GSM8K preservation and math-preservation) shows consistent improvements, but the manuscript does not report statistical significance, variance across seeds, or the magnitude of the 'small new-task trade-off' in the math-preservation case relative to baselines; without these, the robustness of the cross-setting claim cannot be fully assessed."}],"tokens_in":1574,"tokens_out":407,"duration_ms":22394,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that FORA derives a right projector from the top principal directions of input-activation covariances on label-free calibration data, then combines it with a left projector from weight SVD to block capability-relevant directions while allowing a narrow spectral update channel. This is new relative to prior weight-space or output-matching approaches, and the paper shows it improves preservation over those baselines across three settings on Qwen3-1.7B (COGS or GSM8K learned while keeping translation; translation learned while keeping math), with only a small new-task cost in one case. An ablation isolates that the gain comes from using activation-derived rather than weight-derived directions.\n\nThe construction is straightforward and extends to low-rank adaptation. Code is released, which helps. The results are consistent enough on the tested pairs to warrant attention from people working on capability retention during adaptation.\n\nThe soft spot is the untested premise that the chosen calibration inputs produce Q that actually spans the activation subspace the preserved capability relies on. If those inputs miss the syntactic or numerical patterns encountered during new-task training, the projector can leave relevant directions open or constrain irrelevant ones. The abstract gives no direct evidence that Q aligns with capability usage, so the reported edge may be tied to the specific calibration choices and tasks. All experiments are on a single 1.7B model, which limits how far the claim travels.\n\nThis is for researchers focused on practical fine-tuning safeguards. The idea is distinct, the ablation is useful, and the empirical pattern is positive, so it deserves a serious referee to check the calibration assumption and test broader conditions.","headline":"FORA's activation-covariance projector beats weight-space baselines on the reported tasks but the calibration assumption is unverified.","tokens_in":2428,"tokens_out":393,"would_cite":false,"duration_ms":20234,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"FORA protects capabilities during fine-tuning by projecting updates away from activation subspaces derived from covariance rather than weight singular vectors.","keywords":["capability preservation","fine-tuning","activation subspace","orthogonal projection","function space","language model adaptation","covariance estimation"],"falsifier":"A controlled run in which calibration inputs are replaced by data that yields a different Q while the preserved capability remains unchanged, followed by measurement of whether preservation performance drops relative to the original Q.","tokens_in":2726,"feed_emoji":"🛡️","tokens_out":715,"duration_ms":16753,"temperature":0.7,"pith_summary":"The paper claims that existing weight-space methods for preserving capabilities during fine-tuning rely on imperfect proxies and proposes instead to identify the activation directions a capability actually uses. It computes the principal directions Q of the input-activation covariance on label-free calibration data per layer, then builds a right projector that structurally prevents the main update from reading those directions. The resulting update combines this projector with a narrow spectral channel from the weight SVD. Experiments on Qwen3-1.7B across three preservation settings show consistent gains over weight-space projection and regularization baselines, with the advantage traced to the choice of activation-derived rather than weight-derived directions.","feed_headline":"Activation covariance projectors preserve capabilities better than weight-space methods","feed_subtitle":"FORA blocks fine-tuning updates from reading preserved function directions using label-free principal components, with gains shown on Qwen3-","key_machinery":"The right projector P_Q = I - QQ^T, where Q holds the top principal directions of the input-activation covariance, which enforces that the update matrix cannot read activation directions associated with the preserved capability.","core_discovery":"A capability is characterized more faithfully by the activation subspace it induces than by the singular geometry of the weight matrix; FORA forms per-layer principal directions Q of the input-activation covariance from label-free inputs, applies the right projector P_Q = I - QQ^T to bar the high-capacity branch from those directions, and augments with a controlled spectral term U_2 D_δ V_2^T, extending naturally to low-rank adaptation.","pith_inferences":["If the estimated activation subspaces remain stable when new tasks are added sequentially, repeated application of FORA could support continued adaptation without progressive loss of earlier capabilities.","The label-free nature of the covariance estimation suggests the method could be applied in settings where task-specific labels for the preserved capability are unavailable or expensive.","Because the projectors act layer-wise on activation statistics, the approach may interact with other per-layer regularization techniques that also operate on activations."],"forward_implications":["FORA yields higher preservation scores than weight-space projection or standard regularization across the reported COGS, GSM8K, and translation settings on Qwen3-1.7B.","The performance edge is attributable to the source of the projection directions rather than the act of projection itself.","The same projectors apply directly when the update is reparameterized as a low-rank adaptation.","Only a small accuracy cost on the new task appears in the math-preservation setting."],"fun_headline_variants":["Activation subspaces define capability directions better than weights","FORA projects updates using activation covariance principal directions","Function-space projectors protect preserved capabilities in tuning","Activation covariance Q bars high-capacity updates from key directions"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The principal directions of the input-activation covariance computed from label-free calibration inputs accurately identify the activation subspace used by the capability that must be preserved.","fun_headline_variants_meta":{"raw":{"variants":["Activation subspaces define capability directions better than weights","FORA projects updates using activation covariance principal directions","Function-space projectors protect preserved capabilities in tuning","Activation covariance Q bars high-capacity updates from key directions"]},"model":"grok-4.3","cost_usd":0.005801,"raw_usage":{"total_tokens":2810,"prompt_tokens":764,"num_sources_used":0,"completion_tokens":56,"cost_in_usd_ticks":58012000,"prompt_tokens_details":{"text_tokens":764,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1990,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":764,"tokens_out":56,"duration_ms":14965,"temperature":1.0,"reasoning_tokens":1990,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-02T20:17:00.556352+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled run in which calibration inputs are replaced by data that yields a different Q while the preserved capability remains unchanged, followed by measurement of whether preservation performance drops relative to the original Q.","supporting_citations":[],"review_version":2}