Pith. sign in

REVIEW 2 major objections 2 minor 33 references

Fora: From Weight-Space to Function-Space Protection in Capability-Preserving Fine-Tuning

T0 review · 2 major / 2 minor · reviewed 2026-07-02 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FORA protects capabilities during fine-tuning by projecting updates away from activation subspaces derived from covariance rather than weight singular vectors.

desk verdict FORA's activation-covariance projector beats weight-space baselines on the reported tasks but the calibration assumption is unverified. read the letter →

arxiv 2606.31092 v2 pith:GKLLBGFX submitted 2026-06-30 cs.LG

Rui Zhou , Tianci Xie This is my paper · ORCID
classification cs.LG
keywords capabilitypreservationfine-tuningactivationsubspaceorthogonalprojectionfunctionspacelanguagemodeladaptationcovarianceestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that existing weight-space methods for preserving capabilities during fine-tuning rely on imperfect proxies and proposes instead to identify the activation directions a capability actually uses. It computes the principal directions Q of the input-activation covariance on label-free calibration data per layer, then builds a right projector that structurally prevents the main update from reading those directions. The resulting update combines this projector with a narrow spectral channel from the weight SVD. Experiments on Qwen3-1.7B across three preservation settings show consistent gains over weight-space projection and regularization baselines, with the advantage traced to the choice of activation-derived rather than weight-derived directions.

What carries the argument

The right projector P_Q = I - QQ^T, where Q holds the top principal directions of the input-activation covariance, which enforces that the update matrix cannot read activation directions associated with the preserved capability.

What would settle it

A controlled run in which calibration inputs are replaced by data that yields a different Q while the preserved capability remains unchanged, followed by measurement of whether preservation performance drops relative to the original Q.

Watch

Extended reading notes

Core claim

A capability is characterized more faithfully by the activation subspace it induces than by the singular geometry of the weight matrix; FORA forms per-layer principal directions Q of the input-activation covariance from label-free inputs, applies the right projector P_Q = I - QQ^T to bar the high-capacity branch from those directions, and augments with a controlled spectral term U_2 D_δ V_2^T, extending naturally to low-rank adaptation.

Load-bearing premise

The principal directions of the input-activation covariance computed from label-free calibration inputs accurately identify the activation subspace used by the capability that must be preserved.

Editorial extensions

If this is right

  • FORA yields higher preservation scores than weight-space projection or standard regularization across the reported COGS, GSM8K, and translation settings on Qwen3-1.7B.
  • The performance edge is attributable to the source of the projection directions rather than the act of projection itself.
  • The same projectors apply directly when the update is reparameterized as a low-rank adaptation.
  • Only a small accuracy cost on the new task appears in the math-preservation setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the estimated activation subspaces remain stable when new tasks are added sequentially, repeated application of FORA could support continued adaptation without progressive loss of earlier capabilities.
  • The label-free nature of the covariance estimation suggests the method could be applied in settings where task-specific labels for the preserved capability are unavailable or expensive.
  • Because the projectors act layer-wise on activation statistics, the approach may interact with other per-layer regularization techniques that also operate on activations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes FORA (Function-space Orthogonal Residual Adaptation) for capability-preserving fine-tuning of LLMs. It estimates per-layer principal directions Q of input-activation covariance from label-free calibration inputs to form a right projector P_Q = I - QQ^T, pairs it with a left projector P_U from the weight SVD, and constructs the update as ΔW = P_U M P_Q + U_2 D_δ V_2^T (extendable to PEFT via low-rank factors). This bars the high-capacity branch from reading capability-relevant function directions while allowing controlled plasticity. Experiments on Qwen3-1.7B across COGS/GSM8K learning while preserving translation and translation while preserving math report consistent preservation gains over weight-space projection and regularization, with an ablation attributing the benefit to activation-derived rather than weight-derived directions.

Significance. If the results hold under stronger validation of the subspace assumption, the shift from weight-space proxies to function-space characterization of capabilities would be a substantive contribution to continual learning and multi-task adaptation. The controlled ablation isolating projection source, the code release, and the parameter-free derivation of projectors from data (modulo calibration choice) are positive features supporting reproducibility and falsifiability.

major comments (2)
  1. [Method description and ablation study] The central claim that function-space protection via P_Q is more faithful than weight-space methods depends on the premise that the top principal directions Q of the input-activation covariance, estimated from the chosen label-free calibration inputs, span the activation subspace actually used by the preserved capability. The ablation demonstrates gains from activation-derived over weight-derived projectors, yet provides no direct verification (e.g., overlap metrics between Q and activations elicited during capability exercise on held-out translation or math examples) that the calibration distribution covers the relevant syntactic, semantic, or numerical patterns.
  2. [Experiments section] Table or figure reporting the three settings (COGS/GSM8K preservation and math-preservation) shows consistent improvements, but the manuscript does not report statistical significance, variance across seeds, or the magnitude of the 'small new-task trade-off' in the math-preservation case relative to baselines; without these, the robustness of the cross-setting claim cannot be fully assessed.
minor comments (2)
  1. [Method] Notation for the combined projector and the narrow spectral channel (U_2 D_δ V_2^T) is introduced without an explicit equation number or diagram clarifying how the two branches interact during the forward pass.
  2. [Abstract and ablation] The abstract states that the advantage 'comes not from projection itself, but from projecting onto capability-derived rather than weight-derived directions,' but the corresponding ablation description would benefit from an explicit statement of the weight-derived baseline construction to allow direct comparison.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the detailed review. We address each of the major comments below and will revise the manuscript accordingly to incorporate the suggested enhancements.

read point-by-point responses

  1. Referee: [Method description and ablation study] The central claim that function-space protection via P_Q is more faithful than weight-space methods depends on the premise that the top principal directions Q of the input-activation covariance, estimated from the chosen label-free calibration inputs, span the activation subspace actually used by the preserved capability. The ablation demonstrates gains from activation-derived over weight-derived projectors, yet provides no direct verification (e.g., overlap metrics between Q and activations elicited during capability exercise on held-out translation or math examples) that the calibration distribution covers the relevant syntactic, semantic, or numerical patterns.

    Authors: We appreciate this observation. The ablation study provides indirect support by showing superior performance when using activation-derived projectors compared to weight-derived ones. However, we agree that direct verification through overlap metrics on held-out examples would more rigorously validate that the calibration inputs capture the relevant activation subspaces. In the revised manuscript, we will add these overlap analyses for the preserved capabilities in each experimental setting. revision: yes

  2. Referee: [Experiments section] Table or figure reporting the three settings (COGS/GSM8K preservation and math-preservation) shows consistent improvements, but the manuscript does not report statistical significance, variance across seeds, or the magnitude of the 'small new-task trade-off' in the math-preservation case relative to baselines; without these, the robustness of the cross-setting claim cannot be fully assessed.

    Authors: We thank the referee for highlighting this. The reported results are averages, but to better assess robustness, we will include standard deviations across multiple random seeds and conduct statistical significance tests (e.g., paired t-tests) in the updated tables and figures. Furthermore, we will explicitly report the magnitude of the new-task trade-off in the math-preservation setting by comparing the performance drops relative to the baselines. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; explicit construction from independent data-derived projectors validated empirically

full rationale

The paper defines FORA via an explicit algorithmic construction: per-layer estimation of Q as top principal directions of input-activation covariance on label-free calibration inputs, followed by P_Q = I - QQ^T and the update rule ΔW = P_U M P_Q + U_2 D_δ V_2^T (with P_U from weight SVD). This is not derived from a self-referential premise or fitted to the target preservation metric; the projectors are computed directly from calibration data independent of the new-task training or evaluation distributions. Central claims rest on controlled ablations and comparisons against weight-space baselines on external benchmarks (COGS, GSM8K, translation/math preservation), which are falsifiable outside the method definition. No self-citations, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The derivation chain is therefore self-contained and does not reduce to its inputs by construction.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The approach relies on one key domain assumption about what characterizes a capability and on the choice of calibration data as a free parameter; no new entities are postulated.

free parameters (1)
  • choice of calibration inputs
    The principal directions Q depend on the selected label-free calibration inputs, whose selection criteria are not detailed.
assumptions (1)
  • domain assumption A capability is characterized more faithfully by the activation subspace it induces than by the singular geometry of the weight matrix
    This is explicitly argued as the basis for developing function-space protection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fora: From Weight-Space to Function-Space Protection in Capability-Preserving Fine-Tuning." pith.science (2026). https://pith.science/paper/GKLLBGFX

@misc{pith2026260631092,
  author       = {Pith},
  title        = {Pith review of: Fora: From Weight-Space to Function-Space Protection in Capability-Preserving Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GKLLBGFX}},
  note         = {Machine review of arXiv:2606.31092}
}
abstract

Full fine-tuning adapts large language models to new tasks but can erode capabilities they already possess. Existing remedies protect through proxies such as parameter distances, importance penalties, output matching, or dominant singular directions of the weights, but none directly asks which activation directions the preserved capability relies on. We argue that a capability is characterized more faithfully by the activation subspace it induces than by the singular geometry of the weight matrix, and develop function-space protection, instantiated as FORA (Function-space Orthogonal Residual Adaptation). From label-free calibration inputs, FORA estimates, per layer, the principal directions $Q$ of the input-activation covariance and forms a right projector $P_Q = I - QQ^T$. Paired with a left projector $P_U$ from the weight SVD, the update is $\Delta W = P_U M P_Q + U_2 D_{\delta} V_2^T$: a high-capacity branch structurally barred from reading capability-relevant function directions, plus a narrow spectral channel for controlled plasticity. The construction extends to parameter-efficient adaptation via $M \to (\alpha/r) BA$. Across three settings on Qwen3-1.7B, including COGS and GSM8K learned while preserving translation and translation learned while preserving math, FORA consistently improves preservation over weight-space projection and standard regularization, with only a small new-task trade-off in the math-preservation setting. A controlled ablation isolating the projection source shows that the advantage comes not from projection itself, but from projecting onto capability-derived rather than weight-derived directions. Code is available at https://github.com/zrui239/FORA.

Figures

Figures reproduced from arXiv: 2606.31092 by the authors.

Figure 1
Figure 1. Preservation–adaptation trade-off across all three settings (visualizing Tables 2, 3, and 4). Each panel plots a new-task metric (horizontal) against the preserved-capability metric (vertical); the dashed line marks the preserved￾capability target and the shaded band the favorable region. (a) Trans→COGS and (b) Trans→GSM8K plot new-task EM against translation PPL (lower better, axis inverted); (c) Math→Trans plots t… view at source ↗
Figure 2
Figure 2. From weight-space projection to function-space protection. (a) Unconstrained full fine-tuning can overwrite any direction, offering maximal plasticity but no protection. (b) Weight-space projection masks the principal singular directions of W0; these mark where the weight is large, which need not coincide with the capability’s functional footprint. (c) Fora replaces the weight-derived mask with a capability-derived … view at source ↗
Figure 3
Figure 3. Construction and forward computation of Fora. (a) Function-space source. From label-free preserved￾capability inputs we collect per-layer activations Xpres, form the covariance C = 1 N X⊤ presXpres, and take its top-kf eigenvectors Q to build the right projector PQ = I − QQ⊤. (b) Protected forward pass. The output combines the frozen weight W, the function-protected high-capacity branch PUMPQ, and the spectral calib… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Projection source across three settings (appendix visualization of [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages

  1. [1]

    Proceedings of the National Academy of Sciences , volume=

    Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the National Academy of Sciences , volume=. 2017 , doi=

    work page 2017

  2. [2]

    International Conference on Machine Learning (ICML) , pages=

    Explicit Inductive Bias for Transfer Learning with Convolutional Networks , author=. International Conference on Machine Learning (ICML) , pages=. 2018 , url=

    work page 2018

  3. [3]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Learning without Forgetting , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2018 , doi=

    work page 2018

  4. [4]

    International Conference on Learning Representations (ICLR) , year=

    Gradient Projection Memory for Continual Learning , author=. International Conference on Learning Representations (ICLR) , year=

    work page

  5. [5]

    AAAI Conference on Artificial Intelligence , year=

    OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting During Parameter-Efficient Fine-Tuning , author=. AAAI Conference on Artificial Intelligence , year=

    work page

  6. [6]

    Findings of the Association for Computational Linguistics: EMNLP , year=

    Orthogonal Subspace Learning for Language Model Continual Learning , author=. Findings of the Association for Computational Linguistics: EMNLP , year=

    work page

  7. [7]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

    work page

  8. [8]

    International Conference on Machine Learning (ICML) , year=

    DoRA: Weight-Decomposed Low-Rank Adaptation , author=. International Conference on Machine Learning (ICML) , year=

    work page

Show all 33 references
  1. [9]

    Annual Conference of the Nations of the Americas Chapter of the ACL (NAACL) , year=

    MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning , author=. Annual Conference of the Nations of the Americas Chapter of the ACL (NAACL) , year=

  2. [10]

    International Conference on Learning Representations (ICLR) , year=

    AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning , author=. International Conference on Learning Representations (ICLR) , year=

  3. [11]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  4. [12]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  5. [13]

    International Conference on Learning Representations (ICLR) , year=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=

  6. [14]

    Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

    COGS: A Compositional Generalization Challenge Based on Semantic Interpretation , author=. Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

  7. [15]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  8. [16]

    Patel, Arkil and Bhattamishra, Satwik and Goyal, Navin , booktitle=. Are. 2021 , doi=

    2021

  9. [17]

    Parallel Data, Tools and Interfaces in

    Tiedemann, J. Parallel Data, Tools and Interfaces in. International Conference on Language Resources and Evaluation (LREC) , year=

  10. [18]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 Technical Report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  11. [19]

    International Conference on Learning Representations (ICLR) , year=

    Decoupled Weight Decay Regularization , author=. International Conference on Learning Representations (ICLR) , year=

  12. [20]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Attention is All you Need , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  13. [21]

    Psychology of Learning and Motivation , volume=

    Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , author=. Psychology of Learning and Motivation , volume=. 1989 , publisher=

    1989

  14. [22]

    Trends in Cognitive Sciences , volume=

    Catastrophic Forgetting in Connectionist Networks , author=. Trends in Cognitive Sciences , volume=. 1999 , doi=

    1999

  15. [23]

    Neural Networks , volume=

    Continual Lifelong Learning with Neural Networks: A Review , author=. Neural Networks , volume=. 2019 , doi=

    2019

  16. [24]

    European Conference on Computer Vision (ECCV) , year=

    Memory Aware Synapses: Learning What (not) to Forget , author=. European Conference on Computer Vision (ECCV) , year=

  17. [25]

    International Conference on Machine Learning (ICML) , pages=

    Continual Learning Through Synaptic Intelligence , author=. International Conference on Machine Learning (ICML) , pages=. 2017 , url=

    2017

  18. [26]

    Advances in Neural Information Processing Systems (NeurIPS) , pages=

    Gradient Episodic Memory for Continual Learning , author=. Advances in Neural Information Processing Systems (NeurIPS) , pages=

  19. [27]

    Efficient Lifelong Learning with

    Chaudhry, Arslan and Ranzato, Marc'Aurelio and Rohrbach, Marcus and Elhoseiny, Mohamed , booktitle=. Efficient Lifelong Learning with. 2019 , url=

    2019

  20. [28]

    International Conference on Learning Representations (ICLR) , year=

    Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization , author=. International Conference on Learning Representations (ICLR) , year=

  21. [29]

    International Conference on Machine Learning (ICML) , pages=

    Function-Space Regularization in Neural Networks: A Probabilistic Perspective , author=. International Conference on Machine Learning (ICML) , pages=. 2023 , url=

    2023

  22. [30]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Refusal in Language Models Is Mediated by a Single Direction , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  23. [31]

    ACM Computing Surveys , year=

    Continual Learning of Large Language Models: A Comprehensive Survey , author=. ACM Computing Surveys , year=

  24. [32]

    Transactions on Machine Learning Research (TMLR) , year=

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey , author=. Transactions on Machine Learning Research (TMLR) , year=

  25. [33]

    QLoRA: Efficient Finetuning of Quantized

    Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , booktitle=. QLoRA: Efficient Finetuning of Quantized. 2023 , url=

    2023

Pith tools

Reviewed July 2, 2026 · model on record in the stance chip above.