REVIEW 9 cited by
ReFT: Representation Finetuning for Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via updates to a small number of weights. However, much prior interpretability work has shown that representations encode rich semantic information, suggesting that editing representations might be a more powerful alternative. We pursue this hypothesis by developing a family of Representation Finetuning (ReFT) methods. ReFT methods operate on a frozen base model and learn task-specific interventions on hidden representations. We define a strong instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT), and we identify an ablation of this method that trades some performance for increased efficiency. Both are drop-in replacements for existing PEFTs and learn interventions that are 15x--65x more parameter-efficient than LoRA. We showcase LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks, instruction-tuning, and GLUE. In all these evaluations, our ReFTs deliver the best balance of efficiency and performance, and almost always outperform state-of-the-art PEFTs. We release a generic ReFT training library publicly at https://github.com/stanfordnlp/pyreft.
Forward citations
Cited by 9 Pith papers
-
Agent Security Needs Redefinition through a Holistic Framework
Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.
-
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...
-
Infogen: Generating Complex Statistical Infographics from Documents
Infogen generates complex statistical infographics from text via a metadata-then-code pipeline, and the new Infodat benchmark shows it outperforming GPT-4o and fine-tuned open LLMs on the authors' metrics.
-
Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
Fine-tuning a single transformer layer with activation targets scaled by each query's projection onto a refusal direction reduces over-refusal on four benchmarks while preserving safety and general capability.
-
First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers
Single task-vector perturbations around a multitask LoRA point are first-order linear to 1e-2, but pairwise update-order sensitivity is set by a per-pair Lie bracket and can appear inside that window.
-
Dynamically Scaled Activation Steering
DSAS uses a per-token logistic gate to scale any activation-steering method, improving the toxicity-versus-quality Pareto front in LLMs and diffusion models.
-
Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning
CRFT selects critical internal representations via attention and saliency scores and fine-tunes only them, improving GSM8K accuracy over ReFT from 29.0% to 32.8% on LLaMA-2-7B.
-
Drop Dropout on Single-Epoch Language Model Pretraining
Removing dropout from single-epoch language model pretraining improves downstream task performance and model editability across multiple architectures and benchmarks.
-
Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
FAPM prunes task vectors using a combined absolute and relative change magnitude criterion, reducing forgetting on general benchmarks to roughly a third of a percent while keeping downstream accuracy near the fine-tun...
Discussion (0). Sign in to comment.