Task vectors from weight differences allow arithmetic operations to edit pre-trained models, improving multiple tasks simultaneously and enabling analogical inference on unseen tasks.
hub
Smith , title =
26 Pith papers cite this work, alongside 216 external citations. Polarity classification is still indexing.
26
Pith papers citing it
216
external citations · external index
hub tools
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
Fixed-clock optimizer memory turns equal-multiset data shuffle order into an O(η) source of fine-tuning noise, larger than the O(η²) effect in memoryless cases, with a fit-free sizing method derived.
Introduces SolidityBench benchmark and SolidityScore metric for repository-level Solidity code generation, finding supervised fine-tuning outperforms prompting, CoT, ICL, and RAG methods on evaluated LLMs.
OPD updates occupy a relaxed off-principal regime and rapidly lock into a low-dimensional subspace that is functionally sufficient for its performance, distinct from SFT and RLVR trajectories.
BadBone backdoors backbone models with bi-level optimization to make prompt learning on downstream tasks vulnerable while preserving model utility.
Transferability analysis finds that minimal sufficient signals transfer across audio models at rates varying by task, around 26% for music genre classification, with some deepfake models showing distinct behaviors not visible in accuracy metrics.
Reinforcement learning on a reward model trained from human summary comparisons produces summaries humans prefer over supervised fine-tuning or human references on TL;DR and transfers to CNN/DM.
FID variance from training seeds is 3.2 times larger than from sampling seeds on hundreds of SiT models, with 1-2% coefficient of variation that barely shrinks with more compute, leading to a multi-seed evaluation protocol.
MiqraBERT, a finetuned Sentence-BERT model, achieves 2.7-fold better distributional separation of parallel versus non-parallel Biblical Hebrew verses and reduces ambiguous overlap from 24% to 6%, with strong performance on narrative but weak on poetic parallels.
Trained ResNets on CIFAR-10 retain measurable dependence on initialization scale under low-LR SGD (26.5 pp test accuracy spread) but not under Adam, indicating that practical inductive bias is shaped by the forgetting time scale of the optimizer and regularizers.
Data curation alone raises VLM accuracy by more than 11 points on average across many benchmarks while cutting required training compute by up to 87 times.
Instructions trigger a production-centered mechanism in language models, with task-specific information stable in input tokens but varying strongly in output tokens and correlating with behavior.
MADE-IT outperforms baselines in continual model merging accuracy and robustness by using manifold-aware expert evolution with adaptive thresholds and implicit subspace-based routing while pruning redundant experts.
FRPO applies a max-min robust optimization over KL-bounded policy neighborhoods during RLHF to reduce catastrophic forgetting of safety and accuracy under subsequent SFT or RL fine-tuning.
Controlled ablations of 38 models find MLM superior to CLM on representation benchmarks while CLM offers better data efficiency and stability; a biphasic CLM-then-MLM schedule is optimal under fixed compute and improves when initialized from pretrained CLM models.
LIMO achieves 63.3% on AIME24 and 95.6% on MATH500 via supervised fine-tuning on roughly 1% of the data used by prior models, supporting the claim that minimal strategic examples suffice when pre-training has already encoded domain knowledge.
GRAIN is a gradient aggregation method using min-norm objectives to ensure non-negative inner products with group gradients, yielding tighter uniform stability bounds than SGD under smoothness assumptions.
RLVR induces sparse off-principal updates forming near-orthogonal shortcuts that degrade merging, addressed via Sensitivity-aware Resolving Merging using Fisher sensitivity, sparsification, and rescaling.
PromptRad reformulates multi-label radiology report classification as masked language modeling and enriches verbalizers with UMLS synonyms, outperforming baselines with only 32 training examples.
In linear regression, LoRA can achieve lower excess risk than full fine-tuning when the pretraining-downstream difference is low-rank, and small LoRA ranks can improve generalization by acting as regularization.
UAV-VL-R1 combines SFT and multi-stage GRPO reinforcement learning on a new 50,019-sample HRVQA-VL dataset to deliver substantially higher zero-shot accuracy on UAV visual reasoning tasks than both its 2B baseline and a 72B-scale model.
Fine-tuned models under 1B parameters reach micro-F1 of 0.83 on general-domain RE versus 0.69 for GPT-5.4 zero-shot, with similar gains on literary benchmarks.
Audio language models are benchmarked on five semantic and paralinguistic reasoning tasks to reveal limitations in handling spoken audio evidence, accent variation, and domain shifts.
PortBERT releases two RoBERTa models for Portuguese that match or beat prior monolingual and multilingual models on translated GLUE/SuperGLUE tasks while reporting training and inference times.
citing papers explorer
-
Editing Models with Task Arithmetic
Task vectors from weight differences allow arithmetic operations to edit pre-trained models, improving multiple tasks simultaneously and enabling analogical inference on unseen tasks.
-
Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise
Fixed-clock optimizer memory turns equal-multiset data shuffle order into an O(η) source of fine-tuning noise, larger than the O(η²) effect in memoryless cases, with a fit-free sizing method derived.
-
Repository-Level Solidity Code Generation with Large Language Models: From Prompting to Fine-Tuning
Introduces SolidityBench benchmark and SolidityScore metric for repository-level Solidity code generation, finding supervised fine-tuning outperforms prompting, CoT, ICL, and RAG methods on evaluated LLMs.
-
On the Geometry of On-Policy Distillation
OPD updates occupy a relaxed off-principal regime and rapidly lock into a low-dimensional subspace that is functionally sufficient for its performance, distinct from SFT and RLVR trajectories.
-
BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learning
BadBone backdoors backbone models with bi-level optimization to make prompt learning on downstream tasks vulnerable while preserving model utility.
-
If It's Good Enough for You, It's Good Enough for Me: Transferability of Audio Sufficiencies across Models
Transferability analysis finds that minimal sufficient signals transfer across audio models at rates varying by task, around 26% for music genre classification, with some deepfake models showing distinct behaviors not visible in accuracy metrics.
-
Learning to summarize from human feedback
Reinforcement learning on a reward model trained from human summary comparisons produces summaries humans prefer over supervised fine-tuning or human references on TL;DR and transfers to CNN/DM.
-
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
FID variance from training seeds is 3.2 times larger than from sampling seeds on hundreds of SiT models, with 1-2% coefficient of variation that barely shrinks with more compute, leading to a multi-seed evaluation protocol.
-
MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection
MiqraBERT, a finetuned Sentence-BERT model, achieves 2.7-fold better distributional separation of parallel versus non-parallel Biblical Hebrew verses and reduces ambiguous overlap from 24% to 6%, with strong performance on narrative but weak on poetic parallels.
-
Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
Trained ResNets on CIFAR-10 retain measurable dependence on initialization scale under low-LR SGD (26.5 pp test accuracy spread) but not under Adam, indicating that practical inductive bias is shaped by the forgetting time scale of the optimizer and regularizers.
-
20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
Data curation alone raises VLM accuracy by more than 11 points on average across many benchmarks while cutting required training compute by up to 87 times.
-
Instructions Shape Production of Language, not Processing
Instructions trigger a production-centered mechanism in language models, with task-specific information stable in input tokens but varying strongly in output tokens and correlating with behavior.
-
Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution
MADE-IT outperforms baselines in continual model merging accuracy and robustness by using manifold-aware expert evolution with adaptive thresholds and implicit subspace-based routing while pruning redundant experts.
-
Robust Policy Optimization to Prevent Catastrophic Forgetting
FRPO applies a max-min robust optimization over KL-bounded policy neighborhoods during RLHF to reduce catastrophic forgetting of safety and accuracy under subsequent SFT or RL fine-tuning.
-
Should We Still Pretrain Encoders with Masked Language Modeling?
Controlled ablations of 38 models find MLM superior to CLM on representation benchmarks while CLM offers better data efficiency and stability; a biphasic CLM-then-MLM schedule is optimal under fixed compute and improves when initialized from pretrained CLM models.
-
LIMO: Less is More for Reasoning
LIMO achieves 63.3% on AIME24 and 95.6% on MATH500 via supervised fine-tuning on roughly 1% of the data used by prior models, supporting the claim that minimal strategic examples suffice when pre-training has already encoded domain knowledge.
-
GRAIN: Group Aggregation via Min-Norm Objective
GRAIN is a gradient aggregation method using min-norm objectives to ensure non-negative inner products with group gradients, yielding tighter uniform stability bounds than SGD under smoothness assumptions.
-
Sparsity Curse: Understanding RLVR Model Parameter Space from Model Merging
RLVR induces sparse off-principal updates forming near-orthogonal shortcuts that degrade merging, addressed via Sensitivity-aware Resolving Merging using Fisher sensitivity, sparsification, and rescaling.
-
PromptRad: Knowledge-Enhanced Multi-Label Prompt-Tuning for Low-Resource Radiology Report Labeling
PromptRad reformulates multi-label radiology report classification as masked language modeling and enriches verbalizers with UMLS synonyms, outperforming baselines with only 32 training examples.
-
LoRA vs. Full Fine-Tuning: A Theoretical Perspective
In linear regression, LoRA can achieve lower excess risk than full fine-tuning when the pretraining-downstream difference is low-rank, and small LoRA ranks can improve generalization by acting as regularization.
-
UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning
UAV-VL-R1 combines SFT and multi-stage GRPO reinforcement learning on a new 50,019-sample HRVQA-VL dataset to deliver substantially higher zero-shot accuracy on UAV visual reasoning tasks than both its 2B baseline and a 72B-scale model.
-
Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction
Fine-tuned models under 1B parameters reach micro-F1 of 0.83 on general-domain RE versus 0.69 for GPT-5.4 zero-shot, with similar gains on literary benchmarks.
-
Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accents
Audio language models are benchmarked on five semantic and paralinguistic reasoning tasks to reveal limitations in handling spoken audio evidence, accent variation, and domain shifts.
-
PortBERT: Navigating the Depths of Portuguese Language Models
PortBERT releases two RoBERTa models for Portuguese that match or beat prior monolingual and multilingual models on translated GLUE/SuperGLUE tasks while reporting training and inference times.
- Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis
- Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages