Pith. sign in

REVIEW 4 cited by

Less is More: Selective Layer Finetuning with SubTuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2302.06354 v3 pith:7IUGSMZE submitted 2023-02-13 cs.LG cs.AI

Gal Kaplun , Andrey Gurevich , Tal Swisa , Mazor David , Shai Shalev-Shwartz , Eran Malach This is my paper
classification cs.LGcs.AI
keywords finetuningsubtuningtasksmodeldemonstratedifferentfullmethod
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Finetuning a pretrained model has become a standard approach for training neural networks on novel tasks, resulting in fast convergence and improved performance. In this work, we study an alternative finetuning method, where instead of finetuning all the weights of the network, we only train a carefully chosen subset of layers, keeping the rest of the weights frozen at their initial (pretrained) values. We demonstrate that \emph{subset finetuning} (or SubTuning) often achieves accuracy comparable to full finetuning of the model, and even surpasses the performance of full finetuning when training data is scarce. Therefore, SubTuning allows deploying new tasks at minimal computational cost, while enjoying the benefits of finetuning the entire model. This yields a simple and effective method for multi-task learning, where different tasks do not interfere with one another, and yet share most of the resources at inference time. We demonstrate the efficiency of SubTuning across multiple tasks, using different network architectures and pretraining methods.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Backdooring Masked Diffusion Language Models

    cs.LG 2026-05 conditional novelty 8.0 of 10

    SHADOWMASK backdoors MDLMs by modifying the forward corruption process with a trigger-mask mixture, achieving near-100% attack success while preserving clean utility on DiT-based and LLaDA models.

  2. Backdooring Masked Diffusion Language Models

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    SHADOWMASK backdoors MDLMs by replacing the all-mask terminal distribution with a trigger-mask mixture prior, achieving near-100% attack success on DiT and LLaDA-8B models across multiple datasets while resisting fine...

  3. Open-Vocabulary Gaze Object Prediction: Benchmark and Method

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A COCO+GazeFollow-derived benchmark (86 categories) plus a Grounding DINO + gaze-selection pipeline with selective tuning improves open-vocabulary gaze object prediction over existing closed-vocabulary methods.

  4. Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models

    cs.LG 2026-06 unverdicted novelty 5.0 of 10

    Mixture-of-Control adaptively combines local and global control states in transformer fine-tuning by treating per-block states as experts in a sparse MoE setup to improve cross-block communication while keeping memory...

Pith tools