Pith. sign in

REVIEW 9 cited by

Parameter-Efficient Fine-Tuning for Foundation Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.13787 v1 pith:JZCBAAQP submitted 2025-01-23 cs.CL cs.AIcs.LG

Dan Zhang , Tao Feng , Lilong Xue , Yuandong Wang , Yuxiao Dong , Jie Tang This is my paper
classification cs.CLcs.AIcs.LG
keywords peftacrossdiversefine-tuningsurveyunderstandingapplicationscomprehensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This survey delves into the realm of Parameter-Efficient Fine-Tuning (PEFT) within the context of Foundation Models (FMs). PEFT, a cost-effective fine-tuning technique, minimizes parameters and computational complexity while striving for optimal downstream task performance. FMs, like ChatGPT, DALL-E, and LLaVA specialize in language understanding, generative tasks, and multimodal tasks, trained on diverse datasets spanning text, images, and videos. The diversity of FMs guides various adaptation strategies for PEFT. Therefore, this survey aims to provide a comprehensive overview of PEFT techniques applied to diverse FMs and address critical gaps in understanding the techniques, trends, and applications. We start by providing a detailed development of FMs and PEFT. Subsequently, we systematically review the key categories and core mechanisms of PEFT across diverse FMs to offer a comprehensive understanding of trends. We also explore the most recent applications across various FMs to demonstrate the versatility of PEFT, shedding light on the integration of systematic PEFT methods with a range of FMs. Furthermore, we identify potential research and development directions for improving PEFTs in the future. This survey provides a valuable resource for both newcomers and experts seeking to understand and use the power of PEFT across FMs. All reviewed papers are listed at \url{https://github.com/THUDM/Awesome-Parameter-Efficient-Fine-Tuning-for-Foundation-Models}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models

    cs.HC 2026-04 unverdicted novelty 7.0 of 10

    IntentVLM uses forward-inverse modeling in a two-stage video-language setup to reach up to 80% accuracy on open-vocabulary intention recognition benchmarks, beating baselines by 30% and matching human performance.

  2. CURE:Circuit-Aware Unlearning for LLM-based Recommendation

    cs.IR 2026-04 unverdicted novelty 7.0 of 10

    CURE disentangles LLM recommendation circuits into forget-specific, retain-specific, and task-shared modules with tailored update rules to achieve more effective unlearning than weighted baselines.

  3. GoodVibe: Security-by-Vibe for LLM-Based Code Generation

    cs.CR 2026-02 conditional novelty 6.0 of 10

    Retraining only the neurons most responsive to security judgments raises a code LLM's default safe-code rate to 87.5% on C++ (average) with about 1.9 million trainable parameters.

  4. Vision Transformer Finetuning Benefits from Non-Smooth Components

    cs.LG 2026-02 conditional novelty 6.0 of 10

    For vision transformers, components with higher input-output sensitivity (attention and feedforward layers) yield better and more stable fine-tuning accuracy than smoother LayerNorm components.

  5. CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentation

    cs.CV 2025-11 conditional novelty 6.0 of 10

    A Fisher-information-guided dynamic selection over a toolbox of LoRA, adapter, and frequency-adapter modules improves cross-domain remote sensing segmentation over static PEFT methods.

  6. The Hidden Power of Scaling Factor in LoRA Optimization

    cs.AI 2026-06 unverdicted novelty 5.0 of 10

    Alpha in LoRA outperforms learning-rate scaling, follows a square-root law with rank, and enables a minimalist LoRA-alpha method that improves performance across tasks.

  7. A Numerical PDEs Approach to Evolution Equations in Shape Analysis Based on Regularized Morphoelasticity

    math.NA 2026-04 unverdicted novelty 4.0 of 10

    Regularized morphoelasticity yields a high-order elliptic system for continuous shape evolution that is solved by mixed finite elements in FEniCSx within an LDDMM-style optimal-control growth model.

  8. HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST

    cs.CL 2025-09 unverdicted novelty 4.0 of 10

    HiCoLoRA uses hierarchical LoRA with spectral domain-slot clustering, adaptive fusion, and semantic SVD initialization to achieve SOTA zero-shot DST on MultiWOZ and SGD.

  9. C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning

    cs.LG 2025-08 conditional novelty 4.0 of 10

    Adding zeroth- and first-order flatness penalties to continual learning losses yields small consistent accuracy gains across seven methods, with the gated C-Flat++ variant at roughly 30% of the update cost.

Pith tools