Pith. sign in

REVIEW 12 cited by

PPT: Pre-trained Prompt Tuning for Few-shot Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2109.04332 v3 pith:TQR5JVJ7 submitted 2021-09-09 cs.CL

Yuxian Gu , Xu Han , Zhiyuan Liu , Minlie Huang This is my paper · ORCID
classification cs.CL
keywords promptstuningprompttasksdownstreamplmspre-trainedsoft
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Prompts for pre-trained language models (PLMs) have shown remarkable performance by bridging the gap between pre-training tasks and various downstream tasks. Among these methods, prompt tuning, which freezes PLMs and only tunes soft prompts, provides an efficient and effective solution for adapting large-scale PLMs to downstream tasks. However, prompt tuning is yet to be fully explored. In our pilot experiments, we find that prompt tuning performs comparably with conventional full-model fine-tuning when downstream data are sufficient, whereas it performs much worse under few-shot learning settings, which may hinder the application of prompt tuning in practice. We attribute this low performance to the manner of initializing soft prompts. Therefore, in this work, we propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. We name this Pre-trained Prompt Tuning framework "PPT". To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task. Extensive experiments show that tuning pre-trained prompts for downstream tasks can reach or even outperform full-model fine-tuning under both full-data and few-shot settings. Our approach is effective and efficient for using large-scale PLMs in practice.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    GRID combines output-space constrained decoding with gradient-guided prompt compression for task-agnostic, bounded-memory continual prompt tuning.

  2. Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering

    cs.CL 2025-06 conditional novelty 6.0 of 10

    FOCUS improves VQA accuracy by routing easy questions through fast zero-shot answering and hard questions through question-conditioned image segmentation before the final answer.

  3. An Integrated Framework for Contextual Personalized LLM-Based Food Recommendation

    cs.IR 2025-04 unverdicted novelty 6.0 of 10

    The thesis introduces F-RLP, a food-specific LLM-based recommendation framework, alongside a multimedia food logger and the World Food Atlas, as components of a holistic personalized food recommendation system.

  4. CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets

    cs.CV 2025-01 conditional novelty 6.0 of 10

    CM3T shows that multi-head vision adapters plus cross-attention adapters can adapt frozen supervised-pretrained video transformers with a fraction of the trainable parameters of full fine-tuning.

  5. Hymba: A Hybrid-head Architecture for Small Language Models

    cs.CL 2024-11 conditional novelty 6.0 of 10

    Hymba-1.5B, a hybrid-head model with parallel attention and Mamba heads plus learnable meta tokens, achieves state-of-the-art accuracy among sub-2B models and outperforms Llama-3.2-3B with an 11.67x smaller cache and ...

  6. Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Mettle distills frozen transformer layer features into compact meta-tokens via parallel cross-attention and linear projection, cutting training memory dramatically while retaining competitive accuracy on three audio-v...

  7. ReqBrain: Task-Specific Instruction Tuning of LLMs for AI-Assisted Requirements Generation

    cs.SE 2025-05 conditional novelty 5.0 of 10

    ReqBrain, a LoRA-fine-tuned Zephyr-7b-beta model, produces software requirements that human evaluators could not reliably tell apart from human-authored ones, with automatic metrics favoring it over untuned ChatGPT-4o.

  8. MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery

    cs.SE 2025-01 conditional novelty 5.0 of 10

    MPLinker reframes issue-commit link recovery as a masked-language-model cloze task with multi-template averaging and adversarial training, reporting an average F1 of 96.10% on six projects.

  9. Prompt Transfer for Dual-Aspect Cross Domain Cognitive Diagnosis

    cs.LG 2024-12 conditional novelty 5.0 of 10

    PromptCD uses learnable soft prompts shared between source and target domains to improve few-shot cross-domain cognitive diagnosis for both student-aspect and exercise-aspect scenarios.

  10. Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A survey proposes a macro-meso-micro value framework for agentic AI alignment and maps applications, methods, and benchmarks onto it.

  11. ChronoLLM: A Framework for Customizing Large Language Model for Digital Twins generalization based on PyChrono

    cs.SE 2025-01 conditional novelty 4.0 of 10

    Fine-tuning LLMs on PyChrono-specific data improves their success rate at generating runnable simulation code from about 40% to about 85%, compared to prompting general models.

  12. Instance-Aware Graph Prompt Learning

    cs.LG 2024-11 conditional novelty 4.0 of 10

    Instance-aware node prompts built from a compact PHM projector and vector quantization improve few-shot graph classification over static and attention-based prompt baselines.

Pith tools