{"id":"2cdd8c11-3000-467a-8799-cbfc49dfc128","arxiv_id":"2507.06085","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that categorizes prompt tuning methods into direct and transfer learning branches, describes each method's design and limitations, and outlines challenges and future work.","lead":"This survey organizes prompt tuning methods for large language models into direct prompt learning and transfer learning, reviewing 15 representative approaches. It is a reference map for the subfield of parameter-efficient fine-tuning, useful to practitioners choosing among methods.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The taxonomy omits PPT (Gu et al., 2022), a transfer-learning prompt-tuning method cited in Sections 2 and 5; this undercuts the claimed comprehensive categorization more directly than the reader's citation-swap example, which appears factually incorrect.","rationale":"The reader's weakest assumption was that Section 3's citations are swapped, making descriptions unreliable. That specific example does not hold: ACL Anthology 2022.acl-short.8 is the P-Tuning v2 paper (titled 'P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks'), and arXiv:2103.10385 is the original P-Tuning v1 paper ('GPT understands, too'). The text describing deep prompts under P-Tuning v2 and a prompt encoder under P-Tuning matches these papers. Thus the reader's concrete citation evidence is incorrect. However, the survey has a real coverage problem. The reference list includes Gu et al. (2022), 'PPT: Pre-trained prompt tuning for few-shot learning,' and Section 2's transfer-learning discussion cites it as key prior work. PPT is precisely a transfer method: pre-train soft prompts on source data, then transfer to target few-shot tasks. Yet Section 4's transfer branch and the appendix omit PPT entirely. The abstract promises to 'classify existing approaches into two categories' and contribution (1) claims 'comprehensive categorization.' A method the survey itself uses to motivate transfer learning is absent from the taxonomy, so the classification cannot be taken as complete. This is the most load-bearing concern because it directly affects the survey's central value as a guide to the literature. The fix is straightforward: add PPT to Section 4 or explicitly delimit the survey's selection criteria, and audit the bibliography against section coverage. This concern supports the same conditional verdict as the reader, even though the reader's cited example is wrong.","tokens_in":12052,"tokens_out":11525,"duration_ms":121216,"concrete_test":"Run a coverage audit: take every prompt-tuning method named in the bibliography or introduced in Section 2 (notably PPT/Gu et al. 2022) and check whether it appears in Sections 3-4 or the appendix. For PPT specifically, verify whether it satisfies Section 2's own definition of transfer learning (source-task knowledge improving target-task performance). If it does, request the authors to place PPT in Section 4's general transfer branch or to justify its exclusion; if it cannot be placed without altering branch definitions, the taxonomy's completeness claim fails. Independently confirm the Liu et al. 2022b and 2023 references against ACL Anthology and arXiv to establish that the reader's citation-swap example is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the two-branch taxonomy. For that taxonomy to be useful as a survey, it must include the methods that the paper itself identifies as defining transfer learning in prompt tuning. The bibliography contains Gu et al. (2022), 'PPT: Pre-trained prompt tuning for few-shot learning,' and Section 2 explicitly invokes it for 'transfer learning in prompt tuning.' Yet Section 4's transfer branch lists only SPoT, ATTEMPT, TransPrompt, CTPT, and MPT; PPT is never assigned to any cell of the taxonomy, nor does the appendix list it. PPT is exactly a transfer method: it pre-trains soft prompts on a source corpus and transfers them to low-resource target tasks. Its absence is a direct counterexample to the abstract's 'categorizing existing approaches into two categories' and to the contribution's 'comprehensive categorization.' The reader's specific evidence that Section 3's citations are swapped does not survive inspection: 2022b is the ACL 2022 'P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks' paper (P-Tuning v2), and 2023 is 'GPT understands, too' (P-Tuning v1), matching the Section 3 descriptions. The real accuracy risk is coverage, not citation mislabeling.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript is a survey of prompt tuning, a parameter-efficient adaptation technique in which a frozen language model is steered by trainable continuous vectors prepended to the input. The paper's central contribution is a two-branch taxonomy: direct prompt learning and transfer learning. Direct learning is subdivided into general, encoder-based, decomposition, and mixture-of-experts methods; transfer learning is subdivided into general, encoder-based, and decomposition methods. For each method the survey gives a paragraph-level description of the design, insights, advantages, and limitations, and Figure 2 provides schematic comparisons. The paper closes with a discussion of current challenges (computational efficiency, training instability, initialization, model-scale dependence, explainability) and future directions.","tokens_in":12342,"tokens_out":9052,"duration_ms":97820,"significance":"The paper addresses a useful niche: existing surveys tend to cover either all of PEFT or all of prompting, whereas a focused survey of prompt tuning can serve as an entry point for practitioners. The high-level direct-versus-transfer partition is a plausible organizing principle, and the visualizations in Figure 2 help readers see the architectural differences among methods. The paper is clearly written and cites most of the canonical works in the area. That said, the value of a survey depends on its coverage and on the consistency of its taxonomy; the omission of PPT from the transfer-learning branch undercuts the paper's own 'comprehensive categorization' claim. I also checked the P-Tuning references: the citations in Section 3 are internally consistent, so the specific citation-swap concern in the reader's report does not survive inspection.","major_comments":[{"comment":"The transfer-learning branch is incomplete in a way that directly undercuts the paper's central claim. Section 2 defines transfer learning in prompt tuning by citing PPT (Gu et al., 2022), and Section 5.1 cites it again, but Section 4's transfer-learning branch lists only SPoT, ATTEMPT, TransPrompt, CTPT, and MPT, and Appendix A likewise omits PPT from all transfer-learning subsections. Since Section 1 promises a 'comprehensive categorization' and the abstract says existing approaches are classified into two categories, the omission of a canonical transfer method from the taxonomy is an internal inconsistency. Please classify PPT (for example, under general transfer) or explicitly narrow the survey's scope and soften the comprehensiveness claim.","section":"Section 4 and Appendix A"},{"comment":"The 'encoder-based methods' subcategory is applied without a stated criterion. P-Tuning genuinely uses a prompt encoder (LSTM/MLP), but RPT uses a residual MLP reparameterization and Prefix-Tuning uses an MLP-reparameterized prefix; grouping these three together as 'encoder-based' while treating DPT and DePT as 'decomposition-based' is not self-explanatory. Please define the boundary of 'encoder-based' (for example, any learned reparameterization network) or rename the subcategory, since the usefulness of the taxonomy depends on such definitions.","section":"Section 3"}],"minor_comments":[{"comment":"The introduction lists four bullets after saying there are 'three primary contributions'; please renumber or expand the count to four.","section":"Section 1"},{"comment":"The claim that Prompt Tuning is 'the first method that finetunes the input tokens' is historically fragile given Prefix-Tuning, which also tunes continuous prompts albeit at every layer; please soften or qualify this statement.","section":"Section 3"},{"comment":"PT-MoE is described as 'achieving state-of-the-art performance' and 'outperforming other prompt tuning methods' without any numbers or an external comparison being reported in the survey; since PT-MoE is one of the authors' own methods, please add the original paper's evaluation details or qualify the claim.","section":"Section 3 (PT-MoE paragraph)"},{"comment":"Please proofread: 'fiengrained' in the P-Tuning paragraph, 'promptsk' in the SMoP paragraph, and 'Decomposirion' in Appendix A are typos.","section":"Section 3 and Appendix A"},{"comment":"The initialization challenge cites Liu et al. (2022a), which is a few-shot PEFT paper; if the intended reference is specifically about prompt initialization, please choose a more direct citation or clarify the connection.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The main issue is fixable: adding PPT to the taxonomy, or softening the comprehensiveness claim, plus tightening the subcategory definitions. The authors' own PT-MoE is the only method for which the survey makes an unsupported 'state-of-the-art' claim, and I would ask them to substantiate it. The reader's citation-swap allegation did not hold up when checked against the bibliography. The paper is more suited to a workshop than a full journal given its descriptive depth, but the organizing framework is a reasonable contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First off: this is a serviceable survey. The direct-prompt-learning vs transfer-learning split is a reasonable organizing principle, and the per-method write-ups (Prompt Tuning, XPrompt, P-Tuning v2, P-Tuning, RPT, Prefix Tuning, DPT, DePT, SMoP, SPoT, ATTEMPT, TransPrompt, CTPT, MPT) are generally accurate, with useful insights and limitations for each. The visualizations comparing frameworks are a nice touch and will help newcomers. None of this is new research, but as a survey it's a legitimate contribution to the prompt-tuning/PEFT community.\n\nNow the soft spots. The reader's strongest concrete evidence—that P-Tuning v2 and P-Tuning citations are swapped—doesn't survive inspection. In the references, Liu et al. 2022b is the ACL 2022 'P-tuning: Prompt tuning can be comparable...' paper (which is the P-Tuning v2 paper), and Liu et al. 2023 is 'GPT understands, too' (P-Tuning v1). That matches the descriptions in Section 3, so the citation attributions are actually fine.\n\nThe real problem is coverage, not citation mislabeling. The paper cites Gu et al. (2022), PPT—pre-trained prompt tuning—in Section 2 as an example of transfer learning in prompt tuning, but PPT never appears in Section 4's taxonomy or in the appendix list of papers. That's a direct counterexample to the abstract's claim of categorizing existing approaches and to the 'comprehensive categorization' contribution. If PPT is a transfer method, leaving it out undercuts the central organizational claim. There may be other omissions too, but PPT is the one the paper itself invokes and then forgets.\n\nTwo smaller issues. The paper includes the authors' own PT-MoE method in the taxonomy and describes it as state-of-the-art without external benchmarking; it's fine to include your own work, but the SOTA claim should be toned down or clearly labeled as the authors' own evaluation. Also, the introduction promises 'three primary contributions' but lists four bullets, and there are a few typos ('fiengrained', 'promptsk'). Minor, but they add to the impression of haste.\n\nOverall: the taxonomy is plausible and the method summaries are reliable enough to be useful to a reader who wants a map of the field. The omission of PPT from the taxonomy is a real gap but it's fixable, not fatal. The clearer it is about selection criteria, the more the 'comprehensive' claim will hold.\n\nWho is this for: newcomers and practitioners looking for an entry point to prompt tuning; it's less useful for experts who already know the landscape. It deserves a serious referee rather than a desk reject, but I would want the coverage gap and the self-promotion issue addressed before publication. Send it to review with those conditions.","headline":"A useful but overclaimed survey taxonomy: the direct-versus-transfer split works well, but the paper forgets to include PPT in its own taxonomy, and the specific citation swap the reader flagged doesn't hold up.","tokens_in":12789,"tokens_out":3579,"would_cite":false,"duration_ms":34707,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey organizes prompt-tuning methods into two branches—direct prompt learning and transfer learning—and analyzes each method's design, insights, and trade-offs.","keywords":["prompt tuning","parameter-efficient fine-tuning","soft prompts","transfer learning","direct prompt learning","mixture-of-experts","low-rank decomposition","large language models"],"falsifier":"Compare each method entry against the abstract and content of the cited paper; a finding that a substantial share of entries describe a different method than the cited source would show the taxonomy is not a reliable map of the prompt-tuning literature.","tokens_in":11877,"feed_emoji":"🧩","tokens_out":9368,"duration_ms":92298,"temperature":0.7,"pith_summary":"Prompt tuning adapts a frozen language model by prepending trainable continuous vectors to its input. This survey's central claim is that every prompt-tuning method belongs to one of two families: direct prompt learning, which trains soft prompts in a single stage on the target task, and transfer learning, which draws on prompts or knowledge from source tasks. Within those families the survey distinguishes general optimization, encoder-based, decomposition, and mixture-of-experts designs, and it catalogs each method's innovations, advantages, and limitations. The value of the claim, if true, is that a fast-growing and scattered literature acquires a stable map that also highlights open problems in computational efficiency and training stability.","feed_headline":"Prompt-tuning methods fall into two branches, new survey argues","feed_subtitle":"A taxonomy separates direct target-task training from source-task transfer, with decomposition and MoE sub-branches.","key_machinery":"The machinery is the two-branch taxonomy built on the soft prompt, a trainable matrix $P \\in \\mathbb{R}^{m \\times d}$ prepended to the embedded input $[P; E(x)]$ while the language model stays frozen. The split criterion is training regime: a single training stage on the target task versus reliance on source-task prompts or knowledge. The taxonomy does the work of grouping the surveyed methods into coherent families so that design choices—encoders, low-rank decompositions, mixture-of-experts routing, and attention-based prompt mixing—can be compared across the literature.","core_discovery":"The core claim is the taxonomy itself: existing prompt-tuning approaches can be classified into direct prompt learning and transfer learning. Direct prompt learning covers methods that optimize prompt embeddings, prefix key-value pairs, or mixtures of short prompts directly on the target task, subcategorized into general, encoder-based, decomposition, and mixture-of-experts branches. Transfer learning covers methods that initialize, mix, or decompose prompts using knowledge from source tasks, subcategorized into general, encoder-based, and decomposition branches. The survey reads each method through this lens and reports the recurring insights that the taxonomy is meant to surface: soft prompts exhibit redundant low-rank structure, performance depends on model scale, and training is sensitive to initialization and learning rate.","pith_inferences":["Editorial inference: the same two-branch taxonomy could classify prompt-tuning variants outside this survey's list, such as visual or multimodal soft prompts, because the criteria are architectural and data-agnostic.","Editorial inference: the convergence of several methods on low-rank decomposition suggests a testable hypothesis that learned soft prompts are near low-rank; measuring the effective rank of trained prompts across tasks would settle this.","Editorial inference: the taxonomy does not define a precedence rule for hybrids that both transfer from source prompts and train directly on the target, so a reader must classify those by the dominant mechanism."],"forward_implications":["If the taxonomy holds, a new prompt-tuning paper can be located quickly by asking whether it trains directly on the target task or transfers from source tasks.","The survey's challenges section identifies computational efficiency and training stability as the main bottlenecks, giving future prompt-tuning research two concrete targets.","The low-rank decomposition and mixture-of-experts branches point to a cross-method trend: soft prompts have redundancy and can be compressed or routed without sacrificing task performance.","The transfer-learning branches show that prompt reuse can help low-resource and few-shot settings, which is where the survey says the gains matter most."],"supporting_citations":[{"why":"Establishes the soft-prompt formulation and the scale-dependent finding that prompt tuning can approach full fine-tuning on large models; anchors the direct general branch.","marker":"Lester et al., 2021"},{"why":"Introduces layer-wise trainable prefix key-value pairs; anchors the encoder-based direct branch as the survey classifies it.","marker":"Li & Liang, 2021"},{"why":"Introduces the P-Tuning prompt encoder and mixed continuous-discrete prompts; anchors the encoder-based direct branch.","marker":"Liu et al., 2023"},{"why":"Cited as P-Tuning v2, the deep-prompt method that places soft prompts at every layer; anchors the general direct branch.","marker":"Liu et al., 2022b"},{"why":"Presents XPrompt, which prunes soft prompt tokens under the lottery ticket hypothesis; anchors the pruning variant of direct learning.","marker":"Ma et al., 2022"},{"why":"Presents decomposed prompt tuning via low-rank reparameterization; anchors the decomposition branch.","marker":"Xiao et al., 2023"},{"why":"Presents sparse mixture-of-prompts with a gating mechanism; anchors the mixture-of-experts branch.","marker":"Choi et al., 2023"},{"why":"Introduces soft prompt transfer from source tasks with task-similarity retrieval; anchors the transfer general branch.","marker":"Vu et al., 2022"},{"why":"Introduces attention-based mixing of source prompts; anchors transfer via prompt mixing.","marker":"Asai et al., 2022"},{"why":"Introduces decomposition of prompts into shared and task-specific components; anchors the transfer decomposition branch.","marker":"Wang et al., 2023"}],"fun_headline_variants":["Survey splits prompt tuning into two main schools","Soft prompts hide low-rank redundancy, survey says","Direct or transfer? Survey maps prompt-tuning methods","Prompt-tuning taxonomy: direct learning vs transfer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's usefulness assumes that each method description and its citation accurately reflect the cited paper, so that a reader can use the taxonomy as a reliable guide to the literature.","fun_headline_variants_meta":{"raw":{"variants":["Survey splits prompt tuning into two main schools","Soft prompts hide low-rank redundancy, survey says","Direct or transfer? Survey maps prompt-tuning methods","Prompt-tuning taxonomy: direct learning vs transfer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000686,"raw_usage":{"total_tokens":3029,"prompt_tokens":779,"completion_tokens":2250,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":395,"completion_tokens_details":{"reasoning_tokens":2205}},"tokens_in":395,"tokens_out":2250,"duration_ms":17445,"temperature":1.0,"reasoning_tokens":2205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:10:35.517921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare each method entry against the abstract and content of the cited paper; a finding that a substantial share of entries describe a different method than the cited source would show the taxonomy is not a reliable map of the prompt-tuning literature.","supporting_citations":[],"review_version":1}