Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PT-MoE claims to beat prompt tuning and LoRA on QA and math while using 25% fewer parameters than LoRA.

desk verdict A genuinely new PEFT architecture, but the headline 'state-of-the-art' claim rests on single best-of-grid runs with no error bars—real concerns, though the design is sound and worth a serious referee. read the letter →

arxiv 2505.09519 v1 pith:LBZPNRDX submitted 2025-05-14 cs.CL

Zongqian Li , Yixuan Su , Nigel Collier This is my paper · ORCID
classification cs.CL
keywords prompttuningmixture-of-expertsmatrixdecompositionparameter-efficientfine-tuningquestionansweringmathematicalreasoninglow-rankrouting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PT-MoE, a prompt-tuning framework that decomposes each soft prompt into a shared low-rank matrix and prompt-specific matrices, then uses a learned router to select and combine these decomposed prompts for each input. The claim is that this combination of matrix decomposition and mixture-of-experts routing outperforms both plain prompt tuning and the LoRA adapter method on extractive question answering and mathematical problem solving, using only 80K trainable parameters—25% fewer than LoRA. The reported gains are a 1.49 F1-point improvement over prompt tuning and 2.13 points over LoRA on QA, and a 10.75-point accuracy improvement over prompt tuning on math. If correct, PT-MoE offers a single parameter-efficient method that stays competitive across two task families where the baselines each have separate strengths.

What carries the argument

The central object is the decomposed-prompt mixture: each soft prompt $P_i \in \mathbb{R}^{T \times H}$ is written as $A_i B$ with $A_i \in \mathbb{R}^{T \times R}$ and a shared $B \in \mathbb{R}^{R \times H}$; a router computes $w = \mathrm{softmax}(W \overline{x} + b)$ with top-$k$ hard selection and straight-through estimation, and the final prompt is the weighted combination. SVD of text-initialized embeddings provides the initialization. This carries the argument because it is what lets the method share parameters across experts (the shared $B$) while still routing inputs to specialized prompt components (the $A_i$), which the paper claims yields the complementary benefits.

What would settle it

Re-run the full comparison with a fixed training budget per method, at least five random seeds, and report mean and standard deviation; if PT-MoE's margins over prompt tuning (1.49 F1 on QA, 10.75 accuracy on math) fall within the noise, the central SOTA claim is not supported. Alternatively, re-run with the same training step budget for all methods (no per-method grid) and check if the advantage persists.

Watch

Extended reading notes

Core claim

PT-MoE is a prompt tuning variant in which each of $N$ soft prompts is factored as $P_i = A_i B$, with a prompt-specific matrix $A_i$ and a shared matrix $B$ used by all prompts. A router maps the mean token embedding of the input to a distribution over prompts, applies top-$k$ hard selection during training with straight-through estimation, and the final soft prompt is the weighted sum $\sum_i w_i A_i B$, prepended to the frozen language model's input. The authors claim this integration resolves two counter-intuitive observations—that adding a router to prompt tuning (SMoP) only helps in specific domains, and that low-rank decomposition (DPT) can improve performance in specific areas—by combining them. The evidence is a benchmark comparison across 12 QA datasets and 5 math datasets against prompt tuning, DPT, SMoP, ATTEMPT, LoRA, and HydraLoRA, all controlled to a similar parameter budget, with PT-MoE using 80K parameters.

Load-bearing premise

The reported gains are not an artifact of the evaluation setup: each method's training steps were chosen from a small grid and results are single runs without error bars or significance tests, so the 1.49-point F1 and 10.75-point accuracy advantages over prompt tuning could in principle reflect best-of-grid selection or run-to-run variance.

Editorial extensions

If this is right

  • PT-MoE would give practitioners a single parameter-efficient method that works across extractive QA and mathematical reasoning, where plain prompt tuning and LoRA each have domain-specific advantages.
  • The result would demonstrate that matrix decomposition and MoE routing are complementary rather than redundant: decomposition enables parameter sharing through $B$, while routing gives dynamic input-dependent selection.
  • Ablation results suggest concrete design rules: a prompt length around 40 tokens, two experts for in-domain tasks, four for out-of-domain, and selective probationary routing (top-1 selection with confidence-scaled outputs).
  • At 80K parameters, PT-MoE would deliver these gains with 25% fewer parameters than LoRA in the tested setup, altering the parameter-efficiency tradeoff for PEFT.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to apply the same decomposition-plus-routing recipe to LoRA itself—sharing the down-projection and routing among multiple up-projections—and check whether the complementary benefit transfers to adapter-based methods; the paper's HydraLoRA comparison is a different architecture and does not test this.
  • The much larger math gain over prompt tuning (10.75 points) compared to the QA gain (1.49 points) suggests PT-MoE may be addressing a specific failure mode of prompt tuning in multi-step reasoning, possibly through prompt specialization; a targeted analysis of where the errors disappear would test this.
  • A testable hypothesis implied by the design: the shared matrix $B$ captures task-agnostic prompt structure while the $A_i$ encode task-specific behavior. This could be checked by freezing $B$ and retraining only the $A_i$ on a new task, or by measuring the similarity of learned $A_i$ across tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes PT-MoE, a prompt-tuning framework that combines low-rank matrix decomposition of soft prompts with mixture-of-experts routing. Each of N soft prompts is written as P_i = A_i B, with expert-specific A_i and a shared B, and a learned router selects/weights experts per input. The method is evaluated on 12 MRQA QA datasets and 5 math problem-solving datasets with LLaMA-3.2-1B-Instruct (plus a 3B model for one comparison), and it is compared against full fine-tuning, LoRA, HydraLoRA, PT, DPT, SMoP, and ATTEMPT. The paper reports state-of-the-art F1/EM on QA and accuracy on math, with 80k trainable parameters, and includes ablations over prompt length, expert count, parameter count, routing mechanism, and model size. Code is released at a GitHub repository.

Significance. If the reported gains reproduce under careful evaluation, PT-MoE would be a useful contribution: it is a simple, modular combination of decomposition and MoE routing that obtains competitive results with very few trainable parameters, and the paper's broad coverage of 17 datasets plus ablations on architecture choices would be valuable for the PEFT community. The paper also ships a clear algorithm and a public code link, which makes the claims straightforwardly checkable. However, the central 'state-of-the-art' claim rests entirely on empirical comparisons that are currently reported as single runs, and the headline margins are small enough that they may be explained by training-step selection or seed variance. The contribution is interesting and plausible, but the evidence as presented is not yet sufficient to establish the advertised SOTA claim.

major comments (4)
  1. [§5.1, §5.2, Appendix Table 8, Figure 4] All result tables report a single run with no error bars, no confidence intervals, and no significance tests, and training steps are selected per method from a grid (e.g., {500,1000,1500} for PT-based methods and {200,600,1000} for LoRA-based methods in Table 8, with Figure 4 explicitly showing the 'highest performance after training parameter search'). The headline QA gain over PT is 1.49 F1 and the math gain over LoRA is only 0.44 accuracy; both are within the range that run-to-run variance can produce in instruction-tuned LLM fine-tuning. Because the reported margins and the per-method best-of-grid selection are jointly load-bearing for the central SOTA claim, the authors should provide multiple seeds, standard deviations, and paired significance tests, or a fixed-budget comparison with matched training steps, before the claim can be accepted.
  2. [Table 2, §5.1] The average F1 gain of PT-MoE over PT (58.26 vs 56.77) is heavily influenced by DROP, where PT-MoE improves by 7.63 F1 points over PT (48.02 vs 40.39) while most other datasets show much smaller gains. A single favorable run on DROP could drive a large part of the average difference, and the paper does not report any variance information for individual datasets. The authors should report per-dataset error bars or show that the conclusion is robust to excluding DROP, and they should identify whether DROP's discrete-reasoning requirement is the reason for the large gain or whether it is an evaluation artifact.
  3. [§3, Matrix Decomposition; §5.1, Table 2] The PT-MoE soft prompts are initialized via SVD of task-relevant text embeddings, which gives the initial prompt task-related information. The comparison against PT and SMoP does not control for this initialization advantage, so it is unclear how much of the reported improvement comes from the decomposition/MoE architecture rather than from the SVD initialization on task-related text. An ablation that matches initialization across PT, SMoP, DPT, and PT-MoE (e.g., random initialization or the same textual initialization for all methods) is needed to separate the architectural contribution from the initialization confound.
  4. [§5.2, Table 4, §5.4, Routing mechanisms] The mathematical-reasoning comparison has two additional load-bearing weaknesses. First, the 0.44-point average accuracy gain over LoRA (56.91 vs 56.47) is smaller than plausible seed-level noise, and no significance test is provided. Second, the routing ablation shows that the best configuration uses 'selective and probationary' routing, but the main results do not report whether the router confidence multiplier (probationary routing) is applied at inference in all experiments; if it is not, the main accuracy numbers and the ablation numbers measure different inference procedures. The paper should state explicitly whether the multiplier is used in the reported main results and in the 3B model comparison in Table 6.
minor comments (6)
  1. [§4.3] The heading 'Evaluation Metrices' contains a typo; it should be 'Evaluation Metrics'.
  2. [Table 1] The dataset name 'SV AMP' should be written as 'SVAMP', and the reference for 'MATH_PROBLEMS (Nebrelbug, 2024)' is a Hugging Face Hub entry rather than a citable archival source; a stable dataset identifier or a more formal citation would improve reproducibility.
  3. [Algorithm 1] The variable k is used both for the number of tokens per prompt and for the number of top-k router weights (Steps 7-8), which is confusing; rename one of them (e.g., k_tok and k_top).
  4. [Table 5] The polynomial terms in the case study are typeset without superscripts (e.g., y4 instead of y^4), which makes the example harder to read and the claimed error analysis less clear.
  5. [§2] The SMoP reference is listed as 'The 2023 Conference on Empirical Methods in Natural Language Processing' without the usual EMNLP venue name and page/article details; the citation should be completed.
  6. [Figure 3] The legend labels '(N)S: (Non-)Selective' and '(N)P: (Non-)Probationary' are hard to parse; the paper should spell out the four routing configurations explicitly (S/P, S/NP, NS/P, NS/NP) in the figure or caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PT-MoE's central claims are empirical benchmark comparisons against external datasets, not derivations from fitted inputs or self-citation chains.

full rationale

The paper's central claim is an empirical comparison of PT-MoE against PT, LoRA, SMoP, DPT, and other baselines on external benchmarks (MRQA, GSM8K, SVAMP, ASDIV, MAWPS, MATH_PROBLEMS). The proposed method is defined by its architecture (matrix decomposition with MoE routing), and its reported gains are measured rather than derived from the model equations or from fitted parameters renamed as predictions. The motivating observations about SMoP and DPT come from prior published work and from the paper's own baseline tables, and they are used as motivation, not as fitting targets for a theory. The few self-citations in the introduction (Li and Cole, 2025; Li et al., 2024, 2025a,b) are background citations about efficiency and prompt compression; none is load-bearing for the SOTA claim. No uniqueness theorem is invoked, and no architectural choice is justified solely by a self-citation. Concerns about best-of-grid single-run comparisons without error bars are legitimate experimental-validity concerns, but under the review rules those are correctness risks, not circularity. The derivation chain is self-contained: PT-MoE's formulation, training objective, and evaluation are all specified in the paper, and the claimed improvements are external empirical observations rather than consequences of the paper's own definitions. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No invented scientific entities are introduced; PT-MoE is a neural architecture with trainable matrices and a router. The central claim depends on the hyperparameters listed above and on the implicit statistical assumptions recorded as axioms. The evaluation is empirical, so there are no mathematical axioms beyond standard neural network training assumptions.

free parameters (5)
  • Soft prompt length T = 40 tokens
    Chosen as the optimum in the ablation (Figure 3 Left); used for all main results. It is a hand-selected hyperparameter.
  • Low-rank dimension R = 36
    Set manually in Table 9 and smaller than DPT's 39. It controls parameter count and is not derived from a principle.
  • Number of experts N = 2
    Set to 2 in main experiments. Ablation shows in-domain optimum at 2 but out-of-domain optimum at 4 (Figure 3 Center left), so the choice is not robust across domains.
  • Router noise scale sigma = 0.01
    Gaussian exploration noise level chosen by hand in Table 9, matching the SMoP configuration.
  • Training steps = Selected from {500,1000,1500} for PT-methods and {200,600,1000} for LoRA methods
    Per-method grid search; Figure 4 reports the highest performance after the search, so results are best-of-grid and not a fixed protocol.
assumptions (5)
  • domain assumption The frozen backbone LLaMA-3.2-1B-Instruct is a representative base model for PEFT conclusions.
    Main results are on one small model; the 3B experiment covers only math tasks and not QA, so cross-domain and cross-scale generality is assumed.
  • domain assumption Mean-pooled token embeddings carry enough signal for the router to pick the correct prompt.
    Algorithm 1 lines 3-4 compress each input to a single mean vector before routing, discarding word order; no alternative router input is tested.
  • ad hoc to paper Single-run evaluation without error bars is an acceptable estimate of true performance.
    Tables 2-6 report one number per method and dataset, and comparative claims such as +1.49 F1 assume these numbers are not noise.
  • ad hoc to paper Reporting the best training step from a grid for each method gives a fair comparison.
    Per-method grid selection (Table 8, Figure 4) can differentially favor methods; the paper does not analyze sensitivity to the selection rule.
  • ad hoc to paper SVD initialization from task-related text is beneficial and not a confound.
    PT-MoE initializes Ai and B via SVD of prompt-text embeddings (Section 3), but no random-initialization ablation is reported for PT-MoE.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning." pith.science (2026). https://pith.science/paper/LBZPNRDX

@misc{pith2026250509519,
  author       = {Pith},
  title        = {Pith review of: PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBZPNRDX}},
  note         = {Machine review of arXiv:2505.09519}
}
read the original abstract

Parameter-efficient fine-tuning (PEFT) methods have shown promise in adapting large language models, yet existing approaches exhibit counter-intuitive phenomena: integrating router into prompt tuning (PT) increases training efficiency yet does not improve performance universally; parameter reduction through matrix decomposition can improve performance in specific domains. Motivated by these observations and the modular nature of PT, we propose PT-MoE, a novel framework that integrates matrix decomposition with mixture-of-experts (MoE) routing for efficient PT. Results across 17 datasets demonstrate that PT-MoE achieves state-of-the-art performance in both question answering (QA) and mathematical problem solving tasks, improving F1 score by 1.49 points over PT and 2.13 points over LoRA in QA tasks, while enhancing mathematical accuracy by 10.75 points over PT and 0.44 points over LoRA, all while using 25% fewer parameters than LoRA. Our analysis reveals that while PT methods generally excel in QA tasks and LoRA-based methods in math datasets, the integration of matrix decomposition and MoE in PT-MoE yields complementary benefits: decomposition enables efficient parameter sharing across experts while MoE provides dynamic adaptation, collectively enabling PT-MoE to demonstrate cross-task consistency and generalization abilities. These findings, along with ablation studies on routing mechanisms and architectural components, provide insights for future PEFT methods.

Figures

Figures reproduced from arXiv: 2505.09519 by the authors.

Figure 1
Figure 1. Performance comparison of PEFT methods on 12 QA datasets in the MRQA benchmark (upper) and 5 math datasets (lower). ↑ indicates higher is better; ↓ indicates lower is better. updating only a small subset of parameters (Han et al., 2024). Prompt tuning (PT) stands out among PEFT approaches with its unique advan￾tages: minimizing trainable parameters through soft prompt optimization, enabling modular utiliza￾tion thro… view at source ↗
Figure 2
Figure 2. Framework of PT-MoE. Each soft prompt is decomposed into an input-specific matrix Ai and a shared matrix B, with a router adaptively selecting and combining prompt components based on input. The resulting soft prompt is prepended to the input for the frozen LLM. (Wang et al., 2021), and Cross-Task Prompt Tun￾ing (CTPT) that leverages multi-head attention for cross-task knowledge transfer with dimension re￾duction an… view at source ↗
Figure 3
Figure 3. Ablation studies on key components of PT-MoE, showing the influence of (Left) prompt length, (Center left) number of experts, (Center right) trainable parameters, and (Right) routing mechanisms ((N)S: (Non-)Selective, (N)P: (Non-)Probationary) on in-domain (ID) and out-of-domain (OOD) performance. of experts varies by domain type and highlight the importance of balancing expert focus with routing difficulty. Trainab… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Parameter and training efficiency compari￾son across different methods. The x-axis shows training steps for the highest performance after training parame￾ter search, while the y-axis shows the average accuracy on math datasets. Circle sizes indicate the number of train…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper

    cs.LG 2025-08 conditional novelty 5.0 of 10

    MEPT routes each input through one of several per-layer prompt experts plus a shared expert, reporting modest accuracy gains and a 79% cut in activated prompt parameters on SuperGLUE.

Reference graph

Works this paper leans on

40 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Akari Asai, Mohammadreza Salehi, Matthew Peters, and Hannaneh Hajishirzi. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.446 ATTEMPT : Parameter-efficient multi-task tuning via attentional mixtures of soft prompts . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6655--6672, Abu Dhabi, United Arab Emirat...

    doi:10.18653/v1/2022.emnlp-main.446 2022

  2. [2]

    Joon-Young Choi, Junho Kim, Jun-Hyung Park, Wing-Lam Mok, and SangKeun Lee. 2023. https://openreview.net/forum?id=5x5Vxclc1K SM op: Towards efficient and effective prompt tuning with sparse mixture-of-prompts . In The 2023 Conference on Empirical Methods in Natural Language Processing

    work page 2023

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . Preprint, arXiv:2110.14168

    arXiv 2021

  4. [4]

    Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. https://doi.org/10.18653/v1/N19-1246 DROP : A reading comprehension benchmark requiring discrete reasoning over paragraphs . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

    doi:10.18653/v1/n19-1246 2019

  5. [5]

    Ugur Guney, Volkan Cirik, and Kyunghyun Cho

    Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. https://arxiv.org/abs/1704.05179 Searchqa: A new q&a dataset augmented with context from a search engine . Preprint, arXiv:1704.05179

    arXiv 2017

  6. [6]

    Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019. MRQA 2019 shared task: Evaluating generalization in reading comprehension. In Proceedings of 2nd Machine Reading for Reading Comprehension (MRQA) Workshop at EMNLP

    2019

  7. [7]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, ..., and Zhiyu Ma. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    arXiv 2024

  8. [8]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. https://openreview.net/forum?id=lIsCS8b6zj Parameter-efficient fine-tuning for large models: A comprehensive survey . Transactions on Machine Learning Research

    2024

Show all 40 references
  1. [9]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  2. [10]

    Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. https://doi.org/10.1109/CVPR.2017.571 Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension . In 2017 IEEE Conference on ...

    2017 doi

  3. [11]

    Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. https://doi.org/10.18653/v1/N16-1136 MAWPS : A math word problem repository . In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational L...

    2016 doi

  4. [12]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  5. [13]

    Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. https://doi.org/10.18653/v1/D17-1082 RACE : Large-scale R e A ding comprehension dataset from examinations . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages...

    2017 doi

  6. [14]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.243 The power of scale for parameter-efficient prompt tuning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045--3059, Online a...

    2021 doi

  7. [15]

    Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/K17-1034 Zero-shot relation extraction via reading comprehension . In Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 333--342, Va...

    2017 doi

  8. [16]

    Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/v1/2021.acl-long.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    2021 doi

  9. [17]

    Zongqian Li and Jacqueline M Cole. 2025. Auto-generating question-answering datasets with domain-specific knowledge for language models in scientific tasks. Digital Discovery, 4(4):998--1005

    2025

  10. [18]

    Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. 2025 a . https://aclanthology.org/2025.naacl-long.368/ Prompt compression for large language models: A survey . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computati...

    2025

  11. [19]

    Zongqian Li, Ehsan Shareghi, and Nigel Collier. 2025 b . https://arxiv.org/abs/2503.03979 Reasongraph: Visualisation of reasoning paths . Preprint, arXiv:2503.03979

    2025 arXiv

  12. [20]

    Zongqian Li, Yixuan Su, and Nigel Collier. 2024. https://arxiv.org/abs/2408.03094 500xcompressor: Generalized prompt compression for large language models . Preprint, arXiv:2408.03094

    2024 arXiv

  13. [21]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. https://doi.org/10.18653/v1/2022.acl-short.8 P -tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks . In Proceedings of the 60th Annual Meeting of the Associat...

    2022 doi

  14. [22]

    Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023. https://arxiv.org/abs/2103.10385 Gpt understands, too . Preprint, arXiv:2103.10385

    2023 arXiv

  15. [23]

    Fang Ma, Chen Zhang, Lei Ren, Jingang Wang, Qifan Wang, Wei Wu, Xiaojun Quan, and Dawei Song. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.758 XP rompt: Exploring the extreme of prompt tuning . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language...

    2022 doi

  16. [24]

    Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. https://doi.org/10.18653/v1/2020.acl-main.92 A diverse corpus for evaluating and developing E nglish math word problem solvers . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pa...

    2020 doi

  17. [25]

    Nebrelbug. 2024. https://huggingface.co/datasets/nebrelbug/math-problems/tree/main Math problems . Hugging Face Hub

    2024

  18. [26]

    Ioannis Partalas, Eric Gaussier, Axel-Cyrille Ngonga Ngomo, et al. 2013. Results of the first bioasq workshop. In BioASQ@ CLEF, pages 1--8

    2013

  19. [27]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. https://doi.org/10.18653/v1/2021.naacl-main.168 Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling...

    2021 doi

  20. [28]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-...

    2016 doi

  21. [29]

    Anastasiia Razdaibiedina, Yuning Mao, Madian Khabsa, Mike Lewis, Rui Hou, Jimmy Ba, and Amjad Almahairi. 2023. https://doi.org/10.18653/v1/2023.findings-acl.421 Residual prompt tuning: improving prompt tuning with residual reparameterization . In Findings of the Association fo...

    2023 doi

  22. [30]

    Khapra, and Karthik Sankaranarayanan

    Amrita Saha, Rahul Aralikatte, Mitesh M. Khapra, and Karthik Sankaranarayanan. 2018. https://doi.org/10.18653/v1/P18-1156 D uo RC : Towards complex language understanding with paraphrased reading comprehension . In Proceedings of the 56th Annual Meeting of the Association for ...

    2018 doi

  23. [31]

    Zhengxiang Shi and Aldo Lipani. 2024. https://openreview.net/forum?id=KjegfPGRde De PT : Decomposed prompt tuning for parameter-efficient fine-tuning . In The Twelfth International Conference on Learning Representations

    2024

  24. [32]

    Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou ' , and Daniel Cer. 2022. https://doi.org/10.18653/v1/2022.acl-long.346 SP o T : Better frozen model adaptation through soft prompt transfer . In Proceedings of the 60th Annual Meeting of the Association for Computational Lingui...

    2022 doi

  25. [33]

    Chengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang, and Ming Gao. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.221 T rans P rompt: Towards an automatic transferable prompting framework for few-shot text classification . In Proceedings of the 2021 Conference on Empirical M...

    2021 doi

  26. [34]

    Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, and Yoon Kim. 2023. https://openreview.net/forum?id=Nk2pDtuhTq Multitask prompt tuning enables parameter-efficient transfer learning . In The Eleventh International Conference on Learning Representations

    2023

  27. [35]

    Yao Xiao, Lu Xu, Jiaxi Li, Wei Lu, and Xiaoli Li. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.890 Decomposed prompt tuning via low-rank reparameterization . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13335--13347, Singapore. Assoc...

    2023 doi

  28. [36]

    Yige Xu, Zhiwei Zeng, and Zhiqi Shen. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.780 Efficient cross-task prompt tuning for few-shot conversational emotion recognition . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11654--11666, Si...

    2023 doi

  29. [37]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  30. [38]

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. https://openreview.net/forum?id=N8N0hgNDRt Metamath: Bootstrap your own mathematical questions for large language models . In The Twelfth Inte...

    2024

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.