Pith. sign in

REVIEW 3 major objections 6 minor 52 references

RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single learned latent-space offset added to encoder outputs can improve virtually any encoder-decoder PLM, the paper claims.

desk verdict RepCali is a simple additive-shift tuning module with a broad empirical sweep, but the mechanism as written reduces to a constant bias and the central calibration claim is not yet supported. read the letter →

arxiv 2505.08463 v2 pith:INJYUQJT submitted 2025-05-13 cs.CL cs.AI

Fujun Zhang , Xiaoying Fan , XiangDong Su , Guanglai Gao This is my paper · ORCID
classification cs.CLcs.AI
keywords representationcalibrationlatentspaceparameter-efficientfine-tuningencoder-decoderlanguagemodelsplug-and-playadaptationforLLMssequence-to-sequencelearningdownstreamtask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RepCali starts from a diagnosis: after fine-tuning, encoder-decoder PLMs still leave a gap between the distribution of the encoder's hidden states and the distribution the decoder was optimized to consume. The paper proposes closing that gap in latent space with a tiny plug-in block, made of an all-ones matrix passed through a learnable embedding and layer normalization, whose output is scaled and added to every encoder state before the decoder. The paper reports that this block works across 25 PLM-based models, 8 tasks, English and Chinese data, with only 0–0.8% additional parameters, and beats prompt tuning, prefix tuning, adapters, LoRA, and BitFit on a four-task head-to-head comparison. The payoff, if correct, is a nearly free universal upgrade for any encoder-decoder PLM that requires no change to the base model's loss or architecture.

What carries the argument

The load-bearing object is the calibration block, which sits between the encoder and decoder. It has three pieces: ShapeSeed, an input-shaped matrix initialized to all ones; LearnEmb, a learnable embedding layer that maps ShapeSeed to the calibration vector $d_i$; and layer normalization. The block's output is added to the encoder outputs through $p_i = h_i + \lambda\, d_i$, where $\lambda$ controls the strength of the calibration, and the calibrated sequence $p_i$ is what the decoder consumes. This is the entire new machinery: it is position-wise, input-independent, introduces only one extra embedding matrix, and is trained by whatever downstream loss the original model uses.

What would settle it

Probe the input-independence assumption directly: on a trained RepCali model, split the test set by topic or input type, then for each group solve for the group-specific offset $d_g$ that minimizes decoder loss with all other weights frozen. If the optimal $d_g$ vectors differ materially across groups, one shared offset cannot be closing the true encoder-decoder gap, so the calibration mechanism is not what the results are showing.

Watch

Extended reading notes

Core claim

The paper's central claim is that a large share of the remaining gap in encoder-decoder PLMs can be removed by a single learned correction in latent space, inserted between the encoder and decoder. The correction is parameterized by a calibration block: ShapeSeed, an input-shaped matrix initialized to all ones, is mapped by a learnable embedding and layer normalization to a calibration vector $d_i$, and the decoder receives $p_i = h_i + \lambda\, d_i$ instead of the raw encoder output $h_i$. Because the correction is input-independent, once trained the block is effectively a task-specific offset added to every encoder state. The paper reports that 25 PLM-based models improve on 8 downstream tasks, including generation, understanding, summarization, and dialogue tasks, with parameter growth of at most 0.8%, and that the block outperforms prompt tuning, prefix tuning, adapters, LoRA, and BitFit on a four-task T5-base comparison.

Load-bearing premise

The load-bearing premise is that one fixed calibration vector can close the encoder-decoder gap for every input, position, and domain in a task; if the discrepancy is content-dependent, the block is not actually calibrating, and the reported gains would need to be explained by added parameters, optimization dynamics, or regularization.

Editorial extensions

If this is right

  • Any encoder-decoder PLM, from BART-base to a 4.5-billion-parameter T5-3B, can carry the calibration block with at most 0.8% added parameters and no change to the task loss.
  • On the four-task T5-base comparison, RepCali outperforms prompt tuning, prefix tuning, adapters, LoRA, and BitFit, with a 77.55 average score.
  • Inserting the block improves every one of the 25 baseline models across the eight English and Chinese tasks reported, including state-of-the-art systems such as MoKGE, JointGT, GAP, BRIO, and RE-BART.
  • Because the correction happens at the encoder-decoder interface, RepCali is agnostic to the inner structure of the PLM and can be layered on top of existing task-specific models such as MinTL, KB_BART, and keyword-control systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the trained calibration is input-independent, RepCali is equivalent to adding one global bias in the encoder's hidden space; comparing it against per-layer or per-head bias vectors at matched parameter counts would show whether a single surface offset is the right granularity.
  • A content-dependent variant, where $d_i$ is conditioned on a pooled encoder state, would be the natural next test: the paper's diagnosis predicts larger gains exactly where the optimal correction varies across topics.
  • Another testable extension is cross-task transfer: freeze a calibration block trained on one task and reuse it on another with the same encoder-decoder; positive transfer would support the claim that the gap being corrected is a general property of the model rather than a task-specific artifact.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes RepCali, a parameter-efficient fine-tuning method for encoder-decoder pre-trained language models. A calibration block comprising a learnable embedding applied to an all-ones ShapeSeed produces an offset d_i that is added with strength lambda to the encoder hidden states before they are passed to the decoder (Eqs. (1)-(4)). The authors claim this minimizes the discrepancy between the encoder's representation and the decoder's optimal input, and they report improvements over baselines across 25 PLM-based models and 8 tasks, including English and Chinese data, at a parameter overhead of 0-0.8%. The core empirical claim is that adding this block yields consistent downstream gains; the core mechanistic claim is that those gains arise from representation calibration.

Significance. If the empirical results are correct, RepCali would be an attractive plug-and-play component: it is simple, adds very few parameters, and the authors provide a broad sweep over BART, T5, PEGASUS, and derived models on multiple generation and understanding tasks, including a Chinese dialogue summarization dataset. The t-SNE visual analysis is a useful qualitative addition. However, the mechanistic interpretation is not currently established: as defined, the calibration offset is input-independent, and the paper does not provide the controlled experiments needed to distinguish calibration from a constant bias or from the effects of added parameters and changed optimization dynamics. The empirical claims are further weakened by missing variance information and by a training-protocol contradiction. The contribution is potentially publishable, but the load-bearing evidence must be strengthened.

major comments (3)
  1. [§3, Eqs. (2)-(3) and §4.2] Eq. (2) defines d_i = LearnEmb(ShapeSeed) with ShapeSeed an all-ones matrix, so d_i is the same vector for every token and every input. Eq. (3) then adds this fixed vector to every encoder hidden state. Under the §4.2 protocol with a frozen decoder, shifting every encoder output by a constant vector changes each cross-attention key by the same vector, so the softmax attention weights are unchanged and the calibration block reduces to a learned constant bias on decoder hidden states. Because the encoder-decoder discrepancy described in Section 1 is presumably content-dependent, this input-independent offset cannot by itself perform the claimed calibration. The paper needs a control experiment with an equal-parameter constant bias or learned decoder bias to show that the reported gains are not due merely to added parameters or changed optimization. Without that control, the mechanistic claim in Section 5.2 that minimizing encoder-decoder discrepancies drives the improvements is unsupported.
  2. [§4.2 vs Appendix A] Section 4.2 states that 'we froze the entire PLM decoder in NLU tasks' to reduce the fine-tuning parameter size and to validate RepCali's calibration, but Appendix A states that the baseline models were full-model fine-tuned and that 'we also full-model fine-tuned after integrating our representation calibration block into the baseline models.' These protocols are incompatible, and the manuscript does not state which protocol produced Table 1 and Tables 3-11. If the full-model protocol was used, the input-independence argument is moot because the entire model, including the decoder, is updated; if the frozen-decoder protocol was used, the parameter counts and training settings for those runs need to be described. This contradiction must be resolved before the results can be interpreted.
  3. [§4.1-4.2 and Tables 1, 3-11] The claim that RepCali 'significantly improves' performance is not supported by the evidence as reported. The text says results are averaged over three seeds, but no standard deviations, confidence intervals, or significance tests are provided. Many reported improvements are very small (e.g., Table 6 GAP on BLEU-4 is 0.00; Table 7 BRIO-Mul ROUGE-1 is +0.11) and could be within seed noise. In Table 1, baseline numbers are taken from Ding et al. (2022) rather than re-run under the authors' own training protocol (only BitFit is marked as reproduced), so the comparison to prompt, prefix, adapter, and LoRA is not matched. Please re-run all baselines under identical conditions or clearly restrict the comparison to comparable setups, and report seed-level variance or significance tests.
minor comments (6)
  1. [§4.1] Section 4.1 says 'We compare the proposed method with three fine-tuning methods,' but Table 1 lists five baselines (Prompt tuning, Prefix-tuning, Adapter, LoRA, and BitFit); please correct the count or the table.
  2. [§3, Eq. (2)] The statement that ShapeSeed has size batchsize x n with 'n equals the length of token embedding' conflicts with the indexing {d_i}_{i=1}^n, which suggests sequence positions; please clarify the dimensions of ShapeSeed and the operation of LearnEmb.
  3. [Table 2] The parameter count for RepCali appears to omit the LayerNorm scale and bias parameters, which contribute 2 x d_h parameters; please update the formula or clarify what is counted.
  4. [Abstract and Section 5] The abstract and Section 5 say the experiments include both English and Chinese datasets, but the only Chinese dataset in the manuscript appears in Appendix C (CSDS); please state this clearly in the main text or move the CSDS results into the main experimental section.
  5. [Table 10] Several entries in Table 10 show no parameter increase after adding RepCali (e.g., MinTL(T5-small) 102M + RepCali 102M), while Section 5.3 reports 0-0.8% additional parameters; please report exact values or explain the rounding.
  6. [Figure 4] The claim that RepCali produces a 'smoother' and 'more compact' latent space is based on visual inspection only; please add quantitative latent-space metrics or at least state the t-SNE settings and the number of points plotted.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RepCali is an empirical fine-tuning method whose calibrated output is not defined in terms of the results it reports; the input-independent offset is a validity concern, not a circularity burden.

full rationale

The paper's derivation chain is empirical rather than deductive. The calibration block is defined by Eqs. (1)-(4): encoder output h_i is augmented by a learned offset d_i = LearnEmb(ShapeSeed) with ShapeSeed an all-ones matrix, yielding p_i = h_i + λ·d_i, and this p_i is fed to the decoder. The reported downstream metrics are measured on external benchmarks (MultiWOZ, ART, CamRest, WebNLG, XSum, PersonaChat, ROCStories, CSDS) and compared with external baselines. Nothing in the method defines the evaluation metric in terms of the calibration parameters, and no fitted parameter is relabeled as a prediction. The paper invokes no uniqueness theorem, and the authors' self-citations are not load-bearing because the central claim is supported by the experiments in this paper, not by a prior-work citation. The skeptical concern—that an all-ones ShapeSeed makes the offset a single global vector, so the block cannot implement content-dependent calibration—is a substantive limitation of the mechanism, and the discrepancy between Section 4.2's statement that the decoder was frozen in NLU tasks and Appendix A's statement that the models were full-model fine-tuned is an internal consistency problem. Both are validity or reproducibility risks, not circularity: the claim that adding RepCali improves external metrics is testable and falsifiable, and the reported gains are not forced by construction. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claim rests on a single architectural idea: a learned offset generated from an input-independent seed. No free parameters are fitted to test labels beyond training the block itself; the main unstated assumptions are that the gap is global and that the offset does not destabilize the decoder.

free parameters (2)
  • lambda (calibration strength)
    Introduced in Eq. (3) to weight the learned offset; no value or sensitivity analysis is provided anywhere in the paper.
  • ShapeSeed all-ones initialization = all-ones matrix
    The fixed input used to generate d_i; the choice is arbitrary and not ablated, yet it determines the entire calibration signal.
assumptions (3)
  • ad hoc to paper A single input-independent additive vector can close the encoder-decoder distribution gap.
    Eq. (3) sets p_i = h_i + lambda * d_i with d_i from an all-ones ShapeSeed, so the calibration never looks at the input X.
  • domain assumption The encoder-decoder latent mismatch is the primary bottleneck for fine-tuning performance.
    Motivates the method in the introduction via t-SNE visualization; no controlled experiment isolates this mechanism from other fine-tuning effects.
  • domain assumption Adding the offset to encoder outputs does not disrupt the pretrained decoder's cross-attention behavior in a harmful way.
    The method inserts an additive signal before cross-attention; no analysis of decoder sensitivity to the offset is provided.
invented entities (1)
  • ShapeSeed
    purpose: All-ones matrix used as the fixed input to the learnable embedding that produces the calibration offset d_i.
    A new construct with no falsifiable handle outside the paper; because it is input-independent, LearnEmb(ShapeSeed) is effectively a trained constant vector.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models." pith.science (2026). https://pith.science/paper/INJYUQJT

@misc{pith2026250508463,
  author       = {Pith},
  title        = {Pith review of: RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INJYUQJT}},
  note         = {Machine review of arXiv:2505.08463}
}
read the original abstract

Fine-tuning pre-trained language models (PLMs) has become a dominant paradigm in applying PLMs to downstream tasks. However, with limited fine-tuning, PLMs still struggle with the discrepancies between the representation obtained from the PLMs' encoder and the optimal input to the PLMs' decoder. This paper tackles this challenge by learning to calibrate the representation of PLMs in the latent space. In the proposed representation calibration method (RepCali), we integrate a specific calibration block to the latent space after the encoder and use the calibrated output as the decoder input. The merits of the proposed RepCali include its universality to all PLMs with encoder-decoder architectures, its plug-and-play nature, and ease of implementation. Extensive experiments on 25 PLM-based models across 8 tasks (including both English and Chinese datasets) demonstrate that the proposed RepCali offers desirable enhancements to PLMs (including LLMs) and significantly improves the performance of downstream tasks. Comparison experiments across 4 benchmark tasks indicate that RepCali is superior to the representative fine-tuning baselines.

Figures

Figures reproduced from arXiv: 2505.08463 by the authors.

Figure 1
Figure 1. Visualization of the distribution of the output of T5 encoder before and after fine-tuning via the T-SNE algorithm on the Abductive Commonsense Reasoning (αNLG) task. relevant to that task. Therefore, fine-tuning of the PLMs has become a dominant paradigm in apply￾ing PLMs to downstream tasks (Ruder, 2021). As shown in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The detailed architecture of various tuning methods. this paper proposes RepCali, a simple and effective representation calibration method that integrates a well-designed calibration block to the latent space after the PLMs’ encoder and uses the calibrated output as the input of the PLMs’ decoder for PLM fine-tuning. The calibration block only involves shape seed, learnable embedding and layer normal￾ization. It is … view at source ↗
Figure 3
Figure 3. Overview of our representation calibration method. task. With the development and application of LLMs and visual models, researchers have realized that prompt has a large impact on the model’s perfor￾mance. Prompt tuning (Lester et al., 2021) only prepends and updates task-specific trainable param￾eters in the original input embeddings. Chen et al. (2023) proposed a visual prompt framework based on iterative label m… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: We chose three different PLMs, BART, T5, and PEGASUS for visualization and analysis of the latent space. The blue points are the hidden representations obtained after fine-tuning using our calibration method RepCali, and the yellow points are the hidden representations…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 18 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Vinsen Marselino Andreas, Genta Indra Winata, and Ayu Purwarianti. 2022. http://arxiv.org/abs/2201.08687 A comparative study on language models for task-oriented dialogue systems . CoRR, abs/2201.08687

    work page Pith review arXiv 2022

  4. [4]

    S \" o ren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives. 2007. https://doi.org/10.1007/978-3-540-76298-0\_52 Dbpedia: A nucleus for a web of open data . In The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11-15, 2007...

    doi:10.1007/978-3-540-76298-0 2007

  5. [5]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909/ METEOR: an automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, Ann Arbor, Michigan, USA, June 29, 2005, pages 65--72...

    work page 2005

  6. [6]

    Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen - tau Yih, and Yejin Choi. 2020. https://openreview.net/forum?id=Byg1v1HKDB Abductive commonsense reasoning . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net

    2020

  7. [7]

    Pawel Budzianowski, Tsung - Hsien Wen, Bo - Hsiang Tseng, I \ n igo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. https://aclanthology.org/D18-1547/ Multiwoz - A large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, B...

    work page 2018

  8. [8]

    Aochuan Chen, Yuguang Yao, Pin - Yu Chen, Yihua Zhang, and Sijia Liu. 2023. https://doi.org/10.1109/CVPR52729.2023.01834 Understanding and improving visual prompting: A label-mapping perspective . In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 19133--19143. IEEE

    arXiv 2023

Show all 52 references
  1. [9]

    Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, and Mingsheng Long. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/d10d6b28d74c4f0fcab588feeb6fe7d6-Abstract-Conference.html Debiased self-training for semi-supervised learning . In NeurIPS

    2022

  2. [10]

    Jaemin Cho, Min Joon Seo, and Hannaneh Hajishirzi. 2019. https://doi.org/10.18653/V1/D19-1308 Mixture content selection for diverse sequence generation . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint C...

    2019 doi

  3. [11]

    Somnath Basu Roy Chowdhury, Faeze Brahman, and Snigdha Chaturvedi. 2021. Is everything in order? a simple way to order sentences. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10769--10779

    2021

  4. [12]

    Anthony Colas, Mehrdad Alvandipour, and Daisy Zhe Wang. 2022. https://aclanthology.org/2022.coling-1.506 GAP: A graph-aware language model framework for knowledge graph-to-text generation . In Proceedings of the 29th International Conference on Computational Linguistics, COLIN...

    2022

  5. [13]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associat...

    2019 doi

  6. [14]

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi - Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang, Zhiyuan Liu, Hai - Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. 2022. https://doi.org/1...

    2022 doi arXiv

  7. [15]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. http://proceedings.mlr.press/v97/houlsby19a.html Parameter-efficient transfer learning for NLP . In Proceedings of the 36th Int...

    2019

  8. [16]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  9. [17]

    Haozhe Ji and Minlie Huang. 2021. https://doi.org/10.18653/V1/2021.EMNLP-MAIN.347 Discodvt: Generating long text with discourse-aware discrete variational transformer . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtu...

    2021 doi

  10. [18]

    Glass, and Tianxing He

    Jiabao Ji, Yoon Kim, James R. Glass, and Tianxing He. 2022. https://doi.org/10.18653/V1/2022.FINDINGS-ACL.260 Controlling the focus of pretrained language generation models . In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 20...

    2022 doi

  11. [19]

    Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, and Minlie Huang. 2021. https://doi.org/10.18653/V1/2021.FINDINGS-ACL.223 Jointgt: Graph-text joint representation learning for text generation from knowledge graphs . In Findings of the Association for...

    2021 doi

  12. [20]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

    2021 arXiv

  13. [21]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/V1/2020.ACL-MAIN.703 BART: denoising sequence-to-sequence pre-training for natural language generation, translation...

    2020 doi

  14. [22]

    Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, and Jianfeng Gao. 2020. https://doi.org/10.18653/V1/2020.EMNLP-MAIN.378 Optimus: Organizing sentences via pre-trained modeling of a latent space . In Proceedings of the 2020 Conference on Empirical Methods i...

    2020 doi

  15. [23]

    Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/V1/2021.ACL-LONG.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    2021 doi

  16. [24]

    Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/00bb4e415ef117f2dee2fc3b778d806d-Abstract-Conference.html Scaling & shifting your features: A new baseline for efficient model tuning . In Advances in Neural Infor...

    2022

  17. [25]

    Xinnian Liang, Shuangzhi Wu, Chenhao Cui, Jiaqi Bai, Chao Bian, and Zhoujun Li. 2023. Enhancing dialogue summarization with topic-aware global-and local-level centrality. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguis...

    2023

  18. [26]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  19. [27]

    Haitao Lin, Liqun Ma, Junnan Zhu, Lu Xiang, Yu Zhou, Jiajun Zhang, and Chengqing Zong. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.365 CSDS : A fine-grained C hinese dataset for customer service dialogue summarization . In Proceedings of the 2021 Conference on Empirical ...

    2021 doi

  20. [28]

    Haitao Lin, Junnan Zhu, Lu Xiang, Yu Zhou, Jiajun Zhang, and Chengqing Zong. 2022. https://doi.org/10.18653/v1/2022.acl-long.182 Other roles matter! enhancing role-oriented dialogue summarization via role interactions . In Proceedings of the 60th Annual Meeting of the Associat...

    2022 doi

  21. [29]

    Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, and Pascale Fung. 2020. https://doi.org/10.18653/V1/2020.EMNLP-MAIN.273 Mintl: Minimalist transfer learning for task-oriented dialogue systems . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pr...

    2020 doi

  22. [30]

    Qian Liu, Dejian Yang, Jiahui Zhang, Jiaqi Guo, Bin Zhou, and Jian-Guang Lou. 2021. https://doi.org/10.18653/v1/2021.findings-acl.100 Awakening latent grounding from pretrained language models for semantic parsing . In Findings of the Association for Computational Linguistics:...

    2021 doi

  23. [31]

    Radev, and Graham Neubig

    Yixin Liu, Pengfei Liu, Dragomir R. Radev, and Graham Neubig. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.207 BRIO: bringing order to abstractive summarization . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

    2022 doi

  24. [32]

    Gen Luo, Minglang Huang, Yiyi Zhou, Xiaoshuai Sun, Guannan Jiang, Zhiyu Wang, and Rongrong Ji. 2023. https://doi.org/10.48550/ARXIV.2302.08106 Towards efficient visual adaption via structural re-parameterization . CoRR, abs/2302.08106

    2023 doi arXiv

  25. [33]

    Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, and James Henderson. 2021. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. arXiv preprint arXiv:2106.04489

    2021 arXiv

  26. [34]

    Shikib Mehri, Tejas Srinivasan, and Maxine Esk \' e nazi. 2019. https://doi.org/10.18653/V1/W19-5921 Structured fusion networks for dialog . In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, SIGdial 2019, Stockholm, Sweden, September 11-13, 2019, pag...

    2019 doi

  27. [35]

    Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. https://doi.org/10.1145/3485447.3512034 Topic discovery via latent space clustering of pretrained language model representations . In WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25...

    2022

  28. [36]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. https://doi.org/10.18653/v1/D18-1206 Don ' t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization . In Proceedings of the 2018 Conference on Empirical Methods in Natu...

    2018 doi

  29. [37]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, ...

    2002

  30. [38]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. Learn. Res., ...

    2020

  31. [39]

    Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y - Lan Boureau, and Jason Weston. 2021. https://doi.org/10.18653/V1/2021.EACL-MAIN.24 Recipes for building an open-domain chatbot . In Proceedings of the 16th ...

    2021 doi

  32. [40]

    Andreas R \"u ckl \'e , Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2020. Adapterdrop: On the efficiency of adapters in transformers. arXiv preprint arXiv:2010.11918

    2020 arXiv

  33. [41]

    Sebastian Ruder. 2021. Recent Advances in Language Model Fine-tuning . http://ruder.io/recent-advances-lm-fine-tuning

    2021

  34. [42]

    Tianxiao Shen, Myle Ott, Michael Auli, and Marc'Aurelio Ranzato. 2019. http://proceedings.mlr.press/v97/shen19c.html Mixture models for diverse machine translation: Tricks of the trade . In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 J...

    2019

  35. [43]

    Anastasia Shimorina and Claire Gardent. 2018. https://doi.org/10.18653/V1/W18-6543 Handling rare items in data-to-text generation . In Proceedings of the 11th International Conference on Natural Language Generation, Tilburg University, The Netherlands, November 5-8, 2018, page...

    2018 doi

  36. [44]

    Nishant Subramani, Nivedita Suresh, and Matthew Peters. 2022. https://doi.org/10.18653/v1/2022.findings-acl.48 Extracting latent steering vectors from pretrained language models . In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland. Associat...

    2022 doi

  37. [45]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11)

    2008

  38. [46]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461

    2018 arXiv

  39. [47]

    Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2016. A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562

    2016 arXiv

  40. [48]

    Wenhao Yu, Chenguang Zhu, Lianhui Qin, Zhihan Zhang, Tong Zhao, and Meng Jiang. 2022. https://doi.org/10.18653/V1/2022.FINDINGS-ACL.149 Diversifying content generation for commonsense reasoning with mixture of knowledge graph experts . In Findings of the Association for Comput...

    2022 doi

  41. [49]

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022. https://doi.org/10.18653/V1/2022.ACL-SHORT.1 Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models . In Proceedings of the 60th Annual Meeting of the Association for Computational L...

    2022 doi

  42. [50]

    Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. http://proceedings.mlr.press/v119/zhang20ae.html PEGASUS: pre-training with extracted gap-sentences for abstractive summarization . In Proceedings of the 37th International Conference on Machine Learning, ICML 2...

    2020

  43. [51]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. https://doi.org/10.18653/V1/P18-1205 Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational ...

    2018 doi

  44. [52]

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. https://doi.org/10.1145/3209978.3210080 Texygen: A benchmarking platform for text generation models . In The 41st International ACM SIGIR Conference on Research & Development in Informatio...

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.