Pith. sign in

REVIEW 3 major objections 3 minor 35 references

ComMer: a Framework for Compressing and Merging User Data for Personalization

T0 review · 3 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ComMer claims that compressing each user's documents into compact representations, merging them, and feeding the result to a frozen LLM delivers better quality on personalized skill-learning tasks than prompt-based or fine-tuned baselines…

desk verdict Abstract-only look at a coherent compression-and-merge personalization idea whose main claim is untestable until the budget and baselines are pinned down. read the letter →

arxiv 2501.03276 v1 pith:L2R4HGDU submitted 2025-01-05 cs.CL cs.AIcs.IRcs.LG

Yoel Zeldes , Amir Zait , Ilia Labzovsky , Danny Karmon , Efrat Farkash This is my paper · ORCID
classification cs.CLcs.AIcs.IRcs.LG
keywords personalizationlargelanguagemodelspromptcompressiondocumentmergingfrozenmodelskilllearningknowledge-intensivetasksinferencebudget
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces ComMer, a framework that personalizes a frozen large language model by compressing each user's documents into short representations, merging those representations across users, and feeding the merged text into the model. The central claim is that in settings where inference budget is constrained, this compress-and-merge procedure produces better results on personalized skill-learning tasks such as tweet paraphrasing and news headline generation than exposing raw documents through the prompt or fine-tuning the model. The same experiments show that the approach falls short on knowledge-intensive tasks because compression discards too much detailed information. If the claim holds, personalization can be made cheaper at inference time without per-user fine-tuning, at the cost of factual recall.

What carries the argument

The central object is the compress-then-merge representation: each user's documents are compressed into a short text that captures style and task-relevant cues, and those short texts are concatenated into one merged prompt passed to a frozen LLM. The compression step is what keeps the model frozen and inference cost low, while the merging step pools information from multiple users in a single forward pass. This replaces both the long raw-document prompt and any per-user weight update.

What would settle it

Take a single user whose target style appears only in one tweet in the training set, compress all of that user's documents together, and ask the frozen model to paraphrase a new tweet in that style; if the compressed representation loses the only style exemplar and quality collapses, the claim that compression preserves skill cues would be falsified.

Watch

Extended reading notes

Core claim

ComMer's central claim is that a compact, merged representation of user documents is sufficient for a frozen LLM to learn the style and task behavior of that user. When the inference budget is tight, this compressed representation outperforms both prompt-based personalization that spells out the full documents and fine-tuned models on the evaluated skill-learning tasks. The mechanism works by compressing each user's documents separately, then merging the compressed texts into a single prompt-like input. The trade-off is explicit: on knowledge-intensive question answering, the compressed representation loses too much detail and underperforms, revealing the boundary of the method.

Load-bearing premise

The compressed and merged representation of a user's documents keeps the stylistic and task-specific cues that personalization needs, even though it discards most of the original text.

Editorial extensions

If this is right

  • In constrained inference budgets, ComMer yields better quality than prompt-based or fine-tuned baselines on personalized skill-learning tasks, enabling lower per-request computation for style-oriented personalization.
  • The method's value is task-dependent: it works for style-transfer-like skills such as tweet paraphrasing and headline generation but degrades on knowledge-intensive tasks, so deployment should match the task type.
  • Because the LLM stays frozen, keeping personalization current only requires re-compressing and re-merging documents, avoiding expensive re-training when user data changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The compress-then-merge design suggests a general principle: for tasks that hinge on style, register, or formatting, a lossy summary can be more efficient than exhaustive context, whereas tasks that hinge on facts need a fidelity-preserving path; this could extend to code style, email tone, or other personalized formatting tasks.
  • The merging step implies that individual user representations could be cached and combined at serving time, acting as a lightweight personalization cache; the paper does not explore this operational reading.
  • A testable extension would decouple the compression ratio per user: reserving more tokens for users whose style is sparse or unusual could recover some skill-learning quality, an optimization hinted at but not pursued in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces ComMer, a framework that compresses users' documents into compact representations, merges the representations, and feeds them into a frozen LLM for personalization. The abstract reports evaluations on two task types: personalized skill learning (tweet paraphrasing and news headline generation from the LaMP benchmark) and knowledge-intensive question answering (PerLTQA). It claims that ComMer achieves superior quality in skill learning under constrained inference budgets, while acknowledging limitations in knowledge-intensive settings due to loss of detailed information. The reviewable text consists only of the abstract; no methodological or experimental details are available.

Significance. If the central claim is substantiated, ComMer would offer a meaningful efficiency/quality trade-off for personalizing frozen LLMs, reducing inference cost relative to prompt-based methods and avoiding fine-tuning overhead. The paper explicitly identifies a limitation in knowledge-intensive tasks, which is a useful honest boundary. However, the abstract alone cannot establish significance: it reports no effect sizes, error bars, baseline comparisons, significance tests, or precise budget definitions. The claimed advantage is conditional on an undefined 'constrained inference budget scenario,' and the representation modality is unspecified. The work is potentially valuable but currently not assessable from the provided text.

major comments (3)
  1. [Abstract] The central claim that 'in constrained inference budget scenarios ComMer achieves superior quality' is unfalsifiable without a precise definition of the inference budget (e.g., number of tokens, compute, or latency). The abstract does not state the budget threshold or provide budget curves, so the reported advantage could be an artifact of comparing against artificially truncated baselines rather than a property of the compression and merging method.
  2. [Abstract] The sentence 'compressing users' documents into compact representations, which are then merged and fed into a frozen LLM' does not specify whether the merged representation is textual (e.g., a generated summary) or a vector/embedding. If it is an embedding, comparing against text prompts introduces a modality confound; if it is textual, the method reduces to summarization and the specific contribution of the merge step is unclear. This ambiguity must be resolved before the reported results can be interpreted.
  3. [Abstract] The evaluation section names three datasets but reports no quantitative outcomes: no effect sizes, error bars, or statistical significance for the claimed 'superior quality' in skill learning, nor for the 'limitations' in knowledge-intensive settings. Without any reported numbers or baseline comparisons, the abstract's central conclusions are unsupported in the reviewable text.
minor comments (3)
  1. [Abstract] The abstract says 'two types of personalization tasks' but lists three datasets; clarify that there are two task categories (skill learning and knowledge-intensive) and three datasets.
  2. [Abstract] The phrase 'resource and computational constraints' is vague; specify which resources are meant (context window, inference cost, update cost) so the motivation is concrete.
  3. [Abstract] Consider reporting at least one quantitative result and the exact budget configuration in the abstract, so that the main claim is testable from the abstract alone.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified in the abstract-level derivation; the empirical claims rest on external benchmarks and independent baselines.

full rationale

The available manuscript text (the abstract) makes an empirical claim: 'in constrained inference budget scenarios ComMer achieves superior quality in skill learning tasks,' evaluated on LaMP and PerLTQA. No equation, fitted parameter, or self-citation is present that would make the result equal to its input by construction. The compressed document representation is described as an input to a frozen LLM, and the comparison is against prompt-based and fine-tuning alternatives; this is a testable empirical setup rather than a definitional or self-referential derivation. The abstract's vagueness about what 'constrained inference budget' means is a potential experimental-design weakness, but that is a correctness or falsifiability concern, not circularity. Because the visible text shows no step in which a prediction reduces to a fitted input, no load-bearing self-citation, and no uniqueness argument imported from the authors' prior work, the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claims depend on task-relevant information surviving compression and on the chosen benchmarks representing the intended deployment settings. No invented entities or fitted parameters are visible in the abstract.

assumptions (2)
  • domain assumption User documents can be lossily compressed into compact representations that retain enough information for personalized skill learning.
    The whole method rests on this premise. The abstract states it as the design ('compressing users' documents into compact representations'), and the reported knowledge-intensive losses show the assumption does not hold universally.
  • domain assumption The LaMP tweet paraphrasing and news headline generation tasks are valid proxies for personalized skill learning, and PerLTQA is a valid proxy for knowledge-intensive tasks.
    The abstract uses these datasets as evidence for the claims without discussing whether they cover the range of real personalization scenarios.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComMer: a Framework for Compressing and Merging User Data for Personalization." pith.science (2026). https://pith.science/paper/L2R4HGDU

@misc{pith2026250103276,
  author       = {Pith},
  title        = {Pith review of: ComMer: a Framework for Compressing and Merging User Data for Personalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L2R4HGDU}},
  note         = {Machine review of arXiv:2501.03276}
}
read the original abstract

Large Language Models (LLMs) excel at a wide range of tasks, but adapting them to new data, particularly for personalized applications, poses significant challenges due to resource and computational constraints. Existing methods either rely on exposing fresh data to the model through the prompt, which is limited by context size and computationally expensive at inference time, or fine-tuning, which incurs substantial training and update costs. In this paper, we introduce ComMer - Compress and Merge - a novel framework that efficiently personalizes LLMs by compressing users' documents into compact representations, which are then merged and fed into a frozen LLM. We evaluate ComMer on two types of personalization tasks - personalized skill learning, using the tweet paraphrasing dataset and the personalized news headline generation dataset from the LaMP benchmark, and knowledge-intensive, using the PerLTQA dataset. Our experiments demonstrate that in constrained inference budget scenarios ComMer achieves superior quality in skill learning tasks, while highlighting limitations in knowledge-intensive settings due to the loss of detailed information. These results offer insights into trade-offs and potential optimizations in multi-document compression for personalization.

Figures

Figures reproduced from arXiv: 2501.03276 by the authors.

Figure 1
Figure 1. Approaches for adapting LLMs to new data include integrating it through the prompt or modifying the model by updating its existing weights or introducing new trainable weights. Both methods have advantages and drawbacks, while our proposed method, ComMer, combines the benefits of both approaches. several advantages, including reduced computational costs, streamlined and cost-effective adaptation to new data, and imp… view at source ↗
Figure 2
Figure 2. Left: ComMer architecture. Each document is independently compressed into a fixed-size representation by a trainable compressor. These compressions are then merged using mean pool. Finally, the aggregated compression is plugged into a frozen LLM. Right: The compressor architecture. The input is appended with trainable compression embeddings, and processed by a frozen LLM, which is adapted using a trainable LoRA. The… view at source ↗
Figure 3
Figure 3. The trade-off between cost (number of tokens in the prompt) and quality (perplexity on the left, and ROUGE-L on the right), demonstrated using two personlized skill learning tasks: personalized tweet paraphrasing (top) and personalized news headline generation (bottom). Each curve represents models trained with different numbers of embeddings: 4, 8, 16, 32, 64, and 128, ordered from left to right. In the small token… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Perplexity of ComMer as a function of the number of documents follows a power-law relation in both the personalized tweet paraphrasing (left) and the personalized news headline generation (right) tasks. This pattern holds across all numbers of compression embeddings us…
Figure 5
Figure 5. Figure 5: The trade-off between cost (number of tokens in the prompt) and quality (perplexity on the left, and ROUGE-L on the right), demonstrated using PerLTQA. Each curve represents models trained with different numbers of embeddings: 4, 8, 16, 32, 64, and 128, ordered from le…
Figure 6
Figure 6. Figure 6: Evaluation of ComMer on a different number of doc￾uments than those used during training, demonstrated using the personalized tweet paraphrasing task. Each cell shows the perplex￾ity difference relative to the perplexity achieved when using the number of documents the …
Figure 7
Figure 7. Figure 7: Compariosn of different merging operations, demonstrated using the personalized tweet paraphrasing task. Concatenating compressions (ComMer : concat) is less effective than averaging them (ComMer : mean). Concatenating documents before compression (concat → ComMer) yie…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 12 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    E., Hume, T., Carter, S., Henighan, T., and Olah, C

    Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. Towards monosemanticity: Decomposing languag...

    work page 2023

  3. [3]

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A.,...

    1901

  4. [4]

    Adapting language models to compress contexts

    Chevalier, A., Wettig, A., Ajith, A., and Chen, D. Adapting language models to compress contexts. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 3829--3846, Singapore, December 2023. Association for Computational Linguistics. doi:10.18653/v1/2023.emnlp-main.232. U...

    doi:10.18653/v1/2023.emnlp-main.232 2023

  5. [5]

    Task arithmetic with lora for continual learning, 2023

    Chitale, R., Vaidya, A., Kane, A., and Ghotkar, A. Task arithmetic with lora for continual learning, 2023. URL https://arxiv.org/abs/2311.02428

    arXiv 2023

  6. [6]

    P er LTQA : A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering

    Du, Y., Wang, H., Zhao, Z., Liang, B., Wang, B., Zhong, W., Wang, Z., and Wong, K.-F. P er LTQA : A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering. In Wong, K.-F., Zhang, M., Xu, R., Li, J., Wei, Z., Gui, L., Liang, B., and Zhao, R. (eds.), Proceedings of the 10th SIGHAN Workshop on Chinese Languag...

    work page 2024

  7. [7]

    M., and Hegde, C

    Duman Keles, F., Wijewardena, P. M., and Hegde, C. On the computational complexity of self-attention. In Agrawal, S. and Orabona, F. (eds.), Proceedings of The 34th International Conference on Algorithmic Learning Theory, volume 201 of Proceedings of Machine Learning Research, pp.\ 597--619. PMLR, 20 Feb--23 Feb 2023. URL https://proceedings.mlr.press/v20...

    work page 2023

  8. [8]

    Toy models of superposition

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy models of superposition. Transformer Circuits Thread, 2022. URL https://transformer-circuits.pub/2022/toy_model/index.html

    2022

Show all 35 references
  1. [9]

    In-context autoencoder for context compression in a large language model

    Ge, T., Jing, H., Wang, L., Wang, X., Chen, S.-Q., and Wei, F. In-context autoencoder for context compression in a large language model. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uREj4ZuGJE

    2024

  2. [10]

    Arcee ' s M erge K it: A toolkit for merging large language models

    Goddard, C., Siriwardhana, S., Ehghaghi, M., Meyers, L., Karpukhin, V., Benedict, B., McQuade, M., and Solawetz, J. Arcee ' s M erge K it: A toolkit for merging large language models. In Dernoncourt, F., Preo t iuc-Pietro, D., and Shimorina, A. (eds.), Proceedings of the 2024 ...

    2024 doi

  3. [11]

    J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  4. [12]

    LLML ingua: Compressing prompts for accelerated inference of large language models

    Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., and Qiu, L. LLML ingua: Compressing prompts for accelerated inference of large language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ ...

    2023 doi

  5. [13]

    L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression

    Jiang, H., Wu, Q., , Luo, X., Li, D., Lin, C.-Y., Yang, Y., and Qiu, L. L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Associ...

    2024

  6. [14]

    The power of scale for parameter-efficient prompt tuning

    Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp.\ 3045--3059, Online...

    2021 doi

  7. [15]

    A., and Zettlemoyer, L

    Li, M., Gururangan, S., Dettmers, T., Lewis, M., Althoff, T., Smith, N. A., and Zettlemoyer, L. Branch-train-merge: Embarrassingly parallel training of expert language models, 2022. URL https://arxiv.org/abs/2208.03306

    2022 arXiv

  8. [16]

    H., Lu, H., Zhu, Y., Whitehouse, C., and Yu, H

    Lin, C.-C., Wang, X., Clark, J. H., Lu, H., Zhu, Y., Whitehouse, C., and Yu, H. Inducing generalization across languages and tasks using featurized low-rank mixtures, 2024. URL https://arxiv.org/abs/2402.17934

    2024 arXiv

  9. [17]

    ROUGE : A package for automatic evaluation of summaries

    Lin, C.-Y. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp.\ 74--81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/

    2004

  10. [18]

    Llms + persona-plug = personalized llms, 2024 a

    Liu, J., Zhu, Y., Wang, S., Wei, X., Min, E., Lu, Y., Wang, S., Yin, D., and Dou, Z. Llms + persona-plug = personalized llms, 2024 a . URL https://arxiv.org/abs/2409.11901

    2024 arXiv

  11. [19]

    F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P

    Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 0 157--173, 2024 b . doi:10.1162/tacl_a_00638. URL https://a...

    2024 doi

  12. [20]

    Learning to compress prompts with gist tokens

    Mu, J., Li, X., and Goodman, N. Learning to compress prompts with gist tokens. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 19327--19352. Curran Associates, Inc., 2023. U...

    2023

  13. [21]

    Towards modular LLM s by building and reusing a library of L o RA s

    Ostapenko, O., Su, Z., Ponti, E., Charlin, L., Le Roux, N., Caccia, L., and Sordoni, A. Towards modular LLM s by building and reusing a library of L o RA s. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedin...

    2024

  14. [22]

    J., and Veitch, V

    Park, K., Choe, Y. J., and Veitch, V. The linear representation hypothesis and the geometry of large language models. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference...

    2024

  15. [23]

    B., Lozhkov, A., Mitchell, M., Raffel, C., Werra, L

    Penedo, G., Kydlíček, H., allal, L. B., Lozhkov, A., Mitchell, M., Raffel, C., Werra, L. V., and Wolf, T. The fineweb datasets: Decanting the web for the finest text data at scale, 2024. URL https://arxiv.org/abs/2406.17557

    2024 arXiv

  16. [24]

    and Zamani, H

    Salemi, A. and Zamani, H. Comparing retrieval-augmentation and parameter-efficient fine-tuning for privacy-preserving personalization of large language models, 2024. URL https://arxiv.org/abs/2409.09510

    2024 arXiv

  17. [25]

    Optimization methods for personalizing large language models through retrieval augmentation

    Salemi, A., Kallumadi, S., and Zamani, H. Optimization methods for personalizing large language models through retrieval augmentation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, pp.\ 752–762, N...

    2024

  18. [26]

    L a MP : When large language models meet personalization

    Salemi, A., Mysore, S., Bendersky, M., and Zamani, H. L a MP : When large language models meet personalization. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ...

    2024 doi

  19. [27]

    Ziplora: Any subject in any style by effectively merging loras

    Shah, V., Ruiz, N., Cole, F., Lu, E., Lazebnik, S., Li, Y., and Jampani, V. Ziplora: Any subject in any style by effectively merging loras. In Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (eds.), Computer Vision -- ECCV 2024, pp.\ 422--438, C...

    2024

  20. [28]

    Team, G., Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., Mariooryad, S., Ding, Y., Geng, X., Alcober, F., Frostig, R., Omernick, M., Walker, L., Paduraru, C., Sorokin, C., Tacchetti, A., Gaffney, C., Daruki, S., Sercinog...

    2024 arXiv

  21. [29]

    S., Love, J., Tafti, P., Hussenot, L., Sessa, P

    Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., Tafti, P., Hussenot, L., Sessa, P. G., Chowdhery, A., Roberts, A., Barua, A., Botev, A., Castro-Ros, A., Slone, A., Héliou, A., Tacchetti, A., Bulanova, ...

    2024 arXiv

  22. [30]

    Parameter-efficient fine-tuning in large models: A survey of methodologies, 2024

    Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., and Yang, F. Parameter-efficient fine-tuning in large models: A survey of methodologies, 2024. URL https://arxiv.org/abs/2410.19878

    2024 arXiv

  23. [31]

    Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A

    Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., and Schmidt, L. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Ch...

    2022

  24. [32]

    Mixture of lo RA experts

    Wu, X., Huang, S., and Wei, F. Mixture of lo RA experts. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uWvKBCYh4S

    2024

  25. [33]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch

    Yu, L., Yu, B., Yu, H., Huang, F., and Li, Y. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st Int...

    2024

  26. [34]

    E., Cotterell, R., and Sachan, M

    Zhou, W., Jiang, Y. E., Cotterell, R., and Sachan, M. Efficient prompting via dynamic in-context learning, 2023. URL https://arxiv.org/abs/2305.11170

    2023 arXiv

  27. [35]

    Hydra: Model factorization framework for black-box llm personalization, 2024

    Zhuang, Y., Sun, H., Yu, Y., Qiang, R., Wang, Q., Zhang, C., and Dai, B. Hydra: Model factorization framework for black-box llm personalization, 2024. URL https://arxiv.org/abs/2406.02888

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.