Pith. sign in

REVIEW 5 major objections 5 minor 49 references

Treating model weights as a readable modality lets an LLM compose text and trained prefix caches into new task adapters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:45 UTC pith:QWKWOOHW

load-bearing objection A genuinely new mechanism, but the statistical evidence is thinner than the prose: worth a serious referee, not a pass as-is. the 5 major comments →

arxiv 2607.27497 v1 pith:QWKWOOHW submitted 2026-07-29 cs.CL

SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge

Lucio M. Dery , Benedict Aaron Tjandra , Siavash Samiei , Adhiguna Kuncoro , Zohar Yahav , Jiajun Shen , Arthur Szlam This is my paper
classification cs.CL
keywords prefix tuningmodel mergingskill compositionhyper-networkparameter-efficient fine-tuningKV-cachecompositional generalizationagentic adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that an agent's stored parametric skills (trained prefix KV-caches) and its textual knowledge should not be handled as separate silos. It presents SkillSmith, an augmented language model that ingests both the text descriptions of past tasks and the trained prefix weights associated with those tasks, along with a relational rationale, and directly outputs a new prefix weight for a target task. The central claim is that these synthesized weights are a distinctly superior initialization for subsequent fine-tuning, outperforming standard text-only in-context learning, arithmetic weight merging, and direct prefix tuning on the benchmarks studied. A reader should care because this is a concrete recipe for making agents reuse computation across both 'knowing' and 'doing.'

Core claim

SkillSmith treats prefix KV-caches as a third modality, alongside text and the frozen base model. Given a set of source task bundles (each with text metadata and a trained prefix cache), a preamble, and a combination text describing how the source tasks relate to the target, the augmented LLM produces a fixed-length sequence of latent tokens whose KV-caches become the new task's prefix weights via an output MLP. The whole pipeline is trained end-to-end with cross-entropy on the target task, backpropagating through the frozen downstream model. On a synthetic composition benchmark, SkillSmith's generated prefixes outperform arithmetic merging zero-shot and, after fine-tuning, outperform all st

What carries the argument

The load-bearing mechanism is the prefix KV-cache as a portable parametric skill, combined with an LLM acting as a weight-generating hyper-network. Source KV-caches are projected into the language model's embedding space by an input MLP adapter, interleaved with their textual descriptions using control tokens, and followed by a generation block of placeholder tokens. The KV-caches of these placeholder tokens are extracted, inverse-rotated to strip positional encoding, passed through an output MLP, and deployed as the target prefix. Because text can itself be converted into KV-caches by a forward pass, the model learns a shared space in which textual and parametric knowledge are commensurable

Load-bearing premise

The load-bearing premise is that the retrieved or given source task bundles actually carry compositional signal for the target task; the paper's own Section 4.2.2 defines this retrieval setting, and Appendix D shows that when the source pair is selected randomly rather than by retrieval, SkillSmith underperforms direct prefix tuning on 3 of 15 tasks, so if retrieval quality drops, the claimed advantage can disappear.

What would settle it

Give SkillSmith a held-out target with known true source tasks, then corrupt the source KV-caches (replace them with random caches or zero them out) while leaving all text metadata identical. If the generated prefix's downstream negative log-likelihood does not worsen measurably, the model is not actually composing weight information and the central modality claim fails. Conversely, on a target where the retrieved sources share no functional relationship with the target, SkillSmith's advantage over direct prefix tuning should vanish; measuring this would settle how much of the result depends o

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, agents can reuse previously trained prefix weights instead of discarding them, composing them into new task adapters.
  • Language can steer weight-space composition: changing the combination text changes the generated prefix, giving instruction-controlled adaptation.
  • SkillSmith-generated weights are useful primarily as an initialization; after fine-tuning they outperform both arithmetic merging and direct training, so the practical protocol is 'synthesize then fine-tune.'
  • Composition skills learned on synthetic tasks transfer to organic task distributions, so synthetic curricula can bootstrap meta-training when real paired tasks are scarce.
  • Even when the source-task set is retrieved heuristically and noisy, the composition beats direct training on the studied benchmark, suggesting robustness to imperfect retrieval.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The input adapter is agnostic to the exact PEFT format, so a natural extension is to feed LoRA or adapter weights instead of prefix KV-caches; if the same recipe works, the modality claim generalizes beyond prefix tuning.
  • Because SkillSmith's biggest margin appears after fine-tuning, the underlying learning is better described as acquiring a strong prior over task weights rather than emitting finished adapters; this suggests using SkillSmith iteratively in a lifelong-learning loop, feeding its own outputs back as new source bundles.
  • The synthetic-data transfer result hints that the bottleneck for such systems is not the number of real tasks but the diversity of compositional relationships in the meta-training set; constructing curricula over task-pair relationships could be a more efficient route than collecting more tasks.
  • A direct test of the modality claim would be to corrupt or zero out the input KV-caches while keeping all text; the paper's ablation already shows a drop, but varying the corruption level would quantify how much of the signal is truly in the weights as opposed to the text.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SkillSmith is a meta-learned composer that augments an LLM with MLP adapters so that it can ingest, in a single sequence, textual task metadata, learned prefix KV-caches for a set of source tasks, and a textual rationale describing how the sources relate to a target task; it then directly outputs a new prefix cache for the target. The prefix cache is used to condition a frozen Gemma 3 4B model, and training is end-to-end through the downstream cross-entropy loss. The authors introduce Composite-SNI, a synthetic dataset of about 21K composite tasks generated from pairs of SNI tasks, and evaluate on Composite-SNI, SNI, and MMLU-ProX under zero-shot and fine-tuning settings, comparing with ICL, direct prefix tuning, and weight-space merging baselines (LERP, Concat, SVD variants, source transfer). They report Elo ratings aggregated over 15, 10, and 6 held-out tasks and claim that SkillSmith outperforms uni-modal baselines and provides a superior initialization for fine-tuning.

Significance. If the empirical claims survive additional testing, this is a noteworthy step: it demonstrates that an LLM can treat prefix weights as a token-space modality and synthesize them conditioned on text, and it provides a usable recipe (prefix tuning + MLP projection + end-to-end meta-training) plus a large synthetic compositional benchmark. The paper also contributes an unusually thorough baseline suite for prefix-weight merging and publishes raw per-task NLLs in Appendix G, which is valuable for audits. The main weakness is that the headline comparisons rest on very small evaluation sets and single runs, so the quantitative conclusions are not yet established.

major comments (5)
  1. [§5.1, §5.2, Tables 1, 3, 4; Figs. 3–5] The headline claims - zero-shot synthesis outperforms text-only and weight-only baselines, KV caches are actually used, and generated prefixes are a 'distinctly superior initialization' - are all supported only by single-run NLL/Elo comparisons over 15, 10, and 6 held-out tasks. No seeds, confidence intervals, or paired significance tests are reported. The evidence is fragile: in Table 4, Direct Training beats SkillSmith on 79_411 and 330_736, and ICL-Init fine-tuning beats SkillSmith on 330_736; in MMLU-ProX (Table 8), many differences are below 0.05 NLL. A bootstrap over tasks or multi-seed reruns should be added; otherwise the aggregate Elo rankings cannot be distinguished from selection noise.
  2. [§5.1, Table 3] The abstract and Section 5.1 claim that SkillSmith outperforms text-only baselines, but the raw zero-shot NLLs in Table 3 show ICL obtaining lower NLL on 11 of the 15 Composite-SNI tasks (e.g., 1_450, 41_174, 43_122, 96_721, 583_850). The paper itself notes that 'ICL remains highly competitive' in the zero-shot regime. If the Elo aggregate places SkillSmith above ICL, the paper should report the head-to-head SkillSmith-versus-ICL win rate and reconcile it with the raw NLLs; if it does not, the zero-shot text-only outperformance claim should be removed or scoped to weight-space-only baselines.
  3. [§5.2.1, Table 6] The manuscript states that after fine-tuning on SNI 'the win rate between these methods converges to 0.5, indicating no significant performance differences.' This is an explicit limitation of the superior-initialization claim. Table 6 confirms that ICL-Init fine-tuning is better than SkillSmith (Retrieved) on most of the 10 tasks. The conclusion should be restricted to data-sparse/hard regimes (Composite-SNI, MMLU-ProX), and the SNI result should be presented as a ceiling effect rather than as evidence for the central claim.
  4. [Appendix D, Table 2; §5.2.2] The in-the-wild demonstration depends on retrieval. Appendix D shows Random+LLM selection is worse than Direct Training on 3/15 tasks (625_640, 667_794, 79_411), and Retrieval+LLM is only slightly better than Random in Elo. The claim that Retrieval+LLM beats Direct Training on all 15 tasks is a single-run per-task observation. Because a wrong source set can erase SkillSmith's advantage, the paper should report retrieval accuracy and sensitivity (e.g., top-k source sets) with error bars before claiming that SkillSmith 'extracts and composes relevant signal' under noisy retrieval.
  5. [§6.1, Table 1] Table 1 is the only direct evidence that the KV caches contribute beyond text, but it is a single run with no uncertainty. The 92-point Elo difference between 'All Inputs except K-V Caches' (1622) and 'All Inputs' (1714) may be meaningful, but it could also arise from scale or optimization variance. Because this is the central mixed-modality mechanism, please provide repeated runs or at least a per-task comparison of the two configurations; also specify whether the text-only configuration is given the same compute and output-length budget as the full model.
minor comments (5)
  1. [§4.2.1 vs Appendix A.1.2] Section 4.2.1 reports '≈21K composite tasks' while Appendix A.1.2 says the filtration steps produce 'O(90K) tasks'. Please clarify whether the latter is an intermediate count and give the exact final count and split sizes.
  2. [Figure 6] The spider plot lacks axis labels and a legend; 'smaller covered area is better' is not a quantitative basis for comparison. Please add a scale and identify the methods and tasks.
  3. [Tables 5–6 and text] Naming is inconsistent: text uses 'SkillSmith (No SNI Meta-Training)' and 'Pretrain-CSNI', while tables use 'SkillSmith (CSNI-Pre)' and 'SkillSmith (CSNI-Pre-Ret)'. Align the names to avoid confusion.
  4. [§2, §3.1, §4.2.2] Minor typos and spacing issues: 'it’s text examplars', 'a prioriconfidence', 'mix-modality'.
  5. [§3.4] The end-to-end prefix optimization is said to follow Liu et al. (2024), which shares an author with this submission. Please state the incremental contribution relative to that work and disclose the overlap clearly.

Circularity Check

0 steps flagged

No significant circularity: SkillSmith is an empirical meta-learning system evaluated on held-out tasks, and its predictions do not reduce to fitted inputs or self-citations.

full rationale

SkillSmith is a supervised meta-learning architecture: the synthesized prefix KV-cache is produced by a learned map over source bundles and text, then evaluated by downstream NLL on held-out target tasks, including external SNI and MMLU-ProX benchmarks. There is no equation in which the reported prediction equals a fitted input by construction. The output prefix is not obtained by retrieving or averaging source weights; it is generated and scored on independent target data. The only author-overlap citation is Liu et al. (2024), used for differentiable cache augmentation ('The output prefix-weights from SkillSmith are directly optimized on the target task in an end-to-end fashion as in Liu et al. (2024)'). This is an implementation technique, not an invoked uniqueness theorem or a fitted result, so it does not make the central claim circular. The use of Gemini 2.5 both to generate Composite-SNI tasks and to write combination texts is a data-construction overlap that could affect distributional fit, but it is not a definitional reduction: held-out composite tasks and the external benchmarks still provide independent targets. The paper's own caveats (Section 5.1: ICL remains competitive zero-shot; Section 5.2.1: SNI fine-tuning win rates converge to 0.5; Appendix D: random source selection can underperform direct training on 3/15 tasks) are statistical and robustness concerns, not circularity. No constructional equivalence or fitted-parameter-renamed-as-prediction was found.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 1 invented entities

The paper is an empirical ML method, so its ledger is dominated by domain assumptions about what prefix weights mean and whether synthetic meta-training transfers. The most important assumptions are: prefix KV-caches encode composable skills; Composite-SNI is representative enough to transfer; the task-relatedness assumption holds; and NLL is a meaningful evaluation metric. No natural-entity invention (new particle/force) is present; the only invented items are architectural tokens and the model itself.

free parameters (5)
  • Number of source tasks N = 2
    All experiments fix N=2 source bundles; skill composition is tested only for pairs (Section 4.1.1).
  • Output prefix length at evaluation = 32
    During training the output length is sampled from {16, 32, 64, 128}; evaluation fixes it to 32 (Section 4.1.2).
  • KV dropout probability = {0.1, 0.293, 0.5}
    Training regularization hyperparameter chosen by the authors (Appendix F).
  • Source prefix training grid = steps {200,500,1000}; LR 1e-2 to 1e-4
    Source KV-caches are trained and selected by validation NLL; this affects the quality of the input modality (Section 4.1.1).
  • Elo initialization and scale = R0=1500; scale=400
    Bradley-Terry Elo settings used to aggregate win rates across tasks (Section 4.4).
axioms (6)
  • domain assumption Prefix KV-caches are a faithful, composable encoding of a task skill.
    Section 3.1 states this as motivation; it is not proven and is the core bridge between text and weights.
  • domain assumption The meta-training distribution (Gemini-generated Composite-SNI) transfers to organic SNI/MMLU-ProX tasks.
    Section 5.2 relies on this for the in-the-wild claims; it is tested on only 10 and 6 evaluation tasks respectively.
  • domain assumption The problem setting assumes tasks are sufficiently related that a new task can be solved by synthesizing prior experiences.
    Sections 3.1-3.2 assume this; if the source set contains no compositional signal, the method has nothing to exploit, as Appendix D illustrates.
  • domain assumption A single forward pass through a 4B LLM with MLP adapters and fixed placeholder tokens can output a useful target prefix.
    Section 3.3 architecture; empirically probed by the paper but not grounded in any theoretical argument.
  • domain assumption NLL on held-out targets is a valid proxy for task-solving skill.
    All evaluation uses NLL (Section 4.4); no accuracy or task-success metrics are reported.
  • standard math Bradley-Terry/Elo rating model is an appropriate aggregation over tasks.
    Used to convert pairwise task wins into global ratings; standard, but the small number of tasks makes the ratings noisy.
invented entities (1)
  • Control-token vocabulary (<src_start>, <kv_start>, <kv_end>, <gen_start>, <gen_end>) no independent evidence
    purpose: Delineates text and KV-cache segments in the SkillSmith coprocessor input sequence
    Internal structural tokens introduced for this architecture; they have no falsifiable handle outside the paper and are an engineering design choice.

pith-pipeline@v1.3.0-daily-deepseek · 26949 in / 14098 out tokens · 132964 ms · 2026-08-01T06:45:32.273996+00:00 · methodology

0 comments
read the original abstract

Agentic systems driven by large language models (LLMs) regularly feature two key mechanisms to autonomously solve complex problems: synthesizing text-based knowledge and procedures from past experiences and building parametric (weight-space) skill libraries for recurring sub-goals. To date, research has largely treated these as orthogonal pursuits: either organizing textual knowledge through composition and reflection, or consolidating parametric skills via weight-space merging. Consequently, the seamless integration of text and model weights for targeted performance improvements remains largely unexplored. This work bridges this modality gap by treating model weights as an additional modality that an LLM can natively reason over. We instantiate parametric learning via prefix-tuning and augment an LLM to ingest both prefix weights and rich textual data which capture relationships to a target capability. Our augmented LLM, which we call SkillSmith, synthesizes these inputs to perform instruction-steered parametric synthesis, directly outputting new prefix weights that manifest the target skill. We demonstrate that our approach significantly outperforms both text-only and weight-space-only baselines, unlocking performance gains that are out of reach for uni-modal (text-only or weight-only) adaptations.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 9 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2303.08774 , year=

    GPT-4 Technical Report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:1412.6980 , year=

    Adam: A method for stochastic optimization , author=. arXiv preprint arXiv:1412.6980 , year=

    Pith/arXiv arXiv

  3. [3]

    rar: Learning to merge loras via hypernetworks for subject-style conditioned image generation , author=

    Lora. rar: Learning to merge loras via hypernetworks for subject-style conditioned image generation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  4. [4]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  5. [5]

    arXiv preprint arXiv:2412.17747 , year=

    Deliberation in latent space via differentiable cache augmentation , author=. arXiv preprint arXiv:2412.17747 , year=

    Pith/arXiv arXiv

  6. [6]

    Advances in neural information processing systems , volume=

    Faith and fate: Limits of transformers on compositionality , author=. Advances in neural information processing systems , volume=

  7. [7]

    arXiv preprint arXiv:1912.09713 , year=

    Measuring compositional generalization: A comprehensive method on realistic data , author=. arXiv preprint arXiv:1912.09713 , year=

    Pith/arXiv arXiv 1912

  8. [8]

    International conference on machine learning , pages=

    Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  9. [9]

    arXiv preprint arXiv:2508.16153 , year=

    Memento: Fine-tuning llm agents without fine-tuning llms , author=. arXiv preprint arXiv:2508.16153 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2510.04618 , year=

    Agentic context engineering: Evolving contexts for self-improving language models , author=. arXiv preprint arXiv:2510.04618 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2511.02208 , year=

    Training proactive and personalized llm agents , author=. arXiv preprint arXiv:2511.02208 , year=

    arXiv

  12. [12]

    Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=

    Adapterhub: A framework for adapting transformers , author=. Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=

    2020

  13. [13]

    2026 , eprint=

    Adaptive Minds: Empowering Agents with LoRA-as-Tools , author=. 2026 , eprint=

    2026

  14. [14]

    arXiv preprint arXiv:2307.13269 , year=

    Lorahub: Efficient cross-task generalization via dynamic lora composition , author=. arXiv preprint arXiv:2307.13269 , year=

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the 2022 conference on empirical methods in natural language processing , pages=

    Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks , author=. Proceedings of the 2022 conference on empirical methods in natural language processing , pages=

    2022

  16. [16]

    2025 , eprint=

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities , author=. 2025 , eprint=

    2025

  17. [17]

    2024 , url=

    Introducing the next generation of Claude , author=. 2024 , url=

    2024

  18. [18]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  19. [19]

    2022 , eprint=

    SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer , author=. 2022 , eprint=

    2022

  20. [20]

    Transactions on Machine Learning Research , issn=

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey , author=. Transactions on Machine Learning Research , issn=. 2024 , url=

    2024

  21. [21]

    2019 , eprint=

    Parameter-Efficient Transfer Learning for NLP , author=. 2019 , eprint=

    2019

  22. [22]

    Advances in neural information processing systems , volume=

    Reflexion: Language agents with verbal reinforcement learning , author=. Advances in neural information processing systems , volume=

  23. [23]

    Proceedings of the 31st International Conference on Computational Linguistics: Industry Track , pages=

    Lora soups: Merging loras for practical skill composition tasks , author=. Proceedings of the 31st International Conference on Computational Linguistics: Industry Track , pages=

  24. [24]

    arXiv preprint arXiv:2409.16167 , year=

    Merging loras like playing lego: Pushing the modularity of lora to extremes through rank-wise clustering , author=. arXiv preprint arXiv:2409.16167 , year=

    Pith/arXiv arXiv

  25. [25]

    2022 , eprint=

    Towards a Unified View of Parameter-Efficient Transfer Learning , author=. 2022 , eprint=

    2022

  26. [26]

    2023 , eprint=

    Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference , author=. 2023 , eprint=

    2023

  27. [27]

    2021 , eprint=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. 2021 , eprint=

    2021

  28. [28]

    2021 , eprint=

    The Power of Scale for Parameter-Efficient Prompt Tuning , author=. 2021 , eprint=

    2021

  29. [29]

    2021 , eprint=

    Prefix-Tuning: Optimizing Continuous Prompts for Generation , author=. 2021 , eprint=

    2021

  30. [30]

    2022 , eprint=

    ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts , author=. 2022 , eprint=

    2022

  31. [31]

    arXiv preprint arXiv:2405.06682 , year=

    Self-Reflection in LLM Agents: Effects on Problem-Solving Performance , author=. arXiv preprint arXiv:2405.06682 , year=

    Pith/arXiv arXiv

  32. [32]

    2023 , eprint=

    Self-Refine: Iterative Refinement with Self-Feedback , author=. 2023 , eprint=

    2023

  33. [33]

    2025 , eprint=

    MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation , author=. 2025 , eprint=

    2025

  34. [34]

    2026 , eprint=

    Measuring AI Ability to Complete Long Software Tasks , author=. 2026 , eprint=

    2026

  35. [35]

    2025 , eprint=

    SIMA 2: A Generalist Embodied Agent for Virtual Worlds , author=. 2025 , eprint=

    2025

  36. [36]

    2025 , eprint=

    Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline , author=. 2025 , eprint=

    2025

  37. [37]

    2025 , eprint=

    Everything is Context: Agentic File System Abstraction for Context Engineering , author=. 2025 , eprint=

    2025

  38. [38]

    2025 , eprint=

    A-MEM: Agentic Memory for LLM Agents , author=. 2025 , eprint=

    2025

  39. [39]

    2025 , eprint=

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory , author=. 2025 , eprint=

    2025

  40. [40]

    The Twelfth International Conference on Learning Representations , year=

    PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization , author=. The Twelfth International Conference on Learning Representations , year=

  41. [41]

    2026 , url=

    Lakshya A Agrawal and Shangyin Tan and Dilara Soylu and Noah Ziems and Rishi Khare and Krista Opsahl-Ong and Arnav Singhvi and Herumb Shandilya and Michael J Ryan and Meng Jiang and Christopher Potts and Koushik Sen and Alex Dimakis and Ion Stoica and Dan Klein and Matei Zaharia and Omar Khattab , booktitle=. 2026 , url=

    2026

  42. [42]

    2022 , eprint=

    Fusing finetuned models for better pretraining , author=. 2022 , eprint=

    2022

  43. [43]

    2022 , eprint=

    Patching open-vocabulary models by interpolating weights , author=. 2022 , eprint=

    2022

  44. [44]

    2023 , eprint=

    Editing Models with Task Arithmetic , author=. 2023 , eprint=

    2023

  45. [45]

    2025 , eprint=

    Task Prompt Vectors: Effective Initialization through Multi-Task Soft-Prompt Transfer , author=. 2025 , eprint=

    2025

  46. [46]

    Investigating Task Arithmetic for Zero-Shot Information Retrieval , url=

    Braga, Marco and Kasela, Pranav and Raganato, Alessandro and Pasi, Gabriella , year=. Investigating Task Arithmetic for Zero-Shot Information Retrieval , url=. doi:10.1145/3726302.3730216 , booktitle=

    arXiv

  47. [47]

    2025 , eprint=

    Gemini Embedding: Generalizable Embeddings from Gemini , author=. 2025 , eprint=

    2025

  48. [48]

    and Lu, Peihuang and Nocedal, Jorge and Zhu, Ciyou , title =

    Byrd, Richard H. and Lu, Peihuang and Nocedal, Jorge and Zhu, Ciyou , title =. SIAM Journal on Scientific Computing , volume =. 1995 , doi =. https://doi.org/10.1137/0916069 , abstract =

    doi:10.1137/0916069 1995

  49. [49]

    2024 , eprint=

    Model merging with SVD to tie the Knots , author=. 2024 , eprint=

    2024