Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multi-LoRA agents can split their value cache into a shared base cache plus a tiny low-rank adapter cache, preserving accuracy close to the non-shared baseline while approaching fully shared caching speed.

desk verdict A legitimate multi-LoRA cache-sharing system with exact algebra and credible efficiency gains, but the accuracy claim is only demonstrated on ~1k-token trajectories while the headline speedups are at 33k–66k tokens; that gap is the paper's main weakness. read the letter →

arxiv 2602.01053 v2 pith:IY7VYG57 submitted 2026-02-01 cs.LG

Hyesung Jeon , Hyeongju Ha , Jae-Joon Kim This is my paper · ORCID
classification cs.LG
keywords multi-LoRAKVcachesharingLLMagentslow-rankadaptationinferenceefficiencydecompositionFlash-LoRA-Attentionshared-A
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that in multi-LoRA agent systems, the value caches of agents working on the same context differ mainly through small adapter-specific outputs, while the shared pretrained-backbone contribution stays highly similar. It therefore decomposes each value cache into a shared base cache and a compact low-rank adapter cache, and proposes two schemes: BaseShared, which shares the base cache while keeping per-agent low-rank caches, and BaseLRShared, which also shares the low-rank cache by using a common down-projection matrix. A custom attention kernel reorders the relevant matrix products so the low-rank cache never needs to be expanded to full dimension. The reported result is KV-cache memory close to fully shared caching, throughput up to 2.46× the non-shared baseline, time-to-first-token reductions up to 4.44×, and accuracy within 1.5 percentage points of the non-shared baseline on HotpotQA and ScienceQA benchmarks.

What carries the argument

The load-bearing identity is the two-term decomposition of the LoRA value projection, Y_i = X_i W_0 + (X_i A_i) B_i. This separates what can be shared without loss—the base cache, computed once and reused across agents—from what must remain agent-specific, the adapter contribution, stored as the rank-r intermediate X_i A_i. The second mechanism is the matrix-product reordering O = P V_base + (P V_lr) B inside the attention kernel, which accumulates attention-weighted low-rank values first and applies the up-projection only once per query block, avoiding materialization of an L × d_out adapter cache.

What would settle it

Measure the cross-agent base-cache cosine similarity and the accuracy gap of BaseLRShared on a deliberately divergent set of roles (for example, one agent processing code and another processing legal text) or with trajectory lengths far beyond the 2k-token evaluation; if the cosine similarity drops materially below 0.95 or the accuracy loss exceeds about 3 points, the shared-base premise breaks. A second check is to run BaseLRShared on a heterogeneous multi-LoRA system without a shared A, where the paper already shows a 6.15-point drop (Table 7); if that drop compounds with more agents or long

Watch

Extended reading notes

Core claim

The central claim is that a multi-LoRA agent's value cache can be written as Y_i = X_i W_0 + (X_i A_i) B_i, where the base term X_i W_0 is nearly identical across agents (cosine similarity 0.953–0.973) while the adapter term is small (about 15–27× smaller in L1 norm) and nearly orthogonal to the base. Because the adapter contribution is carried by a rank-r intermediate X_i A_i, it can be stored as a thin LR cache and expanded by B_i only at attention time. Sharing the base cache avoids recomputing value projections for context already processed by another agent; sharing the LR cache as well, made possible by a shared-A multi-LoRA architecture, avoids recomputing hidden states for previously

Load-bearing premise

The method assumes that hidden states produced by different agents for the same context are similar enough that one agent's base cache can be reused by all others; the paper justifies this with measured cosine similarities of 0.953–0.973 rather than a worst-case error bound.

Editorial extensions

If this is right

  • Multi-LoRA agent serving can adopt a single shared base cache and, under shared-A, a single low-rank cache, cutting KV cache memory to roughly 1/N of the non-shared scheme.
  • BaseLRShared removes redundant prefill across agents, making total computation scale like fully shared caching, O(L² d_model).
  • Key-cache sharing can be taken for granted in this setting because key similarities exceed 0.98, so the value cache dominates cross-agent divergence.
  • The reordering trick applies whenever the adapter contribution is a rank-r product, giving a general route to low-rank cache expansion that is cheap in the dominant sequence-length factor.
  • The shared-A requirement is both an architectural enabler and a constraint: it improves accuracy, but BaseLRShared's compute savings depend on agents accepting a common down-projection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the shared-base premise rests on measured hidden-state similarity rather than a worst-case error bound, accuracy may degrade as trajectories lengthen, roles become more divergent, or the number of agents grows; an extension would measure cosine similarity and accuracy drop as a function of trajectory length, agent count, and role separation.
  • The base/adapter decomposition is agnostic to how the adapters were trained, so it should transfer to other parameter-efficient fine-tuning variants that leave a low-rank residual, as long as shared-backbone activations remain close.
  • BaseLRShared's shared-A requirement could be relaxed into clustered sharing: group agents whose down-projections are already similar, share LR caches within each cluster, and obtain partial compute savings without forcing a single A.
  • The kernel's reordering may also benefit other low-rank cache-compression methods that store rank-reduced keys or values, since the same associativity trick would let them avoid full-dimension reconstruction during attention.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes LRAgent, a KV cache sharing framework for multi-LoRA LLM agents. It decomposes the value cache into a shared base cache X_i W_0 and an adapter-dependent low-rank cache X_i A_i, stored in rank-r form and expanded by B_i at runtime. Two schemes are presented: BaseShared, which shares the base cache while keeping per-agent LR caches, and BaseLRShared, which additionally shares the LR cache under a shared-A multi-LoRA architecture. A custom kernel, Flash-LoRA-Attention, reorders attention computation to avoid materializing the full-dimension LR cache (Eq. 12). Experiments on HotpotQA and ScienceQA with LLaMA-3.1-8B and Ministral-8B report accuracy drops of at most 1.43 pp versus Non-Shared caching, while emulated-trace measurements show up to 2.46x throughput and 4.44x TTFT improvements over Non-Shared, approaching FullShared efficiency.

Significance. The central algebraic decomposition is exact (Eq. 3 and Eq. 12), and the efficiency evaluation uses a controlled trace (Table 10) with an honest end-to-end latency check (Table 14) that accounts for accuracy-induced trajectory length differences. The paper openly abates the shared-A assumption (Table 7) and provides code. If the accuracy preservation holds in the long-context regime, the work is significant: it shows that multi-LoRA agents can reap most of the benefits of fully shared caching while retaining role-specific behavior, and the kernel-level reordering is a clean and generalizable contribution. The main risk is external validity---the empirical evidence for cross-agent cache similarity and for accuracy preservation is confined to short trajectories, while the headline efficiency gains are demonstrated on long emulated traces.

major comments (3)
  1. [§4.2, Table 15, Tables 4–5] Accuracy is evaluated only on real agent trajectories averaging 1.0–1.2k tokens (Table 15), whereas the headline efficiency gains (up to 2.46x throughput, 4.44x TTFT reduction) are measured on an emulated trace with total sequence lengths up to 66.4k tokens (Appendix C.5, Tables 4–5). No experiment reports accuracy, cache similarity, or error accumulation at those long lengths. Because cross-agent sharing uses hidden states that §3.2 itself states 'are not exactly identical', the paper's claim that BaseLRShared preserves accuracy near the non-shared baseline for long, tool-augmented trajectories is not established in the regime where the efficiency advantages are claimed. This is load-bearing for the abstract and the method's stated purpose; it needs either long-context accuracy experiments or a quantitative error-propagation analysis.
  2. [Appendix A.2, Eq. (5), Table 6] The cosine-similarity 'bound' in Eqs. (5)–(9) assumes exact orthogonality and decorrelation of adapter outputs from base caches. Table 6 reports cos(Y_base,action, ΔY_action)=0.0985 and other nonzero cross terms, directly contradicting Eq. (5). Therefore inequality (8) and the conclusion in Eq. (9) are not proven; they are at best heuristic. Since §3.1 refers to this appendix as a 'concrete derivation' justifying the design choice to share the base cache rather than the full cache, the formal claim should be corrected or replaced with an empirical observation, or a bound should be derived using the actual norms and angles.
  3. [§3.2, Table 7, §3.1] BaseLRShared's efficiency gain depends on sharing both the base cache X_i W_0 and the LR cache X_i A across agents whose inputs X_i differ. The only quantitative support is average cosine similarity (Tables 1–2), which does not bound the L2 error of (X_i−X_j)(W_0 + A B_j); this error propagates through 32–36 layers and up to 45 agent iterations. Table 7 shows the sensitivity: when A is not shared, BaseLRShared drops by 6.15 pp (42.40→36.25 on HotpotQA easy). This demonstrates that the method's accuracy is strongly tied to the exactness of the shared-A premise and the hidden-state similarity it induces. The paper should provide a characterization of admissible divergence (e.g., a norm bound or similarity-versus-length measurement) to support the claim that the method is safe beyond the short tested contexts.
minor comments (4)
  1. [§3.3, Eqs. (13)–(14)] The text says reordering 'reduces the computation of LR cache expansion by approximately a factor of r/d_out'. Comparing Eqs. (13) and (14), the ratio of reordered to naive LR-expansion-related cost is roughly 1/d_out when L≫d_out (or 1/L in the opposite regime), not r/d_out. Please correct the asymptotic statement.
  2. [Table 5, §4.3] At 66.4k tokens for LLaMA-3.1-8B, BaseShared TTFT (67.80 s) is equal to DroidSpeak (67.80 s), not strictly better. The sentence 'BaseShared and BaseLRShared provide up to 1.63x and 4.44x reductions... both exceeding DroidSpeak' is acceptable with 'up to', but the tie at the longest length should be acknowledged or clarified.
  3. [Appendix D.6] The note that OOM at 66.4k for Non-Shared occurs despite 39.84 GB usage on a 48 GB GPU attributes this to 'memory fragmentation'. Please provide evidence (e.g., peak allocation measurements) or state that the OOM is due to fragmentation rather than a simple capacity limit.
  4. [§2.2] The claim that this is 'the first work that explicitly tailors KV cache sharing to multi-LoRA agent settings' is strong given DroidSpeak targets fine-tuned models sharing a backbone. Consider softening to 'the first to exploit the multi-LoRA decomposition' to avoid overclaiming.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the cache decomposition is exact algebra; sharing approximations are empirically measured and externally benchmarked, with no load-bearing self-citation.

full rationale

The paper's central decomposition (Yi = XiW0 + (XiAi)Bi, Eq. 3) is the LoRA definition, so the base/LR cache split is exact and not fitted to outcomes. The non-exact step—reusing XiW0 and, under shared-A, XiA across agents whose hidden states Xi differ—is supported by measured cosine similarities (Tables 1–2), which are observations on HotpotQA, not constants calibrated to the reported accuracy drops. Accuracy is evaluated on external benchmarks against Non-Shared, FullShared, and DroidSpeak; the shared-A design is attributed to independent prior work (HydraLoRA, MTL-LoRA) and its effect is openly ablated in Table 7, including the failure mode when A is not shared. Flash-LoRA-Attention's reordering is associativity, and its overhead analysis is arithmetic. No step reduces to its own input by construction, no fitted parameter is renamed as a prediction, and no load-bearing self-citation or author-imported uniqueness theorem appears. Therefore no circular step can be quoted.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

One design parameter (LoRA rank r=8, ablated 4–32 with negligible accuracy change) plus standard training hyperparameters; no constants are fitted to benchmark outcomes. All load-bearing premises are empirical domain assumptions about cross-agent cache similarity and shared-A training — each is measured in the paper (Tables 1, 2, 7) but none comes with an error-propagation bound. No invented entities (the LR cache is a storage layout, not a new postulated object). Credit: the Table 7 ablation openly shows BaseLRShared's dependence on shared-A.

free parameters (2)
  • LoRA rank r = 8
    Design choice, not fitted to benchmark outcomes. Appendix D.4 ablates r ∈ {4,8,16,32} with marginal accuracy differences (Non-Shared 38.25–39.00). LR-cache size and expansion cost scale with r, so the reported memory/compute savings are conditioned on this choice.
  • Per-agent learning rates = plan 5e-5; action 6e-5 (LLaMA) / 9e-5 (Ministral); reflect 6e-5 / 9e-5
    Grid-searched on training trajectories (Appendix C.3). Standard hyperparameter selection, not a post-hoc fit to benchmark results.
assumptions (6)
  • domain assumption Base cache X_i W_0 is nearly identical across agents for the same context (empirical cosine similarity 0.953–0.973).
    Section 3.1, Table 1. Load-bearing for both schemes: both reuse caches computed from another agent's hidden states. If this similarity dropped, the shared base cache would corrupt value caches.
  • domain assumption Adapter outputs are small (L1 norm 14.8–27.3× smaller) and approximately orthogonal to the base cache and decorrelated across agents.
    Appendix A.1–A.2, Eq. 5. Needed for the claim that sharing the base cache preserves the small agent-specific signal. Table 6 shows the orthogonality is approximate (largest cross term 0.0985).
  • domain assumption Under shared-A training, the LR cache X_i A is highly similar across agents (cosine 0.947–0.963).
    Table 2. Justifies BaseLRShared sharing the LR cache as well. §3.2 acknowledges the LR cache is computed from different agents' hidden states; Table 7 shows the scheme degrades 6.15 pp when A is not shared.
  • domain assumption Sharing the down-projection A across agents improves accuracy and preserves role specificity.
    Section 2.1, citing HydraLoRA (Tian et al., 2024) and MTL-LoRA (Yang et al., 2025c); confirmed in the paper's Table 7 (+0.20 to +2.20 pp across methods). BaseLRShared's compute gains ride on this premise.
  • domain assumption Cache cosine similarity is a valid proxy for end-to-end accuracy preservation.
    Section 3.1 motivates sharing from cosine similarities (Tables 1–2); the real validation is Table 3 accuracy. The proxy is trusted without a formal link between the two quantities.
  • domain assumption The emulated 17-step trace (Table 10) captures real multi-agent workload characteristics.
    Appendix C.5. All headline efficiency numbers (Tables 4–5) come from this fixed trace; Table 14 gives a partial real-benchmark check where BaseLRShared is indeed fastest.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents." pith.science (2026). https://pith.science/paper/IY7VYG57

@misc{pith2026260201053,
  author       = {Pith},
  title        = {Pith review of: LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IY7VYG57}},
  note         = {Machine review of arXiv:2602.01053}
}
read the original abstract

Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only by lightweight adapters. Despite sharing base model weights, each agent independently builds and stores its own KV cache for the same long, tool-augmented trajectories, incurring substantial memory and compute overhead. Existing KV cache sharing methods largely overlook this multi-LoRA setting. We observe that, cache differences across agents are dominated by adapter outputs, while activations from the shared pretrained backbone remain highly similar. Based on this observation, we propose LRAgent, a KV cache sharing framework for multi-LoRA agents. It decomposes the cache into two components, a shared base component derived from pretrained weights and an adapter-dependent component derived from LoRA weights. LRAgent reduces memory overhead by sharing the base component across agents and storing the adapter component in its inherent low-rank form. It also reduces computational overhead by sharing the low-rank cache, enabled by a shared-A multi-LoRA architecture. This avoids redundant computations for contexts that have already been processed by other agents. To efficiently reconstruct adapter contributions at runtime, we introduce Flash-LoRA-Attention, a kernel that reorders attention computation to avoid materializing the low-rank cache to full dimension. LRAgent achieves throughput and time-to-first-token latency close to fully shared caching, while preserving accuracy near the non-shared caching baseline across agentic question-answering benchmarks.

Figures

Figures reproduced from arXiv: 2602.01053 by the authors.

Figure 1
Figure 1. (Left) Relationship between the full cache, base cache, and adapter output. (Right) Layer-wise pairwise cosine similarity of the base and full caches, measured on the same context across three agent pairs using 128 samples of 2k tokens from the HotpotQA dataset [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Agent iteration and cache accumulation for Non-Shared, BaseShared, and BaseLRShared. T0 denotes the system prompt shared across agents, and Ti denotes trajectory context blocks, formed by concatenating model-generated tokens and retrieved context from external sources. BaseShared shares only the base cache and maintains a separate LR cache per agent, whereas BaseLRShared shares both the base and LR caches. 3.2. Base… view at source ↗
Figure 3
Figure 3. Diagram of base and LR cache computation with an initial context of length Lp prefilled by agent i, followed by an additional context of length Lc processed by agent j, under (a) BaseShared and (b) BaseLRShared. BaseShared maintains per-agent LR caches and computes the LR cache using hidden states for all context tokens not yet processed by the current agent, whereas BaseLRShared shares a single LR cache and uses hi… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: System throughput (tokens per second) of BaseShared and BaseLRShared, with Flash-LoRA-Attention (FLA). all experiments on a single NVIDIA A6000 48GB GPU. 4.2. Benchmark Accuracy We demonstrate that both of our cache sharing schemes preserve accuracy more effectively th…
Figure 5
Figure 5. Figure 5: Memory usage (GB) of cache sharing methods on total sequence length of 66.4k on Ministral-8B-Instruct. model generation in multi-hop scenarios. We first demon￾strate the impact of Flash-LoRA-Attention in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: L1 norm of the base cache and adapter output across model layers. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: reports pairwise cosine similarity of the key cache for each model. The average similarity is 0.9922 for LLaMA￾3.1-8B-Instruct and 0.9840 for Ministral-8B-Instruct, and even the minimum similarity across agent pairs is higher than the corresponding average base-cache s…
Figure 8
Figure 8. Figure 8: Agent prompts and an example of an accumulated trajectory on HotpotQA. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Train loss and L2 norm of the gradient for each agent types. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems

    cs.CL 2026-06 unverdicted novelty 7.0 of 10

    Introduces a 3-axis taxonomy (what info, alignment, fusion) for latent communication in multi-agent LLMs and identifies five design patterns from 18 methods.

  2. Low-Scaling Many-Body Green's Function Calculations for Molecular Systems via Interacting-Bath Dynamical Embedding Theory

    physics.chem-ph 2026-04 unverdicted novelty 7.0 of 10

    Molecular ibDET assembles full-system GW/EOM-CCSD Green's functions from small atom-centered impurity problems and recovers IPs/EAs within about 0.1 eV of full-system results.

Reference graph

Works this paper leans on

8 extracted references · 7 linked inside Pith · cited by 2 Pith papers

  1. [45]

    In the benchmark accuracy evaluation, these cases are counted as incorrect

    as out-of-function (OOF). In the benchmark accuracy evaluation, these cases are counted as incorrect. However, from a user-experience perspective, returning no answer can be qualitatively different from returning an incorrect answer, and may be considered a more severe failure. We therefore report OOF incidents in addition to benchmark accuracy. As shown ...

  2. [165]

    acl-long.165/

    URL https://aclanthology.org/2024. acl-long.165/. Qin, Y ., Liang, S., Ye, Y ., Zhu, K., Yan, L., Lu, Y ., et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. InThe Twelfth International Conference on Learning Representations, 2024. Rasal, S. Llm harmony: Multi-agent communication for problem solving.arXiv preprint arXiv:2...

    arXiv 2024

  3. [466]

    acl-long.466/

    URL https://aclanthology.org/2025. acl-long.466/. Bo, X., Chen, X., Dai, Q., Feng, X., Li, R., Wang, L., et al. Reflective multi-agent collaboration based on large language models. InAdvances in Neural Information Pro- cessing Systems, volume 37, pp. 138595–138631, 2024. doi: 10.52202/079017-4397. Chang, C.-C., Lin, C.-Y ., Akhauri, Y ., Lin, W.-C., Wu, K...

    arXiv 2025

  4. [697]

    emnlp-main.697/

    URL https://aclanthology.org/2024. emnlp-main.697/. Yuan, Z., Shang, Y ., Song, Y ., Wu, Q., Yan, Y ., Sun, G., et al. Asvd: Activation-aware singular value decomposition for compressing large language models.arXiv preprint arXiv:2312.05821, 2023. doi: 10.48550/arXiv.2312. 05821. URL https://arxiv.org/abs/2312. 05821. Zhang, C., Goh, X. D., Li, D., Zhang,...

    arXiv doi:10.48550/arxiv.2312.05821 2024

  5. [2023]

    URL https: //arxiv.org/abs/2307.09288

    doi: 10.48550/arXiv.2307.09288. URL https: //arxiv.org/abs/2307.09288. Wang, Y ., Lin, Y ., Zeng, X., and Zhang, G. Multi- lora: Democratizing lora for better multi-task learn- ing.arXiv preprint arXiv:2311.11501, 2023. doi: 10.48550/arXiv.2311.11501. URL https://arxiv. org/abs/2311.11501. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., et...

    arXiv doi:10.48550/arxiv.2307.09288 2023

  6. [2024]

    Talebirad, Y

    doi: 10.48550/arXiv.2408.03314. Talebirad, Y . and Nadiri, A. Multi-agent collaboration: Harnessing the power of intelligent llm agents.arXiv preprint arXiv:2306.03314, 2023. Tian, C., Shi, Z., Guo, Z., Li, L., and Xu, C. Hydralora: An asymmetric lora architecture for efficient fine-tuning. InAdvances in Neural Information Processing Systems, volume 37, p...

    arXiv doi:10.48550/arxiv.2408.03314 2023

  7. [2025]

    URL https: //arxiv.org/abs/2503.18893

    doi: 10.48550/arXiv.2503.18893. URL https: //arxiv.org/abs/2503.18893. Dao, T., Fu, D. Y ., Ermon, S., Rudra, A., and Re, C. Flashat- tention: Fast and memory-efficient exact attention with io-awareness. InAdvances in Neural Information Pro- cessing Systems, volume 35, pp. 16344–16359, 2022. doi: 10.5555/3600270.3601459. URL https://github. com/Dao-AILab/...

    arXiv doi:10.48550/arxiv.2503.18893 2022

  8. [2026]

    URL https: //arxiv.org/abs/2411.02820

    doi: 10.48550/arXiv.2411.02820. URL https: //arxiv.org/abs/2411.02820. Accepted for NSDI 2026. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science question answering. InAdvances in Neural Infor- mation Processing Systems, volume 35, pp....

    arXiv doi:10.48550/arxiv.2411.02820 2026

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.