Pith. sign in

REVIEW 9 cited by

LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.09997 v1 pith:SKKU6SY5 submitted 2024-02-15 cs.AI cs.CLcs.LG

Ziyu Zhao , Leilei Gan , Guoyin Wang , Wangchunshu Zhou , Hongxia Yang , Kun Kuang , Fei Wu This is my paper
classification cs.AIcs.CLcs.LG
keywords lorasloraretrieverloratasksdiverseefficientinputllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Low-Rank Adaptation (LoRA) provides an effective yet efficient solution for fine-tuning large language models (LLM). The modular and plug-and-play nature of LoRA enables the integration of diverse domain-specific LoRAs to enhance the capabilities of LLMs. Previous research on exploiting multiple LoRAs either focuses on specific isolated downstream tasks or fixes the selection of LoRAs during training. However, in real-world scenarios, LLMs receive diverse prompts covering different tasks, and the pool of candidate LoRAs is often dynamically updated. To bridge this gap, we propose LoraRetriever, a retrieve-then-compose framework that adaptively retrieves and composes multiple LoRAs according to the input prompts. LoraRetriever contains three main components: firstly, identifying and retrieving LoRAs relevant to the given input; secondly, formulating strategies for effectively integrating the retrieved LoRAs; and thirdly, developing efficient batch inference to accommodate heterogeneous requests. Experimental results indicate that LoraRetriever consistently outperforms the baselines, highlighting its practical effectiveness and versatility.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SSR-Merge: Subspace Signal Routing for Training-Free LoRA Merging in Diffusion Models

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    SSR-Merge merges LoRAs via subspace construction, inverse correlation decorrelation, and directional steering, shown to match the OLS solution with a streaming implementation that outperforms prior merging methods.

  2. T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts

    cs.CV 2024-12 unverdicted novelty 7.0 of 10

    T2I-FactualBench is a new three-tier benchmark for factuality of knowledge-intensive concepts in T2I models, using multi-round VQA evaluation to show SOTA models need improvement.

  3. Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Language models can use a two-stage sleep process of upward distillation for memory consolidation and RL-based dreaming for unsupervised self-improvement to enable continual learning.

  4. Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

    cs.LG 2026-06 conditional novelty 6.0 of 10

    Sleep-time Knowledge Seeding plus Dreaming lets LLMs expand capacity, distill fragile in-context memories into stable parameters, and self-improve without human labels.

  5. Polaris: Scaling Up Instruction-Guided Image Generation Towards Millions of Personalized Style Needs

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Polaris retrieves and integrates relevant models from a large library of checkpoints and adapters to enable scalable instruction-guided image generation and editing without additional training.

  6. SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    SCALE-LoRA proposes a post-retrieval audit framework using sparse residual composition and disagreement-based reliability signals to improve open-pool LoRA adapter reuse on tasks like BIG-Bench Hard.

  7. Joint Information Extraction Across Classical and Modern Chinese with Tea-MOELoRA

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Tea-MOELORA uses separate task and era gates over LoRA experts to jointly train relation and event extraction across classical and modern Chinese, improving F1 over joint LoRA and existing LoRA-MoE baselines on most datasets.

  8. Advancing Compositional LLM Reasoning with Structured Task Relations in Interactive Multimodal Communications

    cs.LG 2025-07 conditional novelty 5.0 of 10

    ContextLoRA partitions a single LLM's LoRA parameters by task and trains them in dependency-graph order, while ContextGear adds pipeline parallelism to cut training time, together outperforming multi-task LoRA baselin...

  9. Unlocking the Effectiveness of LoRA-FP for Seamless Transfer Implantation of Fingerprints in Downstream Models

    cs.CR 2025-08 conditional novelty 3.0 of 10

    Backdoor fingerprints trained into LoRA adapters on a base LLM transfer to derivative models with 100% trigger success and, in several scenarios, greater robustness than directly injected fingerprints.

Pith tools