Pith. sign in

REVIEW 4 major objections 7 minor 109 references

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EdgeLoRA is a serving system that hosts over 1,000 LoRA adapters on a single edge device while achieving 2–4x the throughput of llama.cpp.

desk verdict A credible multi-tenant LoRA serving system for edge devices with a real artifact; the 2-4x throughput claim is real but measured against a possibly unfair llama.cpp configuration, and the router is validated on the wrong adapter type. read the letter →

arxiv 2507.01438 v1 pith:QWR56BPP submitted 2025-07-02 cs.DC cs.AIcs.LG

Zheyu Shen , Yexiao He , Ziyao Wang , Yuning Zhang , Guoheng Sun , Wanghao Ye , Ang Li This is my paper
classification cs.DCcs.AIcs.LG
keywords Low-RankAdaptationLLMservingEdgedevicesMulti-tenantAdapterselectioncachingBatchinferenceOn-device
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single edge device can serve a large number of personalized large language models at once, provided the models share a frozen base network and differ only in small LoRA adapters. It presents EdgeLoRA, a serving system with three parts: an adapter router that picks the right adapter for each request automatically, a memory manager that keeps frequently used adapters in a pre-allocated cache and loads the rest from disk, and a batched inference path that processes requests for different adapters in one GPU pass. The evaluation reports 2–4x higher throughput than llama.cpp on Jetson and Raspberry Pi hardware, first-token latency near half a second, and support for over 1,000 simultaneous adapters where llama.cpp runs out of memory at 50. If these results hold, multi-tenant personalized LLM inference becomes feasible on devices with 8–64 GB of memory, with no cloud round trip.

What carries the argument

The load-bearing mechanism is batch LoRA inference, which computes $Y = WX + \sum_a B_a A_a X_a$ in a single forward pass: the base-model multiplication $WX$ is batched across all requests, and requests that share adapter $a$ are gathered into a unified sub-batch $X_a$ so the low-rank product $B_a A_a$ runs once per adapter instead of once per request. Surrounding it, the adaptive adapter selection uses a LoRA fine-tuned multi-label classifier (the adapter router) to score adapters for each prompt and picks the top-scoring one that is already cached; the heterogeneous memory manager combines an LRU cache with a pre-allocated memory pool so adapter loads and evictions avoid runtime allocation and fragmentation. Together these components let the server keep hot adapters resident, page cold ones from disk, and fill GPU batches with heterogeneous requests rather than waiting for same-adapter arrivals.

What would settle it

Configure llama.cpp to keep only the base model resident, load LoRA adapters from disk on demand, and batch requests for different adapters together, then rerun the workloads from Tables 4–6 on the same three edge devices; if the 2–4x throughput advantage and the 1,000-adapter capacity gap disappear, the paper's central efficiency claim is falsified.

Watch

Extended reading notes

Core claim

On its own terms, EdgeLoRA is an extension of llama.cpp that turns sequential, per-adapter serving into a batched, cache-aware pipeline. The central claim is that three coordinated mechanisms—automatic adapter selection, heterogeneous memory management, and batch LoRA inference—let one edge device host thousands of LoRA-tuned variants of a shared base model while improving throughput. Concretely, the paper reports 2–4x higher request throughput than llama.cpp across three models and three devices, an increase in resident adapter count from 50 (where llama.cpp runs out of memory) to more than 1,000, and average first-token latency below one second with roughly 98% of requests meeting a 6-second first-token SLO. The system also consumes less power per request than the baseline.

Load-bearing premise

The headline 2–4x throughput comparison assumes llama.cpp is used in its status-quo configuration—all adapters preloaded into memory and requests for different adapters processed sequentially—so a baseline that pages adapters from disk or batches heterogeneous LoRA requests could erode the reported margin.

Editorial extensions

If this is right

  • A Jetson-class device can act as a multi-tenant LLM server, giving each tenant its own fine-tuned behavior without sending prompts to the cloud.
  • The number of adapters a device can host is no longer bounded by RAM but by disk capacity and cache hit rate, since adapters are loaded on demand into a fixed-size pool.
  • EdgeLoRA's reported first-token latency below one second at 1,000 adapters makes per-user personalization usable in interactive applications.
  • Because the throughput gain comes from batching rather than changing the base model, the same design should carry over to other transformer backends that support LoRA, such as Phi-3, Mixtral, or Qwen, which the paper states are compatible.
  • Lower power per request on Jetson devices suggests the batching and caching design is also an energy-efficiency win for thermally constrained deployments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gather/sub-batch scatter pattern for adapter multiplication should transfer to other parameter-efficient methods that add a low-rank delta to a frozen backbone (e.g., prefix tuning or AdaLoRA), though the paper only demonstrates LoRA.
  • The adapter router's accuracy is capped by the best single adapter in the pool; it cannot synthesize a stronger model, so routing only helps when the pool contains a genuinely suitable candidate.
  • The 2–4x margin is measured against a llama.cpp configuration that preloads all adapters and handles different-adapter requests sequentially; a baseline that pages adapters from disk or batches heterogeneous LoRA requests could narrow the gap.
  • Under workloads with almost no adapter locality, the LRU cache would miss frequently and disk loading could dominate, so the thousand-adapter results may not carry over to uniform-access traffic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper describes EdgeLoRA, a serving system built on llama.cpp that automatically selects LoRA adapters with an offline-trained router, manages adapter memory with an LRU cache and a pre-allocated pool, and batches LoRA computations for requests using different adapters. The evaluation uses Llama3.1-8B, Llama3.2-3B, and OpenELM-1.1B on Jetson AGX Orin, Jetson Orin Nano, and Raspberry Pi 5, reporting throughput, first-token latency, SLO attainment, and power consumption on synthetic workloads. The authors claim a 2-4x throughput improvement over llama.cpp, the ability to serve over 1,000 adapters, and high adapter-router benchmark scores; source code and binaries are provided.

Significance. If the throughput and scalability comparisons can be made robust, the paper makes a real contribution to multi-tenant edge serving: it demonstrates that cached, disk-backed adapter management can be combined with grouped LoRA kernels on heterogeneous edge backends, and it ships an artifact that reproduces the main experiments. The adapter-router evaluation is anchored to external benchmarks with a held-out split, so the router-accuracy results are not circular. The main outstanding risk is empirical rather than conceptual: the headline ratio is measured against one baseline configuration whose representativeness is not established, and no uncertainty information is reported, so the size of the effect is not yet firmly established.

major comments (4)
  1. [Section 5, Baselines; Tables 4, 6] The headline 2-4x throughput claim is computed against a single llama.cpp configuration that preloads all adapters and serializes requests by adapter. The paper does not test on-demand adapter loading, per-adapter server instances, or a llama.cpp configuration with comparable batching and slot settings, and it provides no memory-residency figures for the baseline. The 206.28 s first-token latency for OpenELM-1.1B with 20 adapters on S3@Nano in Table 6 is more than two orders of magnitude above the EdgeLoRA value and suggests that the baseline is dominated by memory pressure or an unrepresentative configuration; the manuscript does not diagnose this. Please add at least one alternative llama.cpp configuration (preferably the best of several) with per-trial memory and latency measurements, and report whether the 2-4x ratio persists when the baseline is not preloaded with all adapters.
  2. [Section 5.1, Tables 4-14, Figure 8] All reported values come from a stochastic synthetic workload with Gamma-distributed arrivals and power-law adapter selection, yet every table and figure shows a single measurement with no variance estimate, number of seeds, or confidence interval. Several comparisons that support the paper's claims are small enough to be within run-to-run noise, for example the 0.45 versus 0.44 req/s throughput values in Tables 4 and 7 and the 0.19 versus 0.18 req/s values in Table 4. Please report at least five independent workload repetitions per configuration, with means and standard deviations or confidence intervals, for the headline comparisons and for the scaling curves in Figure 8.
  3. [Section 5.1 and Section 5.2] The paper does not explain how the adapter router trained on six full-parameter Llama-3.1-8B variants over five benchmarks in Section 5.2 is applied to the synthetic adapters used in the serving experiments in Section 5.1. The synthetic workload defines an optimal adapter by power-law sampling, while the router produces top-k ordered candidates from its own scoring; the manuscript does not state whether the synthetic adapters are the same candidate set, how ground-truth optimality maps to router scores, or how often the ground-truth adapter is actually in the router's top-k. Without this mapping, the end-to-end AAS results in Tables 4-10 cannot be separated from a pure cache effect. Please specify the candidate set for the router in each experiment and report router top-k hit rate on the synthetic workloads.
  4. [Section 5.1 and Figure 8] The claim that EdgeLoRA can serve over 1,000 adapters simultaneously is ambiguous because the memory manager only keeps a bounded cache resident and the remaining adapters live on disk. The paper should report the cache capacity, the cache hit rate, and the number of adapter swaps for each n, and state whether simultaneously means addressable in the candidate set or concurrently resident in memory. This matters because the scalability claim of several orders of magnitude more adapters rests on this distinction.
minor comments (7)
  1. [Section 5, Baselines] The second baseline bullet says EdgeLoRA builds upon the full feature set of EdgeLoRA where it should refer to llama.cpp as the base; please correct this.
  2. [Algorithm 1, lines 4-7] The pseudocode describes evaluating every adapter on every dataset and training the router inside the per-request serving path when the router is unavailable, which contradicts Section 4.1 where the router is trained offline; please revise the pseudocode to match the implemented offline-training workflow.
  3. [Section 4.2] The text says that when adapter locality is more unbalanced, the LFU cache could achieve a higher hit rate, but the implementation described immediately afterward uses an LRU policy; please reconcile this inconsistency.
  4. [Section 5.2, Table 12] The table labels the router result as accuracy but reports benchmark scores such as IFEval, BBH, MATH, GPQA, and MMLU-PRO; please define the exact protocol, in particular how prompts are scored and how the selected model is evaluated on the test split.
  5. [Section 5.2] The text says the datasets were selected from the Open LLM Leaderboard and cites reference [87], but reference [87] is the HuggingFace Transformers paper, not the Open LLM Leaderboard; the citation should be corrected.
  6. [Section 5.1, Table 11] The power comparison reports average Watts only; because EdgeLoRA and llama.cpp process different numbers of requests per second, the energy-efficiency claim should be supported by energy per completed request in Joules as well as average power.
  7. [Figure 8] The y-axis label is misspelled as Throught and the latency panels are missing a y-axis title; please fix the figure labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EdgeLoRA's throughput and router claims are empirical, and the sole self-citation is a non-load-bearing related-work mention.

full rationale

The paper's central claims are empirical measurements against an external baseline (llama.cpp) and standard benchmarks (IFEval, BBH, MATH, GPQA, MMLU-PRO), so they do not reduce to fitted parameters or self-citations. The adapter router is trained on an 80% split of benchmark data and evaluated on the held-out 20% split (Section 5.2), which is a genuine generalization test rather than a circular prediction. The only self-citation is FLoRA [85] in the related-work discussion of PEFT; it is descriptive and does not ground any design choice, result, or uniqueness claim. No equation in the paper defines an input in terms of the output it is said to predict; the LoRA composition formula y = Wx + BAx is standard and used only to describe the batching scheme. The baseline configuration of llama.cpp may be contested on fairness grounds, but that is a correctness/experimental-design concern, not a circularity of the derivation chain. Therefore no circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The system's evaluation rests on several unverified modeling choices: the workload generator's power-law and Gamma parameters, the assumption that full fine-tuned model benchmark scores represent LoRA adapter quality, the choice of llama.cpp's preloading configuration as the baseline, and the assumption that Q8_0/Q4_0 adapter quantization does not change the router's label quality. No new physical or conceptual entities are introduced.

free parameters (5)
  • synthetic workload power-law exponent alpha = 1.0 default (0.5-1 tested)
    Hand-set in Table 3 to model adapter locality; Table 7 shows throughput is insensitive but request latency varies with alpha, so the workload model affects the measured user experience.
  • workload burstiness coefficient cv = 1.0 default (tested up to 2)
    Hand-set to model Gamma arrival burstiness; Table 9 shows EdgeLoRA throughput drops from 0.44 to 0.03 req/s as cv goes from 1 to 2, so the headline 4x gain depends on moderate burstiness.
  • request rate R = 0.2 to 1.0 req/s per setting
    Per-device load chosen in Table 3; different R changes queueing and whether latency or throughput dominates, directly affecting reported SLO and latency comparisons.
  • number of slots gamma = 5, 10, 20, or 50 depending on setting
    Manually configured concurrency limit; Table 14 demonstrates throughput scales with the number of slots, so results depend on this tuning knob.
  • top-k adapters k in router = 3
    Algorithm 1 uses k=3 to limit the candidate adapters considered; this affects which cached adapters are selected and thus cache hit behavior and latency.
assumptions (4)
  • domain assumption Power-law adapter popularity, with roughly 10% of adapters generating 80% of invocations, approximates real multi-tenant edge workloads.
    Invoked in Section 4.2 to justify LRU caching and in Section 5.1 to generate synthetic traces, citing CaraServe and AlpaServe; no on-device measurement is provided to confirm this distribution on edge devices.
  • ad hoc to paper Benchmark scores of full-parameter fine-tuned models represent the relative quality of LoRA adapters for routing purposes.
    Section 5.2 builds the router's ground truth from six full fine-tuned models (e.g., Defne-llama3.1-8B), while EdgeLoRA serves LoRA adapters of rank 16-32; no conversion or transfer experiment is described.
  • domain assumption llama.cpp preloading all adapters and serializing different-adapter requests is the appropriate status quo baseline.
    Section 5 'Baselines' defines the comparison this way; the paper does not test llama.cpp with alternative LoRA management modes or compare with other LoRA serving frameworks, so the status quo may be stronger than assumed.
  • domain assumption Q8_0 and Q4_0 quantization of LoRA adapters does not change the relative adapter quality used to train the router.
    End-to-end runs use quantized adapters, but the router's quality labels come from unquantized full models; no quantization ablation is reported, so the router's accuracy on the deployed adapters is unverified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices." pith.science (2026). https://pith.science/paper/QWR56BPP

@misc{pith2026250701438,
  author       = {Pith},
  title        = {Pith review of: EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QWR56BPP}},
  note         = {Machine review of arXiv:2507.01438}
}
read the original abstract

Large Language Models (LLMs) have gained significant attention due to their versatility across a wide array of applications. Fine-tuning LLMs with parameter-efficient adapters, such as Low-Rank Adaptation (LoRA), enables these models to efficiently adapt to downstream tasks without extensive retraining. Deploying fine-tuned LLMs on multi-tenant edge devices offers substantial benefits, such as reduced latency, enhanced privacy, and personalized responses. However, serving LLMs efficiently on resource-constrained edge devices presents critical challenges, including the complexity of adapter selection for different tasks and memory overhead from frequent adapter swapping. Moreover, given the multiple requests in multi-tenant settings, processing requests sequentially results in underutilization of computational resources and increased latency. This paper introduces EdgeLoRA, an efficient system for serving LLMs on edge devices in multi-tenant environments. EdgeLoRA incorporates three key innovations: (1) an adaptive adapter selection mechanism to streamline the adapter configuration process; (2) heterogeneous memory management, leveraging intelligent adapter caching and pooling to mitigate memory operation overhead; and (3) batch LoRA inference, enabling efficient batch processing to significantly reduce computational latency. Comprehensive evaluations using the Llama3.1-8B model demonstrate that EdgeLoRA significantly outperforms the status quo (i.e., llama.cpp) in terms of both latency and throughput. The results demonstrate that EdgeLoRA can achieve up to a 4 times boost in throughput. Even more impressively, it can serve several orders of magnitude more adapters simultaneously. These results highlight EdgeLoRA's potential to transform edge deployment of LLMs in multi-tenant scenarios, offering a scalable and efficient solution for resource-constrained environments.

Figures

Figures reproduced from arXiv: 2507.01438 by the authors.

Figure 1
Figure 1. Multi-tenant LLM Serving on Edge Devices. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The workflow of LoRA. Compared to traditional full-parameter fine-tuning approaches, LoRA achieves a reduction in trainable parameters by several or￾ders of magnitude (up to a 10,000×) while maintaining comparable performance in terms of model accuracy. During the inference phase, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of EdgeLoRA design. Adapter Cache Adapter Pool Adapter Router With Specific Adapter ID? ❌ No ✅ Yes Adapter Score 0 1 ✅ ✅ ✅ ✅ ❌ In Cache? Prompt Selected Adapter Adapter Adapter Adapter Adapter Adapter [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The workflow of adaptive adapter selection. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: further illustrates batching LoRA inference. Consider a scenario with multiple distinct requests {𝑥0, . . . , 𝑥𝑛}, each re￾quiring a unique LoRA adapter. During inference, the inputs for all requests are batched together, and the computations involving the pre-trained …
Figure 7
Figure 7. Figure 7: Slot state machine in our Server Manager. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Throughput and average request latency of [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

109 extracted references · 17 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen- Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Mat...

    arXiv 2024

  2. [2]

    Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977 (2020)

    arXiv 2020

  3. [3]

    Anthropic. 2023. Claude 3 Model Card. https://www.anthropic.com/model- card-claude-3 Accessed: 2024-12-03

    2023

  4. [4]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  5. [5]

    Cunningham

    Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham. 2024. LoRA Learns Less and Forgets Less. arXiv:2405.09673 [cs.LG] https://arxiv.org/abs/2405.09673

    arXiv 2024

  6. [6]

    Raffaello Bonghi. 2023. Jetson-Stats. https://github.com/rbonghi/jetson_stats. Accessed: 2024-12-07

    2023

  7. [7]

    Alexander Borzunov, Max Ryabinin, Artem Chumachenko, Dmitry Baranchuk, Tim Dettmers, Younes Belkada, Pavel Samygin, and Colin A Raffel. 2023. Dis- tributed inference and fine-tuning of large language models over the internet. Advances in neural information processing systems 36 (2023), 12312–12331

    2023

  8. [8]

    Peter F Brown, John Cocke, Stephen A Della Pietra, Vincent J Della Pietra, Frederick Jelinek, John Lafferty, Robert L Mercer, and Paul S Roossin. 1990. A statistical approach to machine translation. Computational linguistics 16, 2 (1990), 79–85

    1990

Show all 109 references
  1. [9]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

    2020 arXiv

  2. [10]

    Rickard Brüel-Gabrielsson, Jiacheng Zhu, Onkar Bhardwaj, Leshem Choshen, Kristjan Greenewald, Mikhail Yurochkin, and Justin Solomon. 2024. Com- press then Serve: Serving Thousands of LoRA Adapters with Little Overhead. arXiv:2407.00066 [cs.DC] https://arxiv.org/abs/2407.00066

    2024 arXiv

  3. [11]

    Branden Butler, Sixing Yu, Arya Mazaheri, and Ali Jannesari. 2024. PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation. arXiv preprint arXiv:2407.11798 (2024)

    2024 arXiv

  4. [12]

    Arnav Chavan, Zhuang Liu, Deepak Gupta, Eric Xing, and Zhiqiang Shen. 2023. One-for-all: Generalized lora for parameter-efficient fine-tuning. arXiv preprint arXiv:2306.07967 (2023)

    2023 arXiv

  5. [13]

    Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krish- namurthy. 2023. Punica: Multi-Tenant LoRA Serving. arXiv:2310.18547 [cs.DC] https://arxiv.org/abs/2310.18547

    2023 arXiv

  6. [14]

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307 (2023)

    2023 arXiv

  7. [15]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 240 (2023), 1–113

    2023

  8. [16]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems 36 (2024)

    2024

  9. [17]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognit...

    2021 arXiv

  10. [18]

    Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang, Maria Lomeli, Patrick Lewis, Gautier Izacard, Edouard Grave, Sebastian Riedel, and Fabio Petroni

  11. [19]

    Wafaa S El-Kassas, Cherif R Salama, Ahmed A Rafea, and Hoda K Mohamed

  12. [20]

    Hugging Face. 2023. Text Generation Inference: Large Language Model Text Generation Inference. https://github.com/huggingface/text-generation-infere nce

    2023

  13. [21]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning. PMLR, 10323–10337

    2023

  14. [22]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 (2022)

    2022 arXiv

  15. [23]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Hao- nan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang...

    2023 doi

  16. [24]

    Georgi Gerganov. 2023. llama.cpp: LLM inference in C/C++. https://github.com /ggerganov/llama.cpp

    2023

  17. [25]

    Georgi Gerganov. 2024. ggml. https://github.com/ggerganov/ggml. Accessed: 2024-12-03

    2024

  18. [26]

    Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. 2023. Multimodal- gpt: A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.04790 (2023)

    2023 arXiv

  19. [27]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, and Others. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  20. [28]

    Alex Graves and Alex Graves. 2012. Long short-term memory. Supervised sequence labelling with recurrent neural networks (2012), 37–45

    2012

  21. [29]

    Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, and Jeffrey P Bigham. 2022. InstructDial: Improving zero and few-shot generalization in dialogue through instruction tuning. arXiv preprint arXiv:2205.12673 (2022)

    2022 arXiv

  22. [30]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. arXiv:2103.03874 [cs.LG] https: //arxiv.org/abs/2103.03874

    2021 arXiv

  23. [31]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning. PMLR, 2790–2799

    2019

  24. [32]

    Chenghao Hu and Baochun Li. 2024. When the Edge Meets Transformers: Distributed Inference with Transformer Models. In 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS) . IEEE, 82–92

    2024

  25. [33]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  26. [34]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guil- laume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-An...

    2024 arXiv

  27. [35]

    Yuxin Jiang, Chunkit Chan, Mingyang Chen, and Wei Wang. 2023. Lion: Ad- versarial distillation of proprietary large language models. arXiv preprint MobiSys ’25, June 23–27, 2025, Anaheim, CA, USA Zheyu Shen, Yexiao He, Ziyao Wang, Yuning Zhang, Guoheng Sun, Wanghao Ye, Ang Li ...

    2023 arXiv

  28. [36]

    Nicolai M Josuttis. 2012. The C++ standard library: a tutorial and reference. (2012)

    2012

  29. [37]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for C...

    2019

  30. [38]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  31. [39]

    2024.{InfiniGen}: Efficient generative inference of large language models with dynamic{KV} cache management

    Wonbeom Lee, Jungi Lee, Junghwan Seo, and Jaewoong Sim. 2024.{InfiniGen}: Efficient generative inference of large language models with dynamic{KV} cache management. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) . 155–172

    2024

  32. [41]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)

    2021 arXiv

  33. [42]

    Suyi Li, Hanfeng Lu, Tianyuan Wu, Minchen Yu, Qizhen Weng, Xusheng Chen, Yizhou Shan, Binhang Yuan, and Wei Wang. 2024. CaraServe: CPU- Assisted and Rank-Aware LoRA Serving for Generative LLM Inference. arXiv:2401.11240 [cs.DC] https://arxiv.org/abs/2401.11240

    2024 arXiv

  34. [43]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)

    2021 arXiv

  35. [44]

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang

  36. [45]

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, et al. 2024. Personal llm agents: Insights and survey about the capability, efficiency and security. arXiv preprint arXiv:2401.05459 (2024)

    2024 arXiv

  37. [46]

    Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, et al. 2023. {AlpaServe}: Statistical multiplexing with model parallelism for deep learning serving. In 17th USENIX Symposium on Operating Syst...

    2023

  38. [47]

    Long Lian, Baifeng Shi, Adam Yala, Trevor Darrell, and Boyi Li. 2023. Llm- grounded video diffusion models. arXiv preprint arXiv:2309.17444 (2023)

    2023 arXiv

  39. [48]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and accel- eration. Proceedings of Machine Learning and Systems 6 ...

    2024

  40. [49]

    Tiedong Liu and Bryan Kian Hsiang Low. 2023. Goat: Fine-tuned llama outper- forms gpt-4 on arithmetic tasks. arXiv preprint arXiv:2305.14201 (2023)

    2023 arXiv

  41. [50]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)

    2021 arXiv

  42. [51]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022

    2021

  43. [52]

    Bei Luo, Raymond YK Lau, Chunping Li, and Yain-Whar Si. 2022. A critical re- view of state-of-the-art chatbot designs and applications.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 12, 1 (2022), e1434

    2022

  44. [53]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems 36 (2023), 21702–21720

    2023

  45. [54]

    Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods. https://github.com/huggingface/peft

    2022

  46. [55]

    Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Hor- ton, Yanzi Jin, Chenfan Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Be- lenko, Peter Zatloukal, and Mohammad Rastegari. 2024. OpenELM: An Ef- ficient Language Model Family with Open Training and Inference Framew...

    2024 arXiv

  47. [56]

    Liang Mi, Weijun Wang, Wenming Tu, Qingfeng He, Rui Kong, Xinyu Fang, Yazhu Dong, Yikang Zhang, Yunchun Li, Meng Li, et al . 2024. V-LoRA: An Efficient and Flexible System Boosts Vision Applications with LoRA LMM. arXiv preprint arXiv:2411.00915 (2024)

    2024 arXiv

  48. [57]

    Microsoft. 2023. DeepSpeed-MII: DeepSpeed Model Implementations for Infer- ence. https://github.com/microsoft/DeepSpeed-MII. Accessed: [Insert date of access]

    2023

  49. [58]

    2023-2025

    MLC team. 2023-2025. MLC-LLM. https://github.com/mlc-ai/mlc-llm

    2023

  50. [59]

    Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Barajas Gonzalez, Jennifer Neville, and Tara Safavi

  51. [60]

    Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Ab- stractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023 (2016)

    2016 arXiv

  52. [61]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

    2024 arXiv

  53. [62]

    Zhang, Mark Harman, and Meng Wang

    Shuyin Ouyang, Jie M. Zhang, Mark Harman, and Meng Wang. 2024. An Empirical Study of the Non-determinism of ChatGPT in Code Generation. ACM Transactions on Software Engineering and Methodology (Sept. 2024). https: //doi.org/10.1145/3697010

    2024 doi

  54. [63]

    arXiv preprint arXiv:2311.09180 (2023)

    Pearl: Personalizing large language model writing assistants with generation-calibrated retrievers. arXiv preprint arXiv:2311.09180 (2023)

    2023 arXiv

  55. [64]

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer

  56. [65]

    James L Peterson and Abraham Silberschatz. 1985. Operating system concepts. Addison-Wesley Longman Publishing Co., Inc

    1985

  57. [66]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. arXiv:2311.12022 [cs.AI] https://arxiv.org/abs/2311.12022

    2023 arXiv

  58. [67]

    Marco Paolieri, Eduardo Quiñones, Francisco J Cazorla, Robert I Davis, and Mateo Valero. 2011. IAˆ 3: An interference aware allocation algorithm for multicore hard real-time systems. In 2011 17th IEEE Real-Time and Embedded Technology and Applications Symposium. IEEE, 280–290

    2011

  59. [68]

    Timo Schick and Hinrich Schütze. 2020. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676 (2020)

    2020 arXiv

  60. [69]

    Tianxiang Shen, Ji Qi, Jianyu Jiang, Xian Wang, Siyuan Wen, Xusheng Chen, Shixiong Zhao, Sen Wang, Li Chen, Xiapu Luo, Fengwei Zhang, and Heming Cui. 2022. SOTER: Guarding Black-box Inference for General Neural Networks at the Edge. In 2022 USENIX Annual Technical Conference (...

    2022

  61. [70]

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, et al

  62. [71]

    Yu Shu, Siwei Dong, Guangyao Chen, Wenhao Huang, Ruihua Zhang, Daochen Shi, Qiqi Xiang, and Yemin Shi. 2023. Llasm: Large language and speech model. arXiv preprint arXiv:2308.15930 (2023)

    2023 arXiv

  63. [72]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiao- qing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023)

    2023 arXiv

  64. [73]

    Felix Stahlberg. 2020. Neural machine translation: A review. Journal of Artificial Intelligence Research 69 (2020), 343–418

    2020

  65. [74]

    Biao Sun, Ziming Huang, Hanyu Zhao, Wencong Xiao, Xinyi Zhang, Yong Li, and Wei Lin. 2024. Llumnix: Dynamic Scheduling for Large Language Model Serving. arXiv preprint arXiv:2406.03243 (2024)

    2024 arXiv

  66. [75]

    Le, Ed H

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. 2022. Challenging BIG-Bench Tasks and Whether Chain- of-Thought Can Solve Them. arXiv:2210.09261 [cs.CL] https://...

    2022 arXiv

  67. [76]

    arXiv preprint arXiv:2311.03285 (2023)

    S-lora: Serving thousands of concurrent lora adapters. arXiv preprint arXiv:2311.03285 (2023)

    2023 arXiv

  68. [77]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpaca: A strong, repli- cable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/...

    2023

  69. [78]

    Yixin Song, Zeyu Mi, Haotong Xie, and Haibo Chen. 2024. Powerinfer: Fast large language model serving with a consumer-grade gpu. In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles . 590–606

    2024

  70. [79]

    Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. 2024. OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data. arXiv preprint arXiv:2410.01560 (2024)

    2024 arXiv

  71. [80]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  72. [81]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  73. [82]

    Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. 2024. MathScale: Scaling Instruction Tuning for Mathematical Reasoning. arXiv:2403.02884 [cs.CL] https://arxiv.org/abs/2403.02884

    2024 arXiv

  74. [83]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen

  75. [84]

    Gemini Team. 2024. Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 [cs.CL] https://arxiv.org/abs/2312.11805

    2024 arXiv

  76. [85]

    Ziyao Wang, Zheyu Shen, Yexiao He, Guoheng Sun, Hongyi Wang, Lingjuan Lyu, and Ang Li. 2024. FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations. arXiv:2409.05976 [cs.LG] https: //arxiv.org/abs/2409.05976

    2024 arXiv

  77. [86]

    Adhika Pramita Widyassari, Supriadi Rustad, Guruh Fajar Shidik, Edi Noer- sasongko, Abdul Syukur, Affandy Affandy, et al. 2022. Review of automatic text summarization techniques & methods.Journal of King Saud University-Computer and Information Sciences 34, 4 (2022), 1029–1046

    2022

  78. [87]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement De- langue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Ma...

    2020

  79. [88]

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou

  80. [89]

    Mengzhou Xia, Zexuan Zhong, and Danqi Chen. 2022. Structured pruning learns compact and accurate models. arXiv preprint arXiv:2204.00408 (2022)

    2022 arXiv

  81. [90]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning . PMLR, 38087–38099

    2023

  82. [91]

    Shang Yang, Junxian Guo, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, and Song Han. 2025. LServe: Efficient Long- sequence LLM Serving with Unified Sparse Attention. arXiv:2502.14866 [cs.CL] https://arxiv.org/abs/2502.14866

    2025 arXiv

  83. [92]

    Zihao Wang, Wei Liu, Qian He, Xinglong Wu, and Zili Yi. 2022. Clip-gen: Language-free training of a text-to-image generator with clip. arXiv preprint arXiv:2203.00386 (2022)

    2022 arXiv

  84. [93]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A Survey on Multimodal Large Language Models. National Science Review (Nov. 2024). https://doi.org/10.1093/nsr/nwae403

    2024 doi

  85. [94]

    Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, et al. 2023. Disc-lawllm: Fine-tuning large language models for intelligent legal services. arXiv preprint arXiv:2309.11325 (2023)

    2023 arXiv

  86. [95]

    Boxiang Yun, Yan Wang, Jieneng Chen, Huiyu Wang, Wei Shen, and Qingli Li

  87. [96]

    Bingyang Wu, Ruidong Zhu, Zili Zhang, Peng Sun, Xuanzhe Liu, and Xin Jin. 2024. {dLoRA}: Dynamically Orchestrating Requests and Adapters for {LoRA}{ LLM} Serving. In 18th USENIX Symposium on Operating Systems De- sign and Implementation (OSDI 24) . 911–927

    2024

  88. [97]

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adalora: Adap- tive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512 (2023)

    2023 arXiv

  89. [98]

    Yue Zhang, Leyang Cui, Deng Cai, Xinting Huang, Tao Fang, and Wei Bi. 2023. Multi-task instruction tuning of llama for specific scenarios: A preliminary study on writing assistance. arXiv preprint arXiv:2305.13225 (2023)

    2023 arXiv

  90. [99]

    Gonzalez, Clark Barrett, and Ying Sheng

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104 [cs.AI] htt...

    2024 arXiv

  91. [100]

    Zhilin Yang, Ye Yuan, Yuexin Wu, William W Cohen, and Russ R Salakhutdinov

  92. [105]

    arXiv preprint arXiv:2103.03604 (2021)

    Spectr: Spectral transformer for hyperspectral pathology image segmen- tation. arXiv preprint arXiv:2103.03604 (2021)

    2021 arXiv

  93. [106]

    Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024. MM-LLMs: Recent Advances in MultiModal Large Language Models. arXiv:2401.13601 [cs.CL] https://arxiv.org/abs/2401.13601

    2024 arXiv

  94. [110]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-Following Evaluation for Large Language Models. arXiv:2311.07911 [cs.CL] https://arxiv.org/abs/2311.0 7911 A Artifact Appendix A.1 Abstract This artifac...

    2023 arXiv

  95. [2016]

    Advances in neural information processing systems 29 (2016)

    Review networks for caption generation. Advances in neural information processing systems 29 (2016)

    2016

  96. [2017]

    Automatic differentiation in PyTorch. (2017)

    2017

  97. [2020]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems 33 EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices MobiSys ’25, June 23–27, 2025, Anaheim, CA, USA (2020...

    2020

  98. [2021]

    Expert systems with applications 165 (2021), 113679

    Automatic text summarization: A comprehensive survey. Expert systems with applications 165 (2021), 113679

    2021

  99. [2022]

    arXiv:2209.13331 [cs.CL] https://arxiv.org/abs/2209.13331

    EditEval: An Instruction-Based Benchmark for Text Improvements. arXiv:2209.13331 [cs.CL] https://arxiv.org/abs/2209.13331

    arXiv

  100. [2023]

    Cureus 15, 6 (2023)

    Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus 15, 6 (2023)

    2023

  101. [2024]

    arXiv:2406.01574 [cs.CL] https://arxiv.org/abs/2406.01574

    MMLU-Pro: A More Robust and Challenging Multi-Task Language Under- standing Benchmark. arXiv:2406.01574 [cs.CL] https://arxiv.org/abs/2406.01574

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.