pith. sign in

arxiv: 2606.03014 · v1 · pith:F5QGCFW5new · submitted 2026-06-02 · 💻 cs.LG · cs.AR

MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency

Saptarshi Mitra , Yifan Zhang , Rachid Karami , Phyo Pyae Moe Aung , Nazmul Takbir , Sreetama Sarkar , Souvik Kundu , Sitao Huang This is my paper

Pith reviewed 2026-06-28 11:31 UTC · model grok-4.3

classification 💻 cs.LG cs.AR
keywords mixture-of-agentsschedulinginteger linear programmingadaptive aggregationLLM inferenceGPU resource managementload balancinginference optimization
0
0 comments X

The pith

MOSAIC accelerates Mixture-of-Agents inference up to 2.3x on four GPUs via ILP scheduling and adaptive aggregation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Mixture-of-Agents systems route each query through multiple expert LLMs and aggregate their outputs to raise reasoning accuracy, yet skill-based routing and long generation lengths create severe load imbalance and GPU idling under conventional schedulers. MOSAIC counters this by solving an integer linear program that jointly decides expert placement and per-worker prompt assignment using offline cost profiles, replicating reasoning experts while pinning lighter models. It adds a second mechanism that measures inter-expert agreement and skips the heavy final aggregator on consensus queries. The resulting schedule cuts expert-stage time by up to 2.5x and aggregator-stage time by up to 4.23x. A reader would care because the method turns a high-accuracy but resource-heavy pattern into something runnable on modest GPU counts without retraining any model.

Core claim

MOSAIC is an ILP-based scheduler that optimizes expert replication across workers and prompt assignment from profiled costs while using inter-expert agreement to bypass the aggregator LLM on consensus queries, delivering 2.5x expert-stage, 4.23x aggregator-stage, and 1.7-2.3x end-to-end speedups on a 4-GPU system with accuracy loss under 0.1 percentage points.

What carries the argument

Integer Linear Program that jointly optimizes expert placement and prompt assignment from offline costs, paired with agreement-based adaptive aggregation that skips the final aggregator on consensus cases.

If this is right

  • Expert replication across workers directly reduces contention on high-demand reasoning models.
  • Pinning lightweight experts eliminates unnecessary migration overhead during assignment.
  • Bypassing the aggregator on agreed outputs removes the dominant latency stage for many queries.
  • Joint ILP optimization of placement and assignment mitigates the load skew that defeats simpler schedulers.
  • End-to-end MoA latency drops enough to approach single-model throughput while retaining the accuracy gain from multiple experts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same profiled-cost ILP could be re-solved periodically if online monitoring detects sustained deviation from the initial profiles.
  • Agreement-based skipping may transfer to other ensemble inference patterns that already compute multiple outputs.
  • Accuracy preservation within 0.1pp is tied to the specific tasks and models tested; different domains may require a calibrated agreement threshold.
  • Scaling the approach beyond four GPUs would require either faster ILP solvers or decomposition heuristics to keep solve time acceptable.

Load-bearing premise

The offline-profiled costs accurately predict runtime behavior under changing prompt lengths and expert demands, and the inter-expert agreement metric correctly identifies queries where skipping the aggregator preserves final answer quality.

What would settle it

Running the system on a workload whose actual generation lengths or expert demands deviate enough from the profiled values that the ILP schedule produces more idling than a baseline round-robin assignment, or measuring accuracy drop when the aggregator is skipped on a new dataset where high agreement still yields incorrect final answers.

Figures

Figures reproduced from arXiv: 2606.03014 by Nazmul Takbir, Phyo Pyae Moe Aung, Rachid Karami, Saptarshi Mitra, Sitao Huang, Souvik Kundu, Sreetama Sarkar, Yifan Zhang.

Figure 1
Figure 1. Figure 1: Output token distribution (a); and mean out [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Solver popularity skew across model pools; (b) Overall state of GPU idling for MMLU-Pro [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparing data-concurrent replication with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Detailed overview of the MOSAIC framework. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-GPU execution timeline of our schedule (bottom row) and the baseline round-robin baseline (top [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: End-to-end wall-time decomposition. For each benchmark a pair of stacked bars compares the round-robin baseline (left) with MOSAIC (right) yields 4.23× speedup (516.49s to 122.09s). For GPQA, skipping ~87% questions give speedup of 1.53× (348s to 227.73s). Overall Speedup End-to-end pipeline runtime is the sum of expert-generation and aggregator wall time. MOSAIC accelerates the two stages by orthogonal me… view at source ↗
Figure 7
Figure 7. Figure 7: Per-question mean output tokens by agree [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
read the original abstract

Mixture-of-Agents (MoA) systems improve reasoning accuracy by routing each query to multiple expert LLMs and aggregating their outputs. Efficiently executing this workload on limited GPU resources has bottlenecks. Skill-based routing creates skewed expert demand, and combining instruction-tuned LLMs with long-reasoning models results in extreme variability in generation lengths. Consequently, traditional scheduling strategies suffer from significant GPU idling and throughput collapse due to load imbalances. We present MOSAIC, a scheduling framework to accelerate MoA workloads. First, we formulate an Integer Linear Program (ILP) based scheduler that jointly optimizes expert placement and per-worker prompt assignment from offline-profiled costs, replicating reasoning experts across workers while pinning lightweight ones. Second, MOSAIC uses confidence-aware adaptive aggregation, leveraging inter-expert agreement to bypass the heavy final aggregator LLM for consensus queries. In our 4-GPU system, MOSAIC achieves up to 2.5x expert-stage, 4.23x aggregator-stage and 1.7~2.3x end-to-end speedups over the baseline scheduler, while matching accuracy within 0.1pp.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces MOSAIC, a scheduling framework for Mixture-of-Agents (MoA) workloads. It formulates an Integer Linear Program (ILP) that jointly optimizes expert placement and per-worker prompt assignment using offline-profiled costs, with replication of reasoning experts and pinning of lightweight ones. It also proposes confidence-aware adaptive aggregation that skips the final aggregator LLM for high inter-expert agreement queries. On a 4-GPU system, it reports up to 2.5x expert-stage, 4.23x aggregator-stage, and 1.7-2.3x end-to-end speedups over a baseline scheduler while keeping accuracy within 0.1 percentage points.

Significance. If the central claims hold under runtime conditions, the work addresses a practically relevant bottleneck in executing MoA systems on limited hardware by combining static optimization with runtime adaptive skipping. The empirical speedups on a 4-GPU setup provide concrete evidence of throughput gains for mixed expert workloads with variable generation lengths.

major comments (3)
  1. [Section 3.2] Section 3.2 (ILP formulation): The scheduler relies on offline-profiled costs for joint expert placement and prompt assignment. Given the abstract's emphasis on extreme variability in generation lengths from mixing model types, no sensitivity analysis or online adjustment mechanism is described to handle deviations between profiled and actual per-prompt execution times; this directly undermines the realizability of the reported 2.5x and 1.7-2.3x speedups.
  2. [Section 4.3] Section 4.3 (experimental results): The accuracy claim of matching within 0.1pp when skipping the aggregator is presented without per-query-type breakdowns, failure-case analysis, or controls for prompt-length variability; this makes it impossible to verify that the inter-expert agreement metric reliably identifies safe consensus cases under the same load-imbalance conditions that motivate the ILP.
  3. [Section 4.1] Section 4.1 (baseline and setup): The baseline scheduler is not defined with sufficient detail (e.g., whether it uses the same profiled costs or a simpler heuristic), and no error bars, multiple random seeds, or statistical significance tests are reported for the speedup numbers; this weakens the load-bearing empirical claims.
minor comments (2)
  1. [Section 3] Notation for the ILP variables and the inter-expert agreement metric could be introduced more clearly with a dedicated table or equation list.
  2. [Section 4] Figure captions for the speedup plots should explicitly state the number of runs and any variability measures.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below with clarifications from the manuscript and indicate planned revisions.

read point-by-point responses

  1. Referee: [Section 3.2] Section 3.2 (ILP formulation): The scheduler relies on offline-profiled costs for joint expert placement and prompt assignment. Given the abstract's emphasis on extreme variability in generation lengths from mixing model types, no sensitivity analysis or online adjustment mechanism is described to handle deviations between profiled and actual per-prompt execution times; this directly undermines the realizability of the reported 2.5x and 1.7-2.3x speedups.

    Authors: The reported speedups (2.5x expert-stage, 1.7-2.3x end-to-end) are measured from actual executions of the ILP-generated schedules on the 4-GPU system. The ILP produces a static placement and assignment using profiled means, after which runtime execution—including variable generation lengths—occurs; the replication of reasoning experts is explicitly intended to mitigate resulting imbalances. Because the speedups reflect real runs rather than simulated profiles, the approach is realizable under the evaluated conditions. We will add a sensitivity analysis subsection quantifying ILP robustness to profile deviations. revision: yes

  2. Referee: [Section 4.3] Section 4.3 (experimental results): The accuracy claim of matching within 0.1pp when skipping the aggregator is presented without per-query-type breakdowns, failure-case analysis, or controls for prompt-length variability; this makes it impossible to verify that the inter-expert agreement metric reliably identifies safe consensus cases under the same load-imbalance conditions that motivate the ILP.

    Authors: The 0.1pp accuracy figure is an aggregate result across the full evaluation set. We agree that finer-grained validation would strengthen the claim regarding the agreement metric under load imbalance. In the revision we will add per-query-type accuracy breakdowns, failure-case analysis for skipped aggregations, and controls that stratify by prompt length. revision: yes

  3. Referee: [Section 4.1] Section 4.1 (baseline and setup): The baseline scheduler is not defined with sufficient detail (e.g., whether it uses the same profiled costs or a simpler heuristic), and no error bars, multiple random seeds, or statistical significance tests are reported for the speedup numbers; this weakens the load-bearing empirical claims.

    Authors: We will expand Section 4.1 to explicitly describe the baseline as a non-ILP scheduler (round-robin assignment without joint placement optimization or profiled-cost modeling). For the empirical results, we will include error bars derived from multiple random seeds and report statistical significance tests in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity: speedups are measured runtime results from ILP using external profiled inputs

full rationale

The paper formulates an ILP scheduler that takes offline-profiled costs as fixed inputs to optimize placement and assignment, then reports empirically measured speedups (2.5x expert-stage, etc.) on a 4-GPU system. No equations, fitted parameters, or self-citations are shown that reduce the claimed speedups or accuracy preservation to definitions or inputs by construction. The profiled costs are external measurements, not derived from the ILP output itself. The adaptive aggregation uses an inter-expert agreement metric as a decision rule, but this is a runtime heuristic whose accuracy impact is measured separately rather than defined into the result. No load-bearing self-citation chains or ansatzes appear in the provided text. This is a standard empirical systems paper whose central claims rest on implementation and benchmarking, not internal redefinition.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no free parameters, axioms, or invented entities can be identified from the provided text.

pith-pipeline@v0.9.1-grok · 5762 in / 1220 out tokens · 32947 ms · 2026-06-28T11:31:53.668184+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

223 extracted references · 23 canonical work pages · 4 internal anchors

  1. [1]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,

    Ainslie, Joshua and Lee-Thorp, James and de Jong, Michiel and Zemlyanskiy, Yury and Lebron, Federico and Sanghai, Sumit. GQA : Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.298

    work page doi:10.18653/v1/2023.emnlp-main.298 2023

  2. [2]

    HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin. H ella S wag: Can a Machine Really Finish Your Sentence?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1472

    work page doi:10.18653/v1/p19-1472 2019

  3. [3]

    Sakaguchi, Keisuke and Bras, Ronan Le and Bhagavatula, Chandra and Choi, Yejin , title =. Commun. ACM , month = aug, pages =. 2021 , issue_date =. doi:10.1145/3474381 , abstract =

    work page doi:10.1145/3474381 2021

  4. [4]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    Pith/arXiv arXiv

  5. [5]

    T ruthful QA : Measuring How Models Mimic Human Falsehoods

    Lin, Stephanie and Hilton, Jacob and Evans, Owain. T ruthful QA : Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.229

    work page doi:10.18653/v1/2022.acl-long.229 2022

  6. [6]

    arXiv preprint arXiv:2503.22879 , year=

    Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models , author=. arXiv preprint arXiv:2503.22879 , year=

    arXiv

  7. [7]

    arXiv preprint arXiv:2511.18890 , year=

    Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models , author=. arXiv preprint arXiv:2511.18890 , year=

    arXiv

  8. [8]

    arXiv preprint arXiv:2507.22448 , year=

    Falcon-h1: A family of hybrid-head language models redefining efficiency and performance , author=. arXiv preprint arXiv:2507.22448 , year=

    arXiv

  9. [9]

    Ninth Annual Conference on Machine Learning and Systems , year=

    Skipkv: Selective skipping of kv generation and storage for efficient inference with large reasoning models , author=. Ninth Annual Conference on Machine Learning and Systems , year=

  10. [11]

    Logical and Symbolic Reasoning in Language Models@ AAAI 2026 , year=

    Don’t Overthink It: Detecting and Truncating Overthinking in LRMs with Lightweight DeBERTa , author=. Logical and Symbolic Reasoning in Language Models@ AAAI 2026 , year=

    2026

  11. [12]

    International Conference on Machine Learning , year=

    Symbolic mixture-of-experts: Adaptive skill-based routing for heterogeneous reasoning , author=. International Conference on Machine Learning , year=

  12. [13]

    International Conference on Learning Representations , volume=

    Mixture-of-agents enhances large language model capabilities , author=. International Conference on Learning Representations , volume=

  13. [14]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Reconcile: Round-table conference improves reasoning via consensus among diverse llms , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  14. [15]

    Forty-first international conference on machine learning , year=

    Improving factuality and reasoning in language models through multiagent debate , author=. Forty-first international conference on machine learning , year=

  15. [16]

    2025 , url=

    Tianzhe Chu and Yuexiang Zhai and Jihan Yang and Shengbang Tong and Saining Xie and Dale Schuurmans and Quoc V Le and Sergey Levine and Yi Ma , booktitle=. 2025 , url=

    2025

  16. [18]

    International Conference on Learning Representations , volume=

    Metamath: Bootstrap your own mathematical questions for large language models , author=. International Conference on Learning Representations , volume=

  17. [19]

    International Conference on Learning Representations , volume=

    Fusing models with complementary expertise , author=. International Conference on Learning Representations , volume=

  18. [20]

    On the impact of fine-tuning on chain-of-thought reasoning , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  19. [22]

    BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack , url =

    Kuratov, Yuri and Bulatov, Aydar and Anokhin, Petr and Rodkin, Ivan and Sorokin, Dmitry and Sorokin, Artyom and Burtsev, Mikhail , booktitle =. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack , url =

  20. [23]

    M ini KV : Pushing the Limits of 2-Bit KV Cache via Compression and System Co-Design for Efficient Long Context Inference

    Sharma, Akshat and Ding, Hangliang and Li, Jianping and Dani, Neel and Zhang, Minjia. M ini KV : Pushing the Limits of 2-Bit KV Cache via Compression and System Co-Design for Efficient Long Context Inference. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.952

    work page doi:10.18653/v1/2025.findings-acl.952 2025

  21. [24]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  22. [25]

    B ench: Extending Long Context Evaluation Beyond 100 K Tokens

    Zhang, Xinrong and Chen, Yingfa and Hu, Shengding and Xu, Zihang and Chen, Junhao and Hao, Moo and Han, Xu and Thai, Zhen and Wang, Shuo and Liu, Zhiyuan and Sun, Maosong. B ench: Extending Long Context Evaluation Beyond 100 K Tokens. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi...

    work page doi:10.18653/v1/2024.acl-long.814 2024

  23. [26]

    2025 , eprint=

    NonGEMM Bench: Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads , author=. 2025 , eprint=

    2025

  24. [27]

    2022 , eprint=

    Efficiently Modeling Long Sequences with Structured State Spaces , author=. 2022 , eprint=

    2022

  25. [28]

    Gu, Albert and Dao, Tri , year =. Mamba:. doi:10.48550/arXiv.2312.00752 , urldate =. arXiv , keywords =:2312.00752 , primaryclass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2312.00752

  26. [29]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Park, Jongho and Park, Jaeseung and Xiong, Zheyang and Lee, Nayoung and Cho, Jaewoong and Oymak, Samet and Lee, Kangwook and Papailiopoulos, Dimitris , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  27. [30]

    Dong, Xin and Fu, Yonggan and Diao, Shizhe and Byeon, Wonmin and Chen, Zijia and Mahabaleshwarkar, Ameya Sunil and Liu, Shih-Yang and Keirsbilck, Matthijs Van and Chen, Min-Hung and Suhara, Yoshi and Lin, Yingyan and Kautz, Jan and Molchanov, Pavlo , year =. Hymba:. doi:10.48550/arXiv.2411.13676 , urldate =. arXiv , keywords =:2411.13676 , primaryclass =

    work page doi:10.48550/arxiv.2411.13676

  28. [31]

    arXiv preprint arXiv:2411.15242 , year=

    The Zamba2 Suite: Technical Report , author=. arXiv preprint arXiv:2411.15242 , year=

    arXiv

  29. [32]

    Glorioso, Paolo and Anthony, Quentin and Tokpanov, Yury and Whittington, James and Pilault, Jonathan and Ibrahim, Adam and Millidge, Beren , year =. Zamba:. doi:10.48550/arXiv.2405.16712 , urldate =. arXiv , keywords =:2405.16712 , primaryclass =

    work page doi:10.48550/arxiv.2405.16712

  30. [33]

    2021 , eprint=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. 2021 , eprint=

    2021

  31. [34]

    2023 , eprint=

    RoFormer: Enhanced Transformer with Rotary Position Embedding , author=. 2023 , eprint=

    2023

  32. [35]

    2024 , eprint=

    Repeat After Me: Transformers are Better than State Space Models at Copying , author=. 2024 , eprint=

    2024

  33. [36]

    2021 , eprint=

    Measuring Massive Multitask Language Understanding , author=. 2021 , eprint=

    2021

  34. [37]

    doi:10.48550/arXiv.2407.08083 , urldate =

    Hatamizadeh, Ali and Kautz, Jan , year =. doi:10.48550/arXiv.2407.08083 , urldate =. arXiv , keywords =:2407.08083 , primaryclass =

    work page doi:10.48550/arxiv.2407.08083

  35. [38]

    Ganji, Darshan C. and Ashfaq, Saad and Saboori, Ehsan and Sah, Sudhakar and Mitra, Saptarshi and AskariHemmat, MohammadHossein and Hoffman, Alexander and Hassanien, Ahmed and L\'eonardon, Mathieu , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , month = jun, year =

  36. [39]

    Accelerating Semantic Image Segmentation on FPGA , year =

    Mitra, Saptarshi , booktitle =. Accelerating Semantic Image Segmentation on FPGA , year =

  37. [40]

    Accelerating and Pruning CNNs for Semantic Segmentation on FPGA , year =

    Mor\`. Accelerating and Pruning CNNs for Semantic Segmentation on FPGA , year =. doi:10.1145/3489517.3530424 , booktitle =

    work page doi:10.1145/3489517.3530424

  38. [41]

    arXiv preprint arXiv:2601.02346 , year=

    Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling , author=. arXiv preprint arXiv:2601.02346 , year=

    arXiv

  39. [42]

    Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture , pages=

    HLX: A Unified Pipelined Architecture for Optimized Performance of Hybrid Transformer-Mamba Language Models , author=. Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture , pages=

  40. [43]

    GPU gems , volume=

    Parallel prefix sum (scan) with CUDA , author=. GPU gems , volume=

  41. [44]

    2017 , eprint=

    Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer , author=. 2017 , eprint=

    2017

  42. [45]

    2022 , eprint=

    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity , author=. 2022 , eprint=

    2022

  43. [46]

    Lieber, Opher and Lenz, Barak and Bata, Hofit and Cohen, Gal and Osin, Jhonathan and Dalmedigos, Itay and Safahi, Erez and Meirom, Shaked and Belinkov, Yonatan and. Jamba:. 2024 , month = mar, number =. doi:10.48550/arXiv.2403.19887 , urldate =. arXiv , keywords =:2403.19887 , primaryclass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2403.19887 2024

  44. [47]

    Nemotron-h: A family of accurate and efficient hybrid mamba-transformer models.arXiv preprint arXiv:2504.03624, 2025

    NVIDIA , year =. Nemotron-. doi:10.48550/arXiv.2504.03624 , urldate =. arXiv , keywords =:2504.03624 , primaryclass =

    work page doi:10.48550/arxiv.2504.03624

  45. [48]

    arXiv:2004.05150 , year=

    Longformer: The Long-Document Transformer , author=. arXiv:2004.05150 , year=

    Pith/arXiv arXiv 2004

  46. [49]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Informer: Beyond efficient transformer for long sequence time-series forecasting , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  47. [50]

    Srihari , title=

    Navid Madani and Sougata Saha and Rohini K. Srihari , title=. CoRR , volume=. 2024 , cdate=

    2024

  48. [51]

    arXiv preprint arXiv:2504.15965 , year=

    From human memory to ai memory: A survey on memory mechanisms in the era of llms , author=. arXiv preprint arXiv:2504.15965 , year=

    Pith/arXiv arXiv

  49. [52]

    arXiv preprint arXiv:2410.15665 , year=

    Long term memory: The foundation of ai self-evolution , author=. arXiv preprint arXiv:2410.15665 , year=

    arXiv

  50. [53]

    arXiv preprint arXiv:2403.08295 , year=

    Gemma: Open models based on gemini research and technology , author=. arXiv preprint arXiv:2403.08295 , year=

    Pith/arXiv arXiv

  51. [54]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  52. [57]

    GitHub repository , howpublished =

    Gu, Albert and Dao, Tri , title =. GitHub repository , howpublished =. 2023 , publisher =

    2023

  53. [58]

    GitHub repository , howpublished =

    Tencent Team , title =. GitHub repository , howpublished =. 2025 , publisher =

    2025

  54. [59]

    arXiv preprint arXiv:2412.16720 , year=

    Openai o1 system card , author=. arXiv preprint arXiv:2412.16720 , year=

    Pith/arXiv arXiv

  55. [60]

    ArXiv , year=

    LLaMA: Open and Efficient Foundation Language Models , author=. ArXiv , year=

  56. [61]

    2023 IEEE Symposium on High-Performance Interconnects (HOTI) , pages=

    Performance characterization of large language models on high-speed interconnects , author=. 2023 IEEE Symposium on High-Performance Interconnects (HOTI) , pages=. 2023 , organization=

    2023

  57. [62]

    arXiv preprint arXiv:1811.09886 , year=

    Deep learning inference in facebook data centers: Characterization, performance optimizations and hardware implications , author=. arXiv preprint arXiv:1811.09886 , year=

    Pith/arXiv arXiv

  58. [63]

    arXiv preprint arXiv:2307.13702 , year=

    Measuring faithfulness in chain-of-thought reasoning , author=. arXiv preprint arXiv:2307.13702 , year=

    Pith/arXiv arXiv

  59. [64]

    2020 , eprint=

    Longformer: The Long-Document Transformer , author=. 2020 , eprint=

    2020

  60. [65]

    2024 IEEE International Symposium on Workload Characterization (IISWC) , pages=

    Understanding Performance Implications of LLM Inference on CPUs , author=. 2024 IEEE International Symposium on Workload Characterization (IISWC) , pages=. 2024 , organization=

    2024

  61. [66]

    18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) , pages=

    \ InfiniGen \ : Efficient generative inference of large language models with dynamic \ KV \ cache management , author=. 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) , pages=

  62. [67]

    FlexGen: high-throughput generative inference of large language models with a single GPU , year =

    Sheng, Ying and Zheng, Lianmin and Yuan, Binhang and Li, Zhuohan and Ryabinin, Max and Chen, Beidi and Liang, Percy and R\'. FlexGen: high-throughput generative inference of large language models with a single GPU , year =. Proceedings of the 40th International Conference on Machine Learning , articleno =

  63. [68]

    Proceedings of the 5th Workshop on Machine Learning and Systems , pages =

    Du, Hongchao and Wu, Shangyu and Kharlamova, Arina and Guan, Nan and Xue, Chun Jason , title =. Proceedings of the 5th Workshop on Machine Learning and Systems , pages =. 2025 , isbn =. doi:10.1145/3721146.3721961 , abstract =

    work page doi:10.1145/3721146.3721961 2025

  64. [69]

    Black, G

    Black, Sid and Gao, Leo and Wang, Phil and Leahy, Connor and Biderman, Stella , title =. doi:10.5281/zenodo.5297715 , url =

    work page doi:10.5281/zenodo.5297715

  65. [70]

    arXiv preprint arXiv:2401.02385 , year=

    Tinyllama: An open-source small language model , author=. arXiv preprint arXiv:2401.02385 , year=

    Pith/arXiv arXiv

  66. [71]

    and Ermon, Stefano and Rudra, Atri and R\'

    Dao, Tri and Fu, Daniel Y. and Ermon, Stefano and Rudra, Atri and R\'. FLASHATTENTION: fast and memory-efficient exact attention with IO-awareness , year =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =

  67. [72]

    and Venieris, Iakovos S

    Panopoulos, Ioannis and Nikolaidis, Sokratis and Venieris, Stylianos I. and Venieris, Iakovos S. , booktitle =. 2023 , volume =. doi:10.1109/ISCC58397.2023.10217850 , url =

    work page doi:10.1109/iscc58397.2023.10217850 2023

  68. [73]

    Ren, Liliang and Liu, Yang and Lu, Yadong and Shen, Yelong and Liang, Chen and Chen, Weizhu , year =. Samba:. doi:10.48550/arXiv.2406.07522 , urldate =. arXiv , keywords =:2406.07522 , primaryclass =

    work page doi:10.48550/arxiv.2406.07522

  69. [74]

    Team, Jamba and Lenz, Barak and Arazi, Alan and Bergman, Amir and Manevich, Avshalom and Peleg, Barak and Aviram, Ben and Almagor, Chen and Fridman, Clara and Padnos, Dan and Gissin, Daniel and Jannai, Daniel and Muhlgay, Dor and Zimberg, Dor and Gerber, Edden M. and Dolev, Elad and Krakovsky, Eran and Safahi, Erez and Schwartz, Erez and Cohen, Gal and Sh...

    work page doi:10.48550/arxiv.2408.12570

  70. [75]

    Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

    Dao, Tri and Gu, Albert , year =. Transformers Are. doi:10.48550/arXiv.2405.21060 , urldate =. arXiv , keywords =:2405.21060 , primaryclass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.21060

  71. [76]

    Waleffe, Roger and Byeon, Wonmin and Riach, Duncan and Norick, Brandon and Korthikanti, Vijay and Dao, Tri and Gu, Albert and Hatamizadeh, Ali and Singh, Sudhakar and Narayanan, Deepak and Kulshreshtha, Garvit and Singh, Vartika and Casper, Jared and Kautz, Jan and Shoeybi, Mohammad and Catanzaro, Bryan , year =. An. doi:10.48550/arXiv.2406.07887 , urldat...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2406.07887

  72. [77]

    Brown and Benjamin Chess and Rewon Child and Scott Gray and Alec Radford and Jeffrey Wu and Dario Amodei , title =

    Jared Kaplan and Sam McCandlish and Tom Henighan and Tom B. Brown and Benjamin Chess and Rewon Child and Scott Gray and Alec Radford and Jeffrey Wu and Dario Amodei , title =. CoRR , volume =. 2020 , url =. 2001.08361 , timestamp =

    Pith/arXiv arXiv 2020

  73. [78]

    arXiv preprint arXiv:2304.04675 , year=

    Multilingual machine translation with large language models: Empirical results and analysis , author=. arXiv preprint arXiv:2304.04675 , year=

    arXiv

  74. [79]

    and Kavukcuoglu, Koray and Kohli, Pushmeet and Hassabis, Demis , year =

    Jumper, John and Evans, Richard and Pritzel, Alexander and Green, Tim and Figurnov, Michael and Ronneberger, Olaf and Tunyasuvunakool, Kathryn and Bates, Russ and. Highly Accurate Protein Structure Prediction with. 2021 , month = aug, journal =. doi:10.1038/s41586-021-03819-2 , abstract =

    work page doi:10.1038/s41586-021-03819-2 2021

  75. [80]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  76. [81]

    2024 , eprint=

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces , author=. 2024 , eprint=

    2024

  77. [82]

    arXiv preprint arXiv:2405.21060 , year=

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality , author=. arXiv preprint arXiv:2405.21060 , year=

    Pith/arXiv arXiv

  78. [83]

    Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=

    Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness , author=. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=. 2017 , publisher =. doi:10.1109/IROS.2017.8206285 , address =

    work page doi:10.1109/iros.2017.8206285 2017

  79. [84]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Deep residual learning for image recognition , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

  80. [85]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

Showing first 80 references.