REVIEW 2 major objections 5 minor 42 references
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read AFD-Ledger claims that when both Attention–FFN disaggregated and collocated deployments are provisioned optimally under the same fixed budget, disaggregation raises throughput in only a minority of studied settings, and a 20-assignment…
desk verdict A solid, honest provisioning study whose headline deployment insights rest on a thinner and differently-stacked physical validation than the claims suggest; worth serious refereeing, but the authors should either broaden validation or temper conclusions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the fixed-budget performance ledger: steady-state decode throughput is $\Theta = N_r b / T$, where $N_r$ is the number of request-serving GPUs, $b$ the resident requests per GPU, and $T$ the TPOT; an AFD-versus-collocated comparison is the product of the ratios of these three factors under a common specification $\Omega$. On top of this, AFD-Ledger uses an analytical execution model (stage latency is the slower of peak-compute time and HBM-transfer time, with pipeline microbatches for AFD) to score deployments, and a two-level search: role-specific hardware estimates rank candidate attention-FFN pairs, a bounded refinement loop fully provisions only up to $Q=20$ assignments, and each assignment gets a complete deployment optimization over EP width, replication, batching, worker ratio, and pipeline depth. The search machinery is what lets the paper claim architecture decisions with 68.8% to 83.5% fewer complete evaluations while still recovering exhaustive optima.
What would settle it
Measure stage-level TPOT and end-to-end throughput for Qwen3-235B and DeepSeek-V3.2 on the H200/A100 PCIe/L40S catalog at the 1K and 4K contexts used in the paper, and compare architecture decisions with AFD-Ledger's predictions; if any setting where the model predicts AFD wins by more than roughly 10% actually shows collocated serving winning (or vice versa), the ordering-preservation assumption that carries the 7-of-36 and complementarity conclusions fails.
Extended reading notes
Core claim
The central discovery is that AFD's reported gains do not automatically translate into deployment-level throughput. The paper reproduces two published AFD results analytically (a 1.404x reconstructed gain versus 1.414x measured for one Qwen3 deployment, and 1.463x versus 1.574x for a DBRX heterogeneous deployment), then shows that after both AFD and collocated families are reprovisioned from scratch under the same specification, the advantage shrinks to near ties (0.990x and 0.947x respectively). Across 36 homogeneous E-grid settings, AFD-Ledger selects AFD in only 7 cases; heterogeneous AFD can reach 1.815x to 1.823x over collocation but only for specific pairs, and the same H200+A100 PCIe pair ranges from 0.483x to 1.815x depending on budget and SLO. The paper's conclusion is that AFD's benefits are deployment-level: they materialize only when expert-memory removal, larger batches, and lower TPOT repay the request-bearing-capacity tax, and future role-specific hardware matters primarily when it unlocks new replica layouts or worker ratios rather than improving one component in isolation.
Load-bearing premise
The analytical execution model, which estimates each stage's latency as the slower of peak compute time and HBM transfer time and omits kernel deficiencies, scheduler latency, and network contention, is accurate enough to preserve which architecture wins across all studied models and catalogs, even though it was validated on only three LongCat 2.0 workloads with 6.6–9.6% throughput error.
Editorial extensions
If this is right
- Operators comparing AFD with collocated serving must reprovision both architectures under the same specification; mechanism-level gains alone do not determine which deployment serves more tokens.
- Homogeneous AFD is the exception, not the rule: in the studied space it wins 7 of 36 settings, and only where batch and TPOT gains repay the request-bearing-capacity tax.
- Heterogeneous AFD can deliver large gains (up to roughly 1.8x) but the winning hardware pair is workload- and budget-dependent; the same H200+A100 PCIe pair ranges from 0.483x to 1.815x.
- Role-specific hardware improvements (cost, memory capacity) are most valuable when they change the deployment organization, such as enabling more replicas; in one DeepSeek case crossing a capability-price boundary raised throughput 19.59x.
- Bounded hardware search with 20 assignments recovers the exhaustive optimum on both tested catalogs, cutting complete deployment evaluations by 68.8% to 83.5%.
Reading between the lines
- If the ordering-preserving property of the analytical model extends beyond the three validated LongCat 2.0 workloads, the same provisioning framework could be applied to other disaggregation architectures (prefill-decode separation, KV-centric pools) that have not yet been compared under full re-provisioning.
- The 7-of-36 win rate is a property of the studied catalogs and SLOs, not a universal bound; the paper's own overhead sweep shows decisions flip to collocated around 20% additional AFD runtime overhead, so tighter SLOs or slower interconnects could shrink AFD's wins further.
- A testable co-design consequence: future AFD-specialized accelerators should be evaluated by the deployments they unlock (replicas, worker ratios, batch sizes) rather than by raw FLOP/s or bandwidth, which suggests memory-capacity-per-dollar and price targets deserve more weight than peak compute.
- A practical extension would be to run the same pair-provisioning comparison on real heterogeneous clusters with more workloads to measure whether the 6.6–9.6% model error window holds outside LongCat; if it does, exhaustive search can be replaced by AFD-Ledger's 20-assignment search with confidence.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents AFD-Ledger, an offline provisioning system that decides, under a fixed model, workload, TPOT SLO, hardware budget, hardware catalog, and runtime assumptions, whether the best Attention–FFN Disaggregation (AFD) deployment or the best collocated deployment achieves higher SLO-feasible decode throughput. The system uses an analytical execution model in which each stage's latency is the maximum of compute time and HBM transfer time, and it searches over hardware assignments with a bounded number of full deployment evaluations. The authors reconstruct two existing AFD results (FastAFD and MegaScale-Infer), show that bounded search recovers the exhaustive analytical optimum on the enumerated cases, validate the analytical model against three physical LongCat 2.0 workloads on Ascend 910C, and derive three deployment insights: homogeneous AFD wins in only 7 of 36 studied settings, heterogeneous AFD requires deployment-level hardware complementarity, and role-specific hardware improvements matter mainly when they enable new deployment organizations.
Significance. If the analytical model is sufficiently faithful, AFD-Ledger is a useful contribution to MoE serving: it upgrades the AFD discussion from mechanism-level evaluations to a full-deployment provisioning comparison, and it gives operators a concrete way to decide when disaggregation is worth the request-bearing-capacity tax. The paper is careful to distinguish reported deployments from independently reprovisioned ones, and it honestly labels retrospective reconstructions that rely on appointed parameters. The strengths are real: the FastAFD reconstruction matches the reported result within 0.7%, the MegaScale proxy within 7.1%, the physical LongCat validation preserves the architecture decision on all three workloads, and bounded search recovers the exhaustive optimum in the eighteen enumerated E-grid cases and all five reference cases. The main limitation is that the headline deployment conclusions are generated by an analytical model that is directly validated on only three workloads from a different model family and hardware stack; this is the load-bearing assumption of the paper and needs to be addressed before the 7-of-36 finding can be taken as a general result.
major comments (2)
- [Sections 5.2.1, 5.4.2, and 6] The analytical execution model of Sections 2.1 and 2.4 is validated only on three LongCat 2.0 workloads on Ascend 910C, while the central deployment findings use Qwen3-235B and DeepSeek-V3.2 on an NVIDIA catalog (H200, H100, A100, L40S, L4). The measured AFD-to-collocated throughput error is 6.6–9.6%, and Figure 14 shows that decisions degrade at roughly 10% additional AFD overhead and flip to collocated around 20%. The margin between the measured error and the flip point is therefore thin, and the robustness test injects a single uniform overhead rather than per-stage or per-device errors from the omitted effects (kernel deficiency, scheduler latency, network contention). The paper explicitly flags model fidelity as a limitation in Section 6, but that limitation is not a peripheral caveat: it is the premise on which the 7-of-36 result and the heterogeneous complementarity conclusions rest. I would like to see either direct validation on at least one Qwen or DeepSeek workload using the relevant NVIDIA devices, or a structured sensitivity analysis that applies per-stage/per-device error perturbations large enough to bracket the 6.6–9.6% validation error plus the omitted runtime overheads. Without such evidence, the deployment insights should be reported as conditional on the analytical model's fidelity rather than as robust findings.
- [Sections 5.2.2 and 5.3] The bounded hardware search is validated by showing that it recovers the best deployment found by exhaustive enumeration of hardware assignments. This is a valid test of search efficiency, but it is a search test, not an accuracy test: the 'global optimum' being recovered is the optimum of the same analytical model used to evaluate all deployments. The abstract and Section 5.2.2 should state this more carefully, because a reader can easily misread 'globally optimal deployment' as a claim about the physical optimum. The actual physical accuracy is established only by the three LongCat 2.0 runs, so the search-quality claims and the model-fidelity claims should be kept separate.
minor comments (5)
- [Section 2.4] Equation (5) and the surrounding text define Θ*_C and Θ*_A but the sentence 'where D_C(Ω) and D_A(Ω) are the legal collocated and AFD plans' is missing a closing period and should be completed.
- [Section 2.4 and Figure 4] The near-tie threshold is described as the error 'observed in physical validation,' but no precise value or rule is given. Please state the actual numerical threshold used to classify a result as a near tie.
- [Section 5.2.2] The text repeats the sentence 'We next evaluate whether bounded hardware exploration recovers exhaustive provisioning...' immediately before Figure 8; one of the two occurrences should be removed.
- [Section 3] The first sentence of Section 3 starts with a lowercase 'it' after the preceding period: 'it first uses inexpensive role-specific estimates...' should be capitalized.
- [Section 2.2] The MegaScale-Infer reconstruction relies on appointed parameters (the 64-L20-equivalent budget, KV configuration, and SLO) that are not reported in the source; the 7.1% agreement is therefore a proxy check rather than a pure validation. The paper labels this correctly in the text, but the abstract's phrase 'reproduces reported gains' should not be read as stronger than the proxy methodology supports.
Circularity Check
No circularity found: the provisioning objective and analytical model are stated independently of the conclusions, and the model is checked against external reports and a physical runtime.
full rationale
AFD-Ledger's derivation chain is self-contained. The throughput objective (Eq. 4) and comparison objective (Eq. 5) are stated from a fixed deployment specification and do not encode the AFD-vs-collocated answer; the same objective and the same analytical execution model are applied to both deployment families. The analytical model (Section 2.1) is not fitted to the conclusions: it uses published model configurations, hardware vendor specifications, and reported operator statistics, and it reproduces externally reported gains (FastAFD 1.404 vs 1.414, MegaScale-Infer proxy 1.463 vs 1.574). The only calibrated quantity, the near-tie band, is taken from the measured 6.6-9.6% physical-validation error and used conservatively to downgrade small analytical differences; it is not used to manufacture a win. The bounded-search claim is checked against exhaustive enumeration under the same model, which is the appropriate ground truth for search quality, and the model itself is separately checked against a physical LongCat 2.0 runtime. Self-citations (e.g., refs [4], [23], [30]) appear in related work and runtime background, not as load-bearing premises for the 7-of-36 or complementarity findings. Section 6 explicitly flags model fidelity and heuristic-search limitations, which are correctness risks, not circular steps. Therefore no circularity is established.
Assumptions & free parameters
free parameters (5)
- Search evaluation budget Q =
20 assignments
- Retention width w and expansion width x =
w=2, x=6
- Refinement-round limit K =
3
- Near-tie threshold =
6.6 percent to 9.6 percent throughput error
- MegaScale proxy appointments =
64 L20-equivalent budget, 571-token context, 150 ms TPOT SLO
assumptions (6)
- domain assumption Steady-state saturated decode throughput equals resident requests divided by TPOT, as in Equation 1.
- domain assumption Stage latency is the slower of peak-compute time and HBM-transfer time, with simple dispatch and combine interconnect costs; no kernel deficiency, scheduler latency, or network contention.
- domain assumption Hardware catalog prices and specifications in Table 1 are accurate market-normalized inputs.
- domain assumption Model weight sizes, KV footprints, and operator FLOP and HBM statistics for Qwen3, DeepSeek, DBRX, and LongCat are correct.
- domain assumption Error measured on three LongCat workloads bounds analytical error for other models, catalogs, and workloads.
- domain assumption Exhaustive enumeration of legal deployment configurations within the deployment specification is complete and correct.
Cite this review
Pith. "Pith review of AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation." pith.science (2026). https://pith.science/paper/WBA2SRVR
@misc{pith2026260804502,
author = {Pith},
title = {Pith review of: AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation},
year = {2026},
howpublished = {\url{https://pith.science/paper/WBA2SRVR}},
note = {Machine review of arXiv:2608.04502}
}
read the original abstract
Attention--Feed-Forward Network (FFN) Disaggregation (AFD) is emerging as a promising architecture for serving Mixture-of-Experts (MoE) language models. While existing AFD systems improve the efficiency of disaggregated execution, they leave a deployment question unanswered: under the same model, workload, time-per-output-token (TPOT) service-level objective (SLO), hardware budget, hardware catalog, and runtime capabilities, does AFD provide higher throughput than the best collocated deployment? Answering this question requires jointly optimizing hardware assignment and deployment organization for both architectures, making exhaustive provisioning prohibitively expensive. We present AFD-Ledger, an offline analytical provisioning system that independently provisions AFD and collocated deployments using an analytical execution model and an evaluation-bounded hardware search. Across deployment spaces where exhaustive provisioning is feasible, AFD-Ledger reduces complete deployment evaluations by 68.8%--83.5% while still recovering the globally optimal deployment. On three physical LongCat 2.0 deployments, it preserves the correct architecture decision while predicting AFD-to-collocated throughput within 6.6%--9.6% of measurement. Using this validated framework, we show that homogeneous AFD improves fixed-budget throughput in only a minority of the studied settings, heterogeneous AFD requires deployment-level hardware complementarity rather than heuristic device selection, and role-specific hardware improvements matter primarily when they enable better deployment organizations by crossing deployment capability--price boundaries.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
RunPod GPU Cloud Pricing.https://www.runpod.io/pricing
2026. RunPod GPU Cloud Pricing.https://www.runpod.io/pricing. Accessed July 5, 2026
work page 2026
-
[2]
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ram- jee. 2024. Taming{Throughput-Latency} tradeoff in{LLM} inference with{Sarathi-Serve}. In18th USENIX symposium on operating systems design and implementation (OSDI 24). 117–134
work page 2024
-
[3]
Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S Gulavani, and Ramachandran Ramjee. 2023. Sarathi: Effi- cient llm inference by piggybacking decodes with chunked prefills. arXiv preprint arXiv:2308.16369(2023)
arXiv 2023
-
[4]
Shaoyuan Chen, Wencong Xiao, Yutong Lin, Mingxing Zhang, Yingdi Shan, Jinlei Jiang, Kang Chen, and Yongwei Wu. 2024. Efficient het- erogeneous large language model decoding with model-attention dis- aggregation.arXiv preprint arXiv:2405.01814(2024)
arXiv 2024
-
[5]
William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch trans- formers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research23, 120 (2022), 1–39
2022
-
[6]
Yicheng Feng, Xin Tan, Yangtao Deng, Yimin Jiang, Yibo Zhu, and Hong Xu. 2026. Frontier: Towards Comprehensive and Accurate LLM Inference Simulation.arXiv preprint arXiv:2605.21312(2026)
arXiv 2026
-
[7]
Yichao Fu, Yuxuan Zhang, Ruitian Wang, Junda Chen, and Hao Zhang
-
[8]
Yipin Guo and Siddharth Joshi. 2026. SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving.arXiv preprint arXiv:2605.01708(2026)
arXiv 2026
Show all 42 references
-
[9]
Jiaao He, Jiezhong Qiu, Aohan Zeng, Zhilin Yang, Jidong Zhai, and Jie Tang. 2021. Fastmoe: A fast mixture-of-expert training system.arXiv preprint arXiv:2103.13262(2021)
2021 arXiv
-
[10]
Cunchen Hu, Heyang Huang, Junhao Hu, Jiang Xu, Xusheng Chen, Tao Xie, Chenxi Wang, Sa Wang, Yungang Bao, Ninghui Sun, et al
-
[11]
Cunchen Hu, Heyang Huang, Liangliang Xu, Xusheng Chen, Jiang Xu, Shuang Chen, Hao Feng, Chenxi Wang, Sa Wang, Yungang Bao, et al
-
[12]
Changho Hwang, Wei Cui, Yifan Xiong, Ziyue Yang, Ze Liu, Han Hu, Zilong Wang, Rafael Salas, Jithin Jose, Prabhat Ram, et al. 2023. Tutel: Adaptive mixture-of-experts at scale.Proceedings of Machine Learning and Systems5 (2023), 269–287
2023
-
[13]
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding.arXiv preprint arXiv:2006.16668 (2020)
2020 arXiv
-
[14]
Inference without interference: Disaggregate llm inference for mixed downstream workloads.arXiv preprint arXiv:2401.11181(2024)
2024 arXiv
-
[15]
Guowei Liu, Hongming Li, Yaning Guo, Yongxi Lyu, Mo Zhou, Yi Liu, Zhaogeng Li, and Yanpeng Wang. 2026. Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems.arXiv preprint arXiv:2602.09721(2026)
2026
-
[16]
Chengyu Qiu, Xiao Fu, Fengcun Li, Yulei Qian, Yuchen Xie, Xunliang Cai, Yingdi Shan, Yongwei Wu, and Mingxing Zhang
Yuhan Liu, Yihua Cheng, Jiayi Yao, Yuwei An, Xiaokun Chen, Shaot- ing Feng, Yuyang Huang, Samuel Shen, Rui Zhang, Kuntai Du, et al. Chengyu Qiu, Xiao Fu, Fengcun Li, Yulei Qian, Yuchen Xie, Xunliang Cai, Yingdi Shan, Yongwei Wu, and Mingxing Zhang
-
[17]
Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. 2025. Deepseek-v3. 2: Pushing the frontier of open large language models.arXiv preprint arXiv:2512.02556(2025)
2025 arXiv
-
[18]
Ziming Liu, Boyu Tian, Guoteng Wang, Zhen Jiang, Peng Sun, Zhen- hua Han, Tian Tang, Xiaohe Hu, Yanmin Jia, Yan Zhang, et al. 2025. Expert-as-a-service: Towards efficient, scalable, and robust large-scale moe serving.arXiv preprint arXiv:2509.17863(2025)
2025
-
[19]
Meituan LongCat Team. 2026. LongCat-2.0.https://huggingface.co/ meituan-longcat/LongCat-2.0. Accessed August 1, 2026
2026
-
[20]
NVIDIA Corporation. 2026. NVIDIA Data Center GPU Product Spec- ifications.https://www.nvidia.com/en-us/data-center/data-center- gpus/. Accessed July 10, 2026
2026
-
[21]
Zedong Liu, Xinyang Ma, Dejun Luo, Hairui Zhao, Bing Lu, Wenjing Huang, Yida Gu, Xingchen Liu, Zheng Wei, Jinyang Liu, et al. 2026. KVServe: Service-Aware KV Cache Compression for Communication- Efficient Disaggregated LLM Serving.arXiv preprint arXiv:2605.13734 (2026)
2026 arXiv
-
[22]
Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, and Ricardo Bianchini. 2024. Splitwise: Efficient generative llm inference using phase splitting. In2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 118–132
2024
-
[23]
Ruoyu Qin, Zheming Li, Weiran He, Jialei Cui, Heyi Tang, Feng Ren, Teng Ma, Shangming Cai, Yineng Zhang, Mingxing Zhang, et al. 2024. Mooncake: A kvcache-centric disaggregated architecture for llm serv- ing.ACM Transactions on Storage(2024)
2024
-
[24]
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yux- iong He. 2022. Deepspeed-moe: Advancing mixture-of-experts infer- ence and training to power next-generation ai scale. InInternational conference on machi...
2022
-
[25]
Xinglin Pan, Shaohuai Shi, Wenxiang Lin, Yuxin Wang, Zhenheng Tang, Wei Wang, and Xiaowen Chu. 2025. Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism. arXiv preprint arXiv:2512.21487(2025)
2025
-
[26]
Meituan LongCat Team, Bei Li, Bingye Lei, Bo Wang, Bolin Rong, Chao Wang, Chao Zhang, Chen Gao, Chen Zhang, Cheng Sun, et al
-
[27]
The SGLang Team. 2025. Deploying DeepSeek with PD Disaggregation and Large-Scale Expert Parallelism on 96 H100 GPUs.https://lmsys. org/blog/2025-05-05-large-scale-ep
2025
-
[28]
Bin Wang, Bojun Wang, Changyi Wan, Guanzhe Huang, Hanpeng Hu, Haonan Jia, Hao Nie, Mingliang Li, Nuo Chen, Siyu Chen, et al
-
[29]
Chendong Song, Meixuan Wang, Hang Zhou, Hong Liang, Yuan Lyu, Zixi Chen, Yuwei Fan, and Zijie Zhou. 2026. Analytical Provision- ing for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads.arXiv preprint arXiv:2601.21351(2026)
2026 arXiv
-
[30]
Ao Xiao, Bangzheng He, Baoquan Zhang, Baoxing Huai, Bingji Wang, Bo Wang, Bo Xu, Boyi Hou, Chan Yang, Changhong Liu, et al. 2025. xdeepserve: Model-as-a-service on huawei cloudmatrix384.arXiv preprint arXiv:2508.02520(2025)
2025
-
[31]
Longcat-flash technical report.arXiv preprint arXiv:2509.01322 (2025)
2025
-
[32]
Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A distributed serving system for {Transformer-Based} generative models. In16th USENIX symposium on operating systems design and implementation (OSDI 22). 521–538
2022
-
[33]
Tianyu Zhang, Peng Zhang, Yusong Gao, and Yun Zhang. 2025. To- gether with SGLang: Best Practices for Serving DeepSeek-R1 on H20- 96G.https://lmsys.org/blog/2025-09-26-sglang-ant-group
2025
-
[34]
Step-3 is large yet affordable: Model-system co-design for cost- effective decoding.arXiv preprint arXiv:2507.19427(2025)
2025
-
[35]
Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee, Tuhin Khare, Sudarshan Srinivasan, Suvinay Subramanian, Souvik Kundu, Madhu Kumar, Midhilesh Elavazhagan, William Won, et al. 2026. How Far Can Disaggregation Go? A Design-Space Ex- ploration of Attention-FFN D...
2026 arXiv
-
[36]
Ruidong Zhu, Ziheng Jiang, Chao Jin, Peng Wu, Cesar A Stuardo, Dongyang Wang, Xinlei Zhang, Huaping Zhou, Haoran Wei, Yang Cheng, et al. 2025. Megascale-infer: Serving mixture-of-experts at scale with disaggregated expert parallelism.arXiv preprint arXiv:2504.02263 (2025)
2025 arXiv
-
[37]
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)
2025 arXiv
-
[40]
Zhexiang Zhang, Ye Wang, Yumiao Zhao, Jiayu Xiao, Qianjing Yang, Xiangyu Wang, Jingzhe Jiang, Qizhen Weng, Ruichuan Chen, Shaohuai Shi, et al . 2025. Janus: Disaggregating Attention and Experts for Scalable MoE Inference.arXiv preprint arXiv:2512.13525(2025)
2025 arXiv
-
[41]
2024.{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xu- anzhe Liu, Xin Jin, and Hao Zhang. 2024.{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving. In18th USENIX Symposium on Operating Systems Design and Implementation (...
2024
-
[2024]
Memserve: Context caching for disaggregated llm serving with elastic memory pool.arXiv preprint arXiv:2406.17565(2024)
2024 arXiv
-
[2025]
Lmcache: An efficient KV cache layer for enterprise-scale LLM inference.arXiv preprint arXiv:2510.09665(2025)
2025
-
[2026]
FastAFD: Open-Source Large-Scale Attention-FFN Disaggrega- tion on Blackwell NVL72.https://github.com/hao-ai-lab/FastAFD Technical blog and open-source release
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.