Pith. sign in

REVIEW 2 major objections 2 minor 41 references

MOCAP balances KV cache across stages and partitions chunks by latency to make chunked pipelining work for long prefill on wafer-scale chips.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-26 06:24 UTC pith:RGKUW6Z5

load-bearing objection MOCAP's MBKR and LBCP are targeted fixes for memory and compute skew in chunked prefill on wafer-scale chips, but the large reported gains rest on redistribution overheads that need explicit scaling data to confirm. the 2 major comments →

arxiv 2606.22968 v1 pith:RGKUW6Z5 submitted 2026-06-22 cs.AR

MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inference

Zichuan Wang , Huizheng Wang , Yuheng Xiao , Haonan Zuo , Taiquan Wei , Jinyi Deng , Chao Li , Yang Hu
show 1 more author
Shouyi Yin
This is my paper
classification cs.AR
keywords LLM inferenceprefill phasewafer-scale chipschunked pipeliningKV cachememory balancinglatency balancing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to prove that standard chunked pipelining fails on wafer-scale chips for long-context prefill because causal attention causes uneven KV cache growth and rising per-chunk attention costs. It introduces two fixes inside the MOCAP framework to restore balance. If those fixes hold, wafer-scale hardware can support longer sequences at lower latency and higher throughput than current GPU pipelines. Readers would care because prefill dominates latency in many LLM applications and communication overhead already limits scale on conventional systems.

Core claim

MOCAP introduces Memory-Balanced KV Reallocation to move KV cache across pipeline stages and Latency-Balanced Chunk Partitioning to adjust chunk boundaries under growing attention cost and reallocation overhead; together these changes produce 76.4 percent lower end-to-end latency and 3.24 times higher throughput than GPipe while extending maximum sequence length by up to 1.31 times versus Terapipe.

What carries the argument

Memory-Balanced KV Reallocation (MBKR) and Latency-Balanced Chunk Partitioning (LBCP) inside a chunked pipeline that redistributes cache and resizes chunks to counteract causal dependency effects.

Load-bearing premise

Redistributing KV cache and re-partitioning chunks adds little enough time and data movement that the pipeline still finishes faster overall.

What would settle it

A direct measurement on wafer-scale hardware showing that total time with MBKR and LBCP exceeds the time of plain GPipe pipelining.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Longer input sequences become feasible without memory overflow on the pipeline stages.
  • End-to-end prefill latency drops by roughly three-quarters relative to prior pipelining methods.
  • Average throughput rises more than threefold under the same hardware.
  • Wafer-scale chips become practical for prefill-only workloads that current GPU clusters cannot handle at scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same redistribution idea could be tested on other high-bandwidth interconnects such as multi-GPU nodes or custom accelerators.
  • If attention cost grows faster than linear with sequence length, LBCP may need dynamic adjustment at runtime rather than static partitioning.
  • Extending the approach to include decode phases would require new rules for how KV cache moves during token generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes MOCAP, a memory-orchestrated chunked pipelining framework for prefill-only LLM inference on wafer-scale chips. It identifies two challenges in applying chunked pipelining to long-context prefill: uneven KV cache accumulation due to causal dependencies across chunks, and increasing attention computation costs for later chunks. MOCAP introduces Memory-Balanced KV Reallocation (MBKR) to redistribute KV cache across pipeline stages and Latency-Balanced Chunk Partitioning (LBCP) to re-partition chunks accounting for attention growth and reallocation overhead. Experiments report 76.4% lower end-to-end latency and 3.24× higher throughput vs. GPipe on average, plus up to 1.31× longer maximum sequence length vs. Terapipe.

Significance. If the overhead claims hold, the work addresses a practical bottleneck in scaling prefill on high-bandwidth WSC hardware and could enable longer-context inference with better pipeline utilization. The targeted mechanisms for memory and latency balancing are a direct response to the identified imbalances; credit is due for focusing on prefill-only workloads and WSC-specific interconnect properties rather than generic GPU assumptions.

major comments (2)
  1. [Abstract (performance claims); MBKR/LBCP descriptions] The central performance claims (76.4% latency reduction, 3.24× throughput) rest on the assumption that MBKR redistribution and LBCP re-partitioning incur negligible time relative to compute. No scaling analysis of bytes moved per stage, interconnect bandwidth fraction consumed, or communication time vs. sequence length is provided to substantiate this; if redistribution volume grows linearly with sequence length and pipeline depth, communication can dominate and erode the reported gains.
  2. [Experimental results / sequence-length evaluation] The 1.31× sequence-length extension vs. Terapipe is presented as a direct benefit of MBKR, yet the manuscript provides no quantitative breakdown of how much of the extension is attributable to reduced memory imbalance versus other factors, nor any sensitivity analysis when interconnect bandwidth is varied.
minor comments (2)
  1. [Experimental setup] Clarify the exact definition of 'end-to-end latency' (does it include host-device transfers or only on-WSC execution?) and confirm whether all baselines use identical chunk sizes and pipeline depths.
  2. [MBKR section] Add a table or figure showing per-stage memory usage before and after MBKR for at least one long-sequence example to make the imbalance correction concrete.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below and indicate planned revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract (performance claims); MBKR/LBCP descriptions] The central performance claims (76.4% latency reduction, 3.24× throughput) rest on the assumption that MBKR redistribution and LBCP re-partitioning incur negligible time relative to compute. No scaling analysis of bytes moved per stage, interconnect bandwidth fraction consumed, or communication time vs. sequence length is provided to substantiate this; if redistribution volume grows linearly with sequence length and pipeline depth, communication can dominate and erode the reported gains.

    Authors: We agree that the current manuscript lacks an explicit scaling analysis of MBKR/LBCP communication overhead. The end-to-end results are measured on the target WSC hardware whose interconnect bandwidth is orders of magnitude higher than conventional GPUs, which keeps redistribution time small relative to attention compute. To directly address the concern, the revised version will add a dedicated subsection with (i) bytes moved per stage as a function of sequence length and pipeline depth, (ii) fraction of interconnect bandwidth consumed, and (iii) a breakdown of communication versus compute time. This analysis will confirm that the overhead remains negligible under the evaluated WSC parameters. revision: yes

  2. Referee: [Experimental results / sequence-length evaluation] The 1.31× sequence-length extension vs. Terapipe is presented as a direct benefit of MBKR, yet the manuscript provides no quantitative breakdown of how much of the extension is attributable to reduced memory imbalance versus other factors, nor any sensitivity analysis when interconnect bandwidth is varied.

    Authors: The reported 1.31× extension is attributed to MBKR’s alleviation of per-stage KV-cache imbalance. We acknowledge that the manuscript does not supply a quantitative attribution breakdown or bandwidth-sensitivity sweeps. The revised manuscript will include (i) per-stage memory-usage plots with and without MBKR, (ii) an ablation isolating the contribution of memory balancing to the maximum feasible sequence length, and (iii) additional experiments sweeping interconnect bandwidth to demonstrate sensitivity. These additions will make the source of the improvement explicit. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical framework with external baselines

full rationale

The paper introduces MBKR and LBCP as engineering techniques to mitigate KV cache imbalance and chunk latency imbalance in chunked pipelining on wafer-scale chips. All reported gains (76.4% latency reduction vs GPipe, 3.24× throughput, 1.31× sequence length vs Terapipe) are presented as direct experimental measurements against named external systems. No equations, fitted parameters, self-citations, or uniqueness theorems appear in the provided text; the derivation chain consists of problem identification followed by proposed fixes and benchmarked outcomes. The evaluation is therefore self-contained against independent baselines and contains no load-bearing self-referential steps.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no information on free parameters, axioms, or invented entities; ledger is therefore empty.

pith-pipeline@v0.9.1-grok · 5879 in / 1094 out tokens · 29795 ms · 2026-06-26T06:24:33.303843+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly used in prefill-only workloads, where end-to-end latency is dominated by the prefill phase. For long-context prefill, communication overhead grows with sequence length and quickly becomes a bottleneck on conventional GPU systems, making wafer-scale chips (WSCs) a promising substrate due to their high communication bandwidth and large aggregate compute and memory capacity. A natural way to accelerate prefill is to partition a long input sequence into multiple chunks and execute them in a finer-grained pipeline across devices. However, directly applying this idea to long-context prefill on WSCs remains challenging. First, causal dependency across chunks causes KV cache to accumulate unevenly across pipeline stages, creating severe memory imbalance and limiting the feasible sequence length. Second, later chunks require more attention computation because each chunk depends on preceding chunks, leading to chunk-level latency imbalance. To address these challenges, we present MOCAP, a memory-orchestrated chunked pipelining framework for prefill-only LLM inference on WSCs. MOCAP introduces Memory-Balanced KV Reallocation (MBKR) to alleviate memory imbalance by redistributing KV cache across pipeline stages, thereby extending the feasible sequence length. It further incorporates Latency-Balanced Chunk Partitioning (LBCP) to balance chunk execution cost under both attention-cost growth and KV reallocation overhead, improving pipeline efficiency. Experimental results show that, compared with GPipe, MOCAP achieves 76.4\% lower end-to-end latency and 3.24$\times$ higher throughput on average. MOCAP also extends the maximum supported sequence length by up to 1.31$\times$ compared with Terapipe.

Figures

Figures reproduced from arXiv: 2606.22968 by Chao Li, Haonan Zuo, Huizheng Wang, Jinyi Deng, Shouyi Yin, Taiquan Wei, Yang Hu, Yuheng Xiao, Zichuan Wang.

Figure 1
Figure 1. Figure 1: (a)(b) LLM inference process. (c) End-to-end latency comparison between HGX B200 GPU systems and wafer-scale chips under equivalent compute and memory ca￾pacity on the GR24 [40]. 1 Introduction Large language models (LLMs) have transformed AI across a broad range of applications. Beyond generative appliccations [41], LLM-style transformer infer￾ence is increasingly used in prefill-only scenarios [1, 40], w… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of microbatch-based pipeline and token-based pipeline. different chunks is inherently imbalanced: later chunks incur higher attention la￾tency due to their longer prefixes, so uniform chunk partitioning creates pipeline bubbles and degrades pipeline efficiency. Therefore, making the chunked prefill pipeline effective requires jointly addressing both KV cache accumulation and chunk-level latency im… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of wafer-scale chip architecture. die contains an array of compute cores equipped with distributed SRAM for local buffering, while HBM stacks are placed around the compute die to provide large-capacity off-die storage and high memory access bandwidth. A key architectural distinction between wafer-scale chips and conventional GPU systems lies in the communication fabric. 1) Wafer-scale chips provid… view at source ↗
Figure 4
Figure 4. Figure 4: Stage-wise memory and execution latency imbalance introduced by chunked pipeline. memory capacity of the system, leaving a large fraction of memory stranded on non-bottleneck stages. Insight: Such skewed memory utilization creates an opportunity to extend sequence headroom through stage-aware KV-cache realloca￾tion. Although KV-cache accumulation is highly skewed, the memory pressure is not simultaneously … view at source ↗
Figure 5
Figure 5. Figure 5: Overview of memory-balanced KV reallocation. However, in chunked pipeline execution, the imbalance pattern changes during request transitions, causing the pairing to become mismatched. In particular, when the hotspot shifts to Stage 3, its original paired stage, Stage 4, differs from it by only one chunk in memory occupancy, which means this head-to-tail strategy provides at most half a chunk of additional… view at source ↗
Figure 6
Figure 6. Figure 6: (a) Performance of GPipe, Terapipe and MOCAP on WSCs. Red crossed marks indicate that the corresponding prefill jobs fail to run due to out-of-memory errors. (b) Maximum sequence length of Terapipe and MOCAP across different models and number of chunks. To address this challenge, we propose Latency-Balanced Chunk Partitioning (LBCP). LBCP first derives a compute-balanced chunk slicing scheme as an initiali… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    In: Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Princi- ples

    Du, K., Wang, B., Zhang, C., Cheng, Y., Lan, Q., Sang, H., et al.: Prefillonly: An inference engine for prefill-only workloads in large language model applications. In: Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Princi- ples. pp. 399–414 (2025)

    2025

  2. [2]

    arXiv preprint arXiv:2406.03868 (2024)

    Fang, J., Wang, H., Yang, Q., Kong, D., Dai, X., Deng, J., et al.: PALM: A efficient performance simulator for tiled accelerators with large-scale model training. arXiv preprint arXiv:2406.03868 (2024)

    work page arXiv 2024

  3. [3]

    arXiv preprint arXiv:2502.04563 (2025)

    He, C., Huang, Y., Mu, P., Miao, Z., Xue, J., Ma, L., et al.: WaferLLM: Large language model inference at wafer scale. arXiv preprint arXiv:2502.04563 (2025)

    work page arXiv 2025

  4. [4]

    IEEE Circuits and Systems Magazine24(1), 52–81 (2024)

    Hu, Y., Lin, X., Wang, H., He, Z., Yu, X., Zhang, J., et al.: Wafer-scale computing: Advancements, challenges, and future perspectives. IEEE Circuits and Systems Magazine24(1), 52–81 (2024)

    2024

  5. [5]

    Advances in neural information processing systems32(2019)

    Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, D., Chen, M., et al.: Gpipe: Efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems32(2019)

    2019

  6. [6]

    In: International Conference on Machine Learning

    Kim, T., Kim, H., Yu, G.I., Chun, B.G.: Bpipe: Memory-balanced pipeline paral- lelism for training large language models. In: International Conference on Machine Learning. pp. 16639–16653. PMLR (2023)

    2023

  7. [7]

    IEEE Transactions on Circuits and Systems I: Regular Papers (2025)

    Lee, S., Kim, B., Park, J., Jeon, D.: Clat: A clustering-based attention transformer accelerator for low-latency text generation in llms. IEEE Transactions on Circuits and Systems I: Regular Papers (2025)

    2025

  8. [8]

    In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA)

    Li, C., Wang, H., Liu, J., Liu, J., Yue, Z., Li, X., et al.: Rethermal: Co-design of thermal-aware static and dynamic scheduling for llm training on liquid-cooled wafer-scale chips. In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA). pp. 1–15. IEEE (2026)

    2026

  9. [9]

    In: International Conference on Machine Learning

    Li, Z., Zhuang, S., Guo, S., Zhuo, D., Zhang, H., Song, D., et al.: Terapipe: Token- level pipeline parallelism for training large-scale language models. In: International Conference on Machine Learning. pp. 6543–6552. PMLR (2021)

    2021

  10. [10]

    In: 2022 IEEE Hot Chips 34 Symposium (HCS)

    Lie, S.: Cerebras architecture deep dive: First look inside the hw/sw co-design for deep learning: Cerebras systems. In: 2022 IEEE Hot Chips 34 Symposium (HCS). pp. 1–34. IEEE Computer Society (2022)

    2022

  11. [11]

    In: Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

    Liu, Y., Pan, Y., Wang, M., Zhao, S., Zhu, H., Han, Y., et al.: Ouroboros: Wafer- scale sram cim with token-grained pipelining for large language model inference. In: Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. pp. 1349–1365 (2026) 14 Authors Suppressed Due to Exc...

    2026

  12. [12]

    In: MICRO- 54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

    Lu, L., Jin, Y., Bi, H., Luo, Z., Li, P., Wang, T., et al.: Sanger: A co-design frame- work for enabling sparse attention using reconfigurable architecture. In: MICRO- 54: 54th Annual IEEE/ACM International Symposium on Microarchitecture. pp. 977–991 (2021)

    2021

  13. [13]

    Available:https://huggingface.co/ meta-llama/Meta-Llama-3-70B

    Meta: Meta-Llama-3-70B (2024), [Online]. Available:https://huggingface.co/ meta-llama/Meta-Llama-3-70B

    2024

  14. [14]

    Available:https://huggingface.co/ meta-llama/Llama-3.1-405B

    Meta AI: Llama-3.1-405B (2024), [Online]. Available:https://huggingface.co/ meta-llama/Llama-3.1-405B

    2024

  15. [15]

    Available:https:// huggingface.co/mistralai/Mistral-Large-Instruct-2407

    Mistral AI: Mistral-Large-Instruct-2407 (2024), [Online]. Available:https:// huggingface.co/mistralai/Mistral-Large-Instruct-2407

    2024

  16. [16]

    In: 2025 IEEE International Solid-State Circuits Conference (ISSCC)

    Moon, S., Li, M., Chen, G.K., Knag, P.C., Krishnamurthy, R.K., Seok, M.: T-REX: A 68-to-567µs/token 0.41-to-3.95µJ/token transformer accelerator with reduced external memory access and enhanced hardware utilization in 16nm FinFET. In: 2025 IEEE International Solid-State Circuits Conference (ISSCC). vol. 68, pp. 406–408. IEEE (2025)

    2025

  17. [17]

    Available:https: //www.nvidia.com/en-us/data-center/dgx-vera-rubin-nvl72/

    NVIDIA: NVIDIA DGX Vera Rubin NVL72 (2026), [Online]. Available:https: //www.nvidia.com/en-us/data-center/dgx-vera-rubin-nvl72/

    2026

  18. [18]

    In: 2021 58th ACM/IEEE Design Au- tomation Conference (DAC)

    Pal, S., Liu, J., Alam, I., Cebry, N., Suhail, H., Bu, S., et al.: Designing a 2048- chiplet, 14336-core waferscale processor. In: 2021 58th ACM/IEEE Design Au- tomation Conference (DAC). pp. 1183–1188. IEEE (2021)

    2048

  19. [19]

    Available:https://huggingface.co/ Qwen/Qwen3-235B-A22B

    Qwen: Qwen3-235B-A22B (2025), [Online]. Available:https://huggingface.co/ Qwen/Qwen3-235B-A22B

    2025

  20. [20]

    arXiv preprint arXiv:2406.19580 (2024)

    Rashidi, S., Won, W., Srinivasan, S., Gupta, P., Krishna, T.: FRED: Flexible reduction-distribution interconnect and communication implementation for wafer- scale distributed training of DNN models. arXiv preprint arXiv:2406.19580 (2024)

    work page arXiv 2024

  21. [21]

    In: 2025 IEEE 75th Electronic Components and Technology Conference (ECTC)

    Shih, P.C., Su, A.J., Tam, K.H., Huang, T.C., Chuang, K., Yeh, J.: SoW-X: A novel system-on-wafer technology for next generation AI server application. In: 2025 IEEE 75th Electronic Components and Technology Conference (ECTC). pp. 1–6. IEEE (2025)

    2025

  22. [22]

    In: 2022 IEEE Hot Chips 34 Symposium (HCS)

    Talpes, E., Williams, D., Sarma, D.D.: Dojo: The microarchitecture of Tesla’s exa- scale computer. In: 2022 IEEE Hot Chips 34 Symposium (HCS). pp. 1–28. IEEE Computer Society (2022)

    2022

  23. [23]

    In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA)

    Tang, X., Hou, J., Jiang, D., Wei, T., Liu, J., Deng, J., et al.: Moentwine: Unleash- ing the potential of wafer-scale chips for large-scale expert parallel inference. In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA). pp. 1–15. IEEE (2026)

    2026

  24. [24]

    In: 2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)

    Wang, H., Fang, J., Tang, X., Yue, Z., Li, J., Qin, Y., et al.: SOFA: A compute- memory optimized sparsity accelerator via cross-stage coordinated tiling. In: 2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO). pp. 1247–1263. IEEE (2024)

    2024

  25. [25]

    In: 2026 IEEE International Symposium on High Performance Computer Archi- tecture (HPCA)

    Wang, H., Wang, H., Wang, Z., Yue, Z., Wang, Y., Li, C., et al.: Pade: A predictor-free sparse attention accelerator via unified execution and stage fusion. In: 2026 IEEE International Symposium on High Performance Computer Archi- tecture (HPCA). pp. 1–19. IEEE (2026)

    2026

  26. [26]

    In: 2026 31st Asia and South Pacific Design Automation Conference (ASP-DAC)

    Wang, H., Wang, H., Wei, S., Hu, Y., Yin, S.: Bitstopper: An efficient transformer attention accelerator via stage-fusion and early termination. In: 2026 31st Asia and South Pacific Design Automation Conference (ASP-DAC). pp. 162–169. IEEE (2026)

    2026

  27. [27]

    In: 2026 31st Asia and South Pacific Design Automation Conference (ASP-DAC)

    Wang, H., Wang, H., Wei, S., Hu, Y., Yin, S.: Lapa: Log-domain prediction-driven dynamic sparsity accelerator for transformer model. In: 2026 31st Asia and South Pacific Design Automation Conference (ASP-DAC). pp. 154–161. IEEE (2026)

    2026

  28. [28]

    IEEE Transactions on Circuits and Systems II: Express Briefs (2025)

    Wang, H., Wang, H., Yue, Z., Liu, J., Wei, T., Hu, S.W.Y., et al.: BETA: A bit- Grained Transformer attention accelerator With efficient early termination. IEEE Transactions on Circuits and Systems II: Express Briefs (2025)

    2025

  29. [29]

    In: 2026 IEEE International Symposium on High Performance Computer Architec- ture (HPCA)

    Wang, H., Wang, Z., Wang, H., Hou, J., Wei, T., Li, C., et al.: Watos: Efficient llm training strategies and architecture co-exploration for wafer-scale chip. In: 2026 IEEE International Symposium on High Performance Computer Architec- ture (HPCA). pp. 1–19. IEEE (2026)

    2026

  30. [30]

    In: Proceedings of the 58th IEEE/ACM International Sympo- sium on Microarchitecture®

    Wang, H., Wang, Z., Yue, Z., Long, Y., Wei, T., Yang, J., et al.: MCBP: A memory- compute efficient LLM inference accelerator leveraging bit-slice-enabled sparsity and repetitiveness. In: Proceedings of the 58th IEEE/ACM International Sympo- sium on Microarchitecture®. pp. 1592–1608 (2025)

    2025

  31. [31]

    IEEE Transactions on Computers (2025)

    Wang, H., Wei, T., Wang, H., Wang, Z., Tang, X., Yue, Z., et al.: Designing spatial architectures for sparse attention: Star accelerator via cross-stage tiling. IEEE Transactions on Computers (2025)

    2025

  32. [32]

    In: 2026 IEEE International Symposium on High Performance Computer Archi- tecture (HPCA)

    Wang, H., Wei, T., Wang, Z., Jiang, D., Yang, Q., Liu, J., et al.: Temp: A memory efficient physical-aware tensor partition-mapping framework on wafer-scale chips. In: 2026 IEEE International Symposium on High Performance Computer Archi- tecture (HPCA). pp. 1–18. IEEE (2026)

    2026

  33. [33]

    Integrated Circuits and Systems (2024)

    Wang, H., Yang, Q., Wei, T., Yu, X., Li, C., Fang, J., et al.: TMAC: Training- targeted mapping and architecture co-exploration for wafer-scale chips. Integrated Circuits and Systems (2024)

    2024

  34. [34]

    In: International Symposium on Advanced Parallel Processing Technologies

    Wei, T., Wang, H., Wang, Z., Yin, S., Hu, Y.: Spatial-aware orchestration of LLM attention on waferscale chips. In: International Symposium on Advanced Parallel Processing Technologies. pp. 386–391. Springer (2025)

    2025

  35. [35]

    0: Modeling hierarchical networks and disaggregated systems for large-model training at scale

    Won, W., Heo, T., Rashidi, S., Sridharan, S., Srinivasan, S., Krishna, T.: Astra- sim2. 0: Modeling hierarchical networks and disaggregated systems for large-model training at scale. In: 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). pp. 283–294. IEEE (2023)

    2023

  36. [36]

    In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA)

    Xu, Z., Kong, D., Liu, J., Jiang, D., Dai, X., Deng, J., et al.: Face: Fully overlapped pd scheduling and multi-level architecture co-exploration on wafer. In: 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA). pp. 1–16. IEEE (2026)

    2026

  37. [37]

    In: Proceedings of the 52nd Annual International Symposium on Computer Architecture

    Xu, Z., Kong, D., Liu, J., Li, J., Hou, J., Dai, X., et al.: WSC-LLM: Efficient LLM service and architecture co-exploration for wafer-scale chips. In: Proceedings of the 52nd Annual International Symposium on Computer Architecture. pp. 1–17 (2025)

    2025

  38. [38]

    In: Proceedings of the 52nd Annual International Symposium on Computer Architecture

    Yang, Q., Wei, T., Guan, S., Li, C., Shang, H., Deng, J., et al.: PD constraint- aware physical/logical topology co-design for network on wafer. In: Proceedings of the 52nd Annual International Symposium on Computer Architecture. pp. 49–64 (2025)

    2025

  39. [39]

    In: 2024ACM/IEEE51stAnnualInternationalSymposiumonComputerArchitecture (ISCA)

    Yue, Z., Wang, H., Fang, J., Deng, J., Lu, G., Tu, F., et al.: Exploiting similarity opportunities of emerging vision ai models on hybrid bonding architecture. In: 2024ACM/IEEE51stAnnualInternationalSymposiumonComputerArchitecture (ISCA). pp. 396–409. IEEE (2024)

    2024

  40. [40]

    Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations

    Zhai, J., Liao, L., Liu, X., Wang, Y., Li, R., Cao, X., et al.: Actions speak louder than words: Trillion-parameter sequential transducers for generative recommenda- tions. arXiv preprint arXiv:2402.17152 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2025)

    Zhang, C., You, Y., Wang, N., Park, J., Zhang, L.: Generative ai through cas lens: An integrated overview of algorithmic optimizations, architectural advances, and automated designs. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2025)

    2025