Pith. sign in

REVIEW 4 major objections 4 minor 27 references

Photonic Fabric Platform for AI Accelerators

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that replacing an accelerator's local HBM stack with a shared photonic memory appliance breaks the fixed memory-to-compute limit, and an analytical simulator validated on H100/H200 GPUs projects up to 3.66x/7.04x…

desk verdict The paper's simulator validation is solid, but the headline speedups rest on a per-GPU bandwidth number that contradicts the paper's own 7.2 Tbps per-port spec; the stress-test check holds up. read the letter →

arxiv 2507.14000 v3 pith:EBWAJVFM submitted 2025-07-18 cs.PF cs.AI

Jing Ding , Trung Diep This is my paper
classification cs.PFcs.AI
keywords photonicfabricmemorydisaggregationAIacceleratorLLMinferenceanalyticalsimulationopticalinterconnectHBMenergyefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the fixed memory-to-compute ratio built into current AI accelerators, not the compute cores themselves, is the main barrier to scaling large language models. It presents the Photonic Fabric Appliance (PFA), a rack-mounted electro-optical system-in-package that pools up to 32 TB of HBM3E-backed DDR5 memory behind 115 Tbps of all-to-all optical switching, so any attached accelerator can address the whole pool at HBM-class bandwidth. Since no PFA hardware exists yet, the authors evaluate the idea with CelestiSim, a lightweight analytical simulator calibrated on H100/H200 microbenchmarks and validated against real LLaMA-3.1-70B inference runs with about 7.6% mean error. The simulator projects up to 3.66x throughput and 1.40x latency gains for 405B-parameter inference, up to 7.04x throughput and 1.41x latency gains for 1T-parameter inference, 60-90% lower data-movement energy for collective training operations, and roughly 22.8x speedup for DLRM embedding pooling. If these projections hold, model size would no longer dictate GPU count, and the memory wall in AI systems would move from the chip package to a shared optical fabric.

What carries the argument

The carrying object is the Photonic Fabric Module (PFM), a 2.5D system-in-package containing an active photonic interposer, a custom ASIC, two HBM3E stacks, and an 8 Tbps embedded network switch; sixteen PFMs form a PFA with 115 Tbps of all-to-all switching and 32 TB of shared memory. The interposer uses germanium-silicon electro-absorption modulators instead of ring resonators or Mach-Zehnder modulators, which the paper says gives thermal stability, sub-100-micrometer device size, and DSP-free operation at an assumed 5-25 pJ/bit. The second mechanism is CelestiSim, a parameterized analytical solver that feeds compute and memory utilization microbenchmarks from H100 and H200 GPUs, model-parallel scheduling rules, and per-bit energy assumptions through a transformer workload model to produce the throughput, latency, and energy projections.

What would settle it

Run the same LLaMA-3.1-70B inference configuration used for validation on a physical single-module PFA prototype attached to an H100-class accelerator: if measured end-to-end latency differs from CelestiSim by more than its reported 7.57% mean absolute percentage error, or if a measured all-reduce transfer exceeds the modeled 25 pJ/bit ceiling, the projected speedups and energy savings are falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that optical memory disaggregation, realized as the Photonic Fabric Appliance, turns memory from a per-chip fixed resource into a cluster-wide addressable pool and thereby removes the dominant overheads of large-model inference: tensor-parallel communication, redundant memory reads, and small-batch memory stalls. In the PFA, a single compute chip attaches through a 7.2 Tbps optical port to 32 TB of shared memory with 26,800 GB/s aggregate bandwidth, so a 405B-parameter model that needs eight GPUs in tensor parallelism on a conventional DGX server can run with no tensor parallelism at all; in one reported configuration the model FLOPs utilization rises from 13.6% to 49.7%. For a 1T-parameter model the simulation reports up to 7.04x throughput and 1.41x latency gains over two interconnected eight-GPU systems. For training, replacing a conventional multi-tier electronic network with the photonic fabric cuts communication-related energy by roughly 60-90%, and for recommendation models a 10 TB embedding table served from shared PFA memory gives 22.8x average speedup over NVLink-connected GPUs. The paper states plainly that these are predictive-modeling results, not measurements from a physical PFA.

Load-bearing premise

The headline numbers depend on the unverified premise that the photonic memory interface will deliver the modeled 26,800 GB/s bandwidth and 5 to 25 pJ/bit energy costs while preserving the same FLOP and memory utilization measured on today's GPUs; Section 9 concedes the evaluation relies on predictive modeling rather than empirical hardware results.

Editorial extensions

If this is right

  • A 405B-parameter model no longer needs eight GPUs in tensor parallelism; one compute chip backed by PFA memory can skip cross-GPU all-reduce and synchronization during decode.
  • At 1T parameters, two interconnected eight-GPU boxes can be replaced by PFA-backed compute, with the largest speedup (7.04x) coming exactly where communication overhead is worst.
  • Training collectives, memory offloading, and pipeline-parallel transfers all become much cheaper per bit, so tensor parallelism can be scaled without the usual linear growth in communication energy.
  • Multi-terabyte DLRM embedding tables move from distributed storage into locally addressable shared memory, changing embedding pooling from a network operation into a memory operation.
  • Tiering multiple PFAs extends the same shared-memory model to 128-256 accelerators, so the claimed benefits are a scaling path rather than a one-box effect.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own breakdown implies that long-output, large-batch serving workloads will realize the largest gains, while short-output prefill-bound workloads gain less; this gives deployment planners a rule for where to prioritize PFA placement.
  • If the assumed photonic per-bit energies hold, the cheapest parallelism strategy for training will shift toward tensor parallelism, so production schedulers should re-optimize their parallelism choices under photonic communication costs rather than electronic ones.
  • The energy claims can be tested before a full 16-module appliance exists: instrument a single PFM plus two accelerators and measure all-reduce energy per byte against the 60-90% reduction CelestiSim predicts.
  • Because the PFA decouples memory technology from the compute die, the same accelerator chiplet may survive HBM4 and later memory generations by upgrading the appliance instead of the GPU, a cost consequence the paper raises only in its outlook.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces the Photonic Fabric Appliance (PFA), a photonic switching and memory-disaggregation system, and uses a new analytical simulator, CelestiSim, to project performance and energy benefits for LLM inference, LLM training collectives, and DLRM inference. CelestiSim is validated on H100/H200 systems with reported MAPE of 7.57% and R² of 0.99 for LLaMA-3.1 70B inference. Using this simulator, the paper claims up to 3.66x throughput and 1.40x latency improvements for 405B-parameter LLM inference, up to 7.04x throughput and 1.41x latency improvements for 1T-parameter inference, 60-90% energy savings for training collectives, and 22.8x speedup for DLRM embedding pooling, relative to DGX-H100 or NVLink/PCIe baselines. The paper's central technical claim is that replacing local HBM with a PFA-provided 32 TB shared memory at full HBM3E bandwidth eliminates tensor-parallelism overheads and memory-capacity limits without changing the GPU core.

Significance. If the PFA's disclosed specifications and CelestiSim's projections were physically consistent and validated, the paper would present a significant contribution to memory-disaggregation architecture for AI accelerators. The simulation framework itself is a genuine strength: CelestiSim's baseline validation is solid, the reported MAPE and R² are strong, and the integration of H100/H200 microbenchmark utilization curves is a concrete, reproducible step. However, the headline speedups and energy savings are not established because the central bandwidth parameter is internally inconsistent with the PFA's own port specifications, and the energy constants are assumed rather than derived. The paper is transparent about the predictive nature of the evaluation, but that transparency does not resolve the load-bearing inconsistencies.

major comments (4)
  1. [§6.1, Table 5; §3.3] The PFA configuration in Table 5 lists '26800 GB/s of interconnect bandwidth,' but Section 3.3 states that each Photonic Fabric Module has an optical port bandwidth of 7.2 Tbps and that the PFA provides 115 Tbps of all-to-all switching. 26,800 GB/s equals 214.4 Tbps, which is roughly double the entire PFA switching capacity; an individual XPU attached to one PFM port is therefore capped at 7.2 Tbps (0.9 TB/s), more than 29x lower than the modeled 26.8 TB/s. This contradiction is load-bearing: in the 405B no-tensor-parallelism configuration, a single GPU must stream 405B FP8 weights from PFA memory every decode step, and at 0.9 TB/s this would take roughly 0.45 s/token versus approximately 15 ms/token at the modeled bandwidth. The claimed 3.66x and 7.04x throughput speedups and 1.40x and 1.41x latency improvements are therefore not a conservative extrapolation of the disclosed PFA specifications; the simulator was run with a bandwidth value that the paper's own specifications make physically unreachable.
  2. [§4.2] The energy savings of 60-90% are forced by the assumed per-bit constants, not measured or derived. The paper assigns 65 pJ/bit for generic adapters, 35 pJ/bit for switches, and 50 pJ/bit for internal NVLink communication, while photonic transceivers, switches, and intra-tray communication are assigned 5, 25, and 10 pJ/bit. These assumptions produce a 5-13x per-bit energy advantage, so the reported 60-90% reduction in communication energy is essentially an algebraic consequence of the chosen inputs. No sensitivity analysis or error bars are provided over these constants, and the model treats all photonic paths as homogeneous despite the paper's own description of multi-tiered DDR5/HBM3E memory with caching. The energy claims should be re-framed as conditional on these unverified constants, and the relative ordering of energy savings across tables should be tested against large perturbations of the per-bit values.
  3. [§7] The DLRM result is presented as a headline claim ('22.8x higher performance than GPUs with NVLink') but is supported by only a terse description and a single bar chart. The section does not state the model of the NVLink baseline (e.g., number of GPUs, whether embeddings are partitioned by row-wise parallelism across 128 GPUs, or the exact TorchRec configuration), does not report the modeled PFA bandwidth or energy used for this workload, and provides no simulation validation for DLRM. Given that the 22.8x figure is one of the paper's central quantitative claims, this level of detail is insufficient to assess its correctness or to distinguish it from artifacts of the same unvalidated bandwidth assumptions identified in Section 6.1.
  4. [§9] Section 9 correctly admits that 'the evaluation of the PFA relies on predictive modeling using CelestiSim, rather than empirical hardware results,' and states that the same H100/H200 bandwidth and FLOP utilization curves are applied to the PFA. However, this limitation is not reflected in the abstract or the headline results, which state the speedups and savings as unconditional findings. Given that the PFA is an unbuilt system, the paper should either (a) present the results as a parameterized case study with explicit ranges over interconnect bandwidth and per-bit energy, or (b) include a sensitivity analysis showing how throughput, latency, and energy vary as the PFA bandwidth is reduced from the assumed 26.8 TB/s to the physically consistent 0.9 TB/s per port. Without such a test, the predictive claims remain ungrounded.
minor comments (4)
  1. [§9] The sentence introducing the next generation says 'the per link data bandwidth is expected to quadruple from 7.2 Tbps to 28.8 Tbps,' but Section 3.3 describes 7.2 Tbps as the port bandwidth of a Photonic Fabric Module, not a per-link bandwidth; the terminology should be made consistent.
  2. [Reference list] Reference [2] contains the typo 'Seqenence' in the title; also, several references are missing publisher or venue information (e.g., [19], [20], [21]), which should be completed for archival consistency.
  3. [Abstract] The abstract states 'the PFA offers up to 32 TB of shared memory alongside 115 Tbps of all-to-all digital switching,' but Section 3.3 adds that this configuration uses 16 PFMs, each with 7.2 Tbps; the relationship between the 115 Tbps switching capacity and the 7.2 Tbps per-port bandwidth should be made explicit early to avoid the confusion raised in the major comments.
  4. [Figure 9 and Figure 10] The left and right panels of Figures 9 and 10 are not labeled in the caption, making it necessary to infer from the text which panel is throughput and which is latency; the captions should explicitly state this.

Circularity Check

2 steps flagged · score 6.0 of 10

Score 6: the headline speedups (3.66x/7.04x) and 60-90% energy savings are arithmetic consequences of chosen inputs — Table 5's 26,800 GB/s (which exceeds the paper's own 115 Tbps total switching capacity and 7.2 Tbps per-XPU port) and assumed 5-25 pJ/bit photonic vs 35-65 pJ/bit electrical energy costs.

  1. fitted input called prediction [Section 4.2 (Power Modeling) and Section 5 (Power Savings of LLM Pre-training)]
    "We assume that adapters and switch energies are parameterized by estimated per-bit costs: 65 pJ/bit for generic adapters, 35 pJ/bit for generic switches, and 50 pJ/bit for internal NVLink communication [28, 29, 30, 31]. Similarly, we assume energy costs of 5 pJ/bit for photonic transceivers, 25 pJ/bit for photonic switches, and 10 pJ/bit for intra-tray photonic communication. ... the analyses consistently indicate approximately 60-90% reductions in communication-related power expenditures."

    The 60-90% saving is the assumed per-bit ratio evaluated in the paper's own equation (E_total = E_s.adapter + sum_i E_switch_i + E_d.adapter): a one-switch path falls from 165 pJ/bit (65+35+65) to 35 pJ/bit (5+25+5), i.e., 79%; NVLink intra-tray falls from 50 to 10 pJ/bit, i.e., 80%. No measurement anchors the photonic constants, and Section 9 concedes the evaluation 'relies on predictive modeling using CelestiSim, rather than empirical hardware results.' Any other assumed photonic constants would proportionally shift the claimed savings, so the headline percentage is the input restated as a result, not an independent discovery.

  2. fitted input called prediction [Section 6.1 (Experiments Setup, Table 5) vs Section 3.3 (PFA Specifications); results in Section 6.2]
    "H100 GPUs with Photonic Fabric ApplianceTM (PFA): A novel architecture featuring 32 TB of photonically-accessible memory, and 26800 GB/s of interconnect bandwidth. ... The PFA has optical port bandwidth of 7.2 Tbps per Photonic Fabric Module, with all-to-all switching totaling 115Tbps per PFA."

    The 3.66x/7.04x throughput and 1.40x/1.41x latency gains are computed from the Table 5 input 26,800 GB/s, not derived from the Section 3.3 specifications. 26,800 GB/s = 214 Tbps, about 1.9x the appliance's entire 115 Tbps (~14.4 TB/s) switching capacity, while each XPU attaches through a single 7.2 Tbps = 0.9 TB/s port, 3.7x below the H100's 3,350 GB/s HBM bandwidth; Section 9's 'conservative scaling of interconnect bandwidth' is inconsistent with this table. The speedups are thus fixed by an input bandwidth choice that the paper's own specifications contradict: the headline gains are the chosen parameter restated as a simulated prediction, so the results are forced by the assumption rather than by the described architecture.

full rationale

CelestiSim is genuinely validated: Section 4.3 reports a mean absolute percentage error of 7.57% and R^2 of 0.99 against TensorRT-LLM inference measurements on H100/H200 across 180 configurations, and the tensor-parallelism overhead analysis in Section 6.3 (Figure 12) uses NVIDIA Nsight profiling of real hardware. The qualitative mechanism — tensor parallelism adds communication overhead and redundant memory access — is independently supported, and the baseline simulation is not circular. The circularity is confined to the two headline quantitative claims. First, the 60-90% reductions in communication-related power expenditures (Section 5) are the ratio of the assumed photonic per-bit constants (5/25/10 pJ/bit) to the assumed electrical constants (65/35/50 pJ/bit) evaluated through the paper's own equation; the photonic values are assumed, not measured or cited, and Section 9 concedes the evaluation 'relies on predictive modeling using CelestiSim, rather than empirical hardware results.' Second, the up-to-3.66x/7.04x throughput and 1.40x/1.41x latency gains are fixed by the Table 5 input of 26,800 GB/s interconnect bandwidth, which the paper's own Section 3.3 specifications contradict: 7.2 Tbps (0.9 TB/s) per Photonic Fabric Module port and 115 Tbps (~14.4 TB/s) total all-to-all switching. 26,800 GB/s is 214 Tbps, about 1.9x the entire appliance switching capacity, and a single XPU attached to one port is capped at 0.9 TB/s — below the H100's 3,350 GB/s HBM bandwidth. The simulated gains are therefore forced by the chosen parameter, not derived from the disclosed PFA design. Because both central claims reduce to their inputs, the score is 6 (predictions reduce by construction) rather than 0-2; because the simulator is externally benchmarked and the qualitative analysis has independent content, it is not a fully self-referential derivation (not 8-10). Self-citation is not the load-bearing issue here: references [10,11] establish only architectural lineage, and the electrical per-bit costs cite NVIDIA manuals.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central results are driven by four assumed parameters: PFA capacity (32 TB), PFA bandwidth (26,800 GB/s), per-bit photonic energies (5/25/10 pJ/bit), and the transferability of H100/H200 utilization curves to PFA. None are measured. The only invented entity is the appliance itself, with no independent evidence. The axioms are the standard analytical-modeling assumptions plus the non-coherent, Megatron-style programming model.

free parameters (4)
  • Photonic per-bit energy constants = 5 pJ/bit transceiver, 25 pJ/bit switch, 10 pJ/bit intra-tray
    Assumed in Section 4.2 and directly drive the 60-90% energy savings claim. No measurement or citation is provided for these numbers.
  • PFA interconnect bandwidth = 26,800 GB/s (7.2 Tbps per module)
    Assumed in Sections 3.3 and 6.1; this value is the main driver of the throughput speedups in the simulator.
  • PFA shared memory capacity = 32 TB
    Assumed design parameter in Section 3.3; enables larger batch sizes and avoids tensor parallelism, contributing to the projected gains.
  • H100/H200 microbenchmark utilization curves = from Figure 6
    Fitted to hardware and used for baseline validation, then carried over to PFA with no evidence that the same utilization holds on a photonic memory interface.
assumptions (4)
  • domain assumption PFA can be integrated into XPUs by replacing a local HBM stack with a chiplet, without changing the GPU core design.
    Stated in Section 3.4 and used throughout; no prototype or measurement supports this integration path.
  • domain assumption The same FLOP and memory bandwidth utilization curves measured on H100/H200 apply to PFA configurations.
    Section 9 explicitly says they 'apply the same bandwidth utilization and FLOP efficiency models to estimate PFA performance.'
  • domain assumption Memory is accessed non-coherently with explicit partitioning, following a Megatron-style programming model.
    Listed in Section 9 as a system-level modeling assumption.
  • standard math Analytical roofline models are a valid approximation of LLM execution on distributed systems.
    The entire CelestiSim approach; similar assumptions are standard in cited prior simulation frameworks.
invented entities (1)
  • Photonic Fabric Appliance (PFA) / Photonic Fabric Module (PFM)
    purpose: Provides 32 TB shared memory and 115 Tbps switching to up to 16 XPUs; the subject of all performance claims.
    No physical prototype or measured data are presented; the claimed performance comes from extrapolating CelestiSim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Photonic Fabric Platform for AI Accelerators." pith.science (2026). https://pith.science/paper/EBWAJVFM

@misc{pith2026250714000,
  author       = {Pith},
  title        = {Pith review of: Photonic Fabric Platform for AI Accelerators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBWAJVFM}},
  note         = {Machine review of arXiv:2507.14000}
}
read the original abstract

This paper presents the Photonic FabricTM and the Photonic Fabric ApplianceTM (PFA), a photonic-enabled switch and memory subsystem that delivers low latency, high bandwidth, and low per-bit energy. By integrating high-bandwidth HBM3E memory, an on-module photonic switch, and external DDR5 in a 2.5D electro-optical system-in-package, the PFA offers up to 32 TB of shared memory alongside 115 Tbps of all-to-all digital switching. The Photonic FabricTM enables distributed AI training and inference to execute parallelism strategies more efficiently. The Photonic Fabric removes the silicon beachfront constraint that limits the fixed memory-to-compute ratio observed in virtually all current XPU accelerator designs. Replacing a local HBM stack on an XPU with a chiplet that connects to the Photonic Fabric increases its memory capacity and correspondingly its memory bandwidth by offering a flexible path to scaling well beyond the limitations of on-package HBM alone. We introduce CelestiSim, a lightweight analytical simulator validated on NVIDIA H100 and H200 systems. It is used to evaluate the performance of LLM reference and energy savings on PFA, without any significant change to the GPU core design. With the PFA, the simulation results show that up to 3.66x throughput and 1.40x latency improvements in LLM inference at 405B parameters, up to 7.04x throughput and 1.41x latency improvements at 1T parameters, and 60-90% energy savings in data movement for heavy collective operations in all LLM training scenarios. While these results are shown for NVIDIA GPUs, they can be applied similarly to other AI accelerator designs (XPUs) that share the same fundamental limitation of fixed memory to compute.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

27 extracted references · 19 canonical work pages

  1. [2]

    Xiao, B., & Su, L. (2024). ISO: Overlap of Computation and Communication within Seqenence For LLM Inference. arXiv preprint arXiv:2409.11155

    arXiv 2024

  2. [3]

    & Zhou, Z

    Wang, S., Wei, J., Sabne, A., Davis, A., Ilbeyi, B., Hechtman, B., ... & Zhou, Z. (2022, December). Overlap communication with dependent computation via decomposition in large deep learning models. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 (pp. 93-106)

    work page 2022

  3. [4]

    C., Skirlo, S., Prabhu, M., Baehr-Jones, T., Hochberg, M., Sun, X., Zhao, S., Larochelle, H., Englund, D., & Soljačić, M

    Shen, Y., Harris, N. C., Skirlo, S., Prabhu, M., Baehr-Jones, T., Hochberg, M., Sun, X., Zhao, S., Larochelle, H., Englund, D., & Soljačić, M. (2017). Deep Learning with Coherent Nanophotonic Circuits. Nature Photonics, 11(7), 441–446

    work page 2017

  4. [5]

    D., Bhaskaran, H., & Pernice, W

    Feldmann, J., Youngblood, N., Wright, C. D., Bhaskaran, H., & Pernice, W. H. P. (2021). Parallel Convolutional Processing Using an Integrated Photonic Tensor Core. Nature, 589(7840), 52–58

    work page 2021

  5. [6]

    These architectural differences are reflected in CelestiSim’s performance modeling, validating its ability to capture hardware-level characteristics

    Specifically, H200 exhibits marginally lower effective memory bandwidth. These architectural differences are reflected in CelestiSim’s performance modeling, validating its ability to capture hardware-level characteristics. 5 Power Savings of LLM Pre-training Power modeling with the CelestiSim demonstrates that migrating from conventional Ethernet-based Cl...

    work page 1979

  6. [7]

    NVIDIA (2025), A New Era in Data Center Networking with NVIDIA Silicon Photonics-based Network Switching

    work page 2025

  7. [8]

    J., & Aflatouni, F

    Ashtiani, F., Geers, A. J., & Aflatouni, F. (2022). An On-Chip Photonic Deep Neural Network for Image Classification. Nature, 606(7912), 501–506

    work page 2022

  8. [12]

    (2023, November)

    Isaev, M., McDonald, N., Dennison, L., & Vuduc, R. (2023, November). Calculon: a methodology and tool for high-level co-design of systems and large language models. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 1-14)

    work page 2023

Show all 27 references
  1. [13]

    (2024, November)

    Bang, J., Choi, Y., Kim, M., Kim, Y., & Rhu, M. (2024, November). vtrain: A simulation framework for evaluating cost-effective and compute-optimal large language model training. In 2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO) (pp. 153-167). IEEE

    2024

  2. [14]

    (2020, August)

    Rashidi, S., Sridharan, S., Srinivasan, S., & Krishna, T. (2020, August). Astra-sim: Enabling sw/hw co-design exploration for distributed dl training platforms. In 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) (pp. 81-92). IEEE

    2020

  3. [15]

    (2024, September)

    Cho, J., Kim, M., Choi, H., Heo, G., & Park, J. (2024, September). LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale. In 2024 IEEE International Symposium on Workload Characterization (IISWC) (pp. 15-29). IEEE

    2024

  4. [16]

    & Tumanov, A

    Agrawal, A., Kedia, N., Mohan, J., Panwar, A., Kwatra, N., Gulavani, B., ... & Tumanov, A. (2024). Vidur: A Large-Scale Simulation Framework For LLM Inference. Proceedings of Machine Learning and Systems, 6, 351-366

    2024

  5. [17]

    Ardalani, N., Pal, S., & Gupta, P. (2024). DeepFlow: A cross-stack pathfinding framework for distributed ai systems. ACM Transactions on Design Automation of Electronic Systems, 29(2), 1-20

    2024

  6. [21]

    https://doi.org/10.48550/ARXIV

    Reducing Activation Recomputation in Large Transformer Models. https://doi.org/10.48550/ARXIV. 2205.05198

    doi arXiv

  7. [27]

    Dao, T., Fu, D., Ermon, S., Rudra, A., & Ré, C. (2022). Flash-attention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35, 16344-16359

    2022

  8. [28]

    Nvidia ConnectX-6 User Manual

    NVIDIA. “Nvidia ConnectX-6 User Manual.” Manualslib, 2022, https://www.manualslib.com/manual/2957118/Nvidia-Connectx-6.html

    2022

  9. [29]

    Nvidia ConnectX-7 User Manual

    NVIDIA, “Nvidia ConnectX-7 User Manual.” Manualslib, 2023, https://www.manualslib.com/manual/3356680/Nvidia-Connectx-7.html

    2023

  10. [30]

    Nvidia Spectrum-3 SN4000 Series Switch Manuals

    NVIDIA. “Nvidia Spectrum-3 SN4000 Series Switch Manuals.” Manualslib, 2023, https://www.manualslib.com/products/Nvidia-Spectrum-3-Sn4000-Series-13172939.html

    2023

  11. [31]

    QM9700/QM9790 1U NDR 400Gb/s InfiniBand Switch Systems User Manual

    NVIDIA. “QM9700/QM9790 1U NDR 400Gb/s InfiniBand Switch Systems User Manual.” Sysgen, 2022, https://www.sysgen.de/media/pdf/6a/98/91/QM9700_QM9790_User_Manual.pdf

    2022

  12. [32]

    & Rao, V

    Mudigere, D., Hao, Y., Huang, J., Jia, Z., Tulloch, A., Sridharan, S., ... & Rao, V. (2022, June). Software-hardware co-design for fast and scalable training of deep learning recommendation models. In Proceedings of the 49th Annual International Symposium on Computer Architect...

    2022

  13. [64]

    Variable Output Length: The output sequence length varies over seven values—32, 64, 128, 256, 512, 1024, and 2048 tokens—while the input sequence length is fixed at 512 tokens

    To evaluate the impact of sequence lengths, we ran two sets of experiments for each model and TP and batch size configuration: Variable Input Length: The input sequence length varies over eight values—1, 32, 64, 128, 256, 512, 1024, and 2048 tokens—while the output sequence le...

  14. [2017]

    In 2017 IEEE 24th International Conference on High Performance Computing (HiPC)

    Parallel Deep Convolutional Neural Network Training by Exploiting the Overlapping of Computation and Communication. In 2017 IEEE 24th International Conference on High Performance Computing (HiPC). 183–192. https://doi.org/10. 1109/HiPC.2017.00030

    2017

  15. [2018]

    In 2018 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)

    SuperSim: Extensible Flit-Level Simulation of Celestial AI Large-Scale Interconnection Networks. In 2018 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 87–98. https://doi.org/10.1109/ISPASS.2018.00017

    2018

  16. [2019]

    https://doi.org/10.48550/ARXIV

    Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. https://doi.org/10.48550/ARXIV. 1909.08053

    1909 doi arXiv

  17. [2021]

    In 2021 IEEE International Symposiumon High-Performance Computer Architecture(HPCA).868–880

    Need for Speed: Experiences Building a Trustworthy System-Level GPU Simulator. In 2021 IEEE International Symposiumon High-Performance Computer Architecture(HPCA).868–880. https: //doi.org/10.1109/HPCA51647.2021.00077

    2021

  18. [2022]

    In 51th International Conference on Parallel Processing (Bordeaux, France) (ICPP 2022)

    ParaGraph: An application-simulator interface and toolkit for hardware-software co-design. In 51th International Conference on Parallel Processing (Bordeaux, France) (ICPP 2022). Association for Computing Machinery, New York, NY, USA, Article 61, 10 pages. https://doi.org/10.1...

    2022

  19. [2023]

    https://arxiv.org/abs/2302.13971

    Llama: Open and efficient foundation language models. https://arxiv.org/abs/2302.13971

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.