Pith. sign in

REVIEW 3 major objections 4 minor 30 references

HCCL offloads all collectives to dedicated message engines on MTIA 300, reaching 940 GB/s intra-rack bandwidth while degrading concurrent compute throughput by less than 0.5%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:36 UTC pith:6XPH2COZ

load-bearing objection A serious industry systems paper on offloaded collectives for an on-package-NIC accelerator: the design is genuinely new, but the headline numbers are device-side only and the pipelining assumption needs an end-to-end test. the 3 major comments →

arxiv 2608.00358 v1 pith:6XPH2COZ submitted 2026-08-01 cs.NI cs.DC

HCCL: Collective Communication for Meta Training and Inference Accelerators

Wesley Bland , Tiago Antunes , Lars Paul Huse , Chidambaram Muthu , Adel Abouchaev , Rabib Alam , Abdullah Alperen , Alexey Andronov
show 58 more authors
Jose Anto Akkara Vineet Badhwar Pavan Balaji Daniel Berkovitch Bartosz Bogdanski Shmeelok Chakraborty Sungjun Cho John Choi James Custer Rodrigo De Castro Nguyen Dinh Pham Matthew Edwards Kristian Evensen Evan Ezell Alex Finestead Seth Goldstein Prankur Gupta Ranwei Hu Adam Incera Anand Jayaraman Prashanth Kannan Soumil Kanwal Martin Karp Sameer Kumar Naina Kuruballi Mahesh Wei Lin Guay Cristian Lumezanu Cory Modlin Dag Georg Moxnes Hoang Nam Nguyen Ashay Narsale Jaden Padua Kirtesh Patil Minh Pham Amin Qassoud Ashwin Ramachandran David Ramon Prados Pallavi Shurpali Gregory R. Steinbrecher John Sundharam Vangelis Tasoulas Fuhou Tian Srinivas Vaidyanathan Vimal Vasudevan Nicolaas Viljoen Daniel Winkelman Yijing Zeng Zhaoqi Zhu Stig Arne Olsen Gilad Goldfarb Rajiv Krishnamurthy Rajeev Nair Jonas Olsson Joseph Provine Sreeram Ravinoothala Shivayogi Ugaji Hongyi Zeng Nairan Zhang
This is my paper
classification cs.NI cs.DC
keywords collective communicationMTIA 300message enginenear-memory computeRDMAcompute-communication overlapinference latencycompiled communication
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

HCCL is a collective communication library co-designed with the MTIA 300 accelerator, and its central claim is that fully offloading collective operations to dedicated message engines and near-memory compute units can saturate the chip's network bandwidth without stealing compute cycles. The paper reports up to 940 GB/s on intra-rack collectives, less than 0.5% degradation to concurrent compute throughput, and sub-6 microsecond collective latency for inference-optimized paths. If correct, this shows that hardware-integrated communication offload can make collective operations nearly free for training and fast enough for latency-sensitive inference. The paper also presents new one-sided and device-triggered collective designs that avoid host scheduling overhead, which matter for real-time serving.

Core claim

The paper's central discovery is that a collective communication library executed entirely on dedicated message engines (MEs) with near-memory compute (NMC) can achieve line-rate bandwidth and near-zero compute interference on an accelerator that integrates backend networking on the chip package. MTIA 300's 16 MEs autonomously execute compiled subgraphs of work queue elements, offloading AllToAllv, AllGather, and AllReduce from the compute grid. Measured results show 940 GB/s intra-rack bandwidth, sub-6 microsecond collective latency using one-sided PUT-based kernels for inference, and under 0.5% throughput loss on a concurrent GEMM. This demonstrates a decoupled communication path that satu

What carries the argument

The key machinery is the compiled communication model: the host compiles each collective into a set of subgraphs containing work queue elements (WQEs) that are dispatched to dedicated message engines for autonomous execution. Each ME contains a RISC-V control core (CPU-M) that unrolls subgraphs into NIC work requests, a NIC interface that writes WQEs directly to express doorbells, and near-memory compute units that perform reductions at line rate. Express doorbells treat the WQE write itself as the doorbell, eliminating separate queue reads and reducing latency. This machinery removes collective execution from the PE grid and enables full compute-communication overlap.

Load-bearing premise

The reported performance numbers assume that the host-side metadata copying and control-core scheduling overhead that were bypassed in the fine-grained timers are fully hidden by pipelining in real workloads; if that hiding fails under load imbalance or host-side jitter, end-to-end performance will be materially lower than the reported figures.

What would settle it

Run the same collective benchmarks on MTIA 300 with a production-style workload that introduces host-side jitter and load imbalance, measuring end-to-end time without bypassing metadata copying and CPU-C scheduling. If the hidden overheads from Figure 10 become exposed and the collective time or compute degradation exceeds the reported 940 GB/s and 0.5% bounds, the paper's central claim would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Collective communication no longer consumes compute resources, so training workloads can overlap large AllReduce and AllToAllv operations with compute almost completely.
  • Bandwidth-bound collectives within a scale-up domain can approach the full 800 GB/s scale-up bandwidth, and topology-aware algorithms can effectively use the asymmetric scale-out links for larger jobs.
  • Inference workloads can achieve sub-6 microsecond collectives using one-sided PUT-based kernels, making offloaded collectives viable for latency-sensitive serving.
  • Device-triggered collectives allow communication to be pre-scheduled and fused with compute kernels in graph mode, avoiding host-side scheduling overhead and reducing exposed latency.
  • The express doorbell design's QP limits are manageable through dynamic QP reuse, a finding that informs similar integrated-NIC architectures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the 940 GB/s and sub-0.5% degradation numbers hold in production, communication could stop being a primary scaling bottleneck for recommendation-model training at rack scale, shifting focus to compute and memory efficiency.
  • The device-triggered collective pattern (attaching a semaphore to a collective so it waits for a compute kernel to signal) could generalize to any accelerator with a programmable network interface, not just MTIA.
  • The paper's benchmarking methodology bypasses host-side metadata copying and control-core scheduling overhead; in production with host jitter or load imbalance, these hidden costs may become exposed, so end-to-end gains may be smaller than the synthetic benchmarks suggest.
  • The compiled subgraph model's separation of collective description from execution could be combined with automated algorithm search to generate topology-optimal collectives for future asymmetric interconnects.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents HCCL, a collective communication library for Meta's MTIA 300 accelerator. HCCL offloads collective execution from the compute grid to dedicated message engines with near-memory compute, using a compiled-communication model in which the host generates subgraphs that are autonomously executed by the message engines. The paper describes the MTIA 300 communication hardware, HCCL's control/data path, topology-aware algorithm selection for asymmetric scale-up/scale-out networks, and optimization paths for training and inference. Headline results are up to 940 GB/s intra-rack collective bandwidth, less than 0.5% degradation to concurrent GEMM throughput, and sub-6 microsecond collective latency for inference-oriented paths.

Significance. If the reported results are reproducible, this is a strong systems contribution: it demonstrates a complete alternative to kernel-resident collective execution, with dedicated NICs, message engines, and near-memory reduction hardware integrated on the accelerator package. The compiled-subgraph model and the device-triggered/one-sided paths are concrete mechanisms for hiding host overhead and enabling fused inference kernels. The paper is also candid about component-level overheads (Figure 10), which is useful. However, the significance is limited by the absence of released code/data/artifacts and by the fact that the headline numbers are measured with device-side timers that exclude host-side metadata copying and control-core scheduling; the end-to-end behavior under production skew is therefore not externally verifiable.

major comments (3)
  1. [Section V-D / V-B] The headline numbers (940 GB/s, <0.5% compute degradation, sub-6us latency) are measured with 'fine-grained event timers inserted before and after each collective's execution, bypassing all metadata copying to the device and any scheduling computational overhead from the control core.' The paper asserts these bypassed costs are hidden by pipelining, but it does not demonstrate this under the failure modes that matter: host-side jitter, rank skew, or a preceding compute kernel that finishes before the metadata copy. Please provide an end-to-end measurement that includes host-side graph generation/copy and CPU-C dispatch, or quantify the exposed overhead under controlled skew; otherwise the 'end-to-end performance' section is only measuring the device-side subset of the path.
  2. [Section V-D, Fig. 11] The paper states a 'total theoretical bandwidth of 1 TB/s (800 GB/s scale-up, 200 GB/s scale-out)' and then reports 'up to 940 GB/s in collective performance for collectives that remain within a single scale-up domain.' A collective confined to the scale-up domain cannot exceed 800 GB/s unless the bandwidth metric counts both directions or uses a nonstandard definition. Define the metric and reconcile 940 GB/s with the 800 GB/s scale-up budget; as written, the headline number is internally inconsistent with the stated link budget.
  3. [Section V-E, Fig. 12] The claim of 'less than 0.5% degradation' is supported only by 'variance of up to 1 TFlop (around 0.5%)' on a single GEMM workload at 16 ranks. No error bars, number of runs, or confidence intervals are reported. Because this is one of the three headline claims, please report the distribution across repeated runs and across at least one additional workload, or reduce the strength of the claim to the specific PARAM GEMM configuration measured.
minor comments (4)
  1. [Section V-F3] The sub-6us result is obtained from PE-grid collective kernels, which consume the compute resources that the ME-offload design is meant to free. Please clarify in the abstract/contribution list whether this path is part of HCCL's inference story or an alternative design; as written, it blurs the central claim about offloaded collectives.
  2. [Fig. 14] The text refers to a 'performance comparison of device-triggered collectives against their eagerly executed counterparts,' but the y-axis units and the exact measured quantities are not stated. Add axis labels and report the numeric speedup or latency values in the text.
  3. [Section V-D] Please define 'on-the-wire bandwidth' and state whether 940 GB/s is a bidirectional aggregate or a per-direction rate. This is needed to interpret Figure 11 and to reconcile the number with the 800 GB/s scale-up link budget.
  4. [Section V-B] The statement 'We do not observe any exposed host-time execution... when validating these workloads' is not backed by a figure, table, or numerical threshold. If this is a measured property, please cite the measurement; if it is a qualitative observation, say so explicitly.

Circularity Check

0 steps flagged

No significant circularity: headline numbers are direct measurements, not predictions derived from fitted inputs or self-citations.

full rationale

This is an empirical systems paper whose central claims — up to 940 GB/s intra-rack collective bandwidth, less than 0.5% compute degradation, and sub-6µs inference-path latency — are presented as measurements of the actual MTIA 300 system (Section V, Figures 11, 12, 16). There is no derivation chain in which an output variable is defined in terms of an input variable, no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors, and no ansatz smuggled in via citation. The self-citations ([18], [25], [26]) provide hardware background, workload context, and prior work references (e.g., AllToAllvDynamic from [25]), but the paper's performance claims are independently measured rather than derived from those papers. Section V-D's statement that data are collected from 'fine-grained event timers inserted before and after each collective's execution, bypassing all metadata copying to the device and any scheduling computational overhead' is a methodological scope caveat, not a circular construction: it narrows what is measured but does not equate the measurement with the claimed result by definition. Similarly, Section V-B's assertion that 'host-side generation time is hidden from the critical path due to the pipelining effect on device-side execution' is an empirical assumption about production overlap, not a definitional equivalence. The paper is self-contained against external benchmarks in the sense that its headline numbers are benchmark results on real hardware, so the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper introduces no free parameters or invented entities; it is an empirical systems paper. Its two main assumptions are the accuracy of the hardware description and the representativeness of the benchmark methodology.

axioms (2)
  • domain assumption MTIA 300 hardware description (Section II) accurately represents the shipped chip: 16 MEs, NMC bandwidth of 2.8 TB/s, 1 TB/s aggregate NIC bandwidth, express doorbells, etc.
    All performance claims depend on the existence and behavior of the described hardware. If, for example, the NMC cannot sustain the stated 128 B/cycle or the NICs do not deliver 800 Gb/s, the reported bandwidths would be unattainable.
  • domain assumption The benchmark methodology (Section V) is representative of real production workloads and the bypassed scheduling overheads (Sections V-C, V-D) are fully hidden in practice.
    The end-to-end numbers are measured with event timers that omit host-side metadata copying and control-core scheduling. The paper argues these are overlapped with compute during real workloads, but this is not proven for all conditions.

pith-pipeline@v1.3.0-daily-deepseek · 15328 in / 5584 out tokens · 58156 ms · 2026-08-04T00:36:23.465090+00:00 · methodology

0 comments
read the original abstract

We present HCCL, a collective communication library co-designed with Meta's MTIA 300 accelerator, the first Meta chip to integrate backend networking directly on chip package. MTIA 300 includes dedicated message engines (MEs) with near-memory compute (NMC) that fully offload collective execution from the compute grid, enabling large overlap between computation and communication. HCCL uses a compiled communication model in which the host generates a complete description of each collective including dependencies. We describe the control and data path architecture, topology-aware algorithm selection across MTIA 300's asymmetric scale-up and scale-out network, and optimizations for both training and inference workloads. For training, HCCL achieves up to 940 GB/s on intra-rack collectives while introducing less than 0.5% degradation to concurrent compute throughput. For inference, we leverage one-sided communication primitives that bypass the scheduling path to minimize collective latency and describe collective designs that improve compute-communication pipelining for latency-sensitive workloads.

Figures

Figures reproduced from arXiv: 2608.00358 by Abdullah Alperen, Adam Incera, Adel Abouchaev, Alexey Andronov, Alex Finestead, Amin Qassoud, Anand Jayaraman, Ashay Narsale, Ashwin Ramachandran, Bartosz Bogdanski, Chidambaram Muthu, Cory Modlin, Cristian Lumezanu, Dag Georg Moxnes, Daniel Berkovitch, Daniel Winkelman, David Ramon Prados, Evan Ezell, Fuhou Tian, Gilad Goldfarb, Gregory R. Steinbrecher, Hoang Nam Nguyen, Hongyi Zeng, Jaden Padua, James Custer, John Choi, John Sundharam, Jonas Olsson, Jose Anto Akkara, Joseph Provine, Kirtesh Patil, Kristian Evensen, Lars Paul Huse, Martin Karp, Matthew Edwards, Minh Pham, Naina Kuruballi Mahesh, Nairan Zhang, Nguyen Dinh Pham, Nicolaas Viljoen, Pallavi Shurpali, Pavan Balaji, Prankur Gupta, Prashanth Kannan, Rabib Alam, Rajeev Nair, Rajiv Krishnamurthy, Ranwei Hu, Rodrigo De Castro, Sameer Kumar, Seth Goldstein, Shivayogi Ugaji, Shmeelok Chakraborty, Soumil Kanwal, Sreeram Ravinoothala, Srinivas Vaidyanathan, Stig Arne Olsen, Sungjun Cho, Tiago Antunes, Vangelis Tasoulas, Vimal Vasudevan, Vineet Badhwar, Wei Lin Guay, Wesley Bland, Yijing Zeng, Zhaoqi Zhu.

Figure 2
Figure 2. Figure 2: Message Engine (ME) architecture. 1) Control Core (CPU-C): Before any work is run on the compute or communication portion of the MTIA 300 chip, it is first submitted through a stream to the CPU-C to ensure it is ready to be dispatched and that the place where it will be executed is ready to receive it. The CPU-C (shown in purple in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 1
Figure 1. Figure 1: The MTIA 300 package with compute, network, and HBM. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: MTIA 300 software stack. III. COMMUNICATION SOFTWARE ARCHITECTURE The software stack for MTIA 300 has been closely co￾designed with the hardware described in Section II and the needs of our applications. A full view of the software stack can be seen in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: An example of an AllReduce ring algorithm with 4 nodes. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Collective distribution for a ranking and recommendation workload [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Collective distribution for a ranking and recommendation workload [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Once the event pipeline is full, we observe a [PITH_FULL_IMAGE:figures/full_fig_p007_10.png] view at source ↗
Figure 10
Figure 10. Figure 10: Breakdown of execution time per component of execution [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: End-to-end test sweep across different job sizes [PITH_FULL_IMAGE:figures/full_fig_p008_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Variance of GEMM performance when scaling collective size at [PITH_FULL_IMAGE:figures/full_fig_p009_12.png] view at source ↗
Figure 15
Figure 15. Figure 15: Execution example of Device-triggered execution. [PITH_FULL_IMAGE:figures/full_fig_p009_15.png] view at source ↗
Figure 13
Figure 13. Figure 13: RDMA Write WQE RTT comparison between PE vs ME [PITH_FULL_IMAGE:figures/full_fig_p009_13.png] view at source ↗
Figure 16
Figure 16. Figure 16: Performance of PUT-based collective implementations [PITH_FULL_IMAGE:figures/full_fig_p010_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Performance of collective operations. “Time %” represents the ratio [PITH_FULL_IMAGE:figures/full_fig_p011_17.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 1 canonical work pages

  1. [1]

    RCCL: ROCm communication collectives library,

    AMD, “RCCL: ROCm communication collectives library,” accessed: 2026-03-10. [Online]. Available: https://github.com/ROCm/rocm- systems/tree/develop/projects/rccl

    2026

  2. [2]

    MPICH: High-performance portable MPI,

    Argonne National Laboratory, “MPICH: High-performance portable MPI,” 2026, accessed: 2026-03-10. [Online]. Available: https://www. mpich.org/

    2026

  3. [3]

    SHMEM user’s guide for C,

    R. Barriuso and A. Knies, “SHMEM user’s guide for C,” Cray Research, Inc., Tech. Rep. SN-2516, 1994

    1994

  4. [4]

    Meta’s second generation ai chip: Model-chip co-design and productionization experiences,

    J. Coburn, C. Tang, S. A. Asal, N. Agrawal, R. Chinta, H. Dixit, B. Dodds, S. Dwarakapuram, A. Firoozshahian, C. Gaoet al., “Meta’s second generation ai chip: Model-chip co-design and productionization experiences,” inProceedings of the 52nd Annual International Sympo- sium on Computer Architecture, 2025, pp. 1689–1702

    2025

  5. [5]

    Pytorch distributed overview,

    W. Constable and W. Feng, “Pytorch distributed overview,”

  6. [6]

    MSCCLang: Microsoft collective communication language,

    M. Cowan, S. Maleki, M. Musuvathi, O. Saarikivi, and Y . Xiong, “MSCCLang: Microsoft collective communication language,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2023, pp. 502–514. [Online]. Available: https://doi.org/10.1145/3575693.3575724

    arXiv 2023

  7. [7]

    MTIA: First generation silicon targeting meta’s recommendation systems,

    A. Firoozshahian, J. Coburn, R. Levenstein, R. Nattoji, A. Kamath, O. Wu, G. Grewal, H. Aepala, B. Jakka, B. Dreyeret al., “MTIA: First generation silicon targeting meta’s recommendation systems,” inProceedings of the 50th Annual International Symposium on Computer Architecture, 2023, pp. 1–13. [Online]. Available: https://doi.org/10.1145/3579371.3589348

    arXiv 2023

  8. [8]

    Open MPI: Goals, concept, and design of a next generation MPI implementation,

    E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V . Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall, “Open MPI: Goals, concept, and design of a next generation MPI implementation,” inProceedings, 11th European PVM/MPI Users’ Group Meeting, Budapest, Hungary, Septembe...

    doi:10.1007/978-3-540-30218-6 2004

  9. [9]

    Scalable Hierarchical Aggregation and Reduction Protocol (SHARP): A Hardware Architecture for Efficient Data Reduction,

    R. L. Graham, L. Levi, D. Bureddy, G. Bloch, G. Shainer, H. Subramoni, S. Chakraborty, and D. K. Panda, “Scalable Hierarchical Aggregation and Reduction Protocol (SHARP): A Hardware Architecture for Efficient Data Reduction,” inProceedings of the First Workshop on Optimization of Communication in HPC (COM-HPC), 2016. [Online]. Available: https://doi.org/1...

    work page doi:10.1109/com-hpc.2016.5 2016

  10. [10]

    Practical lessons from predicting clicks on ads at facebook,

    X. He, J. Pan, O. Jin, T. Xu, B. Liu, T. Xu, Y . Shi, A. Atallah, R. Herbrich, S. Bowerset al., “Practical lessons from predicting clicks on ads at facebook,” inProceedings of the eighth international workshop on data mining for online advertising, 2014, pp. 1–9. [Online]. Available: https://doi.org/10.1145/2648584.2648589

    arXiv 2014

  11. [11]

    oneCCL: Intel oneAPI collective communications library,

    Intel Corporation, “oneCCL: Intel oneAPI collective communications library,” accessed: 2026-03-10. [Online]. Available: https://github.com/ uxlfoundation/oneCCL

    2026

  12. [12]

    [Online]

    Message Passing Interface Forum,MPI-2: Extensions to the Message- Passing Interface, July 1997. [Online]. Available: https://www.mpi- forum.org/docs/mpi-2.0/mpi2-report.pdf

    1997

  13. [13]

    [Online]

    Message Passing Interface Forum,MPI: A Message Passing Interface Standard V ersion 3.0, September 2012. [Online]. Available: https: //www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf

    2012

  14. [14]

    [Online]

    Message Passing Interface Forum,MPI: A Message-Passing Interface Standard V ersion 5.0, June 2025. [Online]. Available: https://www.mpi- forum.org/docs/mpi-5.0/mpi50-report.pdf

    2025

  15. [15]

    Gloo: Collective communications library,

    Meta Platforms, “Gloo: Collective communications library,” accessed: 2026-03-10. [Online]. Available: https://github.com/facebookincubator/ gloo

    2026

  16. [16]

    PARAM Benchmarks,

    Meta Platforms, “PARAM Benchmarks,” accessed: 2026-03-10. [Online]. Available: https://github.com/facebookresearch/param

    2026

  17. [17]

    MSCCL: Microsoft collective communication library,

    Microsoft, “MSCCL: Microsoft collective communication library,” accessed: 2026-03-10. [Online]. Available: https://github.com/microsoft/ msccl

    2026

  18. [18]

    Mtia-3: Meta’s first training chip featuring built-in nics and collective offloading engines,

    MTIA Team, “Mtia-3: Meta’s first training chip featuring built-in nics and collective offloading engines,” inISCA ’26: Proceedings of the 53rd Annual International Symposium on Computer Architecture, 2026

    2026

  19. [19]

    Deep learning recommendation model for personalization and recommendation systems,

    M. Naumov, D. Mudigere, H.-J. M. Shi, J. Huang, N. Sundaraman, J. Park, X. Wang, U. Gupta, C.-J. Wu, A. G. Azzoliniet al., “Deep learning recommendation model for personalization and recommendation systems,”arXiv preprint arXiv:1906.00091, 2019. [Online]. Available: https://doi.org/10.48550/arXiv.1906.00091

    Pith/arXiv arXiv doi:10.48550/arxiv.1906.00091 1906

  20. [20]

    NCCL: NVIDIA collective communications library,

    NVIDIA Corporation, “NCCL: NVIDIA collective communications library,” accessed: 2026-03-10. [Online]. Available: https://developer. nvidia.com/nccl

    2026

  21. [21]

    NVSHMEM: NVIDIA’s implementation of OpenSHMEM for GPU clusters,

    NVIDIA Corporation, “NVSHMEM: NVIDIA’s implementation of OpenSHMEM for GPU clusters,” NVIDIA Developer Documentation, accessed: 2026-03-10. [Online]. Available: https://developer.nvidia.com/ nvshmem [22]NVIDIA BlueField-2 DPU Datasheet, NVIDIA Corporation,

    2026

  22. [23]

    OpenSHMEM application programming interface, version 1.5,

    Open Source Software Solutions, Inc., “OpenSHMEM application programming interface, version 1.5,” Open Source Software Solutions, Inc., Tech. Rep., 2020. [Online]. Available: http://openshmem.org/ specification

    2020

  23. [24]

    Bandwidth optimal all-reduce algorithms for clusters of workstations,

    P. Patarasuk and X. Yuan, “Bandwidth optimal all-reduce algorithms for clusters of workstations,”Journal of Parallel and Distributed Computing, vol. 69, pp. 117–124, 02 2009. [Online]. Available: https://doi.org/10.1016/j.jpdc.2008.09.002

    doi:10.1016/j.jpdc.2008.09.002 2009

  24. [25]

    Collective communication for 100k+ gpus,

    M. Si, P. Balaji, Y . Chen, C.-H. Chu, A. Gangidi, S. Hasan, S. Iyengar, D. Johnson, B. Liu, R. Ren, D. Shah, A. J. Shetty, G. Steinbrecher, Y . Wang, B. Wu, X. Xie, J. Yang, M. Yang, K. Yu, M. Yu, C. Zhao, W. Bland, D. Boyda, S. Gumudavelli, P. Kannan, C. Lumezanu, R. Miao, Z. Qu, V . Ramesh, M. Samoylov, J. Seidel, S. Sundaresan, F. Tian, Q. Tan, S. Zha...

    doi:10.48550/arxiv.2510.20171 2026

  25. [26]

    Four mtia chips in two years: Scaling ai experiences for billions,

    Y . J. Song, A. Tulloch, H. Reddy, C. Tang, and V . Thakkar, “Four mtia chips in two years: Scaling ai experiences for billions,” 2026, accessed: 2026-03-11. [Online]. Available: https://ai.meta.com/blog/meta-mtia- scale-ai-chips-for-billions

    2026

  26. [27]

    torchcomms: a modern PyTorch communications API,

    Team torchcomms at Meta, “torchcomms: a modern PyTorch communications API,” 2025. [Online]. Available: https://pytorch.org/ blog/torchcomms/

    2025

  27. [28]

    Actions speak louder than words: trillion-parameter sequential transducers for generative recommendations,

    J. Zhai, L. Liao, X. Liu, Y . Wang, R. Li, X. Cao, L. Gao, Z. Gong, F. Gu, J. He, Y . Lu, and Y . Shi, “Actions speak louder than words: trillion-parameter sequential transducers for generative recommendations,” inProceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2024. [Online]. Available: https://doi.org/10.485...

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.17152 2024

  28. [29]

    DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction,

    B. Zhang, L. Luo, X. Liu, J. Li, Z. Chen, W. Zhang, X. Wei, Y . Hao, M. Tsang, W. Wanget al., “DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction,”arXiv preprint arXiv:2203.11014, 2022. [Online]. Available: https://doi.org/10.48550/arXiv.2203.11014

    Pith/arXiv arXiv doi:10.48550/arxiv.2203.11014 2022

  29. [2020]

    Available: https://docs.pytorch.org/tutorials/beginner/ dist overview.html

    [Online]. Available: https://docs.pytorch.org/tutorials/beginner/ dist overview.html

  30. [2021]

    Available: https://www.nvidia.com/content/dam/en-zz/ Solutions/Data-Center/documents/datasheet-nvidia-bluefield-2-dpu.pdf

    [Online]. Available: https://www.nvidia.com/content/dam/en-zz/ Solutions/Data-Center/documents/datasheet-nvidia-bluefield-2-dpu.pdf