Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training

Jonghyun Lee; Murali Annavaram; Rachit Rajat; Yongqin Wang

REVIEW 5 major objections 6 minor 36 references

Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GPU TEEs turn gradient synchronization into the dominant cost of distributed ML training, with measured slowdowns averaging 8.68x and reaching 41.64x on four GPUs.

desk verdict Plausible structural account of why GPU TEEs slow down DDP training, but the headline slowdown numbers are emulation estimates that need stronger validation before being treated as measurements. read the letter →

arxiv 2501.11771 v3 pith:H2SJPW4T submitted 2025-01-20 cs.CR cs.DC

Jonghyun Lee , Yongqin Wang , Rachit Rajat , Murali Annavaram This is my paper · ORCID

classification cs.CRcs.DC

keywords GPUTEEconfidentialcomputingdistributeddataparalleltrainingringall-reduceAES-GCMgradientsynchronizationperformancecharacterizationbucket_cap_mb

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to quantify what GPU Trusted Execution Environments (TEEs) cost distributed data-parallel (DDP) training. It argues that because only the CPU and GPU packages are trusted, every gradient message crossing PCIe or NVLink must be AES-GCM encrypted and authenticated, and that this turns the default ring all-reduce into the dominant bottleneck. The paper counts the resulting work as 4×k×(n−1) encryption/authentication operations per training iteration, and measures an average 8.68x and maximum 41.64x per-iteration slowdown on four GPUs. It also shows that enlarging DDP's gradient bucket from 44MB to 400MB cuts the TEE overhead sharply, yet still leaves a 3.03x gap to unsecured training. The practical stakes: cloud ML that promises confidentiality may be unusably slow unless communication designs are rethought for TEEs.

What carries the argument

The load-bearing object is the secured ring all-reduce, together with the count that follows from it. Each of the n−1 scatter-reduce and n−1 all-gather steps requires one encryption plus MAC generation on the sender and one decryption plus authentication on the receiver, making 4×(n−1) crypto operations per all-reduce; with k asynchronous all-reduces per iteration, the total is 4×k×(n−1). The paper's emulation uses AES-GCM with a multi-chaining authentication scheme that divides the serial GHASH authentication chain into parallel chunks, so the measured cost is a software approximation of the TEE's crypto rather than a measurement of the vendor's undisclosed hardware implementation.

What would settle it

Run the identical DDP workloads on genuine multi-GPU GPU TEEs once the vendor's distributed TEE drivers are available, and compare per-iteration times at 2, 4, and 8 GPUs. If GPT2-Xlarge's four-GPU slowdown is not near 41.64x, or if the count 4×k×(n−1) fails to predict the relative scaling across GPU counts, the emulation-based numbers are not representative.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that secure multi-GPU DDP training is dominated by the cryptography that GPU TEEs force onto gradient synchronization. Ring all-reduce is n−1 scatter-reduce steps plus n−1 all-gather steps, and each step encrypts and MACs at the sender and decrypts and authenticates at the receiver; with k asynchronous all-reduces per iteration, that is 4×k×(n−1) cryptographic operations, so the TEE tax grows linearly in both GPU count and model-driven all-reduce frequency. Combining those factors, the authors measure per-iteration runtime increases from 1.97x (ResNet50, 2 GPUs) to 41.64x (GPT2-XL, 4 GPUs), and find that asynchronous overlap — a central DDP optimization — gives almost nothing back because many small transfers multiply fixed crypto costs. Raising the gradient bucket size to 400MB batches gradients into fewer, larger transfers, reducing TEE cost by up to 7.31x in the four-GPU case, but the secure setting still ends up about 3x slower than the unsecured baseline.

Load-bearing premise

The measured slowdowns assume the authors' custom CUDA crypto kernels behave like the real, not-yet-released multi-GPU TEE hardware; the paper validates only single-GPU performance, leaving NVLink crypto, driver involvement, and multi-GPU contention unverified.

Editorial extensions

If this is right

Total TEE crypto work per iteration equals 4×k×(n−1), so doubling the GPU count from 2 to 4 triples the crypto overhead for the same model and all-reduce count.
The default 44MB gradient bucket is a near-worst configuration under a TEE: raising it to 400MB cuts the number of transfers from 145 to 9 in GPT2-XL and reduces runtime by 4.95x with 2 GPUs and 7.31x with 4 GPUs.
Even with the best bucket setting, secure four-GPU DDP remains about 3x slower than unsecured DDP, so the gap is structural, not just a tuning artifact.
At eight GPUs the average slowdown reaches about 16x and GPT2-XL reaches 81.79x, with TEE overhead consuming 98.8% of per-iteration time.
The findings motivate TEE-aware collective-communication design rather than relying on the overlap optimizations that work well without TEEs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 4k(n−1) scaling law should transfer to other collective patterns that also cross the trusted boundary, so sharded-data-parallel all-gather/reduce-scatter and pipeline-parallel send/receive will face the same crypto-per-message tax, proportional to message count rather than just bytes.
If the hardware vendor eventually ships accelerated NVLink crypto or allows software to trust NVLink, the absolute slowdowns would drop, but under the stricter package-only threat model the count-driven overhead remains the fundamental design constraint.
The bucket-size result suggests an automatic, TEE-aware tuning rule: when a TEE is active, frameworks should maximize per-transfer size up to the point where authentication parallelism saturates, instead of optimizing for overlap.
Extending beyond one node, inter-node gradient exchange routes through CPU TEEs, so CPU encryption throughput would join the critical path and likely dominate; the paper notes this but does not measure it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: the paper gives a clean structural explanation for why GPU TEEs make DDP training slow, and the 3–42x slowdown range is plausible. But the headline numbers are estimates from an emulation that is only validated on a single GPU, so the multi-GPU NVLink path is unverified. I'd send it to review with a request for tighter validation and some consistency fixes.

What's genuinely new: this is the first DDP-specific characterization I know of that covers large models (BERT, GPT2) and multiple GPUs. The decomposition into 4×k×(n−1) crypto operations per GPU per iteration is a useful mental model. The bucket_cap_mb mitigation is practical, and the empirical finding that batching reduces overhead by roughly 5–7x is worth knowing.

The paper is also honest: Section 4.1 states clearly that NVIDIA hasn't released drivers for distributed CC, so they implemented AES-GCM as CUDA kernels. The single-GPU validation (within 1.3–16%) is real evidence that the CPU-GPU path is roughly right, but it does not cover the GPU-to-GPU gradient path, where their kernels account for over 93% of runtime in GPT2 models. That's the core caveat. Real NVIDIA hardware may use different GMAC batching or hardware accelerators, so the absolute slowdowns could move. The stress-test note is correct on this point.

Minor but real issues: the max slowdown is reported as 41.64x in the abstract and introduction, but Figure 10a shows 42.36x, and the model name flips between GPT2-Large and GPT2-Xlarge. The “total” crypto-op count in Section 3.3 is per GPU, not system-wide; the wording should be fixed. No error bars are reported. These are fixable.

Who should read it: anyone working on confidential ML systems, especially people deciding whether GPU TEEs are viable for distributed training. It is a useful data point, but cite it as an emulation-based estimate.

Recommendation: accept for peer review. The final version should add error bars, release artifacts, and explicitly frame the multi-GPU results as extrapolations from a validated single-GPU emulation. The structural argument is solid and the topic is timely.

Referee Report

5 major / 6 minor

Summary. The paper presents a characterization of GPU TEE (NVIDIA Confidential Computing) overheads in PyTorch Distributed Data Parallel (DDP) training. It argues that ring all-reduce, the default gradient-synchronization algorithm, incurs encryption, decryption, and MAC authentication at every sender and receiver, leading to a structural crypto-operation count of 4×k×(n−1) for k asynchronous all-reduces across n GPUs. Because NVIDIA has not released drivers for distributed CC in a single CVM, the authors implement AES-GCM encryption/authentication as custom CUDA kernels, validate that implementation against a single-GPU H100 CC system, and then measure 2-, 4-, and 8-GPU training with the emulated kernels. They report an average 8.68× per-iteration slowdown and a maximum slowdown of 41.64×/42.36× on four GPUs, and propose increasing the DDP bucket_cap_mb parameter to batch gradient transfers, which reduces the four-GPU slowdown to 3.03×. The paper's main contribution is a structural breakdown of where TEE overhead arises and a practical hyperparameter mitigation.

Significance. If the emulated measurements faithfully track real NVIDIA CC hardware, the paper provides a useful and timely quantification of a serious practical problem: GPU TEEs impose a large, scaling security tax on multi-GPU DDP training, and the default DDP gradient bucket size is poorly suited to secured settings. The paper's strengths include a structural operation count that is derived from the ring all-reduce algorithm rather than fitted to data, a broad benchmark sweep (ResNet-50/101, BERT-Base/Large, GPT-2 Large/XL), a single-GPU validation against actual H100 CC hardware, and a mitigation analysis that explains why batching reduces per-transfer overhead. The main limitation is that the central multi-GPU numbers are emulation estimates whose fidelity on the NVLink gradient path is not directly validated; the single-GPU validation exercises a different bottleneck. The paper is therefore significant if the emulation assumptions hold, but its headline claims need to be reframed and the internal inconsistencies corrected before the results can be taken as measurements of real NVIDIA CC hardware.

major comments (5)

[4.1–4.2, Figures 9–10] The multi-GPU results are emulation estimates, not direct measurements of NVIDIA CC. Section 4.1 states that the authors implemented AES-GCM as custom CUDA kernels because no drivers support distributed CC in a single CVM, and Section 4.2 validates this emulation only against single-GPU H100 CC training, where the residual error reaches 16% for ResNet and where the CPU-to-GPU transfer path dominates. That validation does not exercise GPU-side encryption/decryption/MAC on the NVLink gradient path, which is precisely the path that accounts for over 93% of runtime in the GPT-2 cases (Figure 9b) and underlies the headline slowdowns. The paper should either validate the multi-GPU kernels against real distributed CC hardware or, if that is not possible, explicitly reclassify all multi-GPU numbers as emulation estimates and change the wording 'measured' and 'observed' throughout.
[3.3, Conclusion] The crypto-operation count is per GPU, but the paper repeatedly presents it as a 'total.' For n GPUs, each GPU performs n−1 sends and n−1 receives per ring all-reduce; per GPU that is 4(n−1) encryption/decryption/authentication operations, and with k all-reduces per iteration the per-GPU total is 4k(n−1). System-wide, however, the total is 4nk(n−1), not 4k(n−1). The examples in Section 3.3 (60 operations for ResNet-50 and 1704 for GPT-2-XL at n=4) are therefore per-GPU values. Because the conclusion and Section 5.3.1 repeat the 'total' wording, the paper should clarify whether it is quantifying per-device or cluster-wide crypto work and adjust the run-time implications accordingly.
[Abstract, Section 3.3, Section 5.2, Figure 10a, Conclusion] The headline maximum slowdown is internally inconsistent. The abstract reports 'a maximum of 41.6x,' the introduction bullet says 'at most 41.64x in GPT-Xlarge,' Section 5.2 and Figure 10a give 42.36x for GPT2-Large and 41.64x for GPT2-Xlarge, and the conclusion repeats 41.64x for GPT2-Xlarge. These are different values for different models and labels. Since the maximum slowdown is a central claimed result, the authors must reconcile the number and the model name so that the figure, text, and summary all agree.
[5.2, Figure 10b] The sentence 'the percentage of TEE overhead in total runtime increases from a minimum of 4.5% in GPT2-Xlarge to a striking 75.1% in Resnet101 compared to two GPU cases' is not interpretable as written and appears to contradict Figure 10b, in which the GPT-2 models should have the largest TEE-overhead share, not the smallest. Either the sentence contains a typo (for example, 'maximum' instead of 'minimum' or a missing model name) or the figure is mislabeled. This must be clarified because the runtime decomposition is the evidence for the claim that TEE overhead dominates training time.
[Figures 9–12] No measure of experimental variability is reported. All runtime comparisons are single numbers without error bars, number of repetitions, or any indication of run-to-run variance. Given that the paper reports precise slowdown factors and even contains an internal discrepancy (41.64x vs 42.36x), the authors should report the mean and standard deviation (or at least a range) over multiple runs for the headline configurations.

minor comments (6)

[Introduction, Section 5.3] The two-GPU mitigation factor is given as 4.95x in the introduction and 4.94x in Section 5.3; these should be reconciled.
[Section 7] The reference to PipeLLM appears as [30] in Section 7, but the bibliography entry for PipeLLM is [31]; also, 'Section.7' in Section 4.1 should be 'Section 7.'
[Section 4.2] The text refers to 'Resnet100'; the model evaluated is ResNet-101.
[Section 6.2] The text contains 'uring FSDP training' which should be 'during FSDP training.'
[Section 5.3.1] The statement that 8 GPUs 'incur a total of 28 encryption and authentication overhead per all-reduce' should say '28 per GPU,' to be consistent with the per-GPU interpretation of the count in Section 3.3.
[Section 6.1] The claim of an average 1.81x slowdown for tree-based all-reduce relative to ring all-reduce is reported without any figure, table, or configuration details; please add the supporting data or clearly mark this as a preliminary result.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central cryptographic-work count follows combinatorially from ring all-reduce, and the slowdown figures are emulation measurements rather than fitted predictions.

full rationale

The derivation chain is self-contained. Section 2.4 defines ring all-reduce as n-1 scatter-reduce plus n-1 all-gather steps; Section 3.2 observes each step imposes one encrypt+MAC on the sender and one decrypt+auth on the receiver; Section 3.3 multiplies by k asynchronous reductions, giving 4×k×(n-1) per GPU. This is a direct count from the algorithm, not a fit or an assumed conclusion. The reported slowdowns (Figures 9-10) are empirical comparisons of CC-on versus CC-off emulation, and the bucket_cap_mb mitigation is evaluated by measurement (Table 2, Figure 11), not derived from the target claim. The only same-author citation is [33], used for the multi-chaining GMAC implementation choice (Section 4.1, Figure 7); it is a disclosed modeling decision, not a load-bearing proof of the paper's conclusions. The single-GPU-only validation of the custom kernels (Section 4.2) and the ambiguity of 'total' when the 4×k×(n-1) count is per GPU are real validity/clarity concerns, but neither makes the claimed result equivalent to its inputs. Score 1 reflects the minor non-load-bearing self-citation and disclosed emulation limitation, not circularity.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim depends on three unverified modeling choices: that the custom kernels equal real TEE crypto on multi-GPU, that multi-chaining equals NVIDIA's GMAC, and that the emulated CPU crypto shortfall is irrelevant. The counting formula itself is assumption-free given ring all-reduce. The only fitted value is the 400 MB bucket size for GPT2-XLarge.

free parameters (1)

bucket_cap_mb optimal value = 400 MB
Chosen empirically for GPT2-XLarge by scanning 44/100/200/400/800 MB in Section 5.3; claimed 'best performance' for both 2-GPU and 4-GPU but not shown to generalize to other models.

assumptions (5)

domain assumption Only CPU and GPU packages are trusted; PCIe and NVLink are untrusted and must be encrypted and authenticated.
Threat model adopted from NVIDIA CC (Section 2.2); defines why AES-GCM is needed on every inter-GPU transfer.
domain assumption NVIDIA GPU TEEs enforce AES-GCM for all traffic crossing package boundaries.
Section 2.2 and Figure 2 describe AES-GCM as the mechanism; NVIDIA implementation details are not public (Section 4.1).
ad hoc to paper Custom CUDA AES-GCM kernels faithfully emulate NVIDIA's GPU TEE crypto for multi-GPU training.
Section 4.1 implements crypto as custom kernels because multi-GPU CC drivers are unavailable; validation in Section 4.2 is single-GPU only.
ad hoc to paper Multi-chaining authentication from Fastrack [33] captures the parallelism of NVIDIA's GMAC implementation.
Section 4.1 adopts multi-chaining to avoid a fully serial GHASH; the actual NVIDIA GMAC scheme is undisclosed.
ad hoc to paper The CPU-side AES-GCM throughput difference (1.82 GB/s vs NVIDIA's claimed 4 GB/s) does not affect multi-GPU inter-GPU conclusions.
Section 4.2 acknowledges the 2.2x difference but dismisses it because CPU-to-GPU transfers are infrequent; error bars are not provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training." pith.science (2026). https://pith.science/paper/H2SJPW4T

@misc{pith2026250111771,
  author       = {Pith},
  title        = {Pith review of: Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2SJPW4T}},
  note         = {Machine review of arXiv:2501.11771}
}

read the original abstract

Confidential computing (CC) or trusted execution enclaves (TEEs) is now the most common approach to enable secure computing in the cloud. The recent introduction of GPU TEEs by NVIDIA enables machine learning (ML) models to be trained without leaking model weights or data to the cloud provider. However, the potential performance implications of using GPU TEEs for ML training are not well characterized. In this work, we present an in-depth characterization study on performance overhead associated with running distributed data parallel (DDP) ML training with GPU Trusted Execution Environments (TEE). Our study reveals the performance challenges in DDP training within GPU TEEs. DDP uses ring-all-reduce, a well-known approach, to aggregate gradients from multiple devices. Ring all-reduce consists of multiple scatter-reduce and all-gather operations. In GPU TEEs only the GPU package (GPU and HBM memory) is trusted. Hence, any data communicated outside the GPU packages must be encrypted and authenticated for confidentiality and integrity verification. Hence, each phase of the ring-all-reduce requires encryption and message authentication code (MAC) generation from the sender, and decryption and MAC authentication on the receiver. As the number of GPUs participating in DDP increases, the overhead of secure inter-GPU communication during ring-all-reduce grows proportionally. Additionally, larger models lead to more asynchronous all-reduce operations, exacerbating the communication cost. Our results show that with four GPU TEEs, depending on the model that is being trained, the runtime per training iteration increases by an average of 8x and up to a maximum of 41.6x compared to DDP training without TEE.

Figures

Figures reproduced from arXiv: 2501.11771 by the authors.

**Figure 2.** Overview of AES-GCM regarding (a) AES-CTR [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Baseline ring all-reduce algorithm in three GPUs. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Computation flow of DDP training of each GPU. All-reduce (AR) operation is launched asynchronously, overlapping [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 6.** Figure 6: Number of model parameters (line) and number [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 7.** Figure 7: Multi-chaining authentication with ciphertext [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

**Figure 8.** Figure 8: Validation of AES-GCM implementation on a [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 9.** Figure 9: DDP training on 2 GPU TEEs. (a) compares the [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

**Figure 10.** Figure 10: DDP training on 4 GPU TEEs. (a) compares the [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

**Figure 11.** Figure 11: Total runtime and total CC cost reduction with [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

**Figure 12.** Figure 12: DDP training on 8 GPU TEEs comparing the [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 30 canonical work pages

[1]

Nvidia collective communication library. NVIDIA. (Accessed: May 1, 2020). [Online]. Available: https://github.com/NVIDIA/nccl

work page 2020
[2]

openssl: The open source toolkit for ssl/tls

Openssl project. openssl: The open source toolkit for ssl/tls. OpenSSL Project. Accessed: 2024-10-18. [Online]. Available: https://www.openssl.org/

work page 2024
[3]

Arm security technology building a secure system using trustzone technology,

“Arm security technology building a secure system using trustzone technology, ” 2016

work page 2016
[4]

[Online]

(2019) Pytorch examples. [Online]. Available: https://github.com/ pytorch/examples

work page 2019
[5]

Amd secure encrypted virtualization (sev),

“Amd secure encrypted virtualization (sev), ” https://www.amd.com/ en/developer/sev.html, AMD, accessed: 2024-10-07

work page 2024
[6]

AMD SEV-TIO: Trusted I/O for Secure Encrypted Virtualiza- tion,

AMD, “AMD SEV-TIO: Trusted I/O for Secure Encrypted Virtualiza- tion, ” 2023

work page 2023
[7]

Biased user history synthesis for personalized long- tail item recommendation,

K. Balasubramanian, A. Alshabanah, E. Markowitz, G. Ver Steeg, and M. Annavaram, “Biased user history synthesis for personalized long- tail item recommendation, ” inProceedings of the 18th ACM Conference on Recommender Systems , 2024, pp. 189–199

work page 2024
[8]

Logical/physical topology-aware col- lective communication in deep learning training,

S. Cho, H. Son, and J. Kim, “Logical/physical topology-aware col- lective communication in deep learning training, ” in 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2023, pp. 56–68

work page 2023

Show all 36 references

[9]

Imagenet: A large-scale hierarchical image database,

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database, ” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

2009
[10]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding, ”arXiv preprint arXiv:1810.04805 , 2018

2018 arXiv
[11]

Accelerate: Training and inference at scale made simple, efficient and adaptable

S. Gugger, L. Debut, T. Wolf, P. Schmid, Z. Mueller, S. Mangrulkar, M. Sun, and B. Bossan, “Accelerate: Training and inference at scale made simple, efficient and adaptable. ” https://github.com/huggingface/ accelerate, 2022

2022
[12]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition, ” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016
[13]

Gpipe: Efficient training of giant neural networks using pipeline parallelism,

Y. Huang, Y. Cheng, A. Bapna, O. Firat, D. Chen, M. Chen, H. Lee, J. Ngiam, Q. V. Le, Y. Wu et al. , “Gpipe: Efficient training of giant neural networks using pipeline parallelism, ” Advances in neural information processing systems , vol. 32, 2019

2019
[14]

Intel trust domain extensions,

“Intel trust domain extensions, ” https://www.intel.com/content/www/ us/en/developer/tools/trust-domain-extensions/overview.html, Intel, accessed: 2024-10-07

2024
[15]

Intel TDX Connect TEE-IO Device Guide,

Intel, “Intel TDX Connect TEE-IO Device Guide, ” 2023

2023
[16]

Hugging face,

S. M. Jain, “Hugging face, ” in Introduction to transformers for NLP: With the hugging face library and models to solve problems . Springer, 2022, pp. 51–67

2022
[17]

Intel software guard extensions: Epid provisioning and attestation services,

S. Johnson, V. Scarlata, C. Rozas, E. Brickell, and F. Mckeen, “Intel software guard extensions: Epid provisioning and attestation services, ” 2016

2016
[18]

Pytorch distributed: Experiences on accelerating data parallel training,

S. Li, Y. Zhao, R. Varma, O. Salpekar, P. Noordhuis, T. Li, A. Paszke, J. Smith, B. Vaughan, P. Damania et al. , “Pytorch distributed: Experiences on accelerating data parallel training, ” arXiv preprint arXiv:2006.15704, 2020

2006 arXiv
[19]

Azure confidential vms with nvidia h100 tensor core gpus,

“Azure confidential vms with nvidia h100 tensor core gpus, ” https://learn.microsoft.com/en-us/azure/confidential-computing/ gpu-options, Microsoft, accessed: 2024-08

2024
[20]

Securing ai inference in the cloud: Is cpu-gpu confidential computing ready?

A. Mohan, M. Ye, H. Franke, M. Srivatsa, Z. Liu, and N. M. Gonzalez, “Securing ai inference in the cloud: Is cpu-gpu confidential computing ready?” in 2024 IEEE 17th International Conference on Cloud Computing (CLOUD). IEEE, 2024, pp. 164–175

2024
[21]

Supporting secure multi-gpu computing with dynamic and batched metadata management,

S. Na, J. Kim, S. Lee, and J. Huh, “Supporting secure multi-gpu computing with dynamic and batched metadata management, ” in 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2024, pp. 204–217

2024
[22]

Crows-pairs: A challenge dataset for measuring social biases in masked language models,

N. Nangia, C. Vania, R. Bhalerao, and S. R. Bowman, “Crows-pairs: A challenge dataset for measuring social biases in masked language models, ”arXiv preprint arXiv:2010.00133 , 2020

2010 arXiv
[23]

Nvidia blackwell architecture technical brief,

“Nvidia blackwell architecture technical brief, ” https://resources. nvidia.com/en-us-blackwell-architecture, NVIDIA, accessed: 2024- 03

2024
[24]

Nvidia confidential computing,

“Nvidia confidential computing, ” https://images.nvidia.com/aem-dam/ en-zz/Solutions/data-center/HCC-Whitepaper-v1.0.pdf, NVIDIA, ac- cessed: 2024-10-07

2024
[25]

Nvidia confidential computing protected pcie,

“Nvidia confidential computing protected pcie, ” https://docs.nvidia. com/570TRD1-trusted-computing-solutions-release-notes.pdf, NVIDIA, accessed: 2025-05

2025
[26]

Nvidia h100 tensor core gpu architecture overview,

“Nvidia h100 tensor core gpu architecture overview, ” https://resources. nvidia.com/en-us-data-center-overview/gtc22-whitepaper-hopper, NVIDIA, accessed: 2022-03

2022
[27]

Nvidia rtx pro blackwell architecture,

“Nvidia rtx pro blackwell architecture, ” https://blogs.nvidia.com/blog/ rtx-pro-6000-blackwell-server-edition/, NVIDIA, accessed: 2025-03

2025
[28]

Language models are unsupervised multitask learners,

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners, ” OpenAI blog , vol. 1, no. 8, p. 9, 2019

2019
[29]

Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,

J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He, “Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters, ” in Proceedings of the 26th ACM SIGKDD inter- national conference on knowledge discovery & data mining , 2020, pp. 3505–3506

2020
[30]

Performance analysis and optimization of nvidia h100 confidential computing for ai workloads,

Y. Tan and Z. Mi, “Performance analysis and optimization of nvidia h100 confidential computing for ai workloads, ” in 2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA). IEEE, 2024, pp. 1426–1432

2024
[31]

Pipellm: Fast and confidential large language model services with speculative pipelined encryption,

Y. Tan, C. Tan, Z. Mi, and H. Chen, “Pipellm: Fast and confidential large language model services with speculative pipelined encryption, ” in Proceedings of the 30th ACM International Conference on Archi- tectural Support for Programming Languages and Operating Systems, Volume...

2025
[32]

Glue: A multi-task benchmark and analysis platform for natural language understanding,

A. Wang, “Glue: A multi-task benchmark and analysis platform for natural language understanding, ” arXiv preprint arXiv:1804.07461 , 2018

2018 arXiv
[33]

Fastrack: Fast io for secure ml using gpu tees,

Y. Wang, R. Rajat, J. Lee, T. Tang, and M. Annavaram, “Fastrack: Fast io for secure ml using gpu tees, ” arXiv preprint arXiv:2410.15240 , 2024

2024 arXiv
[34]

Sampling-bias-corrected neural modeling for large corpus item recommendations,

X. Yi, J. Yang, L. Hong, D. Z. Cheng, L. Heldt, A. Kumthekar, Z. Zhao, L. Wei, and E. Chi, “Sampling-bias-corrected neural modeling for large corpus item recommendations, ” in Proceedings of the 13th ACM conference on recommender systems , 2019, pp. 269–277

2019
[35]

Pytorch fsdp: experiences on scaling fully sharded data parallel,

Y. Zhao, A. Gu, R. Varma, L. Luo, C.-C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer et al. , “Pytorch fsdp: experiences on scaling fully sharded data parallel, ”arXiv preprint arXiv:2304.11277 , 2023

2023 arXiv
[36]

Confidential computing on nvidia hopper gpus: A performance benchmark study,

J. Zhu, H. Yin, P. Deng, A. Almeida, and S. Zhou, “Confidential computing on nvidia hopper gpus: A performance benchmark study, ” arXiv preprint arXiv:2409.03992 , 2024. 11

2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Nvidia collective communication library. NVIDIA. (Accessed: May 1, 2020). [Online]. Available: https://github.com/NVIDIA/nccl

work page 2020

[2] [2]

openssl: The open source toolkit for ssl/tls

Openssl project. openssl: The open source toolkit for ssl/tls. OpenSSL Project. Accessed: 2024-10-18. [Online]. Available: https://www.openssl.org/

work page 2024

[3] [3]

Arm security technology building a secure system using trustzone technology,

“Arm security technology building a secure system using trustzone technology, ” 2016

work page 2016

[4] [4]

[Online]

(2019) Pytorch examples. [Online]. Available: https://github.com/ pytorch/examples

work page 2019

[5] [5]

Amd secure encrypted virtualization (sev),

“Amd secure encrypted virtualization (sev), ” https://www.amd.com/ en/developer/sev.html, AMD, accessed: 2024-10-07

work page 2024

[6] [6]

AMD SEV-TIO: Trusted I/O for Secure Encrypted Virtualiza- tion,

AMD, “AMD SEV-TIO: Trusted I/O for Secure Encrypted Virtualiza- tion, ” 2023

work page 2023

[7] [7]

Biased user history synthesis for personalized long- tail item recommendation,

K. Balasubramanian, A. Alshabanah, E. Markowitz, G. Ver Steeg, and M. Annavaram, “Biased user history synthesis for personalized long- tail item recommendation, ” inProceedings of the 18th ACM Conference on Recommender Systems , 2024, pp. 189–199

work page 2024

[8] [8]

Logical/physical topology-aware col- lective communication in deep learning training,

S. Cho, H. Son, and J. Kim, “Logical/physical topology-aware col- lective communication in deep learning training, ” in 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2023, pp. 56–68

work page 2023