Paper Citation Record · LEDGER

Scaling Deep Learning Training with MPMD Pipeline Parallelism

As of 20 August 2026, this Paper Citation Record lists 33 of 33 outbound references and 0 inbound Pith citation observations for arXiv:2412.14374.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.14374 v1

Coverage vector

measured 33 of 33 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-11T12:21:38.881968Z

measured 33 of 33 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-19T06:32:44.657259+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

33 of 33 outbound references displayed

verified exact0
verified fuzzy12
unresolved20
parse uncertain0
malformed identifier0
metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation cca14982-b4a1-4f08-983e-2a7c1e0bf6ca · outbound

This paper cites URL https://www.top500.org/system/180239.

Scaling Deep Learning Training with MPMD Pipeline Parallelism URL https://www.top500.org/system/180239

Reference 1

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.343082Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 2b7404cb-0d72-4344-9513-d0b078a2de96 · outbound

This paper cites PartIR: Composing SPMD Partitioning Strategies for Machine Learning.

Scaling Deep Learning Training with MPMD Pipeline Parallelism PartIR: Composing SPMD Partitioning Strategies for Machine Learning

Reference 2

Resolution: metadata mismatch; local_arxiv, observed 2026-08-11T12:21:39.241140Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 583b702d-4480-4e21-b82d-241700fc0f2d · outbound

This paper cites E., Thekkath, C.

Scaling Deep Learning Training with MPMD Pipeline Parallelism E., Thekkath, C

Reference 3

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.335359Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 169ba462-9729-485e-9869-e24430089a78 · outbound

This paper cites J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne , S., and Zhang, Q.

Scaling Deep Learning Training with MPMD Pipeline Parallelism J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne , S., and Zhang, Q

Reference 4

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.327203Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 6bd4f832-9cf5-4502-b5ba-978ad4580e4f · outbound

This paper cites an unresolved cited work.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Unresolved cited work

Reference 5

Resolution: unresolved; raw_fallback, observed 2026-08-11T12:21:39.318466Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 50454ee1-f96d-4d8b-8d0a-553ed39ff100 · outbound

This paper cites Training Deep Nets with Sublinear Memory Cost.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Training Deep Nets with Sublinear Memory Cost

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.814422Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 521085b4-422a-4ef0-b961-4f06d2a84a86 · outbound

This paper cites cuDNN: Efficient Primitives for Deep Learning.

Scaling Deep Learning Training with MPMD Pipeline Parallelism cuDNN: Efficient Primitives for Deep Learning

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.817499Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cfe67fb1-a97d-430e-a0da-89b941f13668 · outbound

This paper cites PaLM: Scaling Language Modeling with Pathways.

Scaling Deep Learning Training with MPMD Pipeline Parallelism PaLM: Scaling Language Modeling with Pathways

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.820248Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f474a3f4-9cc6-4e65-9e4b-e784cdc0c0dd · outbound

This paper cites An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale.

Scaling Deep Learning Training with MPMD Pipeline Parallelism An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.311729Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation eee7ab37-6dc2-4689-a7e0-0f1f5b09b8e3 · outbound

This paper cites The Llama 3 Herd of Models.

Scaling Deep Learning Training with MPMD Pipeline Parallelism The Llama 3 Herd of Models

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.826145Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6e0a0107-32a3-4dbc-9bdf-94719833b9ae · outbound

This paper cites Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.829004Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2b855f9a-56f2-4370-b895-1f31fc336153 · outbound

This paper cites NeMo: a toolkit for Conversational AI and Large Language Models.

Scaling Deep Learning Training with MPMD Pipeline Parallelism NeMo: a toolkit for Conversational AI and Large Language Models

Reference 12

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.304871Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation ce303d1b-1216-4043-960c-62ef0b257403 · outbound

This paper cites The Hardware Lottery.

Scaling Deep Learning Training with MPMD Pipeline Parallelism The Hardware Lottery

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.834591Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 16c969e5-03c0-4f46-a657-3984bd7abf54 · outbound

This paper cites DISTMM : Accelerating distributed multimodal model training.

Scaling Deep Learning Training with MPMD Pipeline Parallelism DISTMM : Accelerating distributed multimodal model training

Reference 14

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.297798Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation e11d4ac1-9a8f-4e44-a1eb-fa043afc8243 · outbound

This paper cites X., Lee, H., Ngiam, J., Le, Q.

Scaling Deep Learning Training with MPMD Pipeline Parallelism X., Lee, H., Ngiam, J., Le, Q

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.290838Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation da6ae688-feb0-4673-833b-5932ec6ad8a1 · outbound

This paper cites Megascale: Scaling large language model training to more than 10,000 gpus.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Megascale: Scaling large language model training to more than 10,000 gpus

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.283713Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation d633020d-f208-47aa-bfcf-919c4a64c8ca · outbound

This paper cites Breadth-First Pipeline Parallelism.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Breadth-First Pipeline Parallelism

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.844910Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 25aee9dc-f7e2-40fd-aa0f-a5b604a316ee · outbound

This paper cites Mlir: Scaling compiler infrastructure for domain specific computation.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Mlir: Scaling compiler infrastructure for domain specific computation

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.847921Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2b3b2526-6801-492d-8f4f-685660219c67 · outbound

This paper cites GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding.

Scaling Deep Learning Training with MPMD Pipeline Parallelism GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.850831Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7e5b2747-743f-40fd-b377-275a8e5b7bb4 · outbound

This paper cites TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training.

Scaling Deep Learning Training with MPMD Pipeline Parallelism TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.853492Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 05d3bdda-1b07-48be-be7b-1baa835d2375 · outbound

This paper cites nnScaler : Constraint-Guided parallelization plan generation for deep learning training.

Scaling Deep Learning Training with MPMD Pipeline Parallelism nnScaler : Constraint-Guided parallelization plan generation for deep learning training

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.275275Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 69ef8b5f-b024-4bc5-97b8-d9d8565df6c8 · outbound

This paper cites I., and Stoica, I.

Scaling Deep Learning Training with MPMD Pipeline Parallelism I., and Stoica, I

Reference 22

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.268253Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 415839ce-169e-4de3-ac1a-a2e0c9f7c9b5 · outbound

This paper cites R., Ganger, G.

Scaling Deep Learning Training with MPMD Pipeline Parallelism R., Ganger, G

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.859640Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 506b34d0-76bc-43cb-9d32-bd63359d76c2 · outbound

This paper cites Efficient large-scale language model training on GPU clusters using megatron- LM.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Efficient large-scale language model training on GPU clusters using megatron- LM

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.861312Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7e71a091-1a37-45d6-80f0-d925ea1e9ac6 · outbound

This paper cites Efficiently Scaling Transformer Inference.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Efficiently Scaling Transformer Inference

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.863523Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 23434b05-17ef-4201-a699-71dc0ea260c2 · outbound

This paper cites Zero bubble (almost) pipeline parallelism.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Zero bubble (almost) pipeline parallelism

Reference 26

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.261101Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation b2250dfa-f9ba-4c79-b19d-84aa37f179a4 · outbound

This paper cites Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.867300Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0d66b898-5d83-450e-a330-79555137b4ee · outbound

This paper cites Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.871089Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 35c41776-aef2-47f9-b8e2-de1b42291142 · outbound

This paper cites Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.873748Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 84b218d4-88fd-44d4-bd43-7e7d3c8c1b09 · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Scaling Deep Learning Training with MPMD Pipeline Parallelism Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.875932Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7a1aa935-be64-4cc3-a3ad-1ded6930fefc · outbound

This paper cites GSPMD: General and Scalable Parallelization for ML Computation Graphs.

Scaling Deep Learning Training with MPMD Pipeline Parallelism GSPMD: General and Scalable Parallelization for ML Computation Graphs

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.877852Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19f5bb7c-b741-4659-8308-a0c83c856c75 · outbound

This paper cites P., Gonzalez, J.

Scaling Deep Learning Training with MPMD Pipeline Parallelism P., Gonzalez, J

Reference 32

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T12:21:39.254328Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation a8926a3b-9286-4308-9bc3-b35537842853 · outbound

This paper cites write newline.

Scaling Deep Learning Training with MPMD Pipeline Parallelism write newline

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-11T12:21:38.881968Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.