Paper Citation Record · LEDGER

Attention Is All You Need For Mixture-of-Depths Routing

As of 14 August 2026, this Paper Citation Record lists 24 of 24 outbound references and 0 inbound Pith citation observations for arXiv:2412.20875.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.20875 v1

Coverage vector

measured 24 of 24 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-10T23:12:46.187984Z

measured 24 of 24 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-14T06:32:32.682623+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

24 of 24 outbound references displayed

verified exact0
verified fuzzy7
unresolved17
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 02e9d718-36b9-4932-8269-6b1409f99352 · outbound

This paper cites Neural Machine Translation by Jointly Learning to Align and Translate.

Attention Is All You Need For Mixture-of-Depths Routing Neural Machine Translation by Jointly Learning to Align and Translate

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.087752Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 088ba14c-0a2b-403d-ba18-419097171c2d · outbound

This paper cites (2022)) to column first in order to aggregate row wise scores efficiently.

Attention Is All You Need For Mixture-of-Depths Routing (2022)) to column first in order to aggregate row wise scores efficiently

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.593375Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 00862e64-3b68-48fd-9e4b-379e188947fa · outbound

This paper cites Jordan and R.A.

Attention Is All You Need For Mixture-of-Depths Routing Jordan and R.A

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.649562Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 87190a05-b759-4a1d-9708-0390f408b66a · outbound

This paper cites Decoupled Weight Decay Regularization.

Attention Is All You Need For Mixture-of-Depths Routing Decoupled Weight Decay Regularization

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.138122Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2a86ddda-136d-499a-83cb-3a31e445ab19 · outbound

This paper cites Mixture-of-Depths: Dynamically allocating compute in transformer-based language models.

Attention Is All You Need For Mixture-of-Depths Routing Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.146643Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6704a57f-4422-4318-894b-8b0789abcc2b · outbound

This paper cites Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean.

Attention Is All You Need For Mixture-of-Depths Routing Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.150934Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 27bebf00-eb55-43b2-9abf-4073af15dfb4 · outbound

This paper cites The Computational Limits of Deep Learning.

Attention Is All You Need For Mixture-of-Depths Routing The Computational Limits of Deep Learning

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.154905Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 99890f0c-8a5d-4db2-97d2-f5375c338f5c · outbound

This paper cites YOLOv10: Real-Time End-to-End Object Detection.

Attention Is All You Need For Mixture-of-Depths Routing YOLOv10: Real-Time End-to-End Object Detection

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.159112Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3da45293-e846-4a9e-a604-09ac8a17a533 · outbound

This paper cites Emergent Abilities of Large Language Models.

Attention Is All You Need For Mixture-of-Depths Routing Emergent Abilities of Large Language Models

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.163154Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 134746bd-5b84-4a9d-8c10-c378edd381e7 · outbound

This paper cites Algorithm 1 A-MoD with Flash Attention, modified from Algorithm 1 in Dao et al.

Attention Is All You Need For Mixture-of-Depths Routing Algorithm 1 A-MoD with Flash Attention, modified from Algorithm 1 in Dao et al

Reference 19

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.566158Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 020580f2-c3a5-408a-bb1c-a200fb483b4a · outbound

This paper cites Fine- tuning with A-MoD: Results comparing A-MoD with standard routing and isoFLOP baselines for 12.5% capacity on ImageNet-1k.

Attention Is All You Need For Mixture-of-Depths Routing Fine- tuning with A-MoD: Results comparing A-MoD with standard routing and isoFLOP baselines for 12.5% capacity on ImageNet-1k

Reference 20

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.621253Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation c350d607-1bcf-4e6b-8f02-80b0ee826065 · outbound

This paper cites an unresolved cited work.

Attention Is All You Need For Mixture-of-Depths Routing Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-10T23:12:46.607502Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation abce2d01-e0ec-404e-a0b9-2cab9df573f2 · outbound

This paper cites We compare with the baseline results provided in Table 11 in Bolya et al.

Attention Is All You Need For Mixture-of-Depths Routing We compare with the baseline results provided in Table 11 in Bolya et al

Reference 23

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.579883Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 9c40a882-05a8-4aad-ae4a-64124aa97d76 · outbound

This paper cites Mixture of Nested Experts: Adaptive Processing of Visual Tokens.

Attention Is All You Need For Mixture-of-Depths Routing Mixture of Nested Experts: Adaptive Processing of Visual Tokens

Reference 1991

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.121178Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8253f738-8fa0-4bf4-bb50-462bad4cf955 · outbound

This paper cites Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.

Attention Is All You Need For Mixture-of-Depths Routing Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

Reference 1993

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.134066Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ed50a281-0131-40d9-ae4c-89e9872549e7 · outbound

This paper cites A Survey on Mixture of Experts in Large Language Models.

Attention Is All You Need For Mixture-of-Depths Routing A Survey on Mixture of Experts in Large Language Models

Reference 2013

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.102638Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 10aa2906-3976-4f60-96de-d1435e10a95b · outbound

This paper cites Conditional Computation in Neural Networks for faster models.

Attention Is All You Need For Mixture-of-Depths Routing Conditional Computation in Neural Networks for faster models

Reference 2014

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.093201Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b6685bac-5984-48f2-b2bf-608fa9348939 · outbound

This paper cites Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation.

Attention Is All You Need For Mixture-of-Depths Routing Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Reference 2015

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.098087Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d639b5f7-ce2e-46e4-8a0a-740caa441742 · outbound

This paper cites Training Compute-Optimal Large Language Models.

Attention Is All You Need For Mixture-of-Depths Routing Training Compute-Optimal Large Language Models

Reference 2016

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.116329Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 348de185-fa69-4036-bd7f-3795ac587e3c · outbound

This paper cites Scaling laws for fine-grained mixture of experts.

Attention Is All You Need For Mixture-of-Depths Routing Scaling laws for fine-grained mixture of experts

Reference 2017

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.635097Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 01e2d03d-8dd9-49af-ad4d-cb50e9fd82c5 · outbound

This paper cites Mixtral of Experts.

Attention Is All You Need For Mixture-of-Depths Routing Mixtral of Experts

Reference 2018

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.125395Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f5cdce41-7bdf-4b43-9721-f538f0b5ae2e · outbound

This paper cites A Review of Sparse Expert Models in Deep Learning.

Attention Is All You Need For Mixture-of-Depths Routing A Review of Sparse Expert Models in Deep Learning

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.111458Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ce6f1959-a86d-44eb-9857-b58a40be3fa6 · outbound

This paper cites ST-MoE: Designing Stable and Transferable Sparse Expert Models.

Attention Is All You Need For Mixture-of-Depths Routing ST-MoE: Designing Stable and Transferable Sparse Expert Models

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-10T23:12:46.167415Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7464dd74-b32d-4f9b-860b-c31d48d3842a · outbound

This paper cites On the benefits of learning to route in mixture-of-experts models.

Attention Is All You Need For Mixture-of-Depths Routing On the benefits of learning to route in mixture-of-experts models

Reference 2024

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T23:12:46.662404Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Pith citing papers

No inbound Pith citation observations are available.