Pith. sign in

REVIEW 3 major objections 6 minor 95 references

This paper claims that putting natural-language text into the tokenizer, not just the generator, makes discrete motion codes more compact and text-to-motion generation both more accurate and easier to learn.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:18 UTC pith:SFEFAEBI

load-bearing objection Solid engineering with a real confound: the detokenizer's text conditioning does much of the work, so the paper's central claim about tokenizer-side language guidance isn't actually isolated. the 3 major comments →

arxiv 2602.08337 v2 pith:SFEFAEBI submitted 2026-02-09 cs.CV

Language-Guided Transformer Tokenizer for Human Motion Generation

Sheng Yan , Yong Wang , Xin Du , Junsong Yuan , Mengyuan Liu This is my paper
classification cs.CV
keywords language-guided tokenizationmotion tokenizationtext-to-motion generationtransformer tokenizervector quantizationmulti-scale discrete tokenslanguage-drop guidancehuman motion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to break a trade-off that plagues discrete motion tokenization: more tokens improve reconstruction fidelity, but longer token sequences make the generative model harder to train and sample. Its proposed fix, Language-Guided Tokenization (LG-Tok), feeds the motion caption into both the tokenizer and the detokenizer, so the discrete codes only need to carry the fine-grained details that language does not specify. Because the architecture is a transformer with global attention rather than a convolutional stack, the language and motion streams can align everywhere. The paper reports that this simplifies generative learning—perplexity drops from 146.5 to 103.1 on Motion-X and FID from 0.257 to 0.088—and that a half-token variant stays competitive with the full model. If true, conditioning the tokenizer itself, rather than only the generator, is a practical recipe for high reconstruction quality with fewer tokens.

Core claim

LG-Tok is a vector-quantized autoencoder for human motion in which natural language is not just the conditioning signal for a downstream generator but an active participant in tokenization. A frozen text encoder turns the caption into embeddings; a transformer encodes the concatenation of text, learnable latent tokens, and the raw motion, and only the latent tokens are quantized into multi-scale discrete codes. The detokenizer reconstructs the motion from those codes plus the same text embeddings through cross-attention, and a language-drop scheme removes the text 10% of the time during training, enabling guidance in motion space at inference. The authors' central claim is that this alignmen

What carries the argument

The central mechanism is a transformer tokenizer–detokenizer pair with a learnable query sequence. In the tokenizer, learnable latent tokens are concatenated with the motion stream and text embeddings, self-attention mixes the three streams, and only the latent-token outputs are quantized. The detokenizer uses parallel learnable mask tokens that attend to the dequantized codes and to the text via cross-attention, with long skip connections for fidelity. Multi-scale residual quantization turns the latent features into token sets of increasing temporal resolution, which feed a scalable autoregressive generative transformer that predicts one scale at a time. The language-drop scheme—text remove

Load-bearing premise

The load-bearing premise is that natural-language captions carry enough of a motion's meaning that moving them into the tokenizer reduces the semantic burden on tokens rather than distracting the encoder—and, on Motion-X, that the authors' in-house feature extractor (trained with batch size 32 because smaller batches gave near-zero CLIP scores) and reimplemented baselines are faithful enough for the reported quality gap to hold against the original published numbers.

What would settle it

Train LG-Tok with the same architecture and training budget but shuffle the captions (or replace text embeddings with Gaussian noise); if FID and R-Precision stay at the reported levels, language alignment is not doing the work. On Motion-X, rerun the comparison with the original published feature extractor and baseline checkpoints rather than the authors' reimplementations; if the gap to FID 0.088 / Top-1 0.582 collapses, the result is an artifact of the new evaluator.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Token count can be cut roughly in half without giving up generation quality: LG-Tok-mini (104 total tokens) stays at Top-1 R-Precision 0.521/0.588 and FID 0.085/0.071, close to the full 236-token model.
  • Language becomes part of the tokenizer, so every downstream model trained on the codes inherits a semantic alignment that would otherwise have to be re-learned at the generation stage.
  • Generative models trained on LG-Tok codes are easier to learn, as measured by lower perplexity (160.6 to 155.9 on HumanML3D and 146.5 to 103.1 on Motion-X), which should make training and sampling more stable.
  • Language-drop provides a way to do guidance in motion space after generation, and the paper reports it can even refine free-form, text-unconditional token sequences (FID improving from 4.025 to 3.564 on HumanML3D).
  • Because the detokenizer can be text-masked in temporal regions, the same framework supports motion in-painting and out-painting without a separate editing model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same text-into-tokenizer recipe could transfer to other discrete latent modalities with paired captions—video, audio, or robot trajectories—shrinking codebook sizes wherever semantic burden can be shifted to language.
  • A direct isolation test would be to train LG-Tok with the same transformer but replace the caption embeddings with a frozen random projection of the text; if quality stays high, the gains come from global attention rather than from semantic alignment, whereas a sharp drop would confirm the language signal is load-bearing.
  • The fixed 10% language-drop rate is likely dataset- and codebook-dependent; sweeping it against the codebook size would reveal whether the regularization effect or the guidance mechanism contributes more to the reported FID gains.
  • The paper's own limitations section reports a roughly 10x memory footprint for the attention-based tokenizer and about two days of single-GPU training, so a sparse- or linear-attention variant is the natural next experiment for practical deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes LG-Tok, a VQ-based motion tokenizer with a Transformer encoder/decoder and language guidance injected into both tokenization and detokenization. The authors argue that aligning natural language with motion at the tokenization stage yields compact, semantically informed discrete tokens, which improves reconstruction fidelity and, more importantly, simplifies the subsequent generative model. They report SOTA results on HumanML3D and Motion-X (e.g., Table 2: Top-1 0.542/0.582 and FID 0.057/0.088 versus MARDM's 0.500/0.528 and 0.114/0.147), and they ablate architectural choices, guidance location, text encoder, interaction mechanism, and language-drop. They also analyze t-SNE structure and codebook utilization. The supplementary material describes the full pipeline, a Motion-CLIP-style evaluator trained for Motion-X, baseline reproductions, and additional ablations including the role of language decoding in the detokenizer.

Significance. If the central claim is supported by controlled evidence, the work would provide a practical and transferable recipe: instead of pushing more tokens into the generative model, one can inject language at the tokenizer/detokenizer level to obtain compact, high-level motion tokens. The paper has several strengths: it ships code and checkpoints, it includes architecture-level ablations, it attempts to show generalization to MoMask, and it provides a detailed appendix. However, the current experimental design does not isolate the proposed mechanism: the gains attributed to "language-guided tokenization" are entangled with text conditioning in the detokenizer, which standard baselines do not use. The Motion-X evaluation additionally rests on a self-trained feature extractor with hyperparameters selected by the resulting CLIP-score. These issues are load-bearing for the SOTA claims, though they are addressable with additional controlled experiments.

major comments (3)
  1. [§3.3, Eq. (3); Appendix G, Table 8] The generation pipeline uses text twice: once by the SAR generative model and once by the detokenizer. Standard baselines such as MARDM, MoSa, and MoMask decode without text. Appendix G/Table 8 shows that the detokenizer's language conditioning alone materially improves generation: in the standard text-conditioned setting, FID drops from 0.061 to 0.057 on HumanML3D and from 0.139 to 0.088 on Motion-X; in free-form generation, Motion-X FID drops from 25.093 to 14.715. Thus the detokenizer acts as a "semantic amplifier" that can improve output even when the token sequence itself is not semantically meaningful. Table 5's "w/o text guidance" ablation removes text from both tokenizer and detokenizer simultaneously, so it cannot attribute the gains to tokenization-stage alignment. Table 4e also shows tokenizer-only text guidance gives almost no reconstruction gain (0.064 to 0.063) while detoke
  2. [Appendix D, Table 6; Table 2 (Motion-X block)] The Motion-X evaluation is built on a feature extractor trained by the authors, and the batch size was chosen because it produces a CLIP-score of 0.672 rather than 0.012 (Table 6). Since FID, R-Precision, and related metrics on Motion-X all depend on this evaluator, the Motion-X numbers are not directly comparable to any published numbers; all Motion-X baselines are also author reimplementations. This is a serious comparability problem for the SOTA claim on Motion-X. The authors should release the trained evaluator and the exact evaluation pipeline, cross-check the metrics with an independent or publicly available evaluator, and report the sensitivity of the reported FID/R-Precision to the evaluator hyperparameters. Without this, the Motion-X results should be presented as self-benchmark results, not as SOTA comparisons.
  3. [§4.3, perplexity claim] The paper states that language-guided tokens reduce generative-model perplexity from 146.5 to 103.1 on Motion-X and from 160.6 to 155.9 on HumanML3D, and uses this as evidence that language "simplifies the learning of generative models." However, no definition of perplexity, no computation protocol, and no per-scale or aggregate table is provided. Perplexity over discrete tokens is not uniquely defined in a multi-scale residual VQ setting unless one specifies the token distribution, the conditioning, and the aggregation across scales. Please provide a precise definition and the actual numbers; without this, the claim is unverifiable.
minor comments (6)
  1. [Table 1 and §1] The stated trade-off "more tokens improve reconstruction but worsen generation" is not fully monotonic in Table 1: gFID is 0.230 at 104 tokens, 0.205 at 160, and 0.257 at 236. Please soften the claim or provide more granular data.
  2. [Table 4d] The chosen RoPE base of 100 is reported as "better suited," but the ablation shows RoPE(base=1000) achieves lower reconstruction FID (0.042 vs. 0.049), at a small MPJPE cost (56.3 vs. 56.1). Please explain the selection criterion or justify the choice beyond a single metric.
  3. [§3.4, Eq. (4); Appendix C, Table 5] The guidance scale g is tuned separately per dataset (2.0 on HumanML3D, 1.0 on Motion-X), and the language-drop rate (10%) appears to be fixed without a sensitivity study. The paper should acknowledge that these are dataset-specific hyperparameters and report at least a coarse sweep over the drop rate.
  4. [Table 4g] The checkmark table for interaction methods is difficult to parse, especially the mapping from "In-Context" and "Cross-Attn." columns to the described variants. Consider a simpler layout with explicit variant names.
  5. [§4.2, Table 2 footnote] On HumanML3D, MoSa is marked with '†' (reimplementation), but the text says HumanML3D results are "primarily sourced from [26]." Please clarify which entries are reimplemented and which are taken from the literature, especially for MoSa and MARDM.
  6. [References and typos] There are duplicate/inconsistent reference entries for LLaMA ([72] and [79] both list the LLaMA 3 herd). Also, "We follow the most evaluation protocol proposed in [25, 26]" should read "the most-used evaluation protocol" or similar.

Circularity Check

0 steps flagged

No significant circularity; the reported gains are empirical, though the tokenizer-only contribution is confounded by detokenizer-side text conditioning and the Motion-X evaluator is self-built.

full rationale

The derivation chain (transformer tokenizer -> multi-scale VQ -> SAR -> detokenizer) is not circular: LG-Tok is trained with a reconstruction loss (smooth L1) without text reconstruction, so language-motion alignment is learned rather than defined by a precomputed target. The generation FID and R-Precision numbers are empirical outputs of a stochastic pipeline, not fitted parameters relabeled as predictions. The self-citation to MoSa (Sec. 3.1, Table 2; adopted as the generative backbone and also the primary baseline) is notable, but the central claim is independently supported by large gains over external methods (MARDM, MoMask, T2M-GPT) on HumanML3D, which uses MARDM's feature extractor. The paper's real weaknesses are evaluation confounds rather than circularity: the 'w/o text guidance' ablation (Table 5) removes text from both tokenizer and detokenizer simultaneously, so it cannot isolate tokenizer-stage language guidance; and Appendix G/Table 8 shows the detokenizer-side language decoding alone substantially improves generation, including free-form FID dropping from 25.093 to 14.715. Likewise, the Motion-X feature extractor is self-trained and its batch size was chosen because it produced a CLIP-score of 0.672 versus 0.012 at batch size 8 (Table 6). These are reproducibility and attribution risks, not reductions of the claimed result to its inputs by construction, so they do not warrant a circularity score above 2.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical or conceptual entities are introduced. The free parameters are hyperparameters and evaluator choices; the most load-bearing are the guidance scale and the Motion-X evaluator batch size, which affect the main reported numbers.

free parameters (5)
  • guidance scale g = 2.0 (HumanML3D), 1.0 (Motion-X)
    Swept in Fig. 5; the best value per dataset is used in the main tables.
  • language-drop probability = 0.10
    Chosen by hand in Sec. 3.4; no ablation over the drop rate is provided.
  • RoPE base = 100
    Ablated in Table 4d; base=100 chosen as best for short sequences.
  • Motion-X evaluator batch size = 32
    Selected in Appendix D based on CLIP-score: batch 8 gives 0.012, batch 32 gives 0.672.
  • Scale scheduler S and number of quantizers N = e.g., S=(2,4,...,36), N=10
    Design choices following MoSa; intervals determine token counts (104/160/236).
axioms (5)
  • standard math Multi-head attention, RMSNorm, SwiGLU, and RoPE behave as defined in the cited prior work.
    The architecture relies on standard transformer components without modification.
  • domain assumption The meng67 (67-D) motion representation is sufficient; facial and finger information can be ignored.
    Stated in Sec. 4.1; needed to train tokenizers on both datasets with consistent features.
  • domain assumption Frozen LLaMA-3.2-1B text embeddings are semantically aligned with human motion for this task.
    Used in Sec. 3.3 as the text encoder; no fine-tuning, so alignment is assumed from pretraining.
  • domain assumption The HumanML3D and Motion-X test sets, together with the FID/R-Precision protocol from [25,26], measure the intended quality.
    Evaluation validity is assumed; the Motion-X protocol is reimplemented by the authors.
  • ad hoc to paper A 10% language-drop rate is sufficient to make the detokenizer robust to missing text without losing language conditioning.
    Introduced in Sec. 3.4; the rate is fixed and not swept, so it is an unexamined design choice.

pith-pipeline@v1.3.0-alltime-deepseek · 24178 in / 14576 out tokens · 151646 ms · 2026-08-03T03:18:15.405977+00:00 · methodology

0 comments
read the original abstract

In this paper, we focus on motion discrete tokenization, which converts raw motion into compact discrete tokens--a process proven crucial for efficient motion generation. In this paradigm, increasing the number of tokens is a common approach to improving motion reconstruction quality, but more tokens make it more difficult for generative models to learn. To maintain high reconstruction quality while reducing generation complexity, we propose leveraging language to achieve efficient motion tokenization, which we term Language-Guided Tokenization (LG-Tok). LG-Tok aligns natural language with motion at the tokenization stage, yielding compact, high-level semantic representations. This approach not only strengthens both tokenization and detokenization but also simplifies the learning of generative models. Furthermore, existing tokenizers predominantly adopt convolutional architectures, whose local receptive fields struggle to support global language guidance. To this end, we propose a Transformer-based Tokenizer that leverages attention mechanisms to enable effective alignment between language and motion. Additionally, we design a language-drop scheme, in which language conditions are randomly removed during training, enabling the detokenizer to support language-free guidance during generation. On the HumanML3D and Motion-X generation benchmarks, LG-Tok achieves Top-1 scores of 0.542 and 0.582, outperforming state-of-the-art methods (MARDM: 0.500 and 0.528), and with FID scores of 0.057 and 0.088, respectively, versus 0.114 and 0.147. LG-Tok-mini uses only half the tokens while maintaining competitive performance (Top-1: 0.521/0.588, FID: 0.085/0.071), validating the efficiency of our semantic representations. Code and checkpoints are available at https://eanson023.github.io/LG-Tok/

Figures

Figures reproduced from arXiv: 2602.08337 by Junsong Yuan, Mengyuan Liu, Sheng Yan, Xin Du, Yong Wang.

Figure 1
Figure 1. Figure 1: Comparison between previous CNN-based tokenizers [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Generation quality on HumanML3D. Furthermore, we design a language-drop scheme, in which language conditions are randomly removed during training. This simple yet effective strategy enables the deto￾kenizer to support language-free guidance decoding dur￾ing motion generation [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of our LG-Tok framework. Given an in￾put motion sequence and corresponding natural language descrip￾tion, a frozen text encoder (e.g., LLaMA [72]) extracts text em￾beddings which are concatenated with learnable latent tokens and motions, and fed into a Transformer-based tokenizer to produce high-level semantic motion tokens. The quantizer then quantized these tokens into discrete codes for dow… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparisons on HumanML3D dataset. Our LG-Tok demonstrates superior semantic understanding compared to existing methods. The examples show better spatial awareness (“in the middle”), more realistic posture synthesis (“dodges quickly”), and improved directional control (“then turns to the left”). Methods Reconstruction Generation FID↓ Top 1↑ MPJPE↓ FID↓ MM-Dist↓ Evaluation on HumanML3D dataset T2… view at source ↗
Figure 5
Figure 5. Figure 5: Evaluation sweep over guidance scale g. We evalu￾ate the impact of different guidance scales on generation quality, showing optimal performance at g = 2.0 for HumanML3D and g = 1.0 for Motion-X. formers [73]. RMSNorm, SwiGLU activation, and skip connections all contribute to improved performance. No￾tably, RoPE with base=100 is better suited for our short se￾quence task (∼10s, 196 frames) compared to other… view at source ↗
Figure 6
Figure 6. Figure 6: t-SNE visualization of dequantized embedding space representations. LG-Tok demonstrates well-structured clusters with clear boundaries, indicating effective capture of distinct mo￾tion patterns. for joint self-attention) and Cross-attention (two separate cross-attention between different token types). The results show that In-context concatenation performs better in the tokenizer, while cross-attention exc… view at source ↗
Figure 8
Figure 8. Figure 8: Complete tokenization-generation-detokenization pipeline. (a) Language-Guided Motion Tokenization: The transformer tok￾enizer encodes motion with text guidance, followed by multi-scale quantization that produces multi-scale token sets to enable (b) Scalable Autoregressive Modeling: The SAR model performs scale-by-scale generation conditioned on text, and the transformer detokenizer de￾codes motion from deq… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison for text guidance. Qualitative comparison based on Motion-X checkpoint demonstrating that text￾guided tokenization (LG-Tok) generates motions with improved semantic alignment to text descriptions compared to training without (w/o) text guidance. We compare the qualitative results of our transformer-based tokenizer with text guidance (LG-Tok) against the variant with￾out text guidance… view at source ↗
Figure 10
Figure 10. Figure 10: More qualitative results on HumanML3D. Each motion sequence is visualized with temporal expansion, with yellow dashed boxes highlighting key frames corresponding to the emphasized actions in the text descriptions (shown in blue and red). G. The Role of Language Guidance in the Detokenization Stage during Generation Our language-guided tokenization establishes motion-text alignment during training, enablin… view at source ↗
Figure 11
Figure 11. Figure 11: Motion editing results from LG-Tok. (a) Motion in-painting: given a source motion sequence with overlapping characters (left), our model generates a coherent edited motion where a man bends down and picks something from the ground using his right hand during [5s, 8s] (red dashed box), seamlessly blending with the surrounding context. (b) Motion out-painting: starting from a single source character (left),… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

95 extracted references · 18 linked inside Pith

  1. [1]

    Human motion generation: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    Wentao Zhu, Xiaoxuan Ma, Dongwoo Ro, Hai Ci, Jinlu Zhang, Jiaxin Shi, Feng Gao, Qi Tian, and Yizhou Wang. Human motion generation: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 1

    2023

  2. [2]

    Action- conditioned 3d human motion synthesis with transformer vae

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. Action- conditioned 3d human motion synthesis with transformer vae. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 10985–10995, 2021. 3

    2021

  3. [3]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. InProceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020. 2

    2021

  4. [4]

    Synthesis of compositional animations from textual descriptions

    Anindita Ghosh, Noshaba Cheema, Cennet Oguz, Christian Theobalt, and Philipp Slusallek. Synthesis of compositional animations from textual descriptions. InProceedings of the IEEE/CVF international conference on computer vision, pages 1396–1406, 2021

    2021

  5. [5]

    Dance with melody: An lstm-autoencoder approach to music-oriented dance syn- thesis

    Taoran Tang, Jia Jia, and Hanyang Mao. Dance with melody: An lstm-autoencoder approach to music-oriented dance syn- thesis. InProceedings of the 26th ACM international confer- ence on Multimedia, pages 1598–1606, 2018

    2018

  6. [6]

    Music-driven group choreography

    Nhat Le, Thang Pham, Tuong Do, Erman Tjiputra, Quang D Tran, and Anh Nguyen. Music-driven group choreography. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8673–8682, 2023

    2023

  7. [7]

    Learning individ- ual styles of conversational gesture

    Shiry Ginosar, Amir Bar, Gefen Kohavi, Caroline Chan, Andrew Owens, and Jitendra Malik. Learning individ- ual styles of conversational gesture. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3497–3506, 2019

    2019

  8. [8]

    Listen, denoise, action! audio-driven motion synthesis with diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–20, 2023

    Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. Listen, denoise, action! audio-driven motion synthesis with diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–20, 2023

    2023

  9. [9]

    Taming diffusion models for audio- driven co-speech gesture generation

    Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, and Lequan Yu. Taming diffusion models for audio- driven co-speech gesture generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10544–10553, 2023. 1, 3

    2023

  10. [10]

    Generating human motion from textual descrip- tions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descrip- tions with discrete representations. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14730–14740, 2023. 1, 2, 3, 4, 6, 7, 8

    2023

  11. [11]

    Mmm: Generative masked motion model

    Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1546–1555, 2024. 3, 6, 8

    2024

  12. [12]

    Momask: Generative masked model- ing of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked model- ing of 3d human motions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1900–1910, 2024. 1, 2, 3, 5, 6, 7, 8

    1900

  13. [13]

    Mogents: Motion generation based on spatial-temporal joint modeling.Advances in Neural Information Processing Sys- tems, 37:130739–130763, 2024

    Weihao Yuan, Yisheng He, Weichao Shen, Yuan Dong, Xi- aodong Gu, Zilong Dong, Liefeng Bo, and Qixing Huang. Mogents: Motion generation based on spatial-temporal joint modeling.Advances in Neural Information Processing Sys- tems, 37:130739–130763, 2024. 3

    2024

  14. [14]

    Motion anything: Any to motion generation.arXiv preprint arXiv:2503.06955, 2025

    Zeyu Zhang, Yiran Wang, Wei Mao, Danning Li, Rui Zhao, Biao Wu, Zirui Song, Bohan Zhuang, Ian Reid, and Richard Hartley. Motion anything: Any to motion generation.arXiv preprint arXiv:2503.06955, 2025. 2

    Pith/arXiv arXiv 2025

  15. [15]

    Spatial-temporal multi-scale quantization for flexible motion generation.arXiv preprint arXiv:2508.08991, 2025

    Zan Wang, Jingze Zhang, Yixin Chen, Baoxiong Jia, Wei Liang, and Siyuan Huang. Spatial-temporal multi-scale quantization for flexible motion generation.arXiv preprint arXiv:2508.08991, 2025. 1, 2, 3

    Pith/arXiv arXiv 2025

  16. [16]

    Autoregressive image generation without vec- tor quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024. 1

    2024

  17. [17]

    Visual autoregressive modeling: Scalable im- age generation via next-scale prediction.arXiv preprint arXiv:2404.02905, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable im- age generation via next-scale prediction.arXiv preprint arXiv:2404.02905, 2024. 3, 5

    Pith/arXiv arXiv 2024

  18. [18]

    Lan- guage models are few-shot learners.Advances in neural in- formation processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners.Advances in neural in- formation processing systems, 33:1877–1901, 2020. 3

    1901

  19. [19]

    Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019. 2, 3

    2019

  20. [20]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022. 1, 2, 3

    2022

  21. [21]

    Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts. InEuropean Conference on Computer Vision, pages 580–597. Springer, 2022. 1, 2, 3

    2022

  22. [22]

    Scamo: Exploring the scaling law in au- toregressive motion generation model

    Shunlin Lu, Jingbo Wang, Zeyu Lu, Ling-Hao Chen, Wenxun Dai, Junting Dong, Zhiyang Dou, Bo Dai, and Ruimao Zhang. Scamo: Exploring the scaling law in au- toregressive motion generation model. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 27872–27882, 2025. 1, 2, 3

    2025

  23. [23]

    Mosa: Motion generation with scalable autoregressive modeling.arXiv preprint arXiv:2511.01200,

    Mengyuan Liu, Sheng Yan, Yong Wang, Yingjie Li, Gui-Bin Bian, and Hong Liu. Mosa: Motion generation with scalable autoregressive modeling.arXiv preprint arXiv:2511.01200,

    arXiv

  24. [24]

    Motion-x: A large-scale 3d expressive whole-body human motion dataset.Advances in Neural Information Processing Sys- tems, 36:25268–25280, 2023

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset.Advances in Neural Information Processing Sys- tems, 36:25268–25280, 2023. 2, 5

    2023

  25. [25]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. InProceedings of the IEEE/CVF 9 Conference on Computer Vision and Pattern Recognition, pages 5152–5161, 2022. 2, 5, 4

    2022

  26. [26]

    Rethinking diffusion for text-driven human motion generation: Redundant representations, evaluation, and masked autoregression

    Zichong Meng, Yiming Xie, Xiaogang Peng, Zeyu Han, and Huaizu Jiang. Rethinking diffusion for text-driven human motion generation: Redundant representations, evaluation, and masked autoregression. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 27859– 27871, 2025. 2, 5, 6, 7, 4, 8

    2025

  27. [27]

    Bamm: Bidirectional autoregressive motion model

    Ekkasit Pinyoanuntapong, Muhammad Usama Saleem, Pu Wang, Minwoo Lee, Srijan Das, and Chen Chen. Bamm: Bidirectional autoregressive motion model. InEuropean Conference on Computer Vision, pages 172–190. Springer,

  28. [28]

    Temos: Generating diverse human motions from textual descriptions

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. Temos: Generating diverse human motions from textual descriptions. InEuropean Conference on Computer Vision, pages 480–

  29. [29]

    Motionllm: Multimodal motion-language learning with large language models.arXiv e-prints, pages arXiv–2405, 2024

    Qi Wu, Yubo Zhao, Yifan Wang, Yu-Wing Tai, and Chi- Keung Tang. Motionllm: Multimodal motion-language learning with large language models.arXiv e-prints, pages arXiv–2405, 2024

    2024

  30. [30]

    A recurrent variational autoen- coder for human motion synthesis

    Taku Komura, Ikhsanul Habibie, Daniel Holden, Jonathan Schwarz, and Joe Yearsley. A recurrent variational autoen- coder for human motion synthesis. InThe 28th British Ma- chine Vision Conference, 2017. 2

    2017

  31. [31]

    Motionclip: Exposing human motion generation to clip space

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. InEuropean Conference on Com- puter Vision, pages 358–374. Springer, 2022. 2, 4

    2022

  32. [32]

    Auto-encoding varia- tional bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes.arXiv preprint arXiv:1312.6114, 2013. 2

    Pith/arXiv arXiv 2013

  33. [33]

    An introduction to variational autoencoders.Foundations and Trends® in Ma- chine Learning, 12(4):307–392, 2019

    Diederik P Kingma, Max Welling, et al. An introduction to variational autoencoders.Foundations and Trends® in Ma- chine Learning, 12(4):307–392, 2019. 2

    2019

  34. [34]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18000–18010, 2023. 2, 3

    2023

  35. [35]

    Lead: Latent realignment for human motion diffu- sion

    Nefeli Andreou, Xi Wang, Victoria Fern ´andez Abrevaya, Marie-Paule Cani, Yiorgos Chrysanthou, and Vicky Kalo- geiton. Lead: Latent realignment for human motion diffu- sion. InComputer Graphics Forum, page e70093. Wiley Online Library, 2025

    2025

  36. [36]

    Make-an-animation: Large-scale text- conditional 3d human motion generation

    Samaneh Azadi, Akbar Shah, Thomas Hayes, Devi Parikh, and Sonal Gupta. Make-an-animation: Large-scale text- conditional 3d human motion generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15039–15048, 2023. 2

    2023

  37. [37]

    Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  38. [38]

    Neural machine translation by jointly learning to align and translate.arXiv preprint arXiv:1409.0473, 2014

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate.arXiv preprint arXiv:1409.0473, 2014. 2

    Pith/arXiv arXiv 2014

  39. [39]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. InProceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human language tech- nologies, volume 1 (long and short papers), pages 4171– 4186, 2019. 2, 4

    2019

  40. [40]

    Genm3: Generative pretrained multi- path motion model for text conditional human motion gener- ation.arXiv preprint arXiv:2503.14919, 2025

    Junyu Shi, Lijiang Liu, Yong Sun, Zhiyuan Zhang, Jinni Zhou, and Qiang Nie. Genm3: Generative pretrained multi- path motion model for text conditional human motion gener- ation.arXiv preprint arXiv:2503.14919, 2025. 2, 3

    Pith/arXiv arXiv 2025

  41. [41]

    Kmm: Key frame mask mamba for extended motion generation.arXiv preprint arXiv:2411.06481, 2024

    Zeyu Zhang, Hang Gao, Akide Liu, Qi Chen, Feng Chen, Yi- ran Wang, Danning Li, Rui Zhao, Zhenming Li, Zhongwen Zhou, et al. Kmm: Key frame mask mamba for extended motion generation.arXiv preprint arXiv:2411.06481, 2024. 2, 3

    Pith/arXiv arXiv 2024

  42. [42]

    Language-guided human motion synthesis with atomic actions

    Yuanhao Zhai, Mingzhen Huang, Tianyu Luan, Lu Dong, Ifeoma Nwogu, Siwei Lyu, David Doermann, and Jun- song Yuan. Language-guided human motion synthesis with atomic actions. InProceedings of the 31st ACM Interna- tional Conference on Multimedia, pages 5262–5271, 2023. 3

    2023

  43. [43]

    Text2action: Generative adversarial synthesis from language to action

    Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo, and Songhwai Oh. Text2action: Generative adversarial synthesis from language to action. In2018 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 5915–5920. IEEE, 2018. 3

    2018

  44. [44]

    Lan- guage2pose: Natural language grounded pose forecasting

    Chaitanya Ahuja and Louis-Philippe Morency. Lan- guage2pose: Natural language grounded pose forecasting. In2019 International Conference on 3D Vision (3DV), pages 719–728. IEEE, 2019. 3

    2019

  45. [45]

    Re- modiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Re- modiffuse: Retrieval-augmented motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 364–373, 2023. 3

    2023

  46. [46]

    Motiondif- fuse: Text-driven human motion generation with diffusion model.arXiv preprint arXiv:2208.15001, 2022

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model.arXiv preprint arXiv:2208.15001, 2022. 6, 8

    Pith/arXiv arXiv 2022

  47. [47]

    Guided motion diffusion for controllable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2151–2162, 2023

    2023

  48. [48]

    Emdm: Efficient motion diffusion model for fast and high-quality motion generation

    Wenyang Zhou, Zhiyang Dou, Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, Yuan Liu, Taku Komura, Wenping Wang, and Lingjie Liu. Emdm: Efficient motion diffusion model for fast and high-quality motion generation. InEuropean Conference on Computer Vision, pages 18–38. Springer, 2024

    2024

  49. [49]

    Flame: Free- form language-based motion synthesis & editing

    Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free- form language-based motion synthesis & editing. InPro- ceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8255–8263, 2023

    2023

  50. [50]

    Dif- fusionphase: Motion diffusion in frequency domain.arXiv preprint arXiv:2312.04036, 2023

    Weilin Wan, Yiming Huang, Shutong Wu, Taku Komura, Wenping Wang, Dinesh Jayaraman, and Lingjie Liu. Dif- fusionphase: Motion diffusion in frequency domain.arXiv preprint arXiv:2312.04036, 2023

    Pith/arXiv arXiv 2023

  51. [51]

    Diversemotion: Towards diverse human 10 motion generation via discrete diffusion.arXiv preprint arXiv:2309.01372, 2023

    Yunhong Lou, Linchao Zhu, Yaxiong Wang, Xiaohan Wang, and Yi Yang. Diversemotion: Towards diverse human 10 motion generation via discrete diffusion.arXiv preprint arXiv:2309.01372, 2023. 3

    Pith/arXiv arXiv 2023

  52. [52]

    Physdiff: Physics-guided human motion diffusion model

    Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. InProceedings of the IEEE/CVF international con- ference on computer vision, pages 16010–16021, 2023. 3

    2023

  53. [53]

    Attt2m: Text-driven human motion generation with multi-perspective attention mechanism

    Chongyang Zhong, Lei Hu, Zihao Zhang, and Shihong Xia. Attt2m: Text-driven human motion generation with multi-perspective attention mechanism. InProceedings of the IEEE/CVF international conference on computer vision, pages 509–519, 2023. 3

    2023

  54. [54]

    Lamp: Language-motion pretraining for motion generation, retrieval, and captioning

    Zhe Li, Weihao Yuan, Yisheng He, Lingteng Qiu, Shen- hao Zhu, Xiaodong Gu, Weichao Shen, Yuan Dong, Zi- long Dong, and Laurence T Yang. Lamp: Language-motion pretraining for motion generation, retrieval, and captioning. arXiv preprint arXiv:2410.07093, 2024. 3

    Pith/arXiv arXiv 2024

  55. [55]

    Unitok: A uni- fied tokenizer for visual generation and understanding.arXiv preprint arXiv:2502.20321, 2025

    Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Ze- huan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A uni- fied tokenizer for visual generation and understanding.arXiv preprint arXiv:2502.20321, 2025. 3

    arXiv 2025

  56. [56]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023

    2023

  57. [57]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

  58. [58]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 3, 4

    Pith/arXiv arXiv 2010

  59. [59]

    Max-deeplab: End-to-end panoptic segmentation with mask transformers

    Huiyu Wang, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. Max-deeplab: End-to-end panoptic segmentation with mask transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5463–5474, 2021. 3

    2021

  60. [60]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3

    2021

  61. [61]

    Gener- ating diverse high-fidelity images with vq-vae-2.Advances in neural information processing systems, 32, 2019

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Gener- ating diverse high-fidelity images with vq-vae-2.Advances in neural information processing systems, 32, 2019. 3

    2019

  62. [62]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 11523–11532, 2022. 3

    2022

  63. [63]

    Imagefolder: Autoregres- sive image generation with folded tokens.arXiv preprint arXiv:2410.01756, 2024

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, and Zhe Lin. Imagefolder: Autoregres- sive image generation with folded tokens.arXiv preprint arXiv:2410.01756, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Robust latent matters: Boosting image generation with sampling error synthesis.arXiv preprint arXiv:2503.08354, 2025

    Kai Qiu, Xiang Li, Jason Kuen, Hao Chen, Xiaohao Xu, Jiuxiang Gu, Yinyi Luo, Bhiksha Raj, Zhe Lin, and Mar- ios Savvides. Robust latent matters: Boosting image generation with sampling error synthesis.arXiv preprint arXiv:2503.08354, 2025. 3

    Pith/arXiv arXiv 2025

  65. [65]

    An image is worth 32 tokens for reconstruction and generation.Advances in Neural Information Processing Systems, 37:128940– 128966, 2024

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation.Advances in Neural Information Processing Systems, 37:128940– 128966, 2024. 3

    2024

  66. [66]

    Alitok: Towards sequence modeling alignment be- tween tokenizer and autoregressive model.arXiv preprint arXiv:2506.05289, 2025

    Pingyu Wu, Kai Zhu, Yu Liu, Longxiang Tang, Jian Yang, Yansong Peng, Wei Zhai, Yang Cao, and Zheng-Jun Zha. Alitok: Towards sequence modeling alignment be- tween tokenizer and autoregressive model.arXiv preprint arXiv:2506.05289, 2025. 3

    arXiv 2025

  67. [67]

    Softvq-vae: Efficient 1-dimensional contin- uous tokenizer

    Hao Chen, Ze Wang, Xiang Li, Ximeng Sun, Fangyi Chen, Jiang Liu, Jindong Wang, Bhiksha Raj, Zicheng Liu, and Emad Barsoum. Softvq-vae: Efficient 1-dimensional contin- uous tokenizer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 28358–28370, 2025

    2025

  68. [68]

    Scaling the codebook size of vq-gan to 100,000 with a utilization rate of 99%.Advances in Neural Information Processing Systems, 37:12612–12635, 2024

    Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vq-gan to 100,000 with a utilization rate of 99%.Advances in Neural Information Processing Systems, 37:12612–12635, 2024

    2024

  69. [69]

    Latent denoising makes good visual tokenizers

    Jiawei Yang, Tianhong Li, Lijie Fan, Yonglong Tian, and Yue Wang. Latent denoising makes good visual tokenizers. arXiv preprint arXiv:2507.15856, 2025. 3

    arXiv 2025

  70. [70]

    Language- guided image tokenization for generation

    Kaiwen Zha, Lijun Yu, Alireza Fathi, David A Ross, Cordelia Schmid, Dina Katabi, and Xiuye Gu. Language- guided image tokenization for generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 15713–15722, 2025. 3

    2025

  71. [71]

    Snapmogen: Human motion generation from expressive texts.arXiv preprint arXiv:2507.09122, 2025

    Chuan Guo, Inwoo Hwang, Jian Wang, and Bing Zhou. Snapmogen: Human motion generation from expressive texts.arXiv preprint arXiv:2507.09122, 2025. 3

    arXiv 2025

  72. [72]

    The llama 3 herd of models.arXiv e-prints, pages arXiv–2407,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv e-prints, pages arXiv–2407,

  73. [73]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 4, 7

    2017

  74. [74]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 4

    2022

  75. [75]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InEuropean confer- ence on computer vision, pages 213–229. Springer, 2020. 4

    2020

  76. [76]

    Root mean square layer nor- malization.Advances in neural information processing sys- tems, 32, 2019

    Biao Zhang and Rico Sennrich. Root mean square layer nor- malization.Advances in neural information processing sys- tems, 32, 2019. 4

    2019

  77. [77]

    Glu variants improve transformer.arXiv preprint arXiv:2002.05202, 2020

    Noam Shazeer. Glu variants improve transformer.arXiv preprint arXiv:2002.05202, 2020. 4 11

    Pith/arXiv arXiv 2002

  78. [78]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

  79. [79]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Ab- hinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. 4

    Pith/arXiv arXiv 2024

  80. [80]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 5

    Pith/arXiv arXiv 2022

Showing first 80 references.