Pith. sign in

REVIEW 5 cited by

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2101.08779 v3 pith:774I5L4D submitted 2021-01-21 cs.CV cs.GRcs.MM

Ruilong Li , Shan Yang , David A. Ross , Angjoo Kanazawa This is my paper
classification cs.CVcs.GRcs.MM
keywords dancemotionmusicaistdatasetconditionedchangescross-modal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses -- the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict $N$ future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The effect of grain boundaries on magnetic exchange interactions in iron

    cond-mat.mtrl-sci 2026-04 unverdicted novelty 6.0 of 10

    Grain boundaries in bcc iron induce local antiferromagnetic exchange couplings that phosphorus segregation suppresses, yet realistic grain boundary densities cause only small reductions in Curie temperature as bulk re...

  2. OmniMotion-X: Versatile Multimodal Whole-Body Motion Generation

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A single autoregressive diffusion model, trained on a new 286-hour SMPL-X dataset, generates whole-body motion from text, audio, and spatial-temporal control signals, with reference-motion conditioning.

  3. FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control

    cs.RO 2025-08 conditional novelty 6.0 of 10

    FARM combines frame-accelerated augmentation with a residual mixture-of-experts to track high-dynamic humanoid motions, cutting tracking failures by 42.8% on a new HDHM benchmark.

  4. Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Adding a soft-transition action bank, an action characteristic bank, and adaptive attention fusion to the WAT baseline improves action-conditioned human motion prediction on four benchmarks.

  5. Video-Guided Text-to-Music Generation Using Public Domain Movie Collections

    cs.SD 2025-06 conditional novelty 6.0 of 10

    OSSL is the first self-hosted, mood-annotated video-music dataset, and a video adapter on MusicGen-Medium improves film music generation over text-only baselines.

Pith tools