Pith. sign in

REVIEW 5 major objections 5 minor 84 references

Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By treating every joint as a graph node and scanning the spatial-temporal graph with Mamba-style state space models in both time directions, STG-Mamba claims the best reported music-to-dance results, with a forward/backward…

desk verdict Interesting architecture and a new dataset, but the headline SOTA numbers are probably inflated by comparing 3D baselines against 2D OpenPose skeletons without alignment. read the letter →

arxiv 2507.06689 v1 pith:77OONTNC submitted 2025-07-09 cs.CV

Hao Tang , Ling Shao , Zhenyu Zhang , Luc Van Gool , Nicu Sebe This is my paper
classification cs.CV
keywords music-guideddancevideosynthesisspatial-temporalgraphstatespacemodelMambaskeleton-to-videotranslationself-supervisedregularizationgenerationconditional
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that music-to-dance-video synthesis is best solved in two stages: generate a skeleton sequence from the audio, then render a video from that skeleton plus a conditional image. For the first stage, the authors treat every joint as a graph node and process the graph with a spatial-temporal graph Mamba block, whose three state-space scans capture within-frame joint dependencies and forward/backward temporal correlations. For the second stage, a self-supervised regularization network trained on a newly collected 54,944-clip skeleton-to-video dataset imposes forward and backward consistency losses on the video generator. The reported results are the central evidence: pose Frechet distance (PFD) 86.2 versus 145.9 for EDGE and beat hit rate 81.9% versus 74.8% in music-to-skeleton, and Frechet Video Distance (FVD) 45.4 versus 81.5 for Vid2Vid in skeleton-to-video. If these numbers hold, STG-Mamba would be the best-performing method on the evaluated benchmarks.

What carries the argument

The load-bearing object is the spatial-temporal graph Mamba (STGM) block, a residual block built from three Mamba-style state space models, i.e., sequence models that maintain a compressed hidden state and process tokens in linear time. The spatial graph SSM scans the joints within each frame to capture body-structure dependencies, while the temporal graph forward and backward SSMs scan the same joints across frames in both directions to capture motion dependencies; graph convolution, layer normalization, and residual connections wrap these scans. The second stage uses a Vid2Vid backbone augmented by a self-supervised regularization network whose forward and backward generation strategies enforce the consistency losses $\mathcal{L}_{fsr}$ and $\mathcal{L}_{bsr}$ during training and are dropped at test time.

What would settle it

Run the music-to-skeleton evaluation on the same dataset from [12] with every baseline checkpoint released by its authors and a fixed reimplementation protocol for the rest; if the PFD gap between STG-Mamba and EDGE shrinks below the reported 86.2 versus 145.9, the central claim weakens. For the video stage, re-train and evaluate on an independently collected skeleton-to-video dataset or a released version of the authors' 54,944-clip dataset; if the FVD gap against Vid2Vid does not reproduce, the claimed improvement is a dataset artifact.

Watch

Extended reading notes

Core claim

The central claim is that graph-based selective state space models produce more realistic and beat-consistent dance motion than the transformer and graph-convolutional baselines, and that adding forward and backward self-supervised regularizations to the video translation stage removes temporal inconsistency. The paper's contribution is an architecture: a spatial-temporal graph Mamba block whose three scans replace attention in the skeleton generator, plus a self-supervised regularization network whose forward and backward generation strategies are training-time losses. The ablations carry the argument, showing PFD falling from 653.8 to 86.2 as the three SSM scans are added, and FVD falling from 81.5 to 45.4 as both regularization losses are added.

Load-bearing premise

The main fragile assumption is fair comparison: several baselines were reimplemented by the authors when no code was released, and the skeleton-to-video stage is evaluated on a new dataset that is not publicly released.

Editorial extensions

If this is right

  • If the numbers are reproduced, the recipe for music-to-skeleton becomes a linear-time state space scan over a graph rather than quadratic self-attention, and the paper's measured inference time of 2.2 s versus 3.8-4.8 s for transformer baselines points to cheaper long-clip generation.
  • Modeling skeleton graphs throughout generation, rather than only in a loss term, should reduce the repetitive and distorted poses the paper highlights in baselines.
  • The forward and backward regularization losses are training-only, so they can be layered on top of other motion-transfer backbones without changing inference cost.
  • The collected 54,944-clip dataset, with its mix of dance styles, genders, and indoor/outdoor scenes, provides a larger training bed for skeleton-to-video translation than prior public data.
  • The two-stage design decouples motion from appearance: one generated skeleton sequence can be rendered with different conditional images to produce different dancers dancing the same music.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the same spatial-temporal graph SSM scan could be reused for other structured sequence generation tasks, such as hand gestures from speech or multi-agent motion, where within-node spatial relations and temporal order both matter.
  • A testable extension is to run the same STGM block against diffusion-based dance generators on the same benchmarks, since the paper's baselines are mostly transformer and graph-convolutional models.
  • The inference-time advantage reported for the skeleton stage suggests real-time applications, but end-to-end latency would also require the video rendering stage to match; the paper does not measure that combination.
  • The forward/backward regularization is a training-time plug-in, so it could be added to other motion-transfer backbones than Vid2Vid; each backbone would only need adjacent-frame generation during training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes STG-Mamba, a two-stage system for music-guided dance video synthesis. The first stage, music-to-skeleton translation, uses a sequence of spatial-temporal graph Mamba (STGM) blocks, each containing a spatial graph SSM, a temporal graph forward SSM, and a temporal graph backward SSM, to generate 2D OpenPose skeleton sequences from music. The second stage, skeleton-to-video translation, builds on a Vid2Vid backbone and adds forward and backward self-supervised regularizations to improve video coherence. The authors also collect a new skeleton-to-video dataset of 54,944 video clips. Experiments report large gains over prior work: PFD 86.2 versus 145.9 for EDGE and beat hit rate 81.9% versus 74.8% in music-to-skeleton, and FVD 45.4 versus 81.5 for Vid2Vid in skeleton-to-video, supported by user studies and ablation experiments.

Significance. If the empirical claims hold, this is a meaningful advance: the graph-based SSM design is novel, the linear-complexity motivation is well placed, the ablation study is systematic, and the new dataset is potentially a useful community resource. The reported gains are large and internally consistent, and the qualitative comparisons suggest genuine qualitative differences. However, the manuscript currently lacks the controlled evaluation, statistical support, and implementation detail needed to verify the central superiority claim. The 3D-to-2D baseline mismatch, undocumented reimplementations, missing error bars, and private dataset all bear directly on whether the reported margins are real or an artifact of evaluation setup. The contribution is therefore significant but conditional on these issues being resolved.

major comments (5)
  1. [§4.1, Table 2] The central quantitative claim in Table 2 is not a controlled comparison because the output coordinate spaces of the baselines are not aligned. EDGE, DanceFormer, and FACT are designed for 3D motion data, while STG-Mamba and the MDVS dataset use 2D OpenPose skeletons. The paper says pretrained models were used for some methods and that others were reimplemented, but it never states which methods were reimplemented, which joint format or coordinate convention was used, or whether 3D outputs were projected into 2D. Since PFD, VFD, PVar, and BC are computed in the output skeleton space, a 3D model scored against 2D keypoints can be penalized purely by representation mismatch, which would confound the reported 59.7 PFD advantage over EDGE. To make the claim load-bearing, the authors must specify joint counts and coordinate conventions for every baseline, describe any projection or canonicalization, and either release the evaluation code or evaluate all methods in a common canonical skeleton space.
  2. [§4.1, §4.2, Tables 2 and 3] All main quantitative results are reported as single runs without error bars, multiple seeds, or significance tests. Generative dance models and GAN-based video models exhibit run-to-run variance, so the abstract's word 'significantly' is not statistically supported by the current evidence. The authors should report means and standard deviations over at least three seeds for the primary metrics, and ideally include pairwise significance tests for the comparisons against EDGE and Vid2Vid. This is load-bearing because the claimed margins, particularly the smaller ones in the user studies and some ablation rows, may fall within run-to-run noise.
  3. [§3.1, Eq. (1)] The core STGM block is not defined sufficiently for reproduction or for assessing the claimed O(N) complexity. The three SSM operators SG-SSM, TGF-SSM, and TGB-SSM are described only in prose and by a figure; the paper does not give the state-space equations, the scan order, the graph adjacency construction, or how the selection mechanism is made input-dependent. Without these definitions, the architectural novelty and efficiency claim cannot be checked. Please add explicit mathematical definitions for the SSM update and scan, the graph construction and adjacency, and the key hyperparameters (number of blocks, state dimension, hidden dimension h).
  4. [§4.2, dataset description] The skeleton-to-video experiments are conducted on a new 54,944-clip dataset that is not released and whose source videos are not identified. The two baselines, Pix2pixHD and Vid2Vid, are retrained on this private data, so the FID, LPIPS, and FVD comparisons in Table 3 cannot be independently reproduced. The paper should release the dataset, or a public subset with exact train/test splits, and release the code; at minimum, it must document the complete preprocessing and splitting procedure in sufficient detail. Without this, the strongest video-stage result is not verifiable.
  5. [§4.1, reimplementation statement] For baselines without released code, the manuscript only says 'we try our best to reimplement those models' without naming which methods this applies to or providing any reimplementation details. This leaves open the possibility that under-tuned baselines drive the reported gains. Please list which baselines were reimplemented, their training protocols, and any differences in hyperparameters or data preprocessing relative to the original papers.
minor comments (5)
  1. [§3.1] There is a typo in the text: 'blcok' should be 'block', and the SSM names in Eq. (1) have inconsistent spacing ('T GF SSM' and 'T GBSSM').
  2. [§4.1] The text refers to 'bet consistency score' while Table 2 uses 'BC' and the abstract uses 'beat hit rate'; please use 'beat consistency' consistently throughout.
  3. [Table 2] The inference time comparison only includes FACT, DanceFormer, and EDGE; please specify the hardware used, whether times are averaged over multiple clips, and whether the comparison is single-run or repeated.
  4. [§4.1, Figure 9] The visualization of learned spatial and temporal dependencies is not explained; please state how the response magnitudes are computed and normalized, otherwise the figure is difficult to interpret.
  5. [§3.2, Eqs. (3)–(6)] In the forward and backward generation strategies, it is not clear whether I_{i-1} and I_{i+1} are ground-truth frames or previously generated frames during training, since this affects the definition of the self-supervised losses and the training dynamics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are supported by external baselines, benchmark metrics, and ablation experiments; self-citations appear only in related work and are not load-bearing.

full rationale

The paper's central claim is an empirical performance improvement over prior methods on established metrics (PFD, VFD, PVar, BC, FID, LPIPS, FVD) and on the dataset introduced by Ren et al. [12] for music-to-skeleton translation. The proposed STGM block and the forward/backward self-supervised regularization network are architectural and loss-level contributions whose effects are tested through ablation studies (Tables 4 and 5), not derived by construction from the evaluation metrics. The losses L1 and L2 are standard reconstruction/perceptual/feature-matching objectives; FSR and BSR are consistency regularizers between differently generated video frames, not quantities that make the test-time outputs tautologically equal to training targets. Self-citations in the reference list (e.g., Motion Mamba [60], graph-related works by the authors) are confined to related-work context and do not provide the load-bearing evidence for the reported improvements. The main validity risks, such as the possible use of pretrained 3D-motion baselines (EDGE, DanceFormer, FACT) being evaluated against 2D OpenPose skeletons without explicit format alignment, or the skeleton-to-video dataset not being released, concern fairness and reproducibility rather than circular reasoning. Because no prediction reduces by definition to a fitted quantity and no load-bearing argument is sustained by a self-citation, the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The core claims rest on an empirical pipeline: OpenPose skeletons, a web-crawled dataset, Mamba's SSM formulation, and the authors' own baseline reimplementations. The architecture has several hand-chosen hyperparameters that are not fully reported. No new physical or conceptual entities are introduced.

free parameters (6)
  • loss weight lambda_p for pose perceptual loss = unspecified
    Part of optimization objective L1 in Section 3.1; no value given, so results depend on a hidden choice.
  • loss weight lambda_f for feature matching loss = unspecified
    Part of optimization objective L1 in Section 3.1; no value given, so results depend on a hidden choice.
  • number of STGM blocks = unspecified
    Architecture depth not given in Section 3.1, despite being central to the model.
  • hidden dimension h for audio tokens = unspecified
    Defined in Section 3.1 but no numerical value is provided.
  • lambda_l1 for skeleton-to-video L1 loss = 10
    Given in Section 3.2 as part of L2 optimization objective.
  • lambda_gan for skeleton-to-video adversarial loss = 1
    Given in Section 3.2 as part of L2 optimization objective.
assumptions (4)
  • domain assumption OpenPose provides accurate 2D joint skeletons from dance videos
    Used to extract all skeleton data in Sections 4.1 and 4.2; errors in pose estimation propagate to training and evaluation.
  • domain assumption The collected Internet dataset is representative and free of data leakage
    Section 4.2 says splits avoid overlapping segments, but the dataset itself is not released for independent verification.
  • ad hoc to paper Baseline models without released code are faithfully reimplemented
    Section 4.1 states 'we try our best to reimplement those models'; a sub-par reimplementation would inflate comparative gains.
  • standard math Mamba/SSM layers as described in [20] can be directly applied to graph-structured joint sequences
    The paper relies on the Mamba formulation [20] without deriving graph-specific SSM equations in Section 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis." pith.science (2026). https://pith.science/paper/77OONTNC

@misc{pith2026250706689,
  author       = {Pith},
  title        = {Pith review of: Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/77OONTNC}},
  note         = {Machine review of arXiv:2507.06689}
}
read the original abstract

We propose a novel spatial-temporal graph Mamba (STG-Mamba) for the music-guided dance video synthesis task, i.e., to translate the input music to a dance video. STG-Mamba consists of two translation mappings: music-to-skeleton translation and skeleton-to-video translation. In the music-to-skeleton translation, we introduce a novel spatial-temporal graph Mamba (STGM) block to effectively construct skeleton sequences from the input music, capturing dependencies between joints in both the spatial and temporal dimensions. For the skeleton-to-video translation, we propose a novel self-supervised regularization network to translate the generated skeletons, along with a conditional image, into a dance video. Lastly, we collect a new skeleton-to-video translation dataset from the Internet, containing 54,944 video clips. Extensive experiments demonstrate that STG-Mamba achieves significantly better results than existing methods.

Figures

Figures reproduced from arXiv: 2507.06689 by the authors.

Figure 1
Figure 1. Given an input music, the proposed STG-Mamba first generates [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed STG-Mamba, which consists of two translation mappings. The music-to-skeleton translation consists of a music [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Left: Overview of the proposed spatial-temporal graph Mamba (STGM) block, which contains a temporal graph forward SSM (TGF-SSM), a temporal graph backward SSM (TGB-SSM), and a spatial graph SSM (SG-SSM). Right: Illustration of how these three SSMs capture spatio￾temporal dependences. relationships among joints remains challenging. The Mamba structure allows us to robustly capture these dependencies without being lim… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Different generation strategies for skeleton-to-video translation. Graph convolution enables effective information exchange between nodes in the spatial-temporal graph, facilitating the incorporation of context-aware features into the model. The integration of state sp…
Figure 5
Figure 5. Figure 5: Visual comparisons for music-to-skeleton translation. Red dashed boxes above two frames indicate that the skeletons are repeti￾tive. If the red dashed box is only on a certain frame, it means that the skeleton is unnatural or distorted [PITH_FULL_IMAGE:figures/full_fi…
Figure 6
Figure 6. Figure 6: Visual comparisons for music-to-skeleton translation. Red dashed boxes above two frames indicate that the skeletons are repeti￾tive. If the red dashed box is only on a certain frame, it means that the skeleton is unnatural or distorted. 4 EXPERIMENTS 4.1 Music-to-Skele…
Figure 7
Figure 7. Figure 7: Visual comparisons for music-to-skeleton translation. Red dashed boxes above two frames indicate that the skeletons are repeti￾tive. If the red dashed box is only on a certain frame, it means that the skeleton is unnatural or distorted. [79] to extract human skeletons,…
Figure 8
Figure 8. Figure 8: Our multi-modal results vs. DanceFormer. Zoom-in for the best [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 11
Figure 11. Figure 11: Visual comparisons for skeleton-to-video translation. motion transfer models to convert the generated skele￾ton sequence to realistic video frames. Specifically, Danc￾ing2Music [1] adopts Vid2Vid [23] to achieve this goal, while MDVS [12] uses Pix2pixHD [26] to transl…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 72 canonical work pages

  1. [1]

    Dancing to music,

    H.-Y. Lee, X. Yang, M.-Y. Liu, T.-C. Wang, Y.-D. Lu, M.-H. Yang, and J. Kautz, “Dancing to music,” in NeurIPS, 2019. 1, 2, 7, 8

    2019

  2. [2]

    Making them dance

    J. W. Kim, H. Fouad, and J. K. Hahn, “Making them dance.” in AAAI Fall Symposium: Aurally Informed Performance , 2006. 1

    2006

  3. [3]

    Action2motion: Conditioned generation of 3d human motions,

    C. Guo, X. Zuo, S. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, “Action2motion: Conditioned generation of 3d human motions,” in ACM MM, 2020. 1

    2020

  4. [4]

    Temporally guided music-to-body- movement generation,

    H.-K. Kao and L. Su, “Temporally guided music-to-body- movement generation,” in ACM MM, 2020. 1

    2020

  5. [5]

    Audio to body dynamics,

    E. Shlizerman, L. Dery, H. Schoen, and I. Kemelmacher- Shlizerman, “Audio to body dynamics,” in CVPR, 2018. 1 IEEE TRANSACTIONS ON PATTERN ANAL YSIS AND MACHINE INTELLIGENCE 10

    work page 2018

  6. [6]

    Learning individual styles of conversational gesture,

    S. Ginosar, A. Bar, G. Kohavi, C. Chan, A. Owens, and J. Malik, “Learning individual styles of conversational gesture,” in CVPR,

    work page

  7. [7]

    Chore- onet: Towards music to dance synthesis with choreographic action unit,

    Z. Ye, H. Wu, J. Jia, Y. Bu, W. Chen, F. Meng, and Y. Wang, “Chore- onet: Towards music to dance synthesis with choreographic action unit,” in ACM MM, 2020. 1

    work page 2020

  8. [8]

    Groovenet: Real-time music-driven dance movement generation using artificial neural networks,

    O. Alemi, J. Franc ¸oise, and P . Pasquier, “Groovenet: Real-time music-driven dance movement generation using artificial neural networks,” Networks, vol. 8, no. 17, p. 26, 2017. 1, 2

    work page 2017

Show all 84 references
  1. [9]

    Dancing-to-music character animation,

    T. Shiratori, A. Nakazawa, and K. Ikeuchi, “Dancing-to-music character animation,” in Computer Graphics Forum, 2006. 1, 2

    2006

  2. [10]

    Generative autoregressive networks for 3d dancing move synthesis from music,

    H. Ahn, J. Kim, K. Kim, and S. Oh, “Generative autoregressive networks for 3d dancing move synthesis from music,” IEEE RAL, vol. 5, no. 2, pp. 3500–3507, 2020. 1

    2020

  3. [11]

    Dance with melody: An lstm- autoencoder approach to music-oriented dance synthesis,

    T. Tang, J. Jia, and H. Mao, “Dance with melody: An lstm- autoencoder approach to music-oriented dance synthesis,” in ACM MM, 2018. 1, 2

    2018

  4. [12]

    Self-supervised dance video synthesis conditioned on music,

    X. Ren, H. Li, Z. Huang, and Q. Chen, “Self-supervised dance video synthesis conditioned on music,” in ACM MM, 2020. 1, 2, 3, 5, 6, 7, 8, 9

    2020

  5. [13]

    Choreomaster: choreography-oriented music-driven dance synthesis,

    K. Chen, Z. Tan, J. Lei, S.-H. Zhang, Y.-C. Guo, W. Zhang, and S.-M. Hu, “Choreomaster: choreography-oriented music-driven dance synthesis,” ACM TOG, vol. 40, no. 4, pp. 1–13, 2021. 1

    2021

  6. [14]

    Actional- structural graph convolutional networks for skeleton-based action recognition,

    M. Li, S. Chen, X. Chen, Y. Zhang, Y. Wang, and Q. Tian, “Actional- structural graph convolutional networks for skeleton-based action recognition,” in CVPR, 2019. 1, 2

    2019

  7. [15]

    An attention enhanced graph convolutional lstm network for skeleton-based action recognition,

    C. Si, W. Chen, W. Wang, L. Wang, and T. Tan, “An attention enhanced graph convolutional lstm network for skeleton-based action recognition,” in CVPR, 2019. 1

    2019

  8. [16]

    Spatial temporal graph convolutional networks for skeleton-based action recognition,

    S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in AAAI, 2018. 1, 2

    2018

  9. [17]

    Bailando: 3d dance generation by actor-critic gpt with choreographic memory,

    L. Siyao, W. Yu, T. Gu, C. Lin, Q. Wang, C. Qian, C. C. Loy, and Z. Liu, “Bailando: 3d dance generation by actor-critic gpt with choreographic memory,” in CVPR, 2022. 2

    2022

  10. [18]

    Danceformer: Music conditioned 3d dance generation with parametric motion trans- former,

    B. Li, Y. Zhao, S. Zhelun, and L. Sheng, “Danceformer: Music conditioned 3d dance generation with parametric motion trans- former,” in AAAI, 2022. 2, 7

    2022

  11. [19]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS, 2017. 2

    2017

  12. [20]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023. 2, 3

    2023 arXiv

  13. [21]

    Everybody dance now,

    C. Chan, S. Ginosar, T. Zhou, and A. A. Efros, “Everybody dance now,” in ICCV, 2019. 2

    2019

  14. [22]

    Liquid warping gan: A unified framework for human motion imitation, appearance transfer and novel view synthesis,

    W. Liu, Z. Piao, J. Min, W. Luo, L. Ma, and S. Gao, “Liquid warping gan: A unified framework for human motion imitation, appearance transfer and novel view synthesis,” in ICCV, 2019. 2

    2019

  15. [23]

    Video-to-video synthesis,

    T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, G. Liu, A. Tao, J. Kautz, and B. Catanzaro, “Video-to-video synthesis,” in NeurIPS, 2018. 2, 5, 8, 9

    2018

  16. [24]

    Dance dance gen- eration: Motion transfer for internet videos,

    Y. Zhou, Z. Wang, C. Fang, T. Bui, and T. Berg, “Dance dance gen- eration: Motion transfer for internet videos,” in ICCV Workshop ,

  17. [25]

    Few-shot video-to-video synthesis,

    T.-C. Wang, M.-Y. Liu, A. Tao, G. Liu, B. Catanzaro, and J. Kautz, “Few-shot video-to-video synthesis,” in NeurIPS, 2019. 2

    2019

  18. [26]

    High-resolution image synthesis and semantic manipula- tion with conditional gans,

    T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catan- zaro, “High-resolution image synthesis and semantic manipula- tion with conditional gans,” in CVPR, 2018. 2, 5, 8, 9

    2018

  19. [27]

    An audio-driven dancing avatar,

    F. Ofli, Y. Demir, Y. Yemez, E. Erzin, A. M. Tekalp, K. Balcı, ˙I. Kızo ˘glu, L. Akarun, C. Canton-Ferrer, J. Tilmanne et al. , “An audio-driven dancing avatar,” Springer Journal on Multimodal User Interfaces, vol. 2, no. 2, pp. 93–103, 2008. 2

    2008

  20. [28]

    Music similarity-based approach to generating dance motion sequence,

    M. Lee, K. Lee, and J. Park, “Music similarity-based approach to generating dance motion sequence,” Springer MTA, vol. 62, no. 3, pp. 895–912, 2013. 2

    2013

  21. [29]

    Example-based automatic music- driven conventional dance motion synthesis,

    R. Fan, S. Xu, and W. Geng, “Example-based automatic music- driven conventional dance motion synthesis,” IEEE TVCG, vol. 18, no. 3, pp. 501–515, 2011. 2

    2011

  22. [30]

    Learning to dance: A graph convolutional adversarial network to generate realistic dance mo- tions from audio,

    J. P . Ferreira, T. M. Coutinho, T. L. Gomes, J. F. Neto, R. Azevedo, R. Martins, and E. R. Nascimento, “Learning to dance: A graph convolutional adversarial network to generate realistic dance mo- tions from audio,” Computers & Graphics , vol. 94, pp. 11–21, 2021. 2, 7

    2021

  23. [31]

    Mu- sic2dance: Dancenet for music-driven dance generation,

    W. Zhuang, C. Wang, J. Chai, Y. Wang, M. Shao, and S. Xia, “Mu- sic2dance: Dancenet for music-driven dance generation,” TOMM, vol. 18, no. 2, pp. 1–21, 2022. 2

    2022

  24. [32]

    Ai choreographer: Music conditioned 3d dance generation with aist++,

    R. Li, S. Yang, D. A. Ross, and A. Kanazawa, “Ai choreographer: Music conditioned 3d dance generation with aist++,” in ICCV,

  25. [33]

    Learning to generate diverse dance motions with transformer,

    J. Li, Y. Yin, H. Chu, Y. Zhou, T. Wang, S. Fidler, and H. Li, “Learning to generate diverse dance motions with transformer,” arXiv preprint arXiv:2008.08171, 2020. 2

    2008 arXiv

  26. [34]

    Pb-gcn: Progressive binary graph convolutional networks for skeleton-based action recognition,

    M. Zhao, S. Dai, Y. Zhu, H. Tang, P . Xie, Y. Li, C. Liu, and B. Zhang, “Pb-gcn: Progressive binary graph convolutional networks for skeleton-based action recognition,” Elsevier Neurocomputing , vol. 501, pp. 640–649, 2022. 2

    2022

  27. [35]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017. 2

    2017

  28. [36]

    Relevant region prediction for crowd counting,

    X. Chen, Y. Bin, C. Gao, N. Sang, and H. Tang, “Relevant region prediction for crowd counting,” Elsevier Neurocomputing, 2020. 2

    2020

  29. [37]

    Graph attention networks,

    P . Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P . Lio, and Y. Bengio, “Graph attention networks,” in ICLR, 2018. 2

    2018

  30. [38]

    Graph convolutional networks for text classification,

    L. Yao, C. Mao, and Y. Luo, “Graph convolutional networks for text classification,” in AAAI, 2019. 2

    2019

  31. [39]

    Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection,

    J.-X. Zhong, N. Li, W. Kong, S. Liu, T. H. Li, and G. Li, “Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection,” in CVPR, 2019. 2

    2019

  32. [40]

    Learning to cluster faces on an affinity graph,

    L. Yang, X. Zhan, D. Chen, J. Yan, C. C. Loy, and D. Lin, “Learning to cluster faces on an affinity graph,” in CVPR, 2019. 2

    2019

  33. [41]

    Attention guided graph convolu- tional networks for relation extraction,

    Y. Zhang, Z. Guo, and W. Lu, “Attention guided graph convolu- tional networks for relation extraction,” in ACL, 2019. 2

    2019

  34. [42]

    Bipartite graph rea- soning gans for person pose and facial image synthesis,

    H. Tang, L. Shao, P . H. Torr, and N. Sebe, “Bipartite graph rea- soning gans for person pose and facial image synthesis,” Springer IJCV, vol. 131, no. 3, pp. 644–658, 2023. 2

    2023

  35. [43]

    Fast interactive object annotation with curve-gcn,

    H. Ling, J. Gao, A. Kar, W. Chen, and S. Fidler, “Fast interactive object annotation with curve-gcn,” in CVPR, 2019. 2

    2019

  36. [44]

    Bipartite graph reasoning gans for person image generation,

    H. Tang, S. Bai, P . H. Torr, and N. Sebe, “Bipartite graph reasoning gans for person image generation,” in BMVC, 2020. 2

    2020

  37. [45]

    Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,

    S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in AAAI, 2019. 2

    2019

  38. [46]

    Graphmlp: A graph mlp-like architecture for 3d human pose estimation,

    W. Li, M. Liu, H. Liu, T. Guo, T. Wang, H. Tang, and N. Sebe, “Graphmlp: A graph mlp-like architecture for 3d human pose estimation,” Elsevier PR, vol. 158, p. 110925, 2025. 2

    2025

  39. [47]

    Graph transformer gans with graph masked modeling for architectural layout gener- ation,

    H. Tang, L. Shao, N. Sebe, and L. Van Gool, “Graph transformer gans with graph masked modeling for architectural layout gener- ation,” IEEE TP AMI, 2024. 2

    2024

  40. [48]

    Graph transformer gans for graph-constrained house generation,

    H. Tang, Z. Zhang, H. Shi, B. Li, L. Shao, N. Sebe, R. Timofte, and L. Van Gool, “Graph transformer gans for graph-constrained house generation,” in CVPR, 2023. 2

    2023

  41. [49]

    Bipartite graph diffusion model for human interaction generation,

    B. Chopin, H. Tang, and M. Daoudi, “Bipartite graph diffusion model for human interaction generation,” in WACV, 2024. 2

    2024

  42. [50]

    Knowledge-embedded routing network for scene graph generation,

    T. Chen, W. Yu, R. Chen, and L. Lin, “Knowledge-embedded routing network for scene graph generation,” in CVPR, 2019. 2

    2019

  43. [51]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396 ,

    arXiv

  44. [52]

    Simplified state space layers for sequence modeling,

    J. T. Smith, A. Warrington, and S. W. Linderman, “Simplified state space layers for sequence modeling,” in ICLR, 2023. 3

    2023

  45. [53]

    Hungry hungry hippos: Towards language modeling with state space models,

    D. Y. Fu, T. Dao, K. K. Saab, A. W. Thomas, A. Rudra, and C. R ´e, “Hungry hungry hippos: Towards language modeling with state space models,” in ICLR, 2023. 3

    2023

  46. [54]

    Long range language modeling via gated state spaces,

    H. Mehta, A. Gupta, A. Cutkosky, and B. Neyshabur, “Long range language modeling via gated state spaces,” in ICLR, 2023. 3

    2023

  47. [55]

    Long movie clip classification with state-space video models,

    M. M. Islam and G. Bertasius, “Long movie clip classification with state-space video models,” in ECCV, 2022. 3

    2022

  48. [56]

    S4nd: Modeling images and videos as multidimen- sional signals with state spaces,

    E. Nguyen, K. Goel, A. Gu, G. Downs, P . Shah, T. Dao, S. Baccus, and C. R ´e, “S4nd: Modeling images and videos as multidimen- sional signals with state spaces,” in NeurIPS, 2022. 3

    2022

  49. [57]

    Selective structured state-spaces for long-form video understand- ing,

    J. Wang, W. Zhu, P . Wang, X. Yu, L. Liu, M. Omar, and R. Hamid, “Selective structured state-spaces for long-form video understand- ing,” in CVPR, 2023. 3

    2023

  50. [58]

    Vivim: a video vision mamba for medical video object segmentation,

    Y. Yang, Z. Xing, and L. Zhu, “Vivim: a video vision mamba for medical video object segmentation,” arXiv preprint arXiv:2401.14168, 2024. 3

    2024 arXiv

  51. [59]

    Efficient movie scene detection using state-space transformers,

    M. M. Islam, M. Hasan, K. S. Athrey, T. Braskich, and G. Bertasius, “Efficient movie scene detection using state-space transformers,” in CVPR, 2023. 3

    2023

  52. [60]

    Motion mamba: Efficient and long sequence motion generation,

    Z. Zhang, A. Liu, I. Reid, R. Hartley, B. Zhuang, and H. Tang, “Motion mamba: Efficient and long sequence motion generation,” in ECCV, 2025. 3 IEEE TRANSACTIONS ON PATTERN ANAL YSIS AND MACHINE INTELLIGENCE 11

    2025

  53. [61]

    Mam- baic: State space models for high-performance learned image compression,

    F. Zeng, H. Tang, Y. Shao, S. Chen, L. Shao, and Y. Wang, “Mam- baic: State space models for high-performance learned image compression,” in CVPR, 2025. 3

    2025

  54. [62]

    U-mamba: Enhancing long-range dependency for biomedical image segmentation,

    J. Ma, F. Li, and B. Wang, “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” arXiv preprint arXiv:2401.04722, 2024. 3

    2024 arXiv

  55. [63]

    Vm-unet: Vision mamba unet for medical image segmentation,

    J. Ruan and S. Xiang, “Vm-unet: Vision mamba unet for medical image segmentation,” arXiv preprint arXiv:2402.02491, 2024. 3

    2024 arXiv

  56. [64]

    Swin-umamba: Mamba-based unet with imagenet-based pretraining,

    J. Liu, H. Yang, H.-Y. Zhou, Y. Xi, L. Yu, Y. Yu, Y. Liang, G. Shi, S. Zhang, H. Zheng et al. , “Swin-umamba: Mamba-based unet with imagenet-based pretraining,” arXiv preprint arXiv:2402.03302,

    arXiv

  57. [65]

    Segmamba: Long- range sequential modeling mamba for 3d medical image segmen- tation,

    Z. Xing, T. Ye, Y. Yang, G. Liu, and L. Zhu, “Segmamba: Long- range sequential modeling mamba for 3d medical image segmen- tation,” arXiv preprint arXiv:2401.13560, 2024. 3

    2024 arXiv

  58. [66]

    Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation,

    Y. Wang, J. Zhang, M. Kan, S. Shan, and X. Chen, “Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation,” in CVPR, 2020. 3

    2020

  59. [67]

    Unsupervised intra-domain adaptation for semantic segmentation through self- supervision,

    F. Pan, I. Shin, F. Rameau, S. Lee, and I. S. Kweon, “Unsupervised intra-domain adaptation for semantic segmentation through self- supervision,” in CVPR, 2020. 3

    2020

  60. [68]

    Self- supervised human depth estimation from monocular videos,

    F. Tan, H. Zhu, Z. Cui, S. Zhu, M. Pollefeys, and P . Tan, “Self- supervised human depth estimation from monocular videos,” in CVPR, 2020. 3

    2020

  61. [69]

    Look-into- object: Self-supervised structure modeling for object recognition,

    M. Zhou, Y. Bai, W. Zhang, T. Zhao, and T. Mei, “Look-into- object: Self-supervised structure modeling for object recognition,” in CVPR, 2020. 3

    2020

  62. [70]

    Scaling and benchmarking self-supervised visual representation learning,

    P . Goyal, D. Mahajan, A. Gupta, and I. Misra, “Scaling and benchmarking self-supervised visual representation learning,” in CVPR, 2019. 3

    2019

  63. [71]

    Steering self-supervised feature learning beyond local pixel statistics,

    S. Jenni, H. Jin, and P . Favaro, “Steering self-supervised feature learning beyond local pixel statistics,” in CVPR, 2020. 3

    2020

  64. [72]

    Multi-modal perception attention network with self-supervised learning for audio-visual speaker tracking,

    Y. Li, H. Liu, and H. Tang, “Multi-modal perception attention network with self-supervised learning for audio-visual speaker tracking,” in AAAI, 2022. 3

    2022

  65. [73]

    Stylerig: Rigging stylegan for 3d control over portrait images,

    A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H.-P . Seidel, P . P´erez, M. Zollhofer, and C. Theobalt, “Stylerig: Rigging stylegan for 3d control over portrait images,” in CVPR, 2020. 3

    2020

  66. [74]

    From inference to generation: End-to-end fully self-supervised generation of human face from speech,

    H.-S. Choi, C. Park, and K. Lee, “From inference to generation: End-to-end fully self-supervised generation of human face from speech,” in ICLR, 2020. 3

    2020

  67. [75]

    Video representation learn- ing by recognizing temporal transformations,

    S. Jenni, G. Meishvili, and P . Favaro, “Video representation learn- ing by recognizing temporal transformations,” in ECCV, 2020. 3

    2020

  68. [76]

    Learning by analogy: Reliable supervision from transformations for unsupervised optical flow estimation,

    L. Liu, J. Zhang, R. He, Y. Liu, Y. Wang, Y. Tai, D. Luo, C. Wang, J. Li, and F. Huang, “Learning by analogy: Reliable supervision from transformations for unsupervised optical flow estimation,” in CVPR, 2020. 3

    2020

  69. [77]

    Contrastive self-supervised learning for commonsense reasoning,

    T. Klein and M. Nabi, “Contrastive self-supervised learning for commonsense reasoning,” in ACL, 2020. 3

    2020

  70. [78]

    On the properties of neural machine translation: Encoder-decoder approaches,

    K. Cho, B. Van Merri ¨enboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” in SSST Workshop, 2014. 4

    2014

  71. [79]

    Realtime multi-person 2d pose estimation using part affinity fields,

    Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in CVPR, 2017. 7, 8

    2017

  72. [80]

    Dance revolution: Long-term dance generation with music via curriculum learning,

    R. Huang, H. Hu, W. Wu, K. Sawada, M. Zhang, and D. Jiang, “Dance revolution: Long-term dance generation with music via curriculum learning,” in ICLR, 2021. 7

    2021

  73. [81]

    Edge: Editable dance genera- tion from music,

    J. Tseng, R. Castellon, and K. Liu, “Edge: Editable dance genera- tion from music,” in CVPR, 2023. 7

    2023

  74. [82]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochre- iter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in NeurIPS, 2017. 8

    2017

  75. [83]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P . Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018. 8

    2018

  76. [84]

    Fvd: A new metric for video gen- eration,

    T. Unterthiner, S. van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly, “Fvd: A new metric for video gen- eration,” in ICLR workshops, 2019. 8 Hao Tang is an Assistant Professor at Peking University, China. Previously, he held postdoc- toral positions at both CM...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.