REVIEW 3 major objections 5 minor 1 cited by
Dance Dance ConvLSTM
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that a beat-aligned branched bidirectional ConvLSTM encoder substantially improves automatic Dance Dance Revolution chart generation over the DDC CNN-LSTM baseline.
desk verdict A well-motivated ConvLSTM extension of DDC with credible ablations, but the headline gains rest on a shaky reconstructed baseline and missing error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the beat-aligned branched bidirectional ConvLSTM encoder: two branches of two ConvLSTM layers (16 and 32 units) process 16 beat-based timesteps into the future and past, with 1D max-pooling in the frequency dimension between layers. A ConvLSTM is a recurrent network whose state transitions are convolutions, so its hidden state can preserve local musical motifs while respecting beat position, letting downbeat-to-downbeat dependencies be carried directly instead of learned implicitly by lower LSTM layers. After flattening, the two branches feed ordinary LSTM decoders that receive BPM and integer difficulty as auxiliary inputs, and the decoded branches are concatenated and passed through fully connected layers before the 48-way sigmoid output.
What would settle it
Run the released DDCL code and the original DDC implementation on the same held-out songs from a chart author other than Fraxtil, or recover the original DDC validation split; if the reported F1 edge (0.72 versus 0.58) narrows to noise or reverses, the central claim of substantial improvement fails.
Extended reading notes
Core claim
The paper's central claim is that a branched bidirectional ConvLSTM encoder, aligned to beat positions rather than fixed millisecond windows, is the right replacement for the CNN-LSTM pipeline at the heart of Dance Dance Convolution. In the step placement model, audio is sampled 32 times per beat and encoded separately into the future and past, and the output for each beat is a length-48 binary vector of step placements. A beat-tracking step before generation makes the model tempo-agnostic, unlike DDC's fixed 120 BPM assumption. In the step selection model, audio around each candidate step is encoded by ConvLSTM and concatenated with an LSTM that reads the previous 64 steps, giving the selection stage the musical context DDC lacked. On the Fraxtil dataset, this raises onset F1 from 0.5797 to 0.7245 and step-selection accuracy from 0.4974 to 0.5814, with the largest relative gains on held notes and on lower-difficulty charts.
Load-bearing premise
The load-bearing premise is that the comparison is fair and representative: the same Fraxtil dataset and 8/1/1 split are used for both models, and the rebuilt DDC baseline faithfully stands in for the original; the paper itself notes its DDC baseline underperforms the originally reported DDC numbers and credits differences in validation set composition.
Editorial extensions
If this is right
- Charts can be generated for songs at any detected tempo rather than only 120 BPM, because the audio input is beat-aligned and BPM is computed by onset detection.
- Held-note accuracy roughly doubles, from about 0.20 for DDC to 0.45 for DDCL, so generated charts should be notably more playable on holds.
- Lower-difficulty charts receive the largest placement gains, with F1 around 0.8 for difficulties 3 and 4 where DDC averaged only about 0.52.
- The model is less dependent on threshold tuning at generation time, since a fixed 0.5 threshold is much closer to the tuned optimum than it is for DDC.
- ConvLSTM encoders should help other music information retrieval tasks where beat-relative audio context matters.
Reading between the lines
- A controlled ablation that feeds identical audio features through DDC's CNN and DDCL's ConvLSTM, with the same training budget, would isolate whether the gain comes from the ConvLSTM architecture itself or from the beat-aligned sampling regime.
- The generalization question remains open beyond the 95 charts of a single chart author; training on a multi-author corpus would show whether the DDCL advantage persists across charting styles.
- Because DDC+CNN already improves step selection accuracy from 0.4974 to 0.5589, a no-audio version of the DDCL selection model would clarify how much of the remaining gap is due to audio encoding versus the recurrent selection structure.
- Training separate models for technical versus stream-oriented charts could push both accuracy numbers further, since the paper notes the Fraxtil dataset mixes both disciplines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Dance Dance ConvLSTM (DDCL), a two-stage pipeline for automatic DDR/ITG chart generation that replaces the CNN-LSTM encoder of the earlier Dance Dance Convolution (DDC) system with a beat-aligned, branched, bidirectional ConvLSTM encoder. The step placement model consumes beat-synchronous mel-spectrogram features and auxiliary BPM/difficulty inputs and outputs 48 binary step-placement predictions per beat; the step selection model augments the DDC symbolic autoregressive LSTM with audio context encoded by ConvLSTM. The authors evaluate on the 95-chart Fraxtil dataset with an 8/1/1 split, reporting that DDCL improves step placement F1 from 0.5797 (reconstructed DDC) to 0.7245 (Table 1) and step selection accuracy from 0.4974 to 0.5814 (Table 2), with additional ablations against a Conv3D encoder, a fully bidirectional encoder, and a CNN-augmented DDC step selection model.
Significance. If the reported results are robust, the paper would make a modest but useful contribution by demonstrating that ConvLSTM encoders can exploit beat-aligned audio structure in a music-information-retrieval/generation task, and by extending DDC to variable BPMs. The manuscript has notable strengths: the architecture is described in sufficient detail to be reimplemented, the ablations (Conv3D, fully bidirectional, DDC+CNN) help isolate design choices, and the appendix provides per-difficulty breakdowns that make the aggregate numbers more interpretable. However, the central claim rests on a single small dataset and on a self-reconstructed DDC baseline that the authors themselves say underperforms the original DDC numbers; the evaluation also lacks error bars or significance tests. These issues are load-bearing for the headline improvement, so the contribution is not yet established at the level claimed.
major comments (3)
- [§6.1.1, Tables 1 and A1] The manuscript acknowledges in §6.1.1 that its reconstructed DDC model scores 'somewhat lower' than the original DDC paper and attributes this to 'differences in validation set composition,' but it provides no supporting evidence such as the exact chart-level split, the BPM-alignment procedure, or the evaluation tolerance used for the DDC reconstruction. Because the headline F1 improvement (0.7245 vs. 0.5797) is measured against this reconstructed baseline, the reported gain may be an artifact of a degraded comparator. The authors should reproduce the original DDC numbers on the identical split or supply the split and preprocessing details so that the comparison can be verified directly.
- [Tables 1–2 and §6] The evaluation uses a single dataset of 95 charts by one author, with an 8/1/1 split and no confidence intervals, error bars, or significance tests. Table A1 shows that DDCL is substantially worse than DDC on difficulty-1 charts (F1 0.145 vs. 0.281) and much better on difficulties 3–4, and the authors themselves note that chart-by-chart performance 'varies wildly.' Under these conditions, the aggregate 0.14 F1 gap and 0.084 step-accuracy gap are not established as statistically reliable; the paper should report per-chart paired differences with confidence intervals, or evaluate on an additional dataset, before claiming that DDCL 'substantially increases the accuracy of chart generation.'
- [§5.3 and Table 2] The step selection comparison between DDCL and DDC conflates two changes at once: DDCL adds audio context to the symbolic model, whereas the DDC baseline uses only the 64-step history. The DDC+CNN row provides a partial control, but the central claim that the ConvLSTM encoder specifically drives the improvement requires either a DDCL variant without audio or a DDC+ConvLSTM variant with identical inputs. As presented, the Table 2 headline gap could be due to the additional audio modality rather than the ConvLSTM architecture.
minor comments (5)
- [Throughout] The manuscript contains numerous typographical errors (e.g., 'signifcant,' 'correllate,' 'accuuracy,' 'netrowks,' 'sone,' 'diffculty,' 'indic-tate') that should be corrected in a copyedit pass.
- [§5] The difficulty formula d = floor(BPM/10) - (4 - log2 L) is ambiguous: the units of L are not stated (seconds? minutes? beats?), and it is not clear how the result is guaranteed to be an integer. Please clarify the definition and the intended range.
- [Table 1 caption] The caption does not define the difference between the 'DDC(10ms)' and 'DDC' columns beyond the 20ms/10ms threshold; please state explicitly which column corresponds to the original DDC evaluation protocol and which settings are shared.
- [§9] The code availability statement says only 'this project's github'; a full URL or repository identifier should be provided so the claimed availability can be verified.
- [Figure 6] The color legend for step timing (red downbeats, blue offbeats, light green 16th notes) is described only in prose; adding a visible legend or a more detailed caption would make the qualitative comparison interpretable in print.
Circularity Check
No significant circularity found: all central metrics compare DDCL outputs against held-out human-authored charts on the external Fraxtil dataset; the paper contains no load-bearing self-citations and no fitted quantity is renamed as a prediction.
full rationale
Dance Dance ConvLSTM derives its central claims from a held-out evaluation against human-authored step charts in the shared Fraxtil dataset, using the same 8/1/1 split as the external DDC baseline [5]. No load-bearing step reduces to its own inputs by construction: the beat grid is computed by an external onset-detection algorithm (van de Wetering / ArrowVortex, refs [17,18]) and is validated rather than presupposed by the placement metric; the difficulty input is a stated heuristic formula d = floor(BPM/10) - (4 - log2 L), not a fitted quantity matched to the evaluation labels; and the reported F1 and accuracy numbers compare model outputs to chart ground truth that is produced by neither model. The paper contains no self-citations at all: its single author cites only external works (Donahue et al. [5], Shi et al. [13], van de Wetering [18]), so there is no self-citation chain to be load-bearing. The paper's own flagged limitation, in Section 6.1.1, that its reconstructed DDC scores are 'somewhat lower than that observed in the original DDC paper' and attributable to 'differences in validation set composition', is a baseline-fidelity and statistical-reliability concern (a correctness risk under Hard Rule 5), not a circularity: a possibly weakened baseline is still an external reference, and it could just as plausibly understate as overstate the true performance gap. Likewise the unspecified chart-level split details and the absence of error bars affect the strength of the empirical claim, but they do not make any prediction equivalent to its inputs by definition. The 'Max' metrics with per-chart oracle thresholds are transparently labeled as such and are not the headline numbers, which are reported at a fixed 0.5 threshold. Overall, the evaluation is self-contained against external benchmark data and the derivation chain does not reduce to fitted values or self-citations.
Assumptions & free parameters
free parameters (3)
- BPM detection via van de Wetering algorithm =
not specified
- Difficulty formula d = floor(BPM/10) - (4 - log2(L)) =
d = floor(BPM/10) - (4 - log2(L))
- Classification threshold at 0.5 for step placement =
0.5
assumptions (3)
- domain assumption The Fraxtil dataset of 95 charts by a single author is a representative and sufficient benchmark for chart generation quality.
- domain assumption Binary crossentropy on step placement plus softmax crossentropy on step selection is an appropriate surrogate for chart quality.
- domain assumption Mirroring charts by 4x data augmentation preserves the technicality of charts.
Cite this review
Pith. "Pith review of Dance Dance ConvLSTM." pith.science (2026). https://pith.science/paper/PA2GJG7G
@misc{pith2026250701644,
author = {Pith},
title = {Pith review of: Dance Dance ConvLSTM},
year = {2026},
howpublished = {\url{https://pith.science/paper/PA2GJG7G}},
note = {Machine review of arXiv:2507.01644}
}
read the original abstract
\textit{Dance Dance Revolution} is a rhythm game consisting of songs and accompanying choreography, referred to as charts. Players press arrows on a device referred to as a dance pad in time with steps determined by the song's chart. In 2017, the authors of Dance Dance Convolution (DDC) developed an algorithm for the automatic generation of \textit{Dance Dance Revolution} charts, utilizing a CNN-LSTM architecture. We introduce Dance Dance ConvLSTM (DDCL), a new method for the automatic generation of DDR charts using a ConvLSTM based model, which improves upon the DDC methodology and substantially increases the accuracy of chart generation.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
ITGPT: A Transformer Based Architecture for the Generation of Dance Dance Revolution and In the Groove Charts
ITGPT, a transformer pipeline, generates DDR/ITG arrow charts from audio with better accuracy and roughly 7x lower generation time than the prior ConvLSTM approach.
Reference graph
Works this paper leans on
- [1]
-
[2]
ESSENTIA: an Au- dio Analysis Library for Music Informa- tion Retrieval
D. Bogdanov et al. “ESSENTIA: an Au- dio Analysis Library for Music Informa- tion Retrieval.” In: International Society for Music Information Retrieval Confer- ence (ISMIR’13). 2013, pp. 493–498
work page 2013
-
[3]
Ismail Bounoua et al. “Deep Learning Approaches for Water Stress Forecast- ing in Arboriculture Using Time Series of Remote Sensing Images: Comparative Study between ConvLSTM and CNN- LSTM Models”. In: Technologies 12.6 (2024). issn: 2227-7080. doi: 10 . 3390 / technologies12060077. url: https : / / www.mdpi.com/2227-7080/12/6/77
work page 2024
-
[4]
International Timing Collective. url: https : / / itc . dance / simfile - creation-resources/
-
[5]
Chris Donahue, Zachary C Lipton, and Julian McAuley. “Dance Dance Convolu- tion”. In: Proceedings of the 34th Inter- national Conference on Machine Learning. 2017. 10
work page 2017
-
[6]
TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games
Emily Halina and Matthew Guzdial. “TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games”. In: Proceedings of the 16th International Conference on the Foundations of Dig- ital Games . FDG ’21. Montreal, QC, Canada: Association for Computing Ma- chinery, 2021. isbn: 9781450384223. doi: 10.1145/3472538.3472589 . url: https: //doi.org/10.1...
arXiv 2021
-
[7]
Building musically-relevant au- dio features through mul- tiple timescale representations
Philippe Hamel, Yoshua Bengio, and Dou- glas. Eck. “Building musically-relevant au- dio features through mul- tiple timescale representations.” In: ISMIR (2012)
work page 2012
-
[8]
Adam: A Method for Stochastic Optimization
Diederik Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: International Conference on Learning Representations (Dec. 2014)
work page 2014
Show all 19 references
-
[9]
Procedural Content Generation of Rhythm Games Using Deep Learning Methods
Yubin Liang, Wanxiang Li, and Kokolo Ikeda. “Procedural Content Generation of Rhythm Games Using Deep Learning Methods”. In: Entertainment Computing and Serious Games . Ed. by Erik van der Spek et al. Cham: Springer International Publishing, 2019, pp. 134–145. isbn: 978- 3-030-34644-7
2019
-
[10]
GenerationMania: Learning to Semantically Choreograph
Zhiyu Lin, Kyle Xiao, and Mark O. Riedl. “GenerationMania: Learning to Semantically Choreograph”. In: CoRR abs/1806.11170 (2018). arXiv: 1806 . 11170. url: http : / / arxiv . org / abs / 1806.11170
2018 arXiv
-
[11]
CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network
Mustaqeem and Soonil Kwon. “CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network”. In: Mathemat- ics 8.12 (2020). issn: 2227-7390. doi: 10 . 3390 / math8122133. url: https : //www.mdpi.com/2227-7390/8/12/2133
2020
-
[12]
Dancing Monkeys ( Au- tomated creation of step files for Dance Dance Revolution
Karl O’Keeffe. “Dancing Monkeys ( Au- tomated creation of step files for Dance Dance Revolution”. In: 2003. url: https: //api.semanticscholar.org/CorpusID: 32105004
2003
-
[13]
Convolutional LSTM Network: A Machine Learning Ap- proach for Precipitation Nowcasting
Xingjian Shi et al. Convolutional LSTM Network: A Machine Learning Ap- proach for Precipitation Nowcasting . cite arxiv:1506.04214. 2015. url: http : //arxiv.org/abs/1506.04214
2015 arXiv
-
[14]
Srilm — An Extensi- ble Language Modeling Toolkit
Andreas Stolcke. “Srilm — An Extensi- ble Language Modeling Toolkit”. In: Pro- ceedings of the 7th International Confer- ence on Spoken Language Processing (IC- SLP 2002) 2 (July 2004)
2002
-
[15]
Gen´ eLive! Gen- erating Rhythm Actions in Love Live!
Atsushi Takada et al. “Gen´ eLive! Gen- erating Rhythm Actions in Love Live!” In: AAAI Conference on Artificial In- telligence. 2022. url: https : / / api . semanticscholar . org / CorpusID : 247155107
2022
-
[16]
Dance Dance Gradation: A Generation of Fine-Tuned Dance Charts
Yudai Tsujino and Ryosuke Yamanishi. “Dance Dance Gradation: A Generation of Fine-Tuned Dance Charts”. In: In- ternational Conference on Evolutionary Computation. 2018. url: https : / / api . semanticscholar . org / CorpusID : 52163674
2018
-
[17]
Bram van de Wetering. 2017. url: https: //arrowvortex.ddrnl.com/
2017
-
[18]
Non-causal Beat Tracking for Rhythm Games
Bram van de Wetering. Non-causal Beat Tracking for Rhythm Games . 2016. Appendix 11 Table A1: All Metrics by Fine (Integer) Difficulty and Model Metric Difficulty DDC(10ms) DDC DDCL Conv3D Bidirec. F1 Score 1 0.272102 0.281437 0.145346 0.271256 0.290906 3 0.473902 0.484886 0.7...
2016
-
[2025]
cc/en/playstyles/tech/notation
url: https://itgwiki.dominick. cc/en/playstyles/tech/notation
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.