Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Dance Dance ConvLSTM

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read This paper claims that a beat-aligned branched bidirectional ConvLSTM encoder substantially improves automatic Dance Dance Revolution chart generation over the DDC CNN-LSTM baseline.

desk verdict A well-motivated ConvLSTM extension of DDC with credible ablations, but the headline gains rest on a shaky reconstructed baseline and missing error bars. read the letter →

arxiv 2507.01644 v1 pith:PA2GJG7G submitted 2025-07-02 cs.LG

classification cs.LG
keywords DanceRevolutionConvLSTMstepchartgenerationbeat-alignedaudioencodingmusicinformationretrievaldeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Automatic stepcharting for Dance Dance Revolution needs to put arrows at musically sensible moments and then choose which arrows to place. The paper claims that a beat-aligned branched bidirectional ConvLSTM encoder does both tasks better than the previous CNN-LSTM pipeline known as DDC. On the same Fraxtil dataset, the new model lifts onset placement F1 from about 0.58 to 0.72 and step-selection accuracy from about 0.50 to 0.58. The design also replaces fixed 100-millisecond audio windows with beat-based sampling, so charts can be generated at tempos other than 120 BPM, which matters for making generated charts feel natural to players.

What carries the argument

The central object is the beat-aligned branched bidirectional ConvLSTM encoder: two branches of two ConvLSTM layers (16 and 32 units) process 16 beat-based timesteps into the future and past, with 1D max-pooling in the frequency dimension between layers. A ConvLSTM is a recurrent network whose state transitions are convolutions, so its hidden state can preserve local musical motifs while respecting beat position, letting downbeat-to-downbeat dependencies be carried directly instead of learned implicitly by lower LSTM layers. After flattening, the two branches feed ordinary LSTM decoders that receive BPM and integer difficulty as auxiliary inputs, and the decoded branches are concatenated and passed through fully connected layers before the 48-way sigmoid output.

What would settle it

Run the released DDCL code and the original DDC implementation on the same held-out songs from a chart author other than Fraxtil, or recover the original DDC validation split; if the reported F1 edge (0.72 versus 0.58) narrows to noise or reverses, the central claim of substantial improvement fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a branched bidirectional ConvLSTM encoder, aligned to beat positions rather than fixed millisecond windows, is the right replacement for the CNN-LSTM pipeline at the heart of Dance Dance Convolution. In the step placement model, audio is sampled 32 times per beat and encoded separately into the future and past, and the output for each beat is a length-48 binary vector of step placements. A beat-tracking step before generation makes the model tempo-agnostic, unlike DDC's fixed 120 BPM assumption. In the step selection model, audio around each candidate step is encoded by ConvLSTM and concatenated with an LSTM that reads the previous 64 steps, giving the selection stage the musical context DDC lacked. On the Fraxtil dataset, this raises onset F1 from 0.5797 to 0.7245 and step-selection accuracy from 0.4974 to 0.5814, with the largest relative gains on held notes and on lower-difficulty charts.

Load-bearing premise

The load-bearing premise is that the comparison is fair and representative: the same Fraxtil dataset and 8/1/1 split are used for both models, and the rebuilt DDC baseline faithfully stands in for the original; the paper itself notes its DDC baseline underperforms the originally reported DDC numbers and credits differences in validation set composition.

Editorial extensions

If this is right

  • Charts can be generated for songs at any detected tempo rather than only 120 BPM, because the audio input is beat-aligned and BPM is computed by onset detection.
  • Held-note accuracy roughly doubles, from about 0.20 for DDC to 0.45 for DDCL, so generated charts should be notably more playable on holds.
  • Lower-difficulty charts receive the largest placement gains, with F1 around 0.8 for difficulties 3 and 4 where DDC averaged only about 0.52.
  • The model is less dependent on threshold tuning at generation time, since a fixed 0.5 threshold is much closer to the tuned optimum than it is for DDC.
  • ConvLSTM encoders should help other music information retrieval tasks where beat-relative audio context matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled ablation that feeds identical audio features through DDC's CNN and DDCL's ConvLSTM, with the same training budget, would isolate whether the gain comes from the ConvLSTM architecture itself or from the beat-aligned sampling regime.
  • The generalization question remains open beyond the 95 charts of a single chart author; training on a multi-author corpus would show whether the DDCL advantage persists across charting styles.
  • Because DDC+CNN already improves step selection accuracy from 0.4974 to 0.5589, a no-audio version of the DDCL selection model would clarify how much of the remaining gap is due to audio encoding versus the recurrent selection structure.
  • Training separate models for technical versus stream-oriented charts could push both accuracy numbers further, since the paper notes the Fraxtil dataset mixes both disciplines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Dance Dance ConvLSTM (DDCL), a two-stage pipeline for automatic DDR/ITG chart generation that replaces the CNN-LSTM encoder of the earlier Dance Dance Convolution (DDC) system with a beat-aligned, branched, bidirectional ConvLSTM encoder. The step placement model consumes beat-synchronous mel-spectrogram features and auxiliary BPM/difficulty inputs and outputs 48 binary step-placement predictions per beat; the step selection model augments the DDC symbolic autoregressive LSTM with audio context encoded by ConvLSTM. The authors evaluate on the 95-chart Fraxtil dataset with an 8/1/1 split, reporting that DDCL improves step placement F1 from 0.5797 (reconstructed DDC) to 0.7245 (Table 1) and step selection accuracy from 0.4974 to 0.5814 (Table 2), with additional ablations against a Conv3D encoder, a fully bidirectional encoder, and a CNN-augmented DDC step selection model.

Significance. If the reported results are robust, the paper would make a modest but useful contribution by demonstrating that ConvLSTM encoders can exploit beat-aligned audio structure in a music-information-retrieval/generation task, and by extending DDC to variable BPMs. The manuscript has notable strengths: the architecture is described in sufficient detail to be reimplemented, the ablations (Conv3D, fully bidirectional, DDC+CNN) help isolate design choices, and the appendix provides per-difficulty breakdowns that make the aggregate numbers more interpretable. However, the central claim rests on a single small dataset and on a self-reconstructed DDC baseline that the authors themselves say underperforms the original DDC numbers; the evaluation also lacks error bars or significance tests. These issues are load-bearing for the headline improvement, so the contribution is not yet established at the level claimed.

major comments (3)
  1. [§6.1.1, Tables 1 and A1] The manuscript acknowledges in §6.1.1 that its reconstructed DDC model scores 'somewhat lower' than the original DDC paper and attributes this to 'differences in validation set composition,' but it provides no supporting evidence such as the exact chart-level split, the BPM-alignment procedure, or the evaluation tolerance used for the DDC reconstruction. Because the headline F1 improvement (0.7245 vs. 0.5797) is measured against this reconstructed baseline, the reported gain may be an artifact of a degraded comparator. The authors should reproduce the original DDC numbers on the identical split or supply the split and preprocessing details so that the comparison can be verified directly.
  2. [Tables 1–2 and §6] The evaluation uses a single dataset of 95 charts by one author, with an 8/1/1 split and no confidence intervals, error bars, or significance tests. Table A1 shows that DDCL is substantially worse than DDC on difficulty-1 charts (F1 0.145 vs. 0.281) and much better on difficulties 3–4, and the authors themselves note that chart-by-chart performance 'varies wildly.' Under these conditions, the aggregate 0.14 F1 gap and 0.084 step-accuracy gap are not established as statistically reliable; the paper should report per-chart paired differences with confidence intervals, or evaluate on an additional dataset, before claiming that DDCL 'substantially increases the accuracy of chart generation.'
  3. [§5.3 and Table 2] The step selection comparison between DDCL and DDC conflates two changes at once: DDCL adds audio context to the symbolic model, whereas the DDC baseline uses only the 64-step history. The DDC+CNN row provides a partial control, but the central claim that the ConvLSTM encoder specifically drives the improvement requires either a DDCL variant without audio or a DDC+ConvLSTM variant with identical inputs. As presented, the Table 2 headline gap could be due to the additional audio modality rather than the ConvLSTM architecture.
minor comments (5)
  1. [Throughout] The manuscript contains numerous typographical errors (e.g., 'signifcant,' 'correllate,' 'accuuracy,' 'netrowks,' 'sone,' 'diffculty,' 'indic-tate') that should be corrected in a copyedit pass.
  2. [§5] The difficulty formula d = floor(BPM/10) - (4 - log2 L) is ambiguous: the units of L are not stated (seconds? minutes? beats?), and it is not clear how the result is guaranteed to be an integer. Please clarify the definition and the intended range.
  3. [Table 1 caption] The caption does not define the difference between the 'DDC(10ms)' and 'DDC' columns beyond the 20ms/10ms threshold; please state explicitly which column corresponds to the original DDC evaluation protocol and which settings are shared.
  4. [§9] The code availability statement says only 'this project's github'; a full URL or repository identifier should be provided so the claimed availability can be verified.
  5. [Figure 6] The color legend for step timing (red downbeats, blue offbeats, light green 16th notes) is described only in prose; adding a visible legend or a more detailed caption would make the qualitative comparison interpretable in print.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: all central metrics compare DDCL outputs against held-out human-authored charts on the external Fraxtil dataset; the paper contains no load-bearing self-citations and no fitted quantity is renamed as a prediction.

full rationale

Dance Dance ConvLSTM derives its central claims from a held-out evaluation against human-authored step charts in the shared Fraxtil dataset, using the same 8/1/1 split as the external DDC baseline [5]. No load-bearing step reduces to its own inputs by construction: the beat grid is computed by an external onset-detection algorithm (van de Wetering / ArrowVortex, refs [17,18]) and is validated rather than presupposed by the placement metric; the difficulty input is a stated heuristic formula d = floor(BPM/10) - (4 - log2 L), not a fitted quantity matched to the evaluation labels; and the reported F1 and accuracy numbers compare model outputs to chart ground truth that is produced by neither model. The paper contains no self-citations at all: its single author cites only external works (Donahue et al. [5], Shi et al. [13], van de Wetering [18]), so there is no self-citation chain to be load-bearing. The paper's own flagged limitation, in Section 6.1.1, that its reconstructed DDC scores are 'somewhat lower than that observed in the original DDC paper' and attributable to 'differences in validation set composition', is a baseline-fidelity and statistical-reliability concern (a correctness risk under Hard Rule 5), not a circularity: a possibly weakened baseline is still an external reference, and it could just as plausibly understate as overstate the true performance gap. Likewise the unspecified chart-level split details and the absence of error bars affect the strength of the empirical claim, but they do not make any prediction equivalent to its inputs by definition. The 'Max' metrics with per-chart oracle thresholds are transparently labeled as such and are not the headline numbers, which are reported at a fixed 0.5 threshold. Overall, the evaluation is self-contained against external benchmark data and the derivation chain does not reduce to fitted values or self-citations.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities; its free parameters are the chosen BPM detection algorithm, the handmade difficulty formula, and the fixed classification threshold. The axioms are domain assumptions about the dataset, the loss functions, and the data augmentation, none of which are given independent evidence.

free parameters (3)
  • BPM detection via van de Wetering algorithm = not specified
    The model depends on a tempo estimate computed by an external algorithm, and the paper does not evaluate how BPM estimation errors propagate into beat alignment or final chart quality.
  • Difficulty formula d = floor(BPM/10) - (4 - log2(L)) = d = floor(BPM/10) - (4 - log2(L))
    The formula that maps BPM and song length to a default difficulty is introduced by hand, with no empirical justification or comparison to observed difficulty assignments.
  • Classification threshold at 0.5 for step placement = 0.5
    The main evaluation uses a fixed threshold of 0.5, which the paper acknowledges is not necessarily optimal; a per-chart threshold tuning is reported separately but the authors do not show how this affects chart quality.
assumptions (3)
  • domain assumption The Fraxtil dataset of 95 charts by a single author is a representative and sufficient benchmark for chart generation quality.
    The authors use only this dataset, and the original DDC paper also used it, but the paper notes large variance across difficulties and charting styles within the dataset; this limits the generality of the conclusion.
  • domain assumption Binary crossentropy on step placement plus softmax crossentropy on step selection is an appropriate surrogate for chart quality.
    The loss functions treat each 48-position bin and each 256-way arrow choice independently, but playability, rhythm coherence, and chart enjoyment are not directly optimized.
  • domain assumption Mirroring charts by 4x data augmentation preserves the technicality of charts.
    The paper assumes left-right and up-down mirroring preserves the difficulty and musical quality of a chart, citing an ITG wiki page as support; this is a reasonable but unverified assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dance Dance ConvLSTM." pith.science (2026). https://pith.science/paper/PA2GJG7G

@misc{pith2026250701644,
  author       = {Pith},
  title        = {Pith review of: Dance Dance ConvLSTM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PA2GJG7G}},
  note         = {Machine review of arXiv:2507.01644}
}
read the original abstract

\textit{Dance Dance Revolution} is a rhythm game consisting of songs and accompanying choreography, referred to as charts. Players press arrows on a device referred to as a dance pad in time with steps determined by the song's chart. In 2017, the authors of Dance Dance Convolution (DDC) developed an algorithm for the automatic generation of \textit{Dance Dance Revolution} charts, utilizing a CNN-LSTM architecture. We introduce Dance Dance ConvLSTM (DDCL), a new method for the automatic generation of DDR charts using a ConvLSTM based model, which improves upon the DDC methodology and substantially increases the accuracy of chart generation.

Figures

Figures reproduced from arXiv: 2507.01644 by the authors.

Figure 1
Figure 1. The proposed pipeline takes as input spectrogram information, as above, and returns step placements and selections as below. Charts are expected to correspond to music in a rhythmically fitting manner. In general, steps should land in places where musical context sug￾gests higher activity, while more steps should be placed on higher difficulty levels overall. More 1 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The original DDC architecture. No au￾dio information is used in the selection of steps, and the step placement process is BPM agnostic [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The proposed DDCL architecture. BPM is calculated before the process of step placement through traditional onset detection methods. Difficulty is calculated per chart based on BPM and song length. Audio input is used in the process of step selection. 4 Model description Following the DDC methodology, we construct a two part process for our chart generation pipeline. As a pre-generation alignment step, we compute BPM… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The DDCL step placement model. We employ a branched ConvLSTM encoder struc￾ture, along with two LSTM layers and fully con￾nected layers before output. This model places steps one beat at a time, with 48 individual pre￾dictions for placements between beats. of musical o…
Figure 5
Figure 5. Figure 5: The DDCL step selection model. We employ a branched ConvLSTM encoder struc￾ture for audio input, as well as an LSTM for sequential processing using previously placed steps. We concatenate these outputs before feed￾ing to a dense layer for final output. 5.3 Step selecti…
Figure 6
Figure 6. Figure 6: A comparison of DDC and DDCL chart generation outputs for a snippet from the song Bad Ketchup by Ladyscraper. The color of each arrow represents the timing placement of the arrow (red represents down beats, blue off￾beats, light green 16th notes, etc.) We note the DDC …
Figure 7
Figure 7. Figure 7: Box plot for the accuracy performance of the evaluated step selection models. We note the distribution is roughly even between models, implying the inclusion of audio features results in uniform improvement in this process. features into the DDC framework before the fu…
Figure 9
Figure 9. Figure 9: Box plot representing performance on held notes. These less common steps are better represented in the DDCL model, though neither model achieves great performance. though for better threshold tuning it may be de￾sirable to retain models performing best on .5 thresholde…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ITGPT: A Transformer Based Architecture for the Generation of Dance Dance Revolution and In the Groove Charts

    cs.SD 2026-07 conditional novelty 6.0 of 10

    ITGPT, a transformer pipeline, generates DDR/ITG arrow charts from audio with better accuracy and roughly 7x lower generation time than the prior ConvLSTM approach.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Technical Notation

    ITGwiki authors. Technical Notation

  2. [2]

    ESSENTIA: an Au- dio Analysis Library for Music Informa- tion Retrieval

    D. Bogdanov et al. “ESSENTIA: an Au- dio Analysis Library for Music Informa- tion Retrieval.” In: International Society for Music Information Retrieval Confer- ence (ISMIR’13). 2013, pp. 493–498

  3. [3]

    Deep Learning Approaches for Water Stress Forecast- ing in Arboriculture Using Time Series of Remote Sensing Images: Comparative Study between ConvLSTM and CNN- LSTM Models

    Ismail Bounoua et al. “Deep Learning Approaches for Water Stress Forecast- ing in Arboriculture Using Time Series of Remote Sensing Images: Comparative Study between ConvLSTM and CNN- LSTM Models”. In: Technologies 12.6 (2024). issn: 2227-7080. doi: 10 . 3390 / technologies12060077. url: https : / / www.mdpi.com/2227-7080/12/6/77

  4. [4]

    url: https : / / itc

    International Timing Collective. url: https : / / itc . dance / simfile - creation-resources/

  5. [5]

    Dance Dance Convolu- tion

    Chris Donahue, Zachary C Lipton, and Julian McAuley. “Dance Dance Convolu- tion”. In: Proceedings of the 34th Inter- national Conference on Machine Learning. 2017. 10

  6. [6]

    TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games

    Emily Halina and Matthew Guzdial. “TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games”. In: Proceedings of the 16th International Conference on the Foundations of Dig- ital Games . FDG ’21. Montreal, QC, Canada: Association for Computing Ma- chinery, 2021. isbn: 9781450384223. doi: 10.1145/3472538.3472589 . url: https: //doi.org/10.1...

  7. [7]

    Building musically-relevant au- dio features through mul- tiple timescale representations

    Philippe Hamel, Yoshua Bengio, and Dou- glas. Eck. “Building musically-relevant au- dio features through mul- tiple timescale representations.” In: ISMIR (2012)

  8. [8]

    Adam: A Method for Stochastic Optimization

    Diederik Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: International Conference on Learning Representations (Dec. 2014)

Show all 19 references
  1. [9]

    Procedural Content Generation of Rhythm Games Using Deep Learning Methods

    Yubin Liang, Wanxiang Li, and Kokolo Ikeda. “Procedural Content Generation of Rhythm Games Using Deep Learning Methods”. In: Entertainment Computing and Serious Games . Ed. by Erik van der Spek et al. Cham: Springer International Publishing, 2019, pp. 134–145. isbn: 978- 3-030-34644-7

  2. [10]

    GenerationMania: Learning to Semantically Choreograph

    Zhiyu Lin, Kyle Xiao, and Mark O. Riedl. “GenerationMania: Learning to Semantically Choreograph”. In: CoRR abs/1806.11170 (2018). arXiv: 1806 . 11170. url: http : / / arxiv . org / abs / 1806.11170

  3. [11]

    CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network

    Mustaqeem and Soonil Kwon. “CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network”. In: Mathemat- ics 8.12 (2020). issn: 2227-7390. doi: 10 . 3390 / math8122133. url: https : //www.mdpi.com/2227-7390/8/12/2133

  4. [12]

    Dancing Monkeys ( Au- tomated creation of step files for Dance Dance Revolution

    Karl O’Keeffe. “Dancing Monkeys ( Au- tomated creation of step files for Dance Dance Revolution”. In: 2003. url: https: //api.semanticscholar.org/CorpusID: 32105004

  5. [13]

    Convolutional LSTM Network: A Machine Learning Ap- proach for Precipitation Nowcasting

    Xingjian Shi et al. Convolutional LSTM Network: A Machine Learning Ap- proach for Precipitation Nowcasting . cite arxiv:1506.04214. 2015. url: http : //arxiv.org/abs/1506.04214

  6. [14]

    Srilm — An Extensi- ble Language Modeling Toolkit

    Andreas Stolcke. “Srilm — An Extensi- ble Language Modeling Toolkit”. In: Pro- ceedings of the 7th International Confer- ence on Spoken Language Processing (IC- SLP 2002) 2 (July 2004)

  7. [15]

    Gen´ eLive! Gen- erating Rhythm Actions in Love Live!

    Atsushi Takada et al. “Gen´ eLive! Gen- erating Rhythm Actions in Love Live!” In: AAAI Conference on Artificial In- telligence. 2022. url: https : / / api . semanticscholar . org / CorpusID : 247155107

  8. [16]

    Dance Dance Gradation: A Generation of Fine-Tuned Dance Charts

    Yudai Tsujino and Ryosuke Yamanishi. “Dance Dance Gradation: A Generation of Fine-Tuned Dance Charts”. In: In- ternational Conference on Evolutionary Computation. 2018. url: https : / / api . semanticscholar . org / CorpusID : 52163674

  9. [17]

    Bram van de Wetering. 2017. url: https: //arrowvortex.ddrnl.com/

  10. [18]

    Non-causal Beat Tracking for Rhythm Games

    Bram van de Wetering. Non-causal Beat Tracking for Rhythm Games . 2016. Appendix 11 Table A1: All Metrics by Fine (Integer) Difficulty and Model Metric Difficulty DDC(10ms) DDC DDCL Conv3D Bidirec. F1 Score 1 0.272102 0.281437 0.145346 0.271256 0.290906 3 0.473902 0.484886 0.7...

  11. [2025]

    cc/en/playstyles/tech/notation

    url: https://itgwiki.dominick. cc/en/playstyles/tech/notation

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.