REVIEW 3 major objections 5 minor 33 references
Hookpad Aria: A Copilot for Songwriters
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Hookpad Aria embeds a transformer music copilot in a lead-sheet editor and reports 74k accepted suggestions from 3k users.
desk verdict Real deployed co-creation system with a genuine usage dataset, but the meter-conditioning claim is unsupported by the described input representation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the Anticipatory Music Transformer, a transformer language model that partitions a piece into two sequences, events and controls, with controls shifted five seconds forward and interleaved with events, so the model can 'anticipate' future context and learn fill-in-the-middle. To adapt it to Hookpad, the authors design an encoding between Hookpad's proprietary functional-harmony lead-sheet representation and MIDI notes, and represent beats in absolute time by adding a click track with one note per beat so the model can learn to output notes in lockstep with the user's grid. Fine-tuning examples are built by selecting a random time span and capability, then partitioning melody, harmony, and click-track notes into events and controls accordingly; this single setup supports all four generation capabilities.
What would settle it
A reconstruction or alignment check would settle it: take a set of lead sheets, encode them into the MIDI-plus-click-track format, run the fine-tuned model to generate a fill or continuation, decode back to lead-sheet form, and measure whether the output respects the requested span boundaries, key, meter, and tempo—for instance, whether notes fall on the beat grid defined by the click track and whether the harmony respects the stated key. A high rate of off-grid, out-of-key, or span-violating outputs would indicate the encoding loses the musical information the copilot claims to honor.
Extended reading notes
Core claim
Hookpad Aria is a generative AI system designed to assist musicians in writing Western pop songs, integrated into Hookpad, a web-based editor for lead sheets: symbolic scores describing melody and harmony. The system offers left-to-right continuation, fill-in-the-middle, harmony-from-melody, and melody-from-harmony, all conditioned on the project's meter, key, and tempo, and it displays endless alternative generations for a selected span that users can audit and accept. Built by fine-tuning the Anticipatory Music Transformer on 50k lead sheets, Aria has generated 318k suggestions for 3k unique users since its release in March 2024, of which 74k were accepted. Semi-structured interviews with eight users suggest the tool facilitates ideation, preserves a sense of agency through short reusable suggestions and seamless editor integration, and that users want more control, such as genre, emotional tone, instruments, and structural elements.
Load-bearing premise
The load-bearing premise is that the custom encoding between Hookpad's functional-harmony lead sheets and the MIDI note representation of the Anticipatory Music Transformer, together with the click track used to represent beats in absolute time, preserves enough musical information for the generated suggestions to be coherent and aligned with the user's selected span, key, meter, and tempo.
Editorial extensions
If this is right
- If the system works as claimed, non-sequential composition—filling in a middle span or swapping melody for harmony—is feasible with a single transformer model inside a commercial editor.
- The 318k suggestions and 74k acceptances constitute a real-world interaction dataset that can be mined for implicit feedback on musical quality and style.
- The reported user interviews suggest that short, editable, span-specific suggestions preserve songwriter agency better than end-to-end text-to-music generation.
- The desire for additional controls (genre, emotion, instruments, structure) points to concrete next steps for conditioning symbolic generation models.
- The data flywheel enables A/B testing of generative models in the wild and eventual alignment of the model to user preferences.
Reading between the lines
- The acceptance rate of roughly 23% (74k of 318k) could serve as a preference signal for training a reward model, though the paper does not analyze acceptance by capability or span type.
- The click-track encoding likely constrains the model to the notated meter, so extending it to tempo changes or rubato may require re-encoding; a testable extension is to measure whether generations stay synchronized when the user changes tempo after generation.
- The same events/controls reformulation could be applied to other symbolic tasks with explicit timing constraints, such as drum fills or orchestration, wherever a 'fill this span' interface is natural.
- A stronger validation would pair the flywheel counts with automated alignment checks (e.g., whether accepted suggestions stay within the selected span in key and meter), which the paper does not report.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Hookpad Aria, a generative symbolic music system integrated into the Hookpad web editor for lead-sheet composition. The system is built by fine-tuning the Anticipatory Music Transformer on TheoryTab lead sheets, with a custom encoding between Hookpad's functional-harmony representation and MIDI, plus a click-track pseudo-instrument intended to represent beat timing in absolute time. The paper describes support for left-to-right continuation, fill-in-the-middle, harmony-from-melody, and melody-from-harmony generation, all conditional on the user's project context. It reports deployment statistics (318k suggestions generated for 3k users, 74k accepted) and qualitative findings from eight user interviews, and frames the system as a scalable data flywheel for studying human-AI music co-creation.
Significance. If the described capabilities work as claimed, the paper demonstrates a rare full integration of a state-of-the-art symbolic generative model into a production songwriting editor, with an explicit opt-in data-collection mechanism and unusually large in-the-wild interaction data. The click-track representation is a pragmatically simple way to give an absolute-time model a beat reference, and the interview findings address the under-studied dimensions of agency and ideation in music co-creation. The authors are also honest in Figure 1 that alignment, personalization, and evaluation remain future work. However, the central 'copilot' claim rests on capabilities that are not currently verified: the paper provides no objective evaluation of generation quality, encoding fidelity, or alignment to user-selected global attributes, and the described input representation appears to lack any signal for meter. The contribution is better framed as a deployment and usage report than as a validated generative system.
major comments (3)
- [§2, Figure 1] The claim that generation is 'conditioned ... global attributes (meter, key, and tempo) of their project' is not supported by the described input representation. The only timing mechanism described is a click track with 'one note per beat,' which conveys tempo and beat positions but no downbeat, barline, or meter information; no token, embedding, or control channel for meter is described anywhere in Section 2. In a sparse continuation or fill-in-the-middle context, the model therefore has no way to distinguish 3/4 from 4/4, so the stated meter-conditioning capability cannot be realized. Please either add an explicit meter signal (e.g., barline or beat-number tokens, or a conditioning embedding) or remove/qualify the meter part of the claim.
- [§2, Table 1] As written, Table 1 does not define a missing span to be generated. In the Fill-in-middle row, the events M<te ∪ H<te and controls M≥te ∪ H≥te ∪ C together include every melody and harmony note (since M<te and M≥te partition M, and similarly for H), leaving no notes to fill. The same issue applies to the Harm-to-mel and Mel-to-harm rows, where the indicated events and controls partition all of M and H. The construction should explicitly exclude all notes with start times in [ts,te] from both events and controls, or introduce separate notation for context before ts and after te; otherwise the fine-tuning data do not correspond to infilling.
- [§2, §3] The paper presents no objective evidence that the generated suggestions are coherent or aligned with the user's selected span, key, meter, or tempo. The acceptance count (74k/318k) is an implicit, UI-specific behavior signal, and the eight interviews are self-selected and qualitative; both support user engagement but not the correctness of the generative model. Since Figure 1 itself lists Alignment and Evaluation as future work, please either add at least basic validation (e.g., reconstruction accuracy on held-out TheoryTab spans under the proposed encoding and partitioning, or a small listening/alignment study) or explicitly recast the paper's contribution as a deployment and usage report rather than a validated copilot.
minor comments (5)
- [Abstract, §1] The phrase 'symbolic musical scores that describes melody and harmony' has a subject-verb agreement error; it should be 'that describe'.
- [§1] The brand name 'Github' should be written as 'GitHub'.
- [§2] Please specify the total number of fine-tuning examples generated from the 50k lead sheets, the number of fine-tuning steps, and the details of the random time-span sampling; this information is needed to understand the balance among the four capabilities in Table 1.
- [§3] The qualitative findings would be substantially strengthened by stating how the eight participants were recruited, their musical background and prior experience with AI tools, and the analysis method used (e.g., thematic coding); the current description does not permit assessment of the representativeness or reliability of the reported themes.
- [Figure 1] The 'Global structure' icon is not tied to any described input mechanism; consider replacing it with a block diagram showing how meter, key, and tempo actually enter the model, especially after the meter-conditioning concern is addressed.
Circularity Check
No circular derivation: this system paper reports an integration and usage statistics, with no mathematical claim whose output reduces to its input.
full rationale
This paper is a late-breaking demo extended abstract describing a deployed system, not a derivation of predicted quantities. There is no equation, fitted parameter, or theoretical claim whose conclusion is equivalent to its premise. The load-bearing evidence consists of system design choices (encoding Hookpad lead sheets into the Anticipatory Music Transformer, click track for beats) and aggregate usage numbers (318k suggestions, 74k accepted). The acceptance statistics are offered as evidence of user engagement, not as a predicted quantity derived from the model; they are a measurement, not a circular inference. The only mildly self-referential element is that accepted suggestions are described as future training signal for the flywheel, but the paper does not claim to validate the current system using that signal. The skeptical concern that the click track encodes tempo but no explicit meter signal is a correctness or alignment issue, not a circularity issue: the paper does not define meter into the model's output, and it explicitly lists Alignment as future work in Figure 1. Self-citations to prior work on the Anticipatory Music Transformer provide the backbone model, but the adaptation and user study are independently reported, and no uniqueness theorem or forced conclusion rests on those citations. Overall, the paper is self-contained as a system description and falls at the low end of the circularity scale.
Assumptions & free parameters
free parameters (1)
- Control anticipation window =
5 seconds
assumptions (5)
- domain assumption TheoryTab's 50k lead sheets are representative of the Western pop songwriting that Hookpad users need to generate.
- domain assumption Hookpad's proprietary functional-harmony lead-sheet representation can be converted to and from MIDI notes without loss of musically relevant information.
- domain assumption The click-track pseudo-instrument with one note per beat is sufficient to represent Hookpad's beat-based time in the Anticipatory Music Transformer's absolute-time tokenization.
- domain assumption Implicit feedback (accept vs. ignore) is a valid signal of suggestion quality and user preference.
- domain assumption Findings from interviews with eight self-selected Hookpad Aria users are informative about the broader user base.
invented entities (1)
-
Click-track pseudo-instrument
Cite this review
Pith. "Pith review of Hookpad Aria: A Copilot for Songwriters." pith.science (2026). https://pith.science/paper/2VANHLGZ
@misc{pith2026250208122,
author = {Pith},
title = {Pith review of: Hookpad Aria: A Copilot for Songwriters},
year = {2026},
howpublished = {\url{https://pith.science/paper/2VANHLGZ}},
note = {Machine review of arXiv:2502.08122}
}
read the original abstract
We present Hookpad Aria, a generative AI system designed to assist musicians in writing Western pop songs. Our system is seamlessly integrated into Hookpad, a web-based editor designed for the composition of lead sheets: symbolic music scores that describe melody and harmony. Hookpad Aria has numerous generation capabilities designed to assist users in non-sequential composition workflows, including: (1) generating left-to-right continuations of existing material, (2) filling in missing spans in the middle of existing material, and (3) generating harmony from melody and vice versa. Hookpad Aria is also a scalable data flywheel for music co-creation -- since its release in March 2024, Aria has generated 318k suggestions for 3k users who have accepted 74k into their songs. More information about Hookpad Aria is available at https://www.hooktheory.com/hookpad/aria
Reference graph
Works this paper leans on
-
[1]
Hookpad Aria: A Copilot for Songwriters
INTRODUCTION Recently, a growing body of research on generative mod- els has yielded impressive music generation capabilities in both symbolic [1–6] and acoustic [7–14] domains. These models offer tremendous potential to enrich and accelerate music creation via integration with existing workflows. Beyond the music domain, generative models have been integ...
work page 2024
-
[2]
HOOKPAD ARIA System design. The design of Hookpad Aria centers around two key principles. Firstly, we aim to support song- writers in non-sequential workflows. Accordingly, we de- signed our system to offer unprecedented control capabili- ties for symbolic music generation. In addition to sequen- tial left-to-right generation, we support fill-in-the-middl...
arXiv 2025
-
[3]
FINDINGS FROM ARIA USERS Here we summarize findings from 1-hour semi-structured interviews with eight Hookpad Aria users. Aria facilitates ideation. Participants viewed Hookpad Aria as a fellow songwriter, turning to it when they en- countered creative blocks. For example, they used it to get 1 https://www.hooktheory.com/theorytab Capability Events ( e) C...
-
[4]
We plan to explore these in future work
RESEARCH OPPORTUNITIES The scalable data flywheel created by Aria offers many po- tential avenues for future research, such as aligning mod- els to user feedback [27], A/B testing different generative models for in-the-wild evaluation [28], and producing in- teraction datasets that help us to better study the nature of human-AI music co-creation [29]. We ...
-
[5]
Performance RNN: Generating music with expressive timing and dynamics,
I. Simon and S. Oore, “Performance RNN: Generating music with expressive timing and dynamics,”Magenta Blog, 2017
work page 2017
-
[6]
Music Transformer: Gen- erating music with long-term structure,
C.-Z. A. Huang, A. Vaswani, J. Uszkoreit, I. Simon, C. Hawthorne, N. Shazeer, A. M. Dai, M. D. Hoffman, M. Dinculescu, and D. Eck, “Music Transformer: Gen- erating music with long-term structure,” inICLR, 2018
work page 2018
-
[7]
LakhNES: Improving multi-instrumental music generation with cross-domain pre-training,
C. Donahue, H. H. Mao, Y . E. Li, G. W. Cottrell, and J. McAuley, “LakhNES: Improving multi-instrumental music generation with cross-domain pre-training,” in ISMIR, 2019
work page 2019
- [8]
Show all 33 references
-
[9]
An- ticipatory music transformer,
J. Thickstun, D. Hall, C. Donahue, and P. Liang, “An- ticipatory music transformer,” in TMLR, 2023
2023
-
[10]
Compose & embellish: Well-structured piano performance generation via a two-stage approach,
S.-L. Wu and Y .-H. Yang, “Compose & embellish: Well-structured piano performance generation via a two-stage approach,” in ICASSP, 2023
2023
-
[11]
WaveNet: A generative model for raw audio,
A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” arXiv:1609.03499, 2016
2016 arXiv
-
[12]
The challenge of realistic music generation: modelling raw audio at scale,
S. Dieleman, A. van den Oord, and K. Simonyan, “The challenge of realistic music generation: modelling raw audio at scale,” NeurIPS, 2018
2018
-
[13]
Jukebox: A generative model for music,
P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever, “Jukebox: A generative model for music,” arXiv:2005.00341, 2020
2005 arXiv
-
[14]
Riffusion: Stable diffusion for real-time music generation,
S. Forsgren and H. Martiros, “Riffusion: Stable diffusion for real-time music generation,” 2022. [Online]. Available: https://riffusion.com/about
2022
-
[15]
MusicLM: Gen- erating music from text,
A. Agostinelli, T. I. Denk, Z. Borsos, J. Engel, M. Verzetti, A. Caillon, Q. Huang, A. Jansen, A. Roberts, M. Tagliasacchi et al. , “MusicLM: Gen- erating music from text,” arXiv:2301.11325, 2023
2023 arXiv
-
[16]
SingSong: Generating musi- cal accompaniments from singing,
C. Donahue, A. Caillon, A. Roberts, E. Manilow, P. Es- ling, A. Agostinelli, M. Verzetti, I. Simon, O. Pietquin, N. Zeghidour et al. , “SingSong: Generating musi- cal accompaniments from singing,”arXiv:2301.12662, 2023
2023 arXiv
-
[17]
Simple and control- lable music generation,
J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Syn- naeve, Y . Adi, and A. Défossez, “Simple and control- lable music generation,” NeurIPS, 2024
2024
-
[18]
Music ControlNet: Multiple time-varying controls for music generation,
S.-L. Wu, C. Donahue, S. Watanabe, and N. J. Bryan, “Music ControlNet: Multiple time-varying controls for music generation,” TASLP, 2023
2023
-
[19]
LA VE: LLM-powered agent assistance and language augmentation for video editing,
B. Wang, Y . Li, Z. Lv, H. Xia, Y . Xu, and R. Sodhi, “LA VE: LLM-powered agent assistance and language augmentation for video editing,” in IUI, 2024
2024
-
[20]
3DALL-E: Integrating text-to-image ai in 3d design workflows,
V . Liu, J. Vermeulen, G. Fitzmaurice, and J. Matejka, “3DALL-E: Integrating text-to-image ai in 3d design workflows,” in Proceedings of the 2023 ACM design- ing interactive systems conference, 2023
2023
-
[21]
Word- craft: Story writing with large language models,
A. Yuan, A. Coenen, E. Reif, and D. Ippolito, “Word- craft: Story writing with large language models,” in IUI, 2022
2022
-
[22]
On the design of ai-powered code assis- tants for notebooks,
A. M. McNutt, C. Wang, R. A. Deline, and S. M. Drucker, “On the design of ai-powered code assis- tants for notebooks,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 2023
2023
-
[23]
Kalliamvakou
E. Kalliamvakou. (2022) Research: Quantifying GitHub Copilot’s impact on developer productivity and happiness
2022
-
[24]
Foregrounding artist opinions: A survey study on transparency, ownership, and fairness in ai generative art,
J. Lovato, J. Zimmerman, I. Smith, P. Dodds, and J. Karson, “Foregrounding artist opinions: A survey study on transparency, ownership, and fairness in ai generative art,” arXiv:2401.15497, 2024
2024 arXiv
-
[25]
The Bach Doodle: Approachable music composition with machine learning at scale,
C.-Z. A. Huang, C. Hawthorne, A. Roberts, M. Din- culescu, J. Wexler, L. Hong, and J. Howcroft, “The Bach Doodle: Approachable music composition with machine learning at scale,” in ISMIR, 2019
2019
-
[26]
Cococo: Ai-steering tools for music novices co-creating with generative models
R. Louie, A. Cohen, C.-Z. A. Huang, M. Terry, and C. J. Cai, “Cococo: Ai-steering tools for music novices co-creating with generative models.” in IUI, 2020
2020
-
[27]
The piano inpainting ap- plication,
G. Hadjeres and L. Crestel, “The piano inpainting ap- plication,” arXiv:2107.05944, 2021
2021 arXiv
-
[28]
Composer’s assistant: An in- teractive transformer for multi-track midi infilling,
M. E. Malandro, “Composer’s assistant: An in- teractive transformer for multi-track midi infilling,” arXiv:2301.12525, 2023
2023 arXiv
-
[29]
De- signing live human-ai collaboration for musical impro- visation,
N. Becker, R. Louie, J. Thickstun, and P. Liang, “De- signing live human-ai collaboration for musical impro- visation,” in CHI Workshop on Generative AI and HCI (GenAICHI), 2024
2024
-
[30]
Automatic melody harmo- nization with triad chords: A comparative study,
Y .-C. Yeh, W.-Y . Hsiao, S. Fukayama, T. Kita- hara, B. Genchel, H.-M. Liu, H.-W. Dong, Y . Chen, T. Leong, and Y .-H. Yang, “Automatic melody harmo- nization with triad chords: A comparative study,”Jour- nal of New Music Research , 2021
2021
-
[31]
Training language models to follow in- structions with human feedback,
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wain- wright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow in- structions with human feedback,” NeurIPS, 2022
2022
-
[32]
Chatbot arena: An open platform for evaluating llms by human preference,
W.-L. Chiang, L. Zheng, Y . Sheng, A. N. Angelopou- los, T. Li, D. Li, H. Zhang, B. Zhu, M. Jordan, J. E. Gonzalez et al. , “Chatbot arena: An open platform for evaluating llms by human preference,” arXiv:2403.04132, 2024
2024 arXiv
-
[33]
Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities,
M. Lee, P. Liang, and Q. Yang, “Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities,” in CHI, 2022
2022
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.