Pith. sign in

REVIEW 4 major objections 5 minor 24 references

Revival: Collaborative Artistic Creation through Human-AI Interactions in Musical Creativity

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AI musical agents trained on a small curated corpus can improvise live with human performers, driving music, visuals, and lighting in real time.

desk verdict A well-described practice report on integrating prior musical agents into a live show, but the paper's central claim that the performance demonstrates co-creation is unsupported by any evidence in the manuscript. read the letter →

arxiv 2503.15498 v1 pith:VQBGSKVH submitted 2025-01-19 cs.HC cs.AIcs.MAcs.MMcs.SDeess.AS

Keon Ju M. Lee , Philippe Pasquier , Jun Yuri This is my paper · ORCID
classification cs.HCcs.AIcs.MAcs.MMcs.SDeess.AS
keywords human-AIco-creationliveaudiovisualperformancemusicalagentsmachinelisteningsmalldataself-organizingmapsreal-timeimprovisationaudio-reactivevisuals
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Revival is a live, roughly thirty-minute audiovisual improvisation in which a percussionist and an electronic musician perform alongside three AI agents, and the paper's thesis is that real-time human-AI co-creation works: the agents, trained on a small curated corpus of the collective's compositions and works by deceased composers, respond to the human players as they play. The paper argues that a machine-listening module that reduces incoming audio to 55-dimensional feature vectors—covering duration, loudness, timbre, pitch, harmony, and emotional valence and arousal—enables the agents to segment the live stream and match it to the corpus in real time. If that works as claimed, the performance stands as a concrete demonstration that small, ethically sourced datasets are enough for generative AI to act as a responsive creative partner in live improvisation. The paper also describes how the same audio features drive reactive visuals and lighting, making the audiovisual layer an output of the same musical interaction rather than a separate technical problem.

What carries the argument

The load-bearing mechanism is the machine-listening and corpus-matching loop. Incoming audio is continuously segmented, and each segment is summarized by a 55-dimensional vector comprising duration, mean and standard deviation of loudness, Mel-Frequency Cepstral Coefficients, fundamental frequency, chroma, and valence and arousal from the circumplex model of affect, computed with an FFT window of 8192 samples and a hop size of 512 for high frequency resolution. A self-organizing map organizes the curated corpus, and the real-time matcher aligns incoming segments to the nearest corpus material; one agent, SpireMuse, adds four influence dimensions—rhythmic, spectral, melodic, and harmonic—that let performers weight which features dominate the match. A conductor environment routes OSC messages to the musical agents, the visual synthesizer, and the lighting system, so the same audio feature stream drives both musical response and the audiovisual layer.

What would settle it

Run the same corpus-matching loop on live audio that lies far outside the training data, such as unpitched percussion-only improvisation or heavily distorted noise, and have independent musicians judge whether the agent's chosen corpus segments are musically related to the input; if those judgments are at chance level, the feature-vector premise of the system fails.

Watch

Extended reading notes

Core claim

The central claim is that Revival is a working example of real-time co-creative improvisation between human performers and AI musical agents. The agents are built on systems that use self-organizing maps and variable Markov models, or concatenative synthesis with a factor oracle, to organize a curated reference corpus, and a real-time machine-listening loop converts each incoming audio segment into a 55-dimensional feature vector and matches it against that corpus. The paper contends that this lets the agents dynamically respond to human input and emulate complex musical styles, and that the same feature stream, sent over OSC, drives a generative visual synthesizer and DMX lighting under a human VJ's artistic control. The authors present the work as evidence that small-data, ethically curated AI can be a reactive and creative partner in live performance.

Load-bearing premise

The agents' real-time musical responses hinge on the untested premise that a 55-number summary of each short audio segment, computed with fixed spectral-analysis settings, captures enough musical information to match it meaningfully to the curated corpus.

Editorial extensions

If this is right

  • If the central claim holds, AI musical agents can serve as responsive improvising partners without needing large-scale training corpora.
  • The performance's roughly thirty-minute structure suggests the interaction loop remains stable over extended co-creation, not just short curated exchanges.
  • Because visuals and lighting are driven by the same audio features used for musical matching, the audiovisual design is coupled to the musical content by construction.
  • The four influence dimensions of SpireMuse give performers explicit, real-time control over which musical aspects—rhythm, timbre, melody, or harmony—guide the AI's matching.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not measure whether listeners actually hear the agents as emulating the deceased composers' styles; a listening study comparing perceived style fidelity, musical engagement, and sense of human authorship would test that claim directly.
  • Because the same feature stream drives agents, visuals, and lighting, the architecture suggests a general pattern for real-time generative art: compress the live input, match it to a curated archive, and re-synthesize across media, a recipe that could transfer to dance, spoken word, or installation art.
  • The claimed ethical advantage of small data could be examined independently by comparing the resource footprint and copyright clarity of this setup against large-scale generative music systems.
  • The paper leaves open whether the system would remain coherent with input genres far outside the reference corpus, such as purely percussive improvisation or heavily distorted noise, so the generality of the feature-vector premise remains a testable open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper describes 'Revival,' a live audiovisual performance by the artist collective K-Phi-A, in which a human percussionist and an electronic musician improvise alongside AI musical agents (MASOM and SpireMuse) and an AI-driven visual synthesizer (Autolume). The authors claim that the agents, trained on a small curated corpus of works by deceased composers and the collective's own compositions, dynamically respond to human input in real time and emulate complex musical styles, thereby showcasing human-AI co-creation. The manuscript provides a system description, a review of research-creation methodology, a discussion of challenges, and an appendix with a technical rider and stage plot. No recording, user study, performance log, code, or other direct documentation of the claimed performance is included.

Significance. If the central claim were substantiated, the work would offer a valuable example of real-time human-AI co-creation in musical performance using ethically motivated small-data training. The paper's strengths include the integration of previously developed systems (MASOM, SpireMuse, Autolume), a clear small-data rationale, and a detailed technical setup. However, as submitted, the paper functions more as a performance proposal or system description than as a verified research-creation study: the load-bearing claim that 'Revival showcases the potential of AI and human collaboration' is asserted rather than demonstrated, and no evidence of an actual realized performance is provided.

major comments (4)
  1. [Appendix A.1] The abstract and conclusion state that the 'performance demonstrates' and 'features real-time co-creative improvisation,' but Appendix A.1 says 'we are prepared to perform if needed' and describes the piece as 'designed to last approximately 30 minutes,' which reads as a proposal rather than documentation of a realized work. The manuscript provides no recording, time-aligned audio stems, performance log, or audience study. This missing evidence is load-bearing for the central claim; a published recording or a detailed performance report is required before the claim can be assessed.
  2. [Section 2, Offline/Real-time machine listening] The FFT configuration (8192-sample window, 512-sample hop) is stated to have been 'chosen based on experimental results,' but those results are not presented or cited. The 55-dimensional feature vector and this FFT setting are central to the real-time segmentation and matching that supposedly enable the agents to 'dynamically respond' and 'emulate complex musical styles.' Please report the supporting experiments or provide a citation to a fully documented study, and specify the SpireMuse influence-weight settings used in the performance.
  3. [Section 3] This section is a general literature review on research-creation methodology and AI music and does not provide any evidence about Revival itself. It does not support the paper's central empirical claim, so either the claim must be reduced to a system-description status or this section must be replaced or supplemented with an analysis of the actual performance and interaction data.
  4. [Section 2 and Appendix A] The description of the training data is underspecified: the manuscript says the agents were 'trained in works by deceased composers and the collective's compositions,' but it does not identify the corpus, its size, the composers, or how 'training' applies to the self-organizing maps and Markov models of MASOM and SpireMuse. This is necessary to evaluate the claim that the agents 'emulate complex musical styles.'
minor comments (5)
  1. [Header and Appendix A.1] The workshop is named 'NeurIPS 2024' in the header but 'NIPS workshop' in Appendix A.1; please use a consistent name.
  2. [References] Reference [8] says 'In the proceedings of the conference on the Institute of Electrical and Electronics Engineers (IEEE)'; this should be 'Proceedings of the IEEE.' Reference [12] appears to have an incomplete title; please verify.
  3. [Appendix A.2] The term 'A/Cs' is ambiguous: it likely means AC power outlets, not air conditioners; please clarify.
  4. [Figure 1] Figure 1 (the stage plot) is not referenced in the main text; add a reference or move it closer to the technical description.
  5. [Section 2] The sentence 'The listening module can adjust feature weights, subsequently influencing the matching algorithms' is vague; it would be helpful to give an example of how the weights are adjusted in real time.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Revival is an artwork description that rests on previously published systems and unverified performance claims, not on a derivation that reduces to its own inputs.

full rationale

The manuscript does not present a formal derivation chain, and none of its claims is obtained by an equation, a fitted parameter, or a 'prediction' that is equivalent by construction to its inputs. The AI agents' capabilities are attributed to prior published systems (MASOM [1], SpireMuse [2], Autolume [3]) that are cited as external, peer-reviewed work; relying on those systems is normal engineering support rather than circular reasoning. The 55-dimensional feature vector and the 8192/512 FFT configuration are described as design choices 'chosen based on experimental results,' but the paper makes no quantitative prediction from those choices, so there is no fitted-input-renamed-as-prediction step. The genuine problem with the paper is evidentiary, not circular: Appendix A.1 states that the collective 'is prepared to perform if needed,' which casts doubt on whether the asserted live performance actually took place, and no recording, log, or benchmark is included to verify the 'demonstrates' claims. That is missing external validation, which is a correctness/evidence concern, not a circularity finding. No self-definition, self-citation chain, imported uniqueness theorem, or renamed known result is present, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on the effectiveness of several unvalidated technical components from prior work: feature extraction, affective regression, self-organizing map matching, and Markov model generation. None are evaluated in this paper.

free parameters (3)
  • FFT window size = 8192 samples
    Chosen based on experimental results (Section 2, Offline machine listening) but no experimental comparison is shown; this choice affects frequency resolution and matching.
  • FFT hop size = 512 samples
    Same sentence as window size; affects time resolution and alignment of segmented features.
  • SpireMuse influence weights = not specified
    Section 2, Real-time machine listening: four influence dimensions (rhythmic, spectral, melodic, harmonic) are adjustable parameters that shape matching, but their values for the performance are not given.
assumptions (4)
  • domain assumption The 55-dimensional feature vector is sufficient to characterize musically meaningful audio segments for the corpus matching.
    Invoked in Section 2 (Offline machine listening) without empirical validation in this paper; supports the agents' ability to respond correctly.
  • domain assumption Valence and arousal computed by multivariate linear regression from low-level audio features are valid descriptors of affect in this context.
    Section 2, affective computing module; relies on the circumplex model [9] and prior implementation [1] with no evaluation here.
  • domain assumption Self-organizing map clustering combined with Variable Markov Models produces musically coherent generative output when conditioned on matched segments.
    The musical agents' output quality is assumed from prior publications [1,2]; the paper provides no listening test or analysis.
  • ad hoc to paper The FFT configuration (8192/512) chosen 'based on experimental results' is appropriate even though the results are not shown.
    The paper states this configuration was chosen based on experimental results (Section 2) without providing those results, making it an assumption for the reader.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revival: Collaborative Artistic Creation through Human-AI Interactions in Musical Creativity." pith.science (2026). https://pith.science/paper/VQBGSKVH

@misc{pith2026250315498,
  author       = {Pith},
  title        = {Pith review of: Revival: Collaborative Artistic Creation through Human-AI Interactions in Musical Creativity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQBGSKVH}},
  note         = {Machine review of arXiv:2503.15498}
}
read the original abstract

Revival is an innovative live audiovisual performance and music improvisation by our artist collective K-Phi-A, blending human and AI musicianship to create electronic music with audio-reactive visuals. The performance features real-time co-creative improvisation between a percussionist, an electronic music artist, and AI musical agents. Trained in works by deceased composers and the collective's compositions, these agents dynamically respond to human input and emulate complex musical styles. An AI-driven visual synthesizer, guided by a human VJ, produces visuals that evolve with the musical landscape. Revival showcases the potential of AI and human collaboration in improvisational artistic creation.

Figures

Figures reproduced from arXiv: 2503.15498 by the authors.

Figure 1
Figure 1. Stage plot for our performance setup. * Note: A/Cs are necessary to charge our 4 laptops and electronic multi-drum pads. Summary: our minimum setup is 2.1 audio systems with one 4K Projector screen. Our optional setup is 4.2+ audio systems with two 4K projector screens and 3 DMX lights. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 18 canonical work pages

  1. [1]

    & Pasquier, P

    Tatar, K. & Pasquier, P. (2017). MASOM: A Musical Agent Architecture based on Self-Organizing Maps, Affective Computing, and Variable Markov Models. In the proceedings of the Conference on International Workshop on Musical Metacreation (MuMe)

    2017

  2. [2]

    Thelle, N. J. W., & Pasquier, P. (2021). Spire Muse: A Virtual Musical Partner for Creative Brainstorming. In the proceedings of the Conference on New Interfaces for Musical Expression (NIME)

    work page 2021

  3. [3]

    Kraasch, J., & Pasquier, P. (2022). Autolume-Live: Turning GANs into a Live VJing tool. In the proceedings of the Conference on Computation, Communication, Aesthetics & X (xCoAx)

    work page 2022

  4. [4]

    Liu, P., & Lee, K. J. (2024). ReVerie. In ACM SIGGRAPH 2024 Immersive Pavilion

    work page 2024

  5. [5]

    Rowe, R. (1992). Interactive music systems: machine listening and composing. MIT press

    1992

  6. [6]

    Tatar, K., & Pasquier, P. (2019). Musical agents: A typology and state of the art towards musical metacreation. Journal of New Music Research

    2019

  7. [7]

    J., & Nussbaumer, H

    Nussbaumer, H. J., & Nussbaumer, H. J. (1982). The fast Fourier transform (pp. 80-111). Springer Berlin Heidelberg

    work page 1982

  8. [8]

    Kohonen, T. (1990). The self-organizing map. In the proceedings of the conference on the Institute of Electrical and Electronics Engineers (IEEE)

    1990

Show all 24 references
  1. [9]

    Russell, J. A. (1980). A circumplex model of affect. Journal of personality and social psychology

    1980

  2. [10]

    Vigliensoni, G., Perry, P., & Fiebrink, R. (2022). A small-data mindset for generative AI creative work. In the proceedings of the Machine Learning for Creativity and Design Workshop at the conference on Neural Information Processing Systems (NIPS)

    2022

  3. [11]

    Thelle, N. J. W., & Wærstad, B. I. G. (2023). Co-Creatives Spaces: The machine as a collaborator. In Proceedings of the Conference on New Interfaces for Musical Expression (NIME)

    2023

  4. [12]

    Stévance, S., & Lacasse, S. (2017). creation in Music and the Arts: Towards a Collaborative Interdiscipline. Routledge

    2017

  5. [13]

    Vear, C. (2021). Creative AI and musicking robots. Frontiers in Robotics and AI, 8, 631752

    2021

  6. [14]

    (2015, June)

    Marley, W., & Ward, N. (2015, June). Tightly coupled agents in live performance metacreations. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (pp. 299-302)

    2015

  7. [15]

    T., & Sturm, B

    Ben-Tal, O., Harris, M. T., & Sturm, B. L. (2021). How music AI is useful: engagements with composers, performers and audiences. Leonardo, 54(5), 510-516

    2021

  8. [16]

    Verma, S. (2021). Artificial intelligence and music: History and the future perceptive. International Journal of Applied Research, 7(2), 272-275

    2021

  9. [17]

    (2021, October)

    Smith, J., & Freeman, J. (2021, October). Effects of deep neural networks on the perceived creative autonomy of a generative musical system. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (V ol. 17, No. 1, pp. 91-98)

    2021

  10. [18]

    Deruty, E., Grachten, M., Lattner, S., Nistal, J., & Aouameur, C. (2022). On the development and practice of AI technology for contemporary popular music production. Transactions of the International Society for Music Information Retrieval, 5(1), 35-50

    2022

  11. [19]

    Holzapfel, A., Sturm, B., & Coeckelbergh, M. (2018). Ethical dimensions of music information retrieval technology. Transactions of the International Society for Music Information Retrieval, 1(1), 44-55

    2018

  12. [20]

    L., Iglesias, M., Ben-Tal, O., Miron, M., & Gómez, E

    Sturm, B. L., Iglesias, M., Ben-Tal, O., Miron, M., & Gómez, E. (2019, September). Artificial intelligence and music: open questions of copyright law and engineering praxis. In Arts (V ol. 8, No. 3, p. 115). MDPI

    2019

  13. [21]

    Dash, A., & Agres, K. (2024). AI-based affective music generation systems: A review of methods and challenges. ACM Computing Surveys, 56(11), 1-34

    2024

  14. [22]

    Miranda, E. R. (Ed.). (2021). Handbook of artificial intelligence for music. Cham: Springer

    2021

  15. [23]

    B., Stefanik, C., Stuhlsatz, C., Kacirek, K., & Belfi, A

    Shank, D. B., Stefanik, C., Stuhlsatz, C., Kacirek, K., & Belfi, A. M. (2023). AI composer bias: Listeners like music less when they think it was composed by an AI. Journal of Experimental Psychology: Applied, 29(3), 676

    2023

  16. [24]

    Carnovalini, F., & Rodà, A. (2020). Computational creativity and music generation systems: An introduction to the state of the art. Frontiers in Artificial Intelligence, 3, 14. 5 A Detailed Setup for Performance We offer a comprehensive setup for our music improvisation and au...

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.