Pith. sign in

REVIEW 4 major objections 6 minor 67 references

Two automated pipelines turn any song into a music-video script whose narrative fits the audio, but the rendered clips still lack cross-scene visual consistency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

This paper presents two off-the-shelf model pipelines (CLAP or LALM, an LLM, and a text-to-video model) for generating music videos from arbitrary songs, validated by a preliminary five-participant user study with modest results.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A genuinely new multi-stage pipeline for instrumental music-to-video, but the evaluation doesn't back the abstract's 'visual coherency' claim and the audio-dependence of the scripts is untested. the 4 major comments →

arxiv 2509.00029 v1 pith:I3NAHG6F submitted 2025-08-20 cs.SD cs.AIcs.MMeess.AS

From Sound to Sight: Towards AI-authored Music Videos

Leo Vitasovic , Stella Gra{\ss}hof , Agnes Mercedes Kloft , Ville V. Lehtola , Martin Cunneen , Justyna Starostka , Glenn McGarry , Kun Li
show 1 more author
Sami S. Brandt
This is my paper
classification cs.SD cs.AIcs.MMeess.AS
keywords AI music video generationCLAPlarge audio language modelsLLM scene scriptingtext-to-video generationzero-shot audio analysismusic visualizationcomputational synaesthesia
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes two fully automated pipelines that turn an arbitrary song — instrumental or vocal — into a music video using only off-the-shelf models. The first pipeline uses CLAP to tag audio segments with musical attributes and a reasoning LLM to turn those tags into one-sentence scene descriptions; the second feeds the raw audio to a large audio language model that writes a short story, later broken into scene prompts. In both cases a text-to-video diffusion model renders each scene and the clips are spliced together. The central claim is that the first half of this chain works: both audio-analysis methods capture mood and musical structure, and the resulting scripts are judged thematically fitting even when the videos are not. The paper's own evaluation shows the remaining bottleneck is visual consistency — characters change appearance or species, and color palettes shift between scenes.

Core claim

The paper aims to show that music-video storytelling can be generated from audio alone, without lyrics, handcrafted effects, or human prompt-writing. Its central claim: latent audio features — extracted zero-shot by CLAP or directly by a large audio language model — carry enough musical information (mood, energy, instrumentation, structure) for a reasoning LLM to turn them into concise, stylistically aligned scene scripts. The paper does not claim the visuals cohere: its own evaluation found narratives fitting but clips disjointed, with colors, lighting, and characters shifting across cuts. The contribution is a text-mediated pipeline that shows the script stage works, with the text-to-video

What carries the argument

The machinery is a four-stage audio-to-text-to-video chain. CLAP (contrastive language–audio pretraining) maps audio segments and class labels into one embedding space, returning zero-shot descriptors like 'melodic piano' or 'sad and moody strings'; pipeline 2 instead feeds the raw track to a large audio language model (LALM) that writes a story directly. A reasoning LLM (DeepSeek-R1-Distill-Llama-8B) turns descriptors or story into one-sentence scene prompts, one per segment, with global style guidelines attached. Diffusion text-to-video models (mochi-1 in pipeline 1, WAN 2.1 in pipeline 2) render each scene; clips are spliced over the original audio. The text stage is the load-bearing inte

Load-bearing premise

The pipelines assume that short, single-sentence LLM prompts suffice for off-the-shelf text-to-video models to keep characters, colors, and lighting consistent across scenes — an assumption the paper's own qualitative evaluation (Section 4.2) shows failing, since protagonists change appearance or even species between cuts.

What would settle it

Generate a multi-scene video from a script with one recurring human character and check identity across cuts: if the character's face, clothing, or species changes between scenes in repeated runs — as the paper's own example shows — the visual-consistency half of the claim is refuted. A quantitative version: run a user study large enough to separate pipeline quality from genre preference; the paper's five-participant sample cannot do that.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Instrumental and non-verbal music becomes eligible for automated music-video generation — the pipeline never needs lyrics.
  • The text-as-medium design keeps the process interpretable and steerable: every label, script, and prompt can be inspected or edited, which is the paper's stated path for aligning with artist values and AI governance.
  • Improving character and style persistence in video diffusion models becomes the single highest-leverage next step, since the paper's evidence locates the failure there, not in audio understanding or scripting.
  • Because both pipelines use off-the-shelf models, the final rendering stage can be swapped without redoing audio analysis or script generation.
  • The exploratory comparison (overall mean 2.93 for CLAP vs 2.64 for LALM, on a 7-point scale with 5 participants) tentatively suggests explicit feature tagging currently edges out direct narrative generation, but the paper treats this as too small to conclude.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The documented split — fitting narrative, incoherent images — points to a division of labour the field can exploit: audio-to-text is largely solved as a storytelling problem, so effort should shift to identity-preserving, style-consistent video generation conditioned on multi-sentence context.
  • The rule-based, editor-mimicking segmentation (cuts on frequency change, beat count, or a 7-second cap) is an independently testable component: a controlled comparison against random segmentation could quantify how much of perceived pacing comes from the cutting strategy rather than the visuals.
  • If the script stage generalizes as claimed, the same audio-analysis-plus-LLM chain could be reused for storyboards, album art, or live visuals, swapping only the final renderer.
  • The ethics-by-design framing implies an auditable-authorship claim: because language is the intermediate representation, an artist could in principle steer the whole video by editing only the text — a claim the paper motivates but does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes two fully automated pipelines for generating music videos from arbitrary songs. The CLAP-based pipeline segments audio, extracts semantic class labels with CLAP, and uses a reasoning LLM to write scene-by-scene video scripts; the LALM-based pipeline feeds raw audio to a large audio language model to produce a narrative, which is then decomposed into scene prompts. Both pipelines use off-the-shelf text-to-video models (mochi-1, WAN 2.1) and concatenate the resulting clips with the original audio. The authors report an exploratory user study (N=5) and a two-participant qualitative interview, and conclude that CLAP and LALMs can effectively extract audio features and yield coherent, stylistically aligned video scripts, while acknowledging limitations in visual consistency.

Significance. If the central claim were established, the work would advance music visualization beyond lyric-dependent methods and handcrafted audio-reactive effects, offering an interpretable, text-based intermediate representation. The paper makes a useful design contribution by showing how off-the-shelf models can be chained for this task, and it ships code and intermediate outputs. However, the current evidence does not support the abstract's 'visual coherency' and 'emotional alignment' claims, and the evaluation does not isolate the contribution of audio analysis from LLM priors. The idea is promising but the claims are substantially ahead of the data.

major comments (4)
  1. [Abstract, §5.1 vs. Table 1, §4.2] The abstract and §5.1 claim that the evaluation 'demonstrates visual coherency and emotional alignment with the music.' This is contradicted by the paper's own results: Table 1 shows all dimension means between 2.21 and 3.27 on scales whose upper bounds are 5 or 7, and §4.2 reports 'inconsistent colour palettes, lighting, and overall visual styles,' 'characters would frequently change appearance or even species,' and videos feeling like 'a collection of stock footage.' The conclusion should be reframed as an exploratory finding of storytelling potential, not demonstrated coherency.
  2. [§5.1, §4.1, §4.2, Supp. Block 1] The claim that 'both CLAP and LALMs can effectively extract meaningful audio features' is not supported by the evaluation. The user study rates final videos, not scripts, and the only script-level feedback comes from an unblinded two-participant interview in which participants were shown intermediate pipeline outputs, creating demand characteristics. There is no ablation, baseline, or forced-choice test showing that the generated scripts track the audio rather than reflecting the LLM's generic story priors. The prompt template in Supp. Block 1 injects only coarse, high-level descriptors; a reasoning LLM could produce a plausible three-act story with repeated characters even if those labels were random. A concrete test—e.g., human rating of script–music alignment with music identity blinded, or comparing scripts from mismatched audio—is needed to support the audio-text alignment claim.
  3. [§4.1, Table 1, Figure 4] The evaluation design cannot support the abstract's generalized claims. The sample is five participants; genre is confounded with pipeline (CLAP: Jazz, Vocal Jazz, Irish Folk; LALM: Pop, Vocal Jazz, Heavy Metal); and the rating scales are mixed, with Figure 4 noting Storytelling uses 1–7 while all other dimensions use 1–5, yet Table 1 and the overall rating average across dimensions. This averaging changes the meaning of 'Overall Rating' and makes cross-dimension and cross-pipeline comparisons difficult to interpret. The authors acknowledge these limitations in §4.1, but the abstract and §5.1 do not carry those caveats forward.
  4. [§3.3.2, §5.2] A load-bearing assumption is that single-sentence prompts from an LLM are sufficient for off-the-shelf text-to-video models to produce visually consistent clips across scenes. The paper's own qualitative evaluation (§4.2) and limitations (§5.2) show this assumption fails: characters change appearance or species, and color/lighting are inconsistent. Since the claimed contribution includes 'visual coherency,' the manuscript needs either a mechanism to enforce consistency (e.g., persistent style tokens, reference images, character embeddings) or a substantial de-emphasis of the coherence claim. As written, the central visual claim rests on a known-false assumption.
minor comments (6)
  1. [Supp. Block 1] Typo: 'sccenes' should be 'scenes' in the prompt template.
  2. [§2.2, Supp. Block 4] The specific LALM model is not named. For reproducibility, state the model name, version, and any relevant inference settings.
  3. [§3.3.1] The rule-based segmentation criteria ('significant frequency changes,' 'a certain number of beats') are not operationalized. Provide the thresholds or pseudocode, or point to the code location.
  4. [Table 1, Figure 4] Clarify how the 'Overall Rating' is computed when dimension scales differ. Reporting per-dimension results without a composite average would avoid the scale-mixing problem.
  5. [§4.1] The within-subject design is described as having participants watch six videos, but the videos differ by pipeline and genre. This is more accurately a mixed design with pipeline as a within-subject factor and genre confounded; the text should say so.
  6. [References] Reference [1] appears to contain an unusual title ('Secure & Personalized Music-to-Video Generation via CHARCHA'); verify the title and venue.

Circularity Check

0 steps flagged

No circular derivation: the pipeline is compositional and empirically evaluated; the only self-citation is in future work and not load-bearing.

full rationale

No circular step is present. The paper's derivation chain is compositional rather than self-referential: CLAP/LALM produce audio descriptors, the LLM writes scene scripts, text-to-video models render clips, and the evaluation is a separate user survey whose items are generic (storytelling, visual impression, transitions, emotional consistency, overall impression). The CLAP class labels are explicitly and manually predefined (Sec. 2.1: 'predefined class labels that we manually specify'), so there is no fitted parameter that is later renamed as a prediction. The central claim in Sec. 5.1—that CLAP and LALMs 'effectively extract meaningful audio features' yielding 'coherent, stylistically aligned video scripts'—is under-evidenced (it rests on two qualitative interviews and the paper's own low ratings and stated visual inconsistencies in Secs. 4.2 and 5.2), but that is an empirical attribution/validity weakness, not a circularity. The only self-citation is [35] Kloft et al. in the Future Work discussion of expectation effects; it does not support any pipeline component or result. No equation reduces to its input, no uniqueness theorem is imported, and no known result is renamed. Score 2 reflects one minor, non-load-bearing self-citation; the central contribution remains self-contained.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The central claim rests on the reliability of external models and on the adequacy of one-sentence prompts for coherent video generation. The only hand-tuned quantities are the segmentation durations and the CLAP label vocabulary, which are design choices rather than fitted scientific parameters. No new entities are proposed.

free parameters (3)
  • Randomized segment duration range = 4-8 seconds
    Section 3.3.1: randomized segmentation produces segments of durations randomly varying between 4 and 8 seconds, a hand-chosen range that determines scene count and timing.
  • Rule-based maximum segment duration = 7 seconds
    Section 3.3.1: the rule-based cutter forces a cut when a segment reaches a predefined maximum duration of 7 seconds; this threshold shapes the temporal mapping of scenes.
  • CLAP class label vocabulary = Manually specified label set
    Section 3.1.1: the authors manually select class labels for CLAP (instrumental intensity, tempo range, mood, visual style), which determines the audio descriptors fed to the LLM.
axioms (3)
  • domain assumption Off-the-shelf models (CLAP, LALM, DeepSeek-R1-Distill-Llama-8B, mochi-1, WAN 2.1) perform as documented.
    Sections 2.1-2.4 invoke these models without independent verification; the pipeline's output depends on their zero-shot and generation capabilities.
  • domain assumption Single-sentence textual prompts suffice for coherent text-to-video generation.
    Section 3.3.2 generates each clip from one LLM-written sentence; Section 4.2 and 5.2 show this assumption often fails (inconsistent characters, styles).
  • domain assumption The self-created survey items measure storytelling, visual coherence, and emotional alignment as intended.
    Section 4.1 notes the dimensions were created by the authors and that the sample size prevents internal validity checks.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From Sound to Sight: Towards AI-authored Music Videos." pith.science (2026). https://pith.science/paper/I3NAHG6F

@misc{pith2026250900029,
  author       = {Pith},
  title        = {Pith review of: From Sound to Sight: Towards AI-authored Music Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3NAHG6F}},
  note         = {Machine review of arXiv:2509.00029}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Conventional music visualisation systems rely on handcrafted ad hoc transformations of shapes and colours that offer only limited expressiveness. We propose two novel pipelines for automatically generating music videos from any user-specified, vocal or instrumental song using off-the-shelf deep learning models. Inspired by the manual workflows of music video producers, we experiment on how well latent feature-based techniques can analyse audio to detect musical qualities, such as emotional cues and instrumental patterns, and distil them into textual scene descriptions using a language model. Next, we employ a generative model to produce the corresponding video clips. To assess the generated videos, we identify several critical aspects and design and conduct a preliminary user evaluation that demonstrates storytelling potential, visual coherency and emotional alignment with the music. Our findings underscore the potential of latent feature techniques and deep generative models to expand music visualisation beyond traditional approaches.

Figures

Figures reproduced from arXiv: 2509.00029 by Agnes Mercedes Kloft, Glenn McGarry, Justyna Starostka, Kun Li, Leo Vitasovic, Martin Cunneen, Sami S. Brandt, Stella Gra{\ss}hof, Ville V. Lehtola.

Figure 1
Figure 1. Figure 1: Overview of the method proposed in this paper. Given a selected song, the pipeline generates video clips, which are then merged [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the pipelines. The video for a selected song is generated through four main steps: (1) segmenting the song, (2) audio [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Screenshots captured from final video outputs featuring the songs of specified genres. As hinted by the screenshots, the style and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Participant Ratings by Dimension and Generation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 53 canonical work pages · 3 internal anchors

  1. [1]

    Secure & Personalized Music-to- Video Generation via CHARCHA, 2025

    Mehul Agarwal, Gauri Agarwal, Santiago Benoit, Andrew Lippman, and Jean Oh. Secure & Personalized Music-to- Video Generation via CHARCHA, 2025. Published: Pre- sented at NeurIPS 2024, Creative AI Session 1. 2, 3

    work page 2025

  2. [2]

    ImproveYourVideos: Architectural Im- provements for Text-to-Video Generation Pipeline.IEEE Ac- cess, 13:1986–2003, 2025

    Vladimir Arkhipkin, Zein Shaheen, Viacheslav Vasilev, Elizaveta Dakhova, Konstantin Sobolev, Andrey Kuznetsov, and Denis Dimitrov. ImproveYourVideos: Architectural Im- provements for Text-to-Video Generation Pipeline.IEEE Ac- cess, 13:1986–2003, 2025. 2

    work page 1986

  3. [3]

    The MIT Press, Cambridge, Massachusetts, 2021

    Sofian Audry.Art in the Age of Machine Learning. The MIT Press, Cambridge, Massachusetts, 2021. 2

    work page 2021

  4. [4]

    Boden and Ernest A

    Margaret A. Boden and Ernest A. Edmonds. What is gener- ative art?Digital Creativity, 20(1-2):21–46, 2009. 2

    work page 2009

  5. [5]

    Review of Gottschall (2012): The story- telling animal: How stories make us human.Scientific Study of Literature, 2(2):317–321, 2012

    Marisa Bortolussi. Review of Gottschall (2012): The story- telling animal: How stories make us human.Scientific Study of Literature, 2(2):317–321, 2012. 2

    work page 2012

  6. [6]

    Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?

    Victor Boutin, Thomas Fel, Lakshya Singhal, Rishav Mukherji, Akash Nagaraj, Julien Colin, and Thomas Serre. Diffusion Models as Artists: Are we Closing the Gap be- tween Humans and Machines?, 2023. arXiv:2301.11722 [cs]. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  7. [7]

    Crossmodal associations between naturally occurring tactile and sound textures.Per- ception, 53(4):219–239, 2024

    Vanalata Bulusu and Leslee Lazar. Crossmodal associations between naturally occurring tactile and sound textures.Per- ception, 53(4):219–239, 2024. 2

    work page 2024

  8. [8]

    Cancino-Chac ´on, Maarten Grachten, Werner Goebl, and Gerhard Widmer

    Carlos E. Cancino-Chac ´on, Maarten Grachten, Werner Goebl, and Gerhard Widmer. Computational Models of Ex- pressive Music Performance: A Comprehensive and Critical Review.Frontiers in Digital Humanities, 5:25, 2018. 5

    work page 2018

  9. [9]

    ”scary robots”: Examining public responses to ai

    Stephen Cave, Kate Coughlan, and Kanta Dihal. ”scary robots”: Examining public responses to ai. InProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, page 331–337, New York, NY , USA, 2019. Association for Computing Machinery. 8

    work page 2019

  10. [10]

    Understanding and Creating Art with AI: Review and Outlook

    Eva Cetinic and James She. Understanding and Creating Art with AI: Review and Outlook, 2021. arXiv:2102.09109 [cs]. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  11. [11]

    Dasovich-Wilson, Marc Thompson, and Suvi Saarikallio

    Johanna N. Dasovich-Wilson, Marc Thompson, and Suvi Saarikallio. Exploring Music Video Experiences and Their Influence on Music Perception.Music & Science, 5: 20592043221117651, 2022. Publisher: SAGE Publications Ltd. 2

    work page 2022

  12. [12]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai...

    Pith/arXiv arXiv 2025

  13. [13]

    Pengi: an audio language model for audio tasks

    Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang. Pengi: an audio language model for audio tasks. InProceedings of the 37th International Conference on Neural Information Processing Systems, pages 18090– 18108, Red Hook, NY , USA, 2023. Curran Associates Inc. 4

    work page 2023

  14. [14]

    Berkeley Publishing Group, New York, New York, 2005

    John Dewey.Art as experience. Berkeley Publishing Group, New York, New York, 2005. 2

    work page 2005

  15. [15]

    EasyVid AI Video Maker, 2024

    EasyVid. EasyVid AI Video Maker, 2024. 3

    work page 2024

  16. [16]

    CAN: Creative Adversarial Networks, Generating ”Art” by Learning About Styles and Deviating from Style Norms, 2017

    Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. CAN: Creative Adversarial Networks, Generating ”Art” by Learning About Styles and Deviating from Style Norms, 2017. arXiv:1706.07068 [cs]. 2

    Pith/arXiv arXiv 2017

  17. [17]

    CLAP Learning Audio Concepts from Natural Language Supervision.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. CLAP Learning Audio Concepts from Natural Language Supervision.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023. Conference Name: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Proce...

    work page 2023

  18. [18]

    Rand, and Iyad Rah- wan

    Ziv Epstein, Sydney Levine, David G. Rand, and Iyad Rah- wan. Who Gets Credit for AI-Generated Art?iScience, 23 (9):101515, 2020. 2

    work page 2020

  19. [19]

    Frank, Matthew Groh, Laura Herman, Neil Leach, Robert Mahari, Alex “Sandy” Pentland, Olga Russakovsky, Hope Schroeder, and Amy Smith

    Ziv Epstein, Aaron Hertzmann, the Investigators of Human 9 Creativity, Memo Akten, Hany Farid, Jessica Fjeld, Mor- gan R. Frank, Matthew Groh, Laura Herman, Neil Leach, Robert Mahari, Alex “Sandy” Pentland, Olga Russakovsky, Hope Schroeder, and Amy Smith. Art and the science of generative AI.Science, 380(6650):1110–1111, 2023. Pub- lisher: American Associ...

    work page 2023

  20. [20]

    Sakshi, Oriol Ni- eto, Ramani Duraiswami, and Dinesh Manocha

    Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Ki- ran Reddy Evuru, Utkarsh Tyagi, S. Sakshi, Oriol Ni- eto, Ramani Duraiswami, and Dinesh Manocha. GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities, 2024. arXiv:2406.11768 [cs]. 4

    Pith/arXiv arXiv 2024

  21. [21]

    From Ragtime to Swingtime: Fifty Glittering Years of Stage and Song

    Isidore Goldberg Witmark and Isaac Goldberg Witmark. From Ragtime to Swingtime: Fifty Glittering Years of Stage and Song. Lee Furman, 1 edition, 1939. 2

    work page 1939

  22. [22]

    Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020. 2

    2020

  23. [23]

    Graesser, Murray Singer, and Tom Trabasso

    Arthur C. Graesser, Murray Singer, and Tom Trabasso. Con- structing inferences during narrative text comprehension. Psychological Review, 101(3):371–395, 1994. 2

    work page 1994

  24. [24]

    Beware of fictional ai narratives.Nature Machine Intelligence, 2(11):654–654, 2020

    Isabella Hermann. Beware of fictional ai narratives.Nature Machine Intelligence, 2(11):654–654, 2020. 8

    work page 2020

  25. [25]

    Can Computers Create Art?

    Aaron Hertzmann. Can Computers Create Art?, 2018. arXiv:1801.04486 [cs]. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  26. [26]

    Computers do not make art, people do

    Aaron Hertzmann. Computers do not make art, people do. Communications of the ACM, 63(5):45–48, 2020. 2

    work page 2020

  27. [27]

    Fleet, Mohammad Norouzi, and Tim Salimans

    Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded Dif- fusion Models for High Fidelity Image Generation, 2021. arXiv:2106.15282 [cs]. 4

    Pith/arXiv arXiv 2021

  28. [28]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video Dif- fusion Models, 2022. arXiv:2204.03458 [cs]. 4

    Pith/arXiv arXiv 2022

  29. [29]

    Artificial Intelli- gence, Artists, and Art: Attitudes Toward Artwork Produced by Humans vs

    Joo-Wha Hong and Nathaniel Ming Curran. Artificial Intelli- gence, Artists, and Art: Attitudes Toward Artwork Produced by Humans vs. Artificial Intelligence.ACM Transactions on Multimedia Computing, Communications, and Applications, 15(2s):1–16, 2019. 2

    work page 2019

  30. [30]

    Blaine Horton Jr, Michael W

    C. Blaine Horton Jr, Michael W. White, and Sheena S. Iyen- gar. Bias against AI art can enhance perceptions of human creativity.Scientific Reports, 13(1):19001, 2023. 2

    work page 2023

  31. [31]

    VBench: Com- prehensive Benchmark Suite for Video Generative Models,

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive Benchmark Suite for Video Generative Models,

  32. [32]

    Cross-modal associations between paintings and sounds: Ef- fects of embodiment.Perception, 51(12):871–888, 2022

    Marina Iosifyan, Anton Sidoroff-Dorso, and Judith Wolfe. Cross-modal associations between paintings and sounds: Ef- fects of embodiment.Perception, 51(12):871–888, 2022. 2

    work page 2022

  33. [33]

    Kaiber AI: Generating Videos with Superstu- dio, 2025

    Kaiber Corp. Kaiber AI: Generating Videos with Superstu- dio, 2025. 2

    work page 2025

  34. [34]

    Artificial Intelligence and Copyright: Le- gal Quandary in the Digital Age: Some Musings.SSRN Elec- tronic Journal, 2021

    Jerameel Kevins. Artificial Intelligence and Copyright: Le- gal Quandary in the Digital Age: Some Musings.SSRN Elec- tronic Journal, 2021. Publisher: Elsevier BV . 2

    work page 2021

  35. [35]

    ”AI enhances our performance, I have no doubt this one will do the same”: The Placebo effect is ro- bust to negative descriptions of AI

    Agnes Mercedes Kloft, Robin Welsch, Thomas Kosch, and Steeven Villa. ”AI enhances our performance, I have no doubt this one will do the same”: The Placebo effect is ro- bust to negative descriptions of AI. InProceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–24, Honolulu HI USA, 2024. ACM. 8

    work page 2024

  36. [36]

    The (R)evolution of Music Video in American Music Industry.New Horizons in English Stud- ies, 8:163–176, 2023

    Lidia Knia ´z-Hunek. The (R)evolution of Music Video in American Music Industry.New Horizons in English Stud- ies, 8:163–176, 2023. 2

    work page 2023

  37. [37]

    The Placebo Effect of Artificial Intelli- gence in Human–Computer Interaction.ACM Transactions on Computer-Human Interaction, 29(6):1–32, 2022

    Thomas Kosch, Robin Welsch, Lewis Chuang, and Al- brecht Schmidt. The Placebo Effect of Artificial Intelli- gence in Human–Computer Interaction.ACM Transactions on Computer-Human Interaction, 29(6):1–32, 2022. 8

    work page 2022

  38. [38]

    Robust One Shot Audio to Video Generation

    Neeraj Kumar, Srishti Goel, Ankur Narang, and Mujtaba Hasan. Robust One Shot Audio to Video Generation. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition Workshops (CVPRW), pages 3334–3343,

    work page 2020

  39. [39]

    Seungpil Lee, Woochang Sim, Donghyeon Shin, Wongyu Seo, Jiwon Park, Seokki Lee, Sanha Hwang, Sejin Kim, and Sundong Kim. Reasoning Abilities of Large Language Mod- els: In-Depth Analysis on the Abstraction and Reasoning Corpus.ACM Transactions on Intelligent Systems and Tech- nology, page 3712701, 2025. 4

    work page 2025

  40. [40]

    Lima, Carlos G

    Hugo B. Lima, Carlos G. R. Dos Santos, and Bianchi S. Meiguins. A Survey of Music Visualization Techniques. ACM Computing Surveys, 54(7):1–29, 2022. 2

    work page 2022

  41. [41]

    In ai we trust? effects of agency locus and transparency on uncertainty reduction in human–ai interac- tion.Journal of Computer-Mediated Communication, 26(6): 384–402, 2021

    Bingjie Liu. In ai we trust? effects of agency locus and transparency on uncertainty reduction in human–ai interac- tion.Journal of Computer-Mediated Communication, 26(6): 384–402, 2021. 8

    work page 2021

  42. [42]

    Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation

    Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation. InProceedings of the 61st An- nual Meeting of the Association for Computational Linguis- tics (Volume 1: Long Pa...

    work page 2023

  43. [43]

    Are Emergent Abilities in Large Language Models just In-Context Learning?, 2024

    Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tay- yar Madabushi, and Iryna Gurevych. Are Emergent Abilities in Large Language Models just In-Context Learning?, 2024. arXiv:2309.01809 [cs]. 4

    Pith/arXiv arXiv 2024

  44. [44]

    Art, Creativity, and the Potential of Artificial Intelligence.Arts, 8(1):26, 2019

    Marian Mazzone and Ahmed Elgammal. Art, Creativity, and the Potential of Artificial Intelligence.Arts, 8(1):26, 2019. Publisher: MDPI AG. 2

    work page 2019

  45. [45]

    Windows Media Player, 2025

    Microsoft Corporation. Windows Media Player, 2025. 2

    work page 2025

  46. [46]

    Com- putational Music Structure Analysis (Dagstuhl Seminar 16092)

    Meinard M ¨uller, Elaine Chew, and Juan Pablo Bello. Com- putational Music Structure Analysis (Dagstuhl Seminar 16092). Technical report, Schloss Dagstuhl – Leibniz- Zentrum f ¨ur Informatik, 2016. Artwork Size: 44 pages, 1005371 bytes ISSN: 2192-5283 Issue: 2 Medium: applica- tion/pdf Publication Title: Dagstuhl Reports (DagRep) V ol- ume: 6. 5

    work page 2016

  47. [47]

    AI Music Video Generator, 2025

    Neuralframes. AI Music Video Generator, 2025. 3

    work page 2025

  48. [48]

    Hello GPT-4o by OpenAI, 2025

    OpenAI. Hello GPT-4o by OpenAI, 2025. 3

    work page 2025

  49. [49]

    Synaesthesia.European Neurology, 57(2):120– 124, 2007

    JMS Pearce. Synaesthesia.European Neurology, 57(2):120– 124, 2007. 2 10

    work page 2007

  50. [50]

    Robust Speech Recognition via Large-Scale Weak Supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever. Robust Speech Recognition via Large-Scale Weak Supervision,

    work page

  51. [51]

    Self-supervised Dance Video Synthesis Conditioned on Mu- sic

    Xuanchi Ren, Haoran Li, Zijian Huang, and Qifeng Chen. Self-supervised Dance Video Synthesis Conditioned on Mu- sic. InProceedings of the 28th ACM International Con- ference on Multimedia, pages 46–54, New York, NY , USA,

    work page

  52. [52]

    High-Resolution Image Synthesis With Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis With Latent Diffusion Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 3

    work page 2022

  53. [53]

    Oxford University PressNew York, NY , 1995

    David C Rubin.Memory In Oral Traditions: The Cogni- tive Psychology of Epic, Ballads, and Counting-out Rhymes. Oxford University PressNew York, NY , 1995. 2

    work page 1995

  54. [54]

    Machine Learning Processes As Sources of Ambiguity: Insights from AI Art

    Christian Sivertsen, Guido Salimbeni, Anders Sundnes Løvlie, Steven David Benford, and Jichen Zhu. Machine Learning Processes As Sources of Ambiguity: Insights from AI Art. InProceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–14, Honolulu HI USA, 2024. ACM. 2

    work page 2024

  55. [55]

    Mochi 1 by Genmo Team, 2024

    Genmo Team. Mochi 1 by Genmo Team, 2024. https://huggingface.co/genmo/mochi-1-preview. 4

    work page 2024

  56. [56]

    Revid.ai, 2025

    TMAKER. Revid.ai, 2025. 3

    work page 2025

  57. [57]

    Specterr: Music Video Maker Online, 2025

    Tunebat LLC. Specterr: Music Video Maker Online, 2025. 2

    work page 2025

  58. [58]

    Wan: Open and Advanced Large-Scale Video Generative Models, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fan...

    Pith/arXiv arXiv 2025

  59. [59]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, 2023. arXiv:2201.11903 [cs]. 4

    Pith/arXiv arXiv 2023

  60. [60]

    A Survey on Knowledge Distillation of Large Language Models, 2024

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A Survey on Knowledge Distillation of Large Language Models, 2024. arXiv:2402.13116 [cs]. 4

    Pith/arXiv arXiv 2024

  61. [61]

    Wordcraft: Story Writing With Large Language Models

    Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces, pages 841–852, Helsinki Finland, 2022. ACM. 4

    work page 2022

  62. [62]

    Let’s Play Music: Audio-Driven Performance Video Generation

    Hao Zhu, Yi Li, Feixia Zhu, Aihua Zheng, and Ran He. Let’s Play Music: Audio-Driven Performance Video Generation. In2020 25th International Conference on Pattern Recogni- tion (ICPR), pages 3574–3581, 2021. ISSN: 1051-4651. 3 11 From Sound to Sight: Towards AI-authored Music Videos Supplementary Material

    work page 2021

  63. [66]

    SCENE #:

    Prompt Flow 6.1. CLAP-based Approach In the following, we present the prompt structure for the CLAP-based approach in pseudocode in Block 1, an exam- ple prompt in Block 2, and the corresponding response in Block 3. Block 1. Prompt structure for the LLM used in the CLAP-based approach number_of_scenes = length(segments) final_prompt = "You need to think o...

    work page

  64. [67]

    All items were rated on a 7-point Likert scale: where 1 = Strongly Disagree, and 7 = Strongly Agree

    Exploratory Generative Pipeline Evaluation: Survey Items In the following, we present the survey items used for the preliminary evaluation described in subsection 4.1. All items were rated on a 7-point Likert scale: where 1 = Strongly Disagree, and 7 = Strongly Agree. Storytelling 1.The video featured consistent characters. 2.I felt connected to at least ...

    work page

  65. [2020]

    Association for Computing Machinery. 3

  66. [2022]

    arXiv:2212.04356 [eess]. 3

    Pith/arXiv arXiv

  67. [2023]

    arXiv:2311.17982 [cs]. 4

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.