Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning a small open multimodal model on synthetic sheet music with a compact symbolic note format yields up to eight times the GPT-evaluated accuracy of GPT-4o baselines on OMR and six times on chord estimation.

desk verdict Worth a serious look, but the headline claims about music-sheet understanding rest on synthetic-only evidence and a dataset that doesn't add up in the text. read the letter →

arxiv 2506.23009 v3 pith:TIQXH57A submitted 2025-06-28 cs.CV

Jian Chen , Wenye Ma , Penghang Liu , Wei Wang , Tengwei Song , Ming Li , Chenguang Wang , Jiayu Qin
show 2 more authors
Ruiyi Zhang Changyou Chen
This is my paper
classification cs.CV
keywords multimodallargelanguagemodelssheetmusicunderstandingopticalrecognitionvisualquestionansweringsyntheticdatagenerationparameter-efficientfine-tuningoutputrepresentationdesignchordestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a small open multimodal language model can learn to read sheet music when trained on a large, synthetically generated question-answering dataset, and that the text format used for musical notes is decisive. The authors build MusiXQA, a benchmark of MusiXTeX-rendered sheets with ground-truth annotations for note pitch and duration, chords, clefs, key and time signatures, titles, and layout, and show that current frontier models such as GPT-4o perform near randomly on OMR and chord estimation. They then fine-tune Phi-3-Vision with LoRA to produce Phi-3-MusiX; using a compact symbolic representation called kern+, it reaches 68.4 GPT-accuracy on OMR and 84.9 on chord estimation, roughly eight times and six times the strongest GPT-4o baseline respectively. The same model trained to emit verbose JSON does much worse, supporting the claim that representation format, not just scale or supervision, drives structured prediction quality.

What carries the argument

The central machinery is a closed loop of synthetic generation and compact symbolic output. MusiXTeX renders randomly sampled, theory-constrained configurations (clefs, key and time signatures, tempo, chord labels, 10 to 20 bars) into high-resolution images, and the QA templates convert ground truth from the LaTeX source directly into questions. On the output side, kern+, a compact extension of the **kern notation where a note is written like qC4 (duration symbol followed by pitch name with octave), carries the argument by making content tokens dominate format tokens. The paper attributes the JSON model's premature convergence and near-random chord performance to format-token dominance.

What would settle it

Run the released Phi-3-MusiX model on the same OMR and chord QA templates over a held-out set of real published sheet music, both engraved and scanned, that was not generated by MusiXTeX; if its GPT-evaluated accuracy on OMR falls to near the single-digit level of the zero-shot GPT-4o baseline rather than staying above 60, the central transfer claim is falsified.

Watch

Extended reading notes

Core claim

MusiXQA is offered as the first large-scale, balanced, synthetic benchmark for visual music-sheet understanding, and Phi-3-MusiX as the first MLLM specifically fine-tuned for it. The paper's central discovery is that a compact, content-only symbolic encoding of notes is what unlocks accurate structured recognition: with kern+, the fine-tuned model reaches 68.4 GPT-evaluated accuracy on OMR and 84.9 on chord estimation, versus 9.2 and 19.6 for the same model trained to output JSON, and versus 8.4 and 13.0 for the best GPT-4o pipeline that even receives oracle OMR text. The explanation offered is token efficiency: JSON spends most of its token budget on punctuation and keys, so the model learns format before content, while kern+ forces the loss to concentrate on pitches and durations.

Load-bearing premise

The headline gains are measured only on MusiXTeX sheets whose musical content was created by the paper's own chord-based heuristics, so the whole result assumes those synthetic sheets stand in for real sheet music; the Limitation section concedes they are not real compositions.

Editorial extensions

If this is right

  • If the gains hold, a roughly 4-billion-parameter open model fine-tuned on synthetic data can outperform much larger proprietary models on note-level sheet-music extraction, making specialized OCR pipelines unnecessary for clean typeset sheets.
  • A compact, content-centric output tokenization should be preferred over verbose structured formats for any MLLM task whose correctness hinges on a few precise tokens, such as tables, timestamps, or chemical formulas.
  • Current GPT-4o-style baselines, even with retrieval and an external OMR module, mostly mirror answer formats rather than recognize symbols, so benchmark scores need semantic metrics like G-Acc, not just string similarity.
  • Because the training signal is synthetic and template-based, the same pipeline can be scaled to other notation systems, such as guitar tablature, percussion, or historical notations, without manual annotation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the representation-format result generalizes, it suggests that MLLM structured-output failures often diagnosed as reasoning errors may actually be tokenization errors: JSON-like formats let the model learn braces and keys without learning content.
  • A natural next test is to measure Phi-3-MusiX on real engraved, scanned, or handwritten sheets; the paper does not do this, and the synthetic-to-real gap is the main open question.
  • The chord-estimation result hints that the model may be learning harmonic inference rather than only symbol reading; one could probe this by holding out chord types or transposing keys and checking whether accuracy transfers.
  • The manuscript contains a numeric inconsistency: the abstract reports 9,600 sheets while Section 3.1 reports 96,000; this does not change the mechanism but should be resolved before relying on the scale claims.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MusiXQA, a large synthetic dataset of music sheet images rendered with MusiXTeX, together with template-based QA pairs covering OCR, layout understanding, OMR, and chord estimation. The authors benchmark several open and proprietary MLLMs, report that GPT-4o and related baselines perform poorly on the OMR and chord tasks, and propose Phi-3-MusiX, a LoRA fine-tuned Phi-3-Vision model. They report large improvements over the GPT-4o baselines, especially when notes are represented in the proposed kern+ format rather than JSON, and they attribute the gain partly to the compactness of the output representation. The paper also includes a limitation section acknowledging that the music is generated by chord-based heuristics rather than real compositions.

Significance. If the claims hold, the paper would provide a useful synthetic resource for studying music sheet understanding in MLLMs, as well as an interesting empirical finding about output-token format in structured prediction. Several strengths deserve credit: the annotation pipeline derives ground truth directly from the MusiXTeX source, avoiding post-processing errors; the generation framework supports controlled sampling across clefs, keys, time signatures, tempo, and layout variations; the comparison between JSON and kern+ is a concrete, reproducible experimental design; and the paper honestly discloses the synthetic nature of the data in Section 6. The central limitation is that all reported gains are measured in-distribution on the same synthetic pipeline used for training, so the external validity of the headline 'music sheet understanding' claim is untested. The dataset-statistics contradictions also prevent verification of the benchmark scale.

major comments (4)
  1. [§6, §3.3, §5] The central claim that Phi-3-MusiX learns 'music sheet understanding' is supported only by evaluations on synthetic MusiXTeX sheets generated with the same chord-based heuristics used for training. Section 6 concedes that the music is generated via heuristics rather than real compositions, and no evaluation is reported on existing real-sheet benchmarks such as PrIMuS, DeepScores, Camera-PrIMuS, or MUSCIMA++, nor on a held-out set of real engraved scores. As it stands, the 8x/6x G-Acc gains in Table 1 may reflect in-distribution overfitting to the synthetic rendering and annotation style rather than transferable music-reading ability. I ask the authors to either (a) add a real-sheet evaluation or a careful synthetic-to-real transfer experiment, or (b) substantially soften the 'music sheet understanding' claim to 'synthetic music sheet understanding' throughout the abstract and conclusion.
  2. [§1 vs. §3.1] The dataset size is reported inconsistently: Section 1 states 9,600 sheets and 'over 130,000' QA pairs, while Section 3.1 reports 96k sheets, 1.3M bars, 11.7M notes, and 670k + 337k + 288k + 47k = 1.342M QA pairs. This is a factor-of-ten discrepancy in sheets and roughly a factor-of-ten discrepancy in QA pairs. Because the paper's contribution is explicitly the dataset and because the training split of the model is drawn from this dataset, the reader cannot determine which numbers are correct. Please correct the abstract/introduction or the statistics section, and specify whether the 90/10 split in Section 4.1 applies to sheets, QA pairs, or both.
  3. [Table 1 caption, §4.1, §4.5] The Table 1 caption refers to 'GPT-4o + RAG + OMR' as using 'oracle OMR results,' but Section 4.1 states that the OMR output comes from the Oemer model, and Section 4.5 explicitly notes that Oemer is lossy and omits accidentals. Calling a lossy model's output 'oracle' is misleading and inflates the apparent strength of the baseline that Phi-3-MusiX is compared against. If an oracle OMR baseline is intended, it should use ground-truth annotations from the MusiXTeX source; otherwise the caption and the main-text description should consistently say 'OMR-augmented' rather than 'oracle.'
  4. [§4.4, §5] The fine-tuned model is trained and evaluated on the same MusiXQA distribution, so the reported improvements are closed-loop with respect to that distribution. The zero-shot GPT-4o baselines provide some independent grounding that the task is not trivially easy, but they do not establish that the learned representation generalizes beyond the specific synthetic annotation and rendering style. I recommend that the authors report at least one cross-dataset or human-annotated real-score condition, or explicitly describe the contribution as a synthetic benchmark with in-distribution gains, rather than as general visual music understanding.
minor comments (6)
  1. [Title] The title contains a typo: 'ADV ANCING' should be 'ADVANCING.'
  2. [Figure 3] In Figure 3, the label 'Temp o' should be 'Tempo,' and the example for the chord label shows inconsistent quoting ('Bb chord' with straight and curly quotes) that should be normalized.
  3. [§4.2] The description of the JSON representation says an underscore marks 'the start of a slur' and a dot indicates dotted rhythms, but Figure 4 uses an underscore after a duration and a dot after a note value; please clarify the exact token semantics in one consistent place.
  4. [§4.3] The GPT-based evaluation uses GPT-4o as a binary judge, but the paper does not report human agreement or a small validation of the judge on a sample. Given that the same model family is also used as a baseline, a brief sanity-check of the judge would strengthen the metric.
  5. [§4.1, Table 1] The table header lists 'PNLS' for all tasks, and the text says PNLS is computed between model answers and ground truths, but the exact normalization and partial alignment procedure is only referenced to [43]. A one-sentence definition or formula would make the metric self-contained.
  6. [§3.1] The phrase '1.3 million bars / measures' is inconsistent with the reported 10–20 bars per sheet and 96k sheets (which would give 0.96M–1.92M bars, depending on the count convention); please reconcile the statistics and clarify whether repeats or different clefs are counted separately.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline result is a standard held-out supervised evaluation, and the synthetic-data concern is an external-validity limitation rather than a circular derivation.

full rationale

The paper's central claim is that Phi-3-MusiX, fine-tuned on a 90% split of the MusiXQA dataset, outperforms GPT-4o baselines on the held-out 10% test split. This is a conventional supervised-learning evaluation: the test images and their labels are not used during training, so the reported gains are genuine out-of-sample predictions within the benchmark. The fact that both training and test sheets are generated by the same MusiXTeX pipeline and the same chord-based heuristics is a limitation on external validity, not circularity; the paper itself concedes this in Section 6: 'a current limitation of the dataset is that the music is generated using chord-based heuristics rather than real musical compositions.' No fitted parameter is renamed as a prediction, no quantity is defined in terms of the quantity it is claimed to predict, and no uniqueness or ansatz is imported from the authors' prior work to force the conclusion. The PNLS metric and the observation that output format matters cite prior work by overlapping authors ([43], [47]), but these citations are not load-bearing derivational premises: the metric is defined and the format comparison is demonstrated by the paper's own Table 1 and Figure 5. The discrepancy between the 9,600-sheet / 130k-QA counts in Section 1 and the 96k-sheet / 1.34M-QA counts in Section 3.1 is a reporting-consistency concern, not a circularity. The benchmark includes external baselines (GPT-4o, Paligemma2, Phi-3-V) that provide independent grounding for task difficulty, and the central result does not reduce to any of the paper's own inputs by construction. Therefore no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central results rest on several hand-chosen generation parameters, the assumption that synthetic MusiXTeX output exactly matches ground truth, the use of GPT-4o as semantic judge, and the untested representativeness of synthetic sheets. No new physical entities are introduced; kern+ is a text encoding, not an invented entity.

free parameters (4)
  • Bar count per sheet = 10 to 20
    Sampled uniformly in Section 3.4; controls layout complexity and the test distribution.
  • Notes per bar multiplier = 1 to 3 times beat count
    Handpicked in Section 3.3; controls note density and OMR difficulty.
  • Tempo range = 50 to 140 BPM
    Sampled in Section 3.2; affects OCR tasks but not musical difficulty.
  • Chord label and bar index annotation probability = 50%
    Chosen in Section 3.4; determines how many sheets have explicit chord labels, directly shaping the chord estimation task distribution.
assumptions (4)
  • domain assumption Structurally valid random notation is sufficient for music sheet QA; musical coherence is not required.
    Section 3 states this principle and uses it to justify generating notes heuristically rather than using real compositions.
  • domain assumption MusiXTeX source exactly determines the rendered image and hence the ground truth answers.
    Section 3.5 claims annotations are extracted directly from MusiXTeX to avoid post-processing errors, assuming the typesetter renders source without ambiguity.
  • domain assumption GPT-4o is a reliable binary judge of semantic correctness for music QA answers.
    Section 4.3 uses GPT-4o-based G-Acc without human agreement or calibration against ground truth.
  • domain assumption Synthetic sheets are representative of the general domain of music sheet understanding.
    This is the external validity assumption; acknowledged only in Section 6 as a limitation, but the conclusions in Section 5 are stated without this caveat.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/TIQXH57A

@misc{pith2026250623009,
  author       = {Pith},
  title        = {Pith review of: MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TIQXH57A}},
  note         = {Machine review of arXiv:2506.23009}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable visual reasoning abilities in natural images, text-rich documents, and graphic designs. However, their ability to interpret music sheets remains underexplored. To bridge this gap, we introduce MusiXQA, the first comprehensive dataset for evaluating and advancing MLLMs in music sheet understanding. MusiXQA features high-quality synthetic music sheets generated via MusiXTeX, with structured annotations covering note pitch and duration, chords, clefs, key/time signatures, and text, enabling diverse visual QA tasks. Through extensive evaluations, we reveal significant limitations of current state-of-the-art MLLMs in this domain. Beyond benchmarking, we developed Phi-3-MusiX, an MLLM fine-tuned on our dataset, achieving significant performance gains over GPT-based methods. The proposed dataset and model establish a foundation for future advances in MLLMs for music sheet understanding. Code, data, and model will be released upon acceptance.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LEGATO 2: Toward Multimodal Sheet Music Recognition and Understanding

    cs.CV 2026-07 conditional novelty 6.5 of 10

    System-by-system autoregressive OMR with text-aware ABC transcription outperforms prior neural and rule-based systems and boosts VLM sheet-music QA.

  2. Music I Care About: Automated Multimodal Benchmarking of LLM Music Perception Skills on (Almost) Any Music

    cs.SD 2026-07 unverdicted novelty 6.0 of 10

    A meta-benchmark that auto-generates multimodal music-perception multiple-choice tests from user symbolic music, demonstrated on ChoraleBricks with text-only and white-noise controls.

Reference graph

Works this paper leans on

49 extracted references · 30 canonical work pages · cited by 2 Pith papers

  1. [1]

    Acrobat AI Assistant, 2024

    Adobe Inc. Acrobat AI Assistant, 2024. https: //www.adobe.com/acrobat/generative-ai-pdf. html

    work page 2024

  2. [2]

    Mmmu: A mas- sive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A mas- sive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567, 2024

    2024

  3. [3]

    The basics of reading music

    Kevin Meixner. The basics of reading music. url: http://readsheetmusic. info/readingmusic. shtml, 2015

    work page 2015

  4. [4]

    Reading sheet music facilitates sensorimotor mu- desynchronization in musicians

    Lawrence Paul Behmer Jr and Kelly J Jantzen. Reading sheet music facilitates sensorimotor mu- desynchronization in musicians. Clinical Neurophysi- ology, 122(7):1342–1347, 2011

    work page 2011

  5. [5]

    Optical Music Recognition: State of the Art and Major Challenges

    Elona Shatri and György Fazekas. Optical music recognition: State of the art and major challenges. arXiv preprint arXiv:2006.07885, 2020

    work page Pith review arXiv 2006

  6. [6]

    Understanding optical music recognition.ACM Computing Surveys (CSUR), 53(4):1–35, 2020

    Jorge Calvo-Zaragoza, Jan Haji ˇc Jr, and Alexander Pacha. Understanding optical music recognition.ACM Computing Surveys (CSUR), 53(4):1–35, 2020

    work page 2020

  7. [7]

    Optical music recognition: state-of-the-art and open issues

    Ana Rebelo, Ichiro Fujinaga, Filipe Paszkiewicz, An- dre RS Marcal, Carlos Guedes, and Jaime S Cardoso. Optical music recognition: state-of-the-art and open issues. International Journal of Multimedia Informa- tion Retrieval, 1:173–190, 2012

    work page 2012

  8. [8]

    The challenge of opti- cal music recognition

    David Bainbridge and Tim Bell. The challenge of opti- cal music recognition. Computers and the Humanities, 35:95–121, 2001

    work page 2001

Show all 49 references
  1. [9]

    Optical music recognition using pro- jections

    Ichiro Fujinaga. Optical music recognition using pro- jections. 1988

    1988

  2. [10]

    Gui agents: A survey

    Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namy- ong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, et al. Gui agents: A survey. arXiv preprint arXiv:2412.13501, 2024. 8

    2024

  3. [11]

    Natural language understand- ing and inference with mllm in visual question answer- ing: A survey

    Jiayi Kuang, Ying Shen, Jingyou Xie, Haohao Luo, Zhe Xu, Ronghao Li, Yinghui Li, Xianfeng Cheng, Xika Lin, and Yu Han. Natural language understand- ing and inference with mllm in visual question answer- ing: A survey. ACM Computing Surveys, 2024

    2024

  4. [12]

    Internvl: Scal- ing up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scal- ing up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer v...

    2024

  5. [13]

    Mllm-tool: A multimodal large language model for tool agent learning

    Chenyu Wang, Weixin Luo, Qianyu Chen, Haonan Mai, Jindi Guo, Sixun Dong, Zhengxin Li, Lin Ma, Shenghua Gao, et al. Mllm-tool: A multimodal large language model for tool agent learning. arXiv preprint arXiv:2401.10727, 2024

    2024 arXiv

  6. [14]

    Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision- language benchmark

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision- language benchmark. In Forty-first International Con- ference on Machine Learning, 2024

    2024

  7. [15]

    Pp-ocr: A practi- cal ultra lightweight ocr system

    Yuning Du, Chenxia Li, Ruoyu Guo, Xiaoting Yin, Weiwei Liu, Jun Zhou, Yifan Bai, Zilin Yu, Yehua Yang, Qingqing Dang, et al. Pp-ocr: A practi- cal ultra lightweight ocr system. arXiv preprint arXiv:2009.09941, 2020

    2009 arXiv

  8. [16]

    Tex- tocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

    Amanpreet Singh, Guan Pang, Mandy Toh, Jing Huang, Wojciech Galuba, and Tal Hassner. Tex- tocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8802–8812, 2021

    2021

  9. [17]

    CVC-MUSCIMA: A ground-truth of hand- written music score images for writer identification and staff removal

    Alicia Fornés, Anjan Dutta, Albert Gordo, and Josep Lladós. CVC-MUSCIMA: A ground-truth of hand- written music score images for writer identification and staff removal. International Journal on Document Analysis and Recognition, 15(3):243–251, 2012

    2012

  10. [18]

    Knowledge dis- covery in optical music recognition: Enhancing infor- mation retrieval with instance segmentation

    Elona Shatri and George Fazekas. Knowledge dis- covery in optical music recognition: Enhancing infor- mation retrieval with instance segmentation. arXiv preprint arXiv:2408.15002, 2024

    2024 arXiv

  11. [19]

    Deepscores-a dataset for segmentation, detection and classification of tiny objects

    Lukas Tuggener, Ismail Elezi, Jurgen Schmidhuber, Marcello Pelillo, and Thilo Stadelmann. Deepscores-a dataset for segmentation, detection and classification of tiny objects. In 2018 24th International Conference on Pattern Recognition (ICPR) , pages 3704–3709. IEEE, 2018

    2018

  12. [20]

    End-to- end neural optical music recognition of monophonic scores

    Jorge Calvo-Zaragoza and David Rizo. End-to- end neural optical music recognition of monophonic scores. Applied Sciences, 8(4):606, 2018

    2018

  13. [21]

    Doremi: First glance at a universal omr dataset

    Elona Shatri and György Fazekas. Doremi: First glance at a universal omr dataset. arXiv preprint arXiv:2107.07786, 2021

    2021 arXiv

  14. [22]

    A uni- fied representation framework for the evaluation of optical music recognition systems

    Pau Torras, Sanket Biswas, and Alicia Fornés. A uni- fied representation framework for the evaluation of optical music recognition systems. International Jour- nal on Document Analysis and Recognition (IJDAR), 27(3):379–393, 2024

    2024

  15. [23]

    Practical end-to-end optical music recognition for pianoform music

    Jiˇrí Mayer, Milan Straka, Jan Hajiˇc, and Pavel Pecina. Practical end-to-end optical music recognition for pianoform music. In International Conference on Document Analysis and Recognition , pages 55–73. Springer, 2024

    2024

  16. [24]

    Breezewhite/oemer: v0.1.7, October 2023

    Yoyo, Christian Liebhardt, and Sayooj Samuel. Breezewhite/oemer: v0.1.7, October 2023

    2023

  17. [25]

    Optical music recognition in manuscripts from the ricordi archive

    Federico Simonetta, Rishav Mondal, Luca Andrea Ludovico, and Stavros Ntalampiras. Optical music recognition in manuscripts from the ricordi archive. In Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures, AM ’24, page 260–269, New York...

    2024

  18. [26]

    Optical music recognition with convolutional sequence-to-sequence models

    Eelco van der Wel and Karen Ullrich. Optical music recognition with convolutional sequence-to-sequence models. In Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017, Suzhou, China, October 23-27, 2017 , pages 731–737, 2017

    2017

  19. [27]

    Tromr:transformer-based polyphonic optical music recognition

    Yixuan Li, Huaping Liu, Qiang Jin, Miaomiao Cai, and Peng Li. Tromr:transformer-based polyphonic optical music recognition. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023

    2023

  20. [28]

    Sheet music transformer: End-to-end optical music recognition beyond monophonic transcription, 2024

    Antonio Ríos-Vila, Jorge Calvo-Zaragoza, and Thierry Paquet. Sheet music transformer: End-to-end optical music recognition beyond monophonic transcription, 2024

    2024

  21. [29]

    Sheet music trans- former++: End-to-end full-page optical music recog- nition for pianoform sheet music

    Antonio Rıos-Vila, Jorge Calvo-Zaragoza, David Rizo, and Thierry Paquet. Sheet music trans- former++: End-to-end full-page optical music recog- nition for pianoform sheet music. arXiv preprint arXiv:2405.12105, 2024

    2024 arXiv

  22. [30]

    Chatmusician: Under- standing and generating music intrinsically with llm

    Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, et al. Chatmusician: Under- standing and generating music intrinsically with llm. arXiv preprint arXiv:2402.16153, 2024

    2024 arXiv

  23. [31]

    Mu- sicagent: An ai agent for music understanding and generation with large language models

    Dingyao Yu, Kaitao Song, Peiling Lu, Tianyu He, Xu Tan, Wei Ye, Shikun Zhang, and Jiang Bian. Mu- sicagent: An ai agent for music understanding and generation with large language models. arXiv preprint arXiv:2310.11954, 2023. 9

    2023 arXiv

  24. [32]

    Paligemma 2: A family of versatile vlms for transfer

    Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bit- ton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, et al. Paligemma 2: A family of versatile vlms for transfer. arXiv preprint arXiv:2412.03555, 2024

    2024 arXiv

  25. [33]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  26. [34]

    Deepseek- v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek- v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  27. [35]

    Musical scales and the generalized circle of fifths

    John Clough and Gerald Myerson. Musical scales and the generalized circle of fifths. The american mathematical monthly, 93(9):695–701, 1986

    1986

  28. [36]

    MusiXTEX

    Daniel Taupin, Ross Mitchell, and Andreas Egler. MusiXTEX. using TEX to write polyphonic or in- strumental music. EuroTEX, 92:257–272, 1993

    1993

  29. [37]

    Harmonic experience: Tonal harmony from its natural origins to its modern expression

    William Allaudin Mathieu. Harmonic experience: Tonal harmony from its natural origins to its modern expression. Simon and Schuster, 1997

    1997

  30. [38]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

  31. [39]

    Trins: Towards multimodal language models that can read

    Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, and Tong Sun. Trins: Towards multimodal language models that can read. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 22584– 22594, 2024

    2024

  32. [40]

    Llava-read: Enhanc- ing reading ability of multimodal language models

    Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, and Tong Sun. Llava-read: Enhanc- ing reading ability of multimodal language models. arXiv preprint arXiv:2407.19185, 2024

    2024 arXiv

  33. [41]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  34. [42]

    Music information processing using the humdrum toolkit: Concepts, examples, and lessons

    David Huron. Music information processing using the humdrum toolkit: Concepts, examples, and lessons. Computer Music Journal, 26(2):11–26, 2002

    2002

  35. [43]

    MMR: Evaluating reading ability of large multimodal models

    Jian Chen, Ruiyi Zhang, Yufan Zhou, Ryan Rossi, Jiuxiang Gu, and Changyou Chen. MMR: Evaluating reading ability of large multimodal models. arXiv preprint arXiv:2408.14594, 2024

    2024 arXiv

  36. [44]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  37. [45]

    Retrieval-augmented generation for knowledge- intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge- intensive nlp tasks. Advances in neural information processing system...

    2020

  38. [46]

    Layoutgpt: Compositional visual planning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems, 36:18225–18250, 2023

    2023

  39. [47]

    Text- lap: Customizing language models for text-to-layout planning

    Jian Chen, Ruiyi Zhang, Yufan Zhou, Jennifer Healey, Jiuxiang Gu, Zhiqiang Xu, and Changyou Chen. Text- lap: Customizing language models for text-to-layout planning. arXiv preprint arXiv:2410.12844, 2024

    2024 arXiv

  40. [48]

    mplug-docowl 1.5: Unified structure learning for ocr-free document understanding

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403.12895, 2024

    2024 arXiv

  41. [49]

    Information not found

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 67(12):220102, 2024. 10 A Scale Details ♯ / ...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.