Pith. sign in

REVIEW 4 major objections 6 minor 26 references

Proceedings of the 6th International Workshop on Reading Music Systems

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single transformer with task prompts, SMIReT, can handle every sheet-music retrieval task end-to-end on 17th-century prints, with music error near 6 percent and region-classification F1 at 97.

desk verdict A workshop proceedings volume, not a single paper: the YOLO layout-analysis study is solid and reusable, the SMIReT multitask work is promising but under-evidenced, and the rest is a useful state-of-the-field snapshot. read the letter →

arxiv 2411.15741 v1 pith:DFXRLUPI submitted 2024-11-24 cs.CV cs.IRcs.LG

Jorge Calvo-Zaragoza , Alexander Pacha , Elona Shatri (Eds.) This is my paper · ORCID
classification cs.CVcs.IRcs.LG
keywords opticalmusicrecognitionsheetinformationretrievalmultitasktransformerstaskpromptingstafflayoutanalysissemanticreconstructionmensuralnotationscoredatasets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

These proceedings of the 6th International Workshop on Reading Music Systems present a field shifting from single-purpose recognition pipelines toward unified end-to-end models, cheaper layout analysis, and human-in-the-loop tools for historical collections. The strongest claim is that one autoregressive transformer, SMIReT, can perform every task in a newly proposed Sheet Music Information Retrieval challenge on the MOTTECTA corpus of 297 printed 17th-century mensural pages: full-page parsing, music-only and text-only recognition, region detection and classification, partial transcription of user-specified regions, and pattern-search queries. The reported numbers are music symbol error rates of 6.05 for full parsing and 5.92 for music-only recognition, text error of 10.08, region-classification F1 of 97.00, and pattern-match accuracy of 73.80, with the weak spots being spatial localization (intersection-over-union 70.23) and selective transcription (error rate 41.55). A second load-bearing result is that a YOLOv8m detector matches or slightly beats the previous Faster R-CNN measure detector on staff layout analysis while running more than 20 times faster and using more than 4 times less memory, which would make layout indexing cheap enough for library-scale digitization. A sympathetic reader would care because these results, if they hold, mean one model could replace several task-specific services and that efficient multitask reading of historical music is no longer out of reach.

What carries the argument

The carrying mechanism is task prompting over a unified multimodal vocabulary inside an autoregressive transformer: a short sequence of prompt tokens is prepended to the decoder input, redirecting the same encoder features and the same vocabulary toward parsing, layout analysis, OCR, selective transcription, or pattern queries. The vocabulary is deliberately multimodal, binding music symbols in an agnostic shape-plus-position encoding, text characters, absolute-position bounding-box tokens in the style of the Pix2Seq detection procedure cited in the paper, and special tokens for region categories. Curriculum learning carries the model from the synthetic PRIMENS incipit collection to the real MOTTECTA corpus, feeding pages with progressively more staves and text before interleaving synthetic and target samples; this transfer step is the load-bearing premise of the multitask claim. For the volume's layout-analysis result, the equivalent mechanism is a single-stage YOLOv8m detector that replaces the two-stage region-proposal pipeline and avoids the confidence collapse the older model shows when a grand staff coincides with a full system.

What would settle it

Train SMIReT on the same MOTTECTA fine-tuning schedule from random initialization, omitting the synthetic-pretraining curriculum, and compare music symbol error rate, region intersection-over-union, and pattern-match accuracy on the held-out test pages: if those numbers stay near 6, 70, and 74, the transfer premise is not load-bearing and the multitask claim must be re-attributed; if they collapse, the premise is confirmed. A second check sits ready inside the graph-neural-network paper: its candidate graphs contain only 80-91 percent of ground-truth edges for two of its datasets, so no recall figure from that pipeline can exceed those fractions no matter how the edge classifier is tuned.

Watch

Extended reading notes

Core claim

In the authors' own framing, the central discovery is that task unification for music documents is feasible: SMIReT adapts the Sheet Music Transformer, an autoregressive encoder-decoder that generates one output token at a time conditioned on a convolutional feature map of the page and on all tokens generated so far, by prepending a prompt-token sequence to the decoder input. With task prompts, a single unified vocabulary covering music symbols (encoded agnostically as shape plus staff position), text characters, bounding boxes written as absolute position tokens, and region-category tokens, plus a curriculum that moves from synthetic images of mensural incipits to real 17th-century pages, the same weights produce full parsing, OMR, OCR, layout recognition, selective OMR, and pattern-matching queries; the authors state that the model learns all of them successfully with acceptable performance. They read the uneven error profile as a feasibility result with clear next targets: text recognition degrades by 51.78 percent when mixed with music, localization lags classification, and the hardest task is the one where the user points at pixels and asks for a partial transcription. Around this result the volume documents adjacent findings: a graph-neural-network pipeline reconstructs music-notation graphs but is seed-sensitive and its candidate graphs already cap recall, a tree-based evaluation format with tiered metrics is proposed so that systems can be compared fairly, general multimodal language models can identify tonality and texture from score images but cannot yet transcribe them, and the YOLO layout result makes cheap page analysis practical.

Load-bearing premise

The entire multitask result rests on one unisolated premise: that curriculum pretraining on synthetic images of early mensural notation transfers enough to real 17th-century pages for a single vocabulary and a few prompt tokens to cover all six task families; the paper asserts this curriculum in words only, reports no ablation isolating it, and its most visible weak point is the 41.55 error rate of the selective-transcription task, where the premise must do the most work.

Editorial extensions

If this is right

  • A single SMIReT-style model could replace separate OMR, OCR, and layout services in a digitization workflow, removing the compute and maintenance overhead the paper identifies as a practical obstacle.
  • Adding a new sheet-music reading task would need no new architecture: a new prompt token and matching training data extend the same model, exactly the mechanism used for the six task families evaluated here.
  • With YOLO-class layout detection running at 0.83 seconds per page on a CPU, staff and system indexing of million-page collections becomes feasible as a cheap gate before expensive full-page recognition is invoked.
  • Comparisons between OMR systems become meaningful if the field adopts a shared representation like the proposed Music Tree Notation with its tree-edit-distance metric, replacing per-methodology evaluation.
  • The failure modes the papers identify, especially selective transcription at 41.55 error rate, put a concrete target on attention mechanisms that can ground instructions in specific regions of the score image.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper describes the curriculum verbally and reports no ablation that isolates the synthetic pretraining; a natural experiment is to train SMIReT on the target corpus alone. If the synthetic pretraining is truly load-bearing, that version should be clearly worse; if it is not, the multitask credit belongs to the unified vocabulary and fine-tuning schedule rather than the curriculum.
  • The contrast between pattern-match queries (73.80 accuracy) and selective OMR (41.55 error rate) is revealing because both consume bounding-box information; the difference suggests the bottleneck is instruction-following under dense, per-symbol guidance rather than localization itself, which could be tested by varying how many regions a query specifies.
  • The out-of-domain asymmetry in the layout paper, where YOLO handles grand staffs well but staffs and measures poorly on handwritten pages, points to a cheap extension: adding synthetic handwritten staff lines to the training mix would show whether the gap is data or architecture.
  • The GNN paper already computes the ceiling for its own approach, with candidate graphs containing only 80-91 percent of ground-truth edges for two datasets; folding music-grammar rules into candidate-graph construction, as the authors suggest, is the directly testable route past that cap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This submission is the proceedings of the 6th International Workshop on Reading Music Systems (WoRMS 2024), consisting of a set of short contributed papers on optical music recognition and music-reading systems. The topics range from an exploratory study of multimodal LLMs on score images, to a multitask transformer for sheet-music information retrieval (SMIReT), a graph-neural-network approach to semantic reconstruction, a YOLO-based staff layout detector, a tree-based evaluation representation (MTN), and several system and tool reports. The most defensible archival contribution is the YOLO paper, which harmonizes a 7,013-image corpus, evaluates on one in-domain and three out-of-domain settings, and releases models and scripts; it demonstrates comparable or better accuracy than Faster R-CNN at roughly 26x lower latency and 4.4x lower memory. The most ambitious claim, that SMIReT can perform all proposed SMIR tasks successfully, is made on the basis of a single table of results without single-task baselines or an ablation of the synthetic pretraining stage.

Significance. If the stronger claims hold, the volume contributes practically useful resources to the OMR community. The YOLO paper stands out because it ships harmonized datasets, converted annotation scripts, trained models, and a reproducible evaluation protocol, which is exactly the kind of infrastructure the field needs; its finding that modern one-stage detection is substantially cheaper and often more accurate for staff layout is directly actionable. The MTN paper is also valuable as a community-level proposal for a common evaluation representation, and its public dataset and toolkit are a concrete step away from per-methodology metrics. The SMIReT paper addresses a timely question, namely whether an end-to-end multitask transformer can unify parsing, layout, and query tasks, but the current evidence is preliminary: the absence of baselines and ablations prevents the reader from attributing the reported numbers to the proposed multitask mechanism. Several other contributions are explicitly preliminary or experience reports; they are useful as workshop documentation but not as completed research claims.

major comments (4)
  1. [SMIReT paper, Section IV-B, Table I] The central claim that SMIReT can perform all SMIR tasks successfully rests on seven metrics reported for a single trained model, with no single-task baseline. In particular, the OMR SER of 5.92 cannot be attributed to the multitask prompting and unified vocabulary unless it is compared with a Sheet Music Transformer trained on the same MOTTECTA split with the same curriculum but without the added tasks and prompts. Without such a baseline, the 51.78% relative CER increase in full parsing (10.08 to 15.30) is also uninterpretable as an effect of task interference. Please add at least an SMT-only OMR baseline and, ideally, a sequential fine-tuning baseline.
  2. [SMIReT paper, Section III-C and Section IV-A] The load-bearing premise that curriculum pretraining on synthetically rendered PRIMENS incipits transfers to the 297-page MOTTECTA corpus is untested: the training procedure is described only verbally, and no ablation removes the PRIMENS stage. If the synthetic stage is essential, the paper should demonstrate this experimentally; if it is not, the curriculum-learning narrative should be removed or substantially weakened. As written, the claim that the approach is made viable by synthetic pretraining goes beyond the evidence presented.
  3. [MLLM paper, Section II-B and Section III, Table I] The empirical basis for the paper's title question is three score crops selected for simplicity and cultural familiarity, with outputs scored by the authors on a subjective three-level scale and prompts generated with the assistance of the models themselves. This design cannot support general conclusions about MLLM capability for music score reading; it is an anecdotal pilot. The authors should either enlarge the sample with a stratified selection and report quantitative agreement, or explicitly frame all conclusions as observations on three specific images.
  4. [GNN paper, Section VI] The paper's own data show that the k=20 candidate graph contains only 80% of ground-truth edges for the MUSCIMA++ measure-cut set and 91% for the DoReMi measure-cut set. Since the GNN only prunes edges, these numbers set an upper bound on achievable recall and should be stated whenever the reported MER values are interpreted. The conclusion that GNNs can effectively recover relations between musical primitives should be qualified by this ceiling, which also suggests that candidate-graph construction is itself a load-bearing component of the pipeline.
minor comments (6)
  1. [SMIReT paper, Equation (1)] The displayed expression is not a coherent probability statement: as written, the left-hand side is equated to a sum over the vocabulary, which appears to be a typesetting omission of an argmax or of a distribution over tokens.
  2. [SMIReT paper, Section IV-A] There are several typos and spacing inconsistencies, including 'effort hat has been put', 'adress', and the inconsistent rendering of 'MOTTECTA'; these should be corrected in a revision.
  3. [MLLM paper, Section I] The sentence 'no previuos work has evaluated this scenario' is too strong unless it is restricted to the specific combination of models and tasks tested here; at minimum, the claim should be scoped and the typo 'previuos' fixed.
  4. [YOLO paper, Abstract and author affiliation] The abstract contains the typo 'MeausreDetector', and the affiliation line contains 'Linquistics' for 'Linguistics'; these should be corrected.
  5. [Suzipu paper, Section III-B] The phrase 'nearest neighbors retrieved using K-means' conflates clustering with nearest-neighbor search; the intended procedure appears to be a k-nearest-neighbor lookup in the UMAP embedding space.
  6. [MTN paper, Section III-C] The reference to the tree edit distance algorithm should be 'Zhang and Shasha', not 'Zhang and Sasha'.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: headline metrics are measured against external ground truth; only minor, non-load-bearing self-citation chains appear.

full rationale

The central results of this volume are empirical measurements against external ground truth, not quantities derived from the models' own definitions. SMIReT (Ríos-Vila et al., pp. 7-11) builds on the authors' Sheet Music Transformer (refs 22-23) and on the same group's MOTTECTA/PRIMENS corpora (ref 27); this is a genuine self-citation chain, but the multitask prompting mechanism (Eqs. 2-3) is a new extension, and every Table I metric is computed on the fixed 20% MOTTECTA test split, so no metric is entailed by construction or by the cited SMT. The paper itself authors the task set ('we propose which tasks—based on state of the art—should be considered to compose the SMIR challenge'), but this is benchmark definition, not reduction: 'success' is judged by SER/CER/IoU against labeled pages. Query tasks are generated 'through the information given in the dataset', yet the model must still localize and transcribe from the image. What the paper lacks—single-task baselines, a PRIMENS-pretraining ablation, and its own flagged selective-OMR SER 41.55 and 70.23 IoU shortcomings—weakens the multitask claim (a correctness risk) but is not circularity. The GNN paper (de Lambertye & Pacha) openly reports that its k=20 candidate graphs contain only 80% (MUSCIMA) and 91% (DoReMi) of ground-truth edges, capping achievable recall; its k was explored against dataset ground truth ('k=13 is sufficient to include every relation from the ground truth'), a disclosed, mildly test-informed hyperparameter choice, and Section V admits 'different seeds led to vastly different outcomes.' These are self-reported ceilings, not hidden derivations. The suzipu tool (Repolusk & Veas) reuses the authors' previously published classifier (CER 10.4% from [13])—a prior external published result, legitimately cited. The YOLO study (Dvořák et al.) 'train[s] both architectures on the same datasets' and publishes the models; speed and memory claims are direct measurements. The MTN evaluation (Torras et al.) discards surplus predicted measures ('the extra ones are just discarded'), which biases reported accuracy but does not reduce the metric to an input. Overall, no equation in these papers is equivalent to its input by construction, and no load-bearing premise is justified solely by an unverified self-citation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 3 invented entities

The ledger records hand-set or fitted quantities and domain assumptions across the visible papers. The most consequential free parameter is the kNN candidate-graph size k (13/20) in the GNN paper, because the authors' own inclusion measurements show k=20 captures only 80-91% of ground-truth edges on two datasets, capping any achievable recall. The SMIReT paper's key unstated premise is that synthetic PRIMENS curriculum data transfers to real 17th-century MOTTECTA pages; no ablation isolates this. The MLLM paper's uncontrolled prompts and tiny iconic sample are ad hoc to the claims drawn from them. No genuinely new physical or mathematical entities are introduced; the new conceptual objects (SMIR task family, SMIReT, MTN) each carry a falsifiable handle in the form of a public dataset or held-out evaluation, so they are listed with independent_evidence true.

free parameters (4)
  • k in kNN candidate graph = 13 for MusiGraph; 20 for MUSCIMA++ and DoReMi
    Hand-chosen in the GNN paper (Section III-B and VI); the authors verify k=13 suffices for MusiGraph but k=20 leaves only 80-91% of ground-truth edges inside the candidate graph for the other datasets, directly capping achievable recall.
  • Temperature scaling temperatures = 1.4574 (pitch); 1.2430 (secondary)
    Fitted on validation sets in the suzipu annotation paper (Section III-A) to calibrate OMR classifier confidences; standard calibration, not part of a derivation.
  • UMAP random_state = 42
    Chosen by hand in the suzipu similarity visualization (Section III-B); affects nearest-neighbor results but not the central claim.
  • MLLM prompt wording
    The MLLM paper states prompts were 'carefully formulated with the help of the MLLM itself' (Section II-C), an uncontrolled variable that prevents replicating the Q1-Q4 results.
assumptions (3)
  • domain assumption Related music primitives are spatially close, so kNN graphs suffice as candidate edge sets.
    GNN paper Section III-B justifies the candidate graph construction; the paper's own inclusion-rate measurements (80-91%) show this assumption is only partially true.
  • domain assumption The MOTTECTA corpus labels (regions and text) are complete and correct, and PRIMENS synthetic incipits are representative of 17th-century mensural layout.
    SMIReT paper Sections IV-A and III-C rely on these datasets for train/test and for the curriculum; no quality audit of the ground truth is reported.
  • ad hoc to paper A tiny sample of three culturally well-known score crops is representative for probing general MLLM capability on OMR.
    MLLM paper Section II-B ('tiny set of samples... relatively simple in terms of graphic complexity'); results are used to draw general conclusions about five MLLMs.
invented entities (3)
  • SMIR (Sheet Music Information Retrieval) task family independent evidence
    purpose: Frames OMR, OCR, layout recognition, and query tasks as one challenge addressed by a single model.
    Defined for the first time in the SMIReT paper (Section II); testable through the proposed task suite on MOTTECTA, though no community benchmark yet exists.
  • SMIReT model independent evidence
    purpose: Multitask autoregressive transformer unifying parsing, layout, and query tasks via task prompting.
    Evaluated on a held-out MOTTECTA test split (Table I), so its claims are falsifiable even though no code or weights are released.
  • Music Tree Notation (MTN) format independent evidence
    purpose: Tree-based visual-primitive representation to serve as a common OMR evaluation target.
    Shipped with a converter and the COMREF dataset of 435k measure images (MTN paper Sections II and IV), enabling external adoption and checking.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Proceedings of the 6th International Workshop on Reading Music Systems." pith.science (2026). https://pith.science/paper/DFXRLUPI

@misc{pith2026241115741,
  author       = {Pith},
  title        = {Pith review of: Proceedings of the 6th International Workshop on Reading Music Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DFXRLUPI}},
  note         = {Machine review of arXiv:2411.15741}
}
read the original abstract

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 6th International Workshop on Reading Music Systems, held Online on November 22nd 2024.

Figures

Figures reproduced from arXiv: 2411.15741 by the authors.

Figure 1
Figure 1. Sample of images used for evaluating the capabilities of [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 1
Figure 1. Architecture of the SMIReT model with some examples of task prompting. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the SMIReT performance on a test [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figures from the paper (30 more)
Figure 1
Figure 1. Figure 1: Music Notation Graph II. RELATED WORK The term Music Notation Graph (MuNG) has first been used by Hajic et al. [5] to build the MUSCIMA++ dataset. ˇ This format has then been adopted by other datasets such as MusiGraph [1], and DoReMi [6]. In Pacha et al. [3], the auth…
Figure 2
Figure 2. Figure 2: Training Pipeline Generally, it’s beneficial to have the most detailed granularity, as it can be simplified to a reduced set of classes while maintaining potentially relevant information (e.g., the direction of a stem might help to determine the voice in a polyphonic s…
Figure 3
Figure 3. Figure 3: Illustration of the jump back on learning rate change mechanism: Epoch 10 yields the best validation loss with an initial learning rate. After 10epochs without improvement, the learning rate is reduced and the snapshot from epoch 10 gets loaded. Proceedings of the 6th …
Figure 4
Figure 4. Figure 4: Example of model 2’s prediction on the MUSCIMA++ dataset (cut by measure). [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Example of model 3’s prediction on the DoReMi dataset (cut by measure). In this example, we see an error in the [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Example of the model ensemble prediction on the MusiGraph dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 1
Figure 1. Figure 1: Overview of front covers from MZKBlank resized to squares. [PITH_FULL_IMAGE:figures/full_fig_p021_1.png]
Figure 2
Figure 2. Figure 2: Visualization of R-CNN’s overlap problem. Each cell contains [PITH_FULL_IMAGE:figures/full_fig_p022_2.png]
Figure 1
Figure 1. Figure 1: Example showing a fragment of a measure in which the annotation format for attributes and staff-modifying elements is shown. Rectangular nodes [PITH_FULL_IMAGE:figures/full_fig_p025_1.png]
Figure 2
Figure 2. Figure 2: The pipeline through which the COMREF dataset has been generated. [PITH_FULL_IMAGE:figures/full_fig_p027_2.png]
Figure 1
Figure 1. Figure 1: A small staff fragment of a real sheet music image [PITH_FULL_IMAGE:figures/full_fig_p030_1.png]
Figure 2
Figure 2. Figure 2: Screenshot of the piano keyboard, simulating a real one. The [PITH_FULL_IMAGE:figures/full_fig_p031_2.png]
Figure 3
Figure 3. Figure 3: Screenshot of the piano tab after a melody has been played and [PITH_FULL_IMAGE:figures/full_fig_p032_3.png]
Figure 1
Figure 1. Figure 1: A multimodal score and its sources The project’s work consists in designing tools to produce and manage MMS, including a powerful OMR system which the privileged mean to obtain a pivot. They are briefly sum￾marized below. B. Producing the pivot via optical recognition …
Figure 2
Figure 2. Figure 2: Aligning sources: A multimodal score with three documents [PITH_FULL_IMAGE:figures/full_fig_p035_2.png]
Figure 3
Figure 3. Figure 3: Overview of DMOS: combination of low level detectors and high [PITH_FULL_IMAGE:figures/full_fig_p035_3.png]
Figure 4
Figure 4. Figure 4: The OMR process: Detection of graphical content [PITH_FULL_IMAGE:figures/full_fig_p036_4.png]
Figure 6
Figure 6. Figure 6: The collaborative process, phase 2: checking the transcription context [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]
Figure 7
Figure 7. Figure 7: COLLABSCORE interface showing three synchronized sources on La Coccinelle with the Dezrann libraries: the original image, the pivot score, and a YouTube performance. IV. CONCLUSION The COLLABSCORE project addresses many challenges in modeling and interlinking multimoda…
Figure 1
Figure 1. Figure 1: The Suzipu Intelligent Assistant window al￾lows for visualization and automatic labeling of notation instances. Proceedings of the 6th International Workshop on Reading Music Systems, 2024 39 [PITH_FULL_IMAGE:figures/full_fig_p040_1.png]
Figure 2
Figure 2. Figure 2: Reliability scores for the individual OMR classifiers [PITH_FULL_IMAGE:figures/full_fig_p041_2.png]
Figure 3
Figure 3. Figure 3: UMAP embedding space visualization for both the [PITH_FULL_IMAGE:figures/full_fig_p042_3.png]
Figure 1
Figure 1. Figure 1: Snippet of the split-view component of OMMR4all-2024. Displayed is the transcription result next to the original image. Different overlays (e.g. [PITH_FULL_IMAGE:figures/full_fig_p048_1.png]
Figure 2
Figure 2. Figure 2: The images were cropped from the OMMR4all-2024 overlay editor. The purple areas mark drop capitals. Green areas are music regions. Red areas [PITH_FULL_IMAGE:figures/full_fig_p049_2.png]
Figure 1
Figure 1. Figure 1: Segmentation of handwritten music sheets into smaller [PITH_FULL_IMAGE:figures/full_fig_p052_1.png]
Figure 2
Figure 2. Figure 2: Handwritten music sheet generated by the ESRGAN [PITH_FULL_IMAGE:figures/full_fig_p053_2.png]
Figure 3
Figure 3. Figure 3: Graph showing the generator loss (GLoss) and discrim [PITH_FULL_IMAGE:figures/full_fig_p054_3.png]
Figure 4
Figure 4. Figure 4: t-SNE clustering plot showing the distribution of [PITH_FULL_IMAGE:figures/full_fig_p054_4.png]
Figure 6
Figure 6. Figure 6: Examples of generated ESRGAN scores with a resolution of 256x512 before post-processing. [PITH_FULL_IMAGE:figures/full_fig_p056_6.png]
Figure 7
Figure 7. Figure 7: Examples of ESRGAN generated scores after resizing and post-processing. [PITH_FULL_IMAGE:figures/full_fig_p057_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 13 canonical work pages

  1. [1]

    J. Y . Zhu, T. Park, P. Isola, and A. A. Efros, ”Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proc. IEEE Int. Conf. Computer Vision (ICCV) , 2017, pp. 2223–2232

    2017

  2. [2]

    Karras, S

    T. Karras, S. Laine, and T. Aila, ”A style-based generator architecture for generative adversarial networks,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR) , 2019, pp. 4401–4410

    2019

  3. [3]

    Shatri and G

    E. Shatri and G. Fazekas, ”DoReMi: First glance at a universal OMR dataset,” arXiv preprint, arXiv:2107.07786, Jul. 2021

    arXiv 2021

  4. [4]

    Shatri and G

    E. Shatri and G. Fazekas, ”Knowledge Discovery in Optical Music Recognition: Enhancing Information Retrieval with Instance Segmenta- tion,” in Proc. Int. Conf. Knowledge Discovery and Information Retrieval (KDIR), 2024

    2024

  5. [5]

    Brock, J

    A. Brock, J. Donahue, and K. Simonyan, ”Large-scale GAN training for high-fidelity natural image synthesis,” arXiv preprint, arXiv:1809.11096, 2018

    arXiv 2018

  6. [6]

    Shatri and G

    E. Shatri and G. Fazekas, ”Optical music recognition: State of the art and major challenges,” arXiv preprint, arXiv:2006.07885, 2020

    arXiv 2006

  7. [7]

    Shatri, K

    E. Shatri, K. Palavala, and G. Fazekas, ”Synthesising Handwritten Music with GANs: A Comprehensive Evaluation of CycleWGAN, ProGAN, and DCGAN,” to appear in 2nd Workshop on AI Music Generation (AIMG 2024), IEEE Big Data , Washington D.C., 2024

    work page 2024

  8. [8]

    Hande, E

    P. Hande, E. Shatri, B. Timms, and G. Fazekas, ”Towards Artificially Generated Handwritten Sheet Music Datasets,” in Proc. 5th Int. Work- shop on Reading Music Systems , 2023, p. 25

    2023

Show all 26 references
  1. [9]

    Haji ˇc and P

    J. Haji ˇc and P. Pecina, ”The MUSCIMA++ Dataset for Handwritten Optical Music Recognition,” in Proc. 14th IAPR Int. Conf. Document Analysis and Recognition (ICDAR) , 2017, pp. 39–46, doi: 10.1109/IC- DAR.2017.16

    2017 doi

  2. [10]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, ”Generative adversarial networks,” Communications of the ACM , vol. 63, no. 11, pp. 139–144, 2014, doi: 10.1145/3422622

    2014 doi

  3. [11]

    Creswell, T

    A. Creswell, T. White, V . Dumoulin, K. Arulkumaran, B. Sengupta, and A. Bharath, ”Generative Adversarial Networks: An Overview,” IEEE Signal Process. Mag. , vol. 35, no. 1, pp. 53–65, 2017, doi: 10.1109/MSP.2017.2765202

    2017

  4. [12]

    Li, ”Generative Adversarial Network for Musical Notation Recog- nition during Music Teaching,” Computational Intelligence and Neuro- science, 2022, doi: 10.1155/2022/8724688

    N. Li, ”Generative Adversarial Network for Musical Notation Recog- nition during Music Teaching,” Computational Intelligence and Neuro- science, 2022, doi: 10.1155/2022/8724688

    2022 doi

  5. [13]

    S. Lee, U. Hwang, S. Min, and S. Yoon, ”Polyphonic Music Genera- tion with Sequence Generative Adversarial Networks,” arXiv preprint, arXiv:1710.11418, 2017

    2017 arXiv

  6. [14]

    Transfer Learning with Pix2Pix GAN for Generating Realistic Photographs from Viewed Sketch Arts

    Raghavendra, M., & Sarappadi, P., 2022. Transfer Learning with Pix2Pix GAN for Generating Realistic Photographs from Viewed Sketch Arts. Journal of Southwest Jiaotong University . https://doi.org/10.35741/issn. 0258-2724.57.4.17

    2022 doi

  7. [15]

    H. Dong, W. Hsiao, L. Yang, and Y . Yang, ”MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Gen- eration and Accompaniment,” in Proc. AAAI Conf. Artificial Intelligence, 2017, pp. 34–41, doi: 10.1609/aaai.v32i1.11312

    2017 doi

  8. [16]

    H. Chen, Q. Xiao, and X. Yin, ”Generating Music Algorithm with Deep Convolutional Generative Adversarial Networks,” in Proc. IEEE Int. Conf. Electronics Technology (ICET) , 2019, pp. 576–580, doi: 10.1109/ELTECH.2019.8839521

    2019

  9. [17]

    M. Liu, X. Huang, J. Yu, T. Wang, and A. Mallya, ”Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications,” Proc. IEEE , vol. 109, no. 5, pp. 839–862, 2020, doi: 10.1109/JPROC.2021.3049196

    2020

  10. [18]

    Clabaut, M

    ´E. Clabaut, M. Lemelin, M. Germain, Y . Bouroubi, and T. St-Pierre, ”Model Specialization for the Use of ESRGAN on Satellite and Air- borne Imagery,” Remote Sens. , vol. 13, no. 20, p. 4044, 2021, doi: 10.3390/rs13204044

    2021 doi

  11. [19]

    Z. Zhu, Y . Lei, Y . Qin, C. Zhu, and Y . Zhu, ”IRE: Improved Image Super-Resolution Based on Real-ESRGAN,” IEEE Access, vol. 11, pp. 45334–45348, 2023, doi: 10.1109/ACCESS.2023.3256086

    2023

  12. [20]

    Rabbi, N

    J. Rabbi, N. Ray, M. Schubert, S. Chowdhury, and D. Chao, ”Small- object detection in Remote Sensing Images with End-to-End Edge- Enhanced GAN and Object Detector Network,” Remote Sens., vol. 12, no. 9, p. 1432, 2020, doi: 10.20944/preprints202003.0313.v1

    2020

  13. [21]

    Wang et al., ”ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc

    X. Wang et al., ”ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc. Eur. Conf. Computer Vision (ECCV) , 2018, pp. 63–79, doi: 10.1007/978-3-030-11021-5 5

    2018 doi

  14. [22]

    Le-Tien, T

    T. Le-Tien, T. Nguyen-Thanh, H. Xuan, G. Nguyen-Truong, and V . Ta- Quoc, ”Deep Learning-Based Approach Implemented to Image Super- Resolution,” J. Adv. Inf. Technol., vol. 11, no. 4, pp. 209–216, 2020, doi: 10.12720/jait.11.4.209-216

    2020 doi

  15. [23]

    X. Wang, L. Xie, C. Dong, and Y . Shan, ”Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data,” in Proc. IEEE/CVF Int. Conf. Computer Vision Workshops (ICCVW) , 2021, pp. 1905–1914, doi: 10.1109/ICCVW54120.2021.00217

    2021

  16. [24]

    Singh, A

    N. Singh, A. F, M. Rastogi, and R. Prasad, ”Performance Anal- ysis of Conditional GANs-based Image-to-Image Translation Mod- els for Low-Light Image Enhancement,” in Proc. Int. Conf. Sig- nal Process. and Communication (ICSC) , 2022, pp. 468–474, doi: 10.1109/ICSC56524.2022.10009340

    2022

  17. [25]

    Proposal to Improve the Image Quality of Short-Acquisition Time-Dedicated Breast Positron Emission Tomography Using the Pix2pix Generative Adversarial Net- work

    Fujioka, T., Satoh, Y ., Imokawa, T., Mori, M., Yamaga, E., Takahashi, K., Kubota, K., Onishi, H., & Tateishi, U., 2022. Proposal to Improve the Image Quality of Short-Acquisition Time-Dedicated Breast Positron Emission Tomography Using the Pix2pix Generative Adversarial Net- ...

    2022 doi

  18. [26]

    Calvo-Zaragoza, A

    J. Calvo-Zaragoza, A. Gallego, and A. Pertusa, ”Recognition of Hand- written Music Symbols with Convolutional Neural Codes,” in Proc. 14th IAPR Int. Conf. Document Analysis and Recognition (ICDAR), 2017, pp. 691–696, doi: 10.1109/ICDAR.2017.118. APPENDIX Proceedings of the 6 t...

    2017 doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.