CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction

Emmanouil Benetos; Haiwen Xia; Hewei Gao; Mingshuo Ding; Ruibin Yuan; Simon Dixon; Sungkyun Chang; Weixiong Chen; Yinghao Ma; Yizhi Li

REVIEW 3 major objections 4 minor 2 cited by

CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction

T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a single parameter-efficient reward model can judge generated music against compositional multimodal instructions—text, lyrics, and reference audio—as well as or better than frontier multimodal LLMs, and can serve as

desk verdict A solid, useful benchmark/dataset paper for music reward models; the stress-test ceiling argument is mathematically wrong and shouldn't drive a rejection. read the letter →

arxiv 2603.00610 v3 pith:MRJXLS3E submitted 2026-02-28 cs.SD cs.AIcs.LGcs.MMeess.AS

Yinghao Ma , Haiwen Xia , Hewei Gao , Weixiong Chen , Yuxin Ye , Yuchen Yang , Sungkyun Chang , Mingshuo Ding

show 4 more authors

Yizhi Li Ruibin Yuan Simon Dixon Emmanouil Benetos

This is my paper · ORCID

classification cs.SDcs.AIcs.LGcs.MMeess.AS

keywords musicrewardmodelscompositionalmultimodalinstructionpreferencedatasetsbenchmarkinference-timescalingbest-of-nfilteringmusicalityevaluationtext-to-musicalignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that music evaluation can be unified under 'compositional multimodal instruction' (CMI), where a generated track must be judged against any mix of text, lyrics, and reference audio. To support this, it builds two preference datasets (a 110k-pair pseudo-labeled set and a 4k-pair human-annotated set) and a benchmark spanning five evaluation tasks. The central experimental claim is that CMI-RM, a roughly 30M-parameter model, reaches 78.20% agreement with human preferences on the CMI-Pref test set and 73.43% on Music Arena, outperforming much larger general-purpose audio-language models such as Gemini-3-Pro. If true, one small open reward model can proxy human judgment across modality combinations within a single architecture, and can improve music generation through best-of-N reranking.

What carries the argument

The key machinery is the two-tower compositional reward architecture: one tower encodes the multimodal prompt (text, lyrics, optional reference audio) via frozen MuQ-MuLan encoders and a 4-layer Prompt Transformer; the other encodes the evaluation audio; a single-layer Joint Transformer and a lightweight MLP produce two scalar scores, one for musicality and one for alignment. Training proceeds in two stages: Bradley-Terry preference learning on pseudo-labeled pairs with 0.2 label smoothing (to counter over-confidence from noisy labels), followed by fine-tuning on human annotations using both pairwise and regression losses. This design lets the model handle any subset of input modalities with

What would settle it

A re-annotation of a random 100–200 pairs from CMI-Pref by a completely different panel of expert annotators, followed by a check of whether CMI-RM's pairwise choices agree with the new labels at the same rate, would directly test whether the model's apparent alignment reflects stable human taste or just the idiosyncrasies of the original 31 annotators.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a compact, two-tower reward model with frozen MuQ-MuLan encoders and a lightweight Prompt Transformer can jointly predict musicality and instruction-alignment scores for music generated under optional text, lyrics, and reference-audio conditions. After pre-training on 110k consistency-filtered pseudo-labels from Qwen3-Omni and fine-tuning on a mixture of human-annotated CMI-Pref and MusicEval data, the model achieves state-of-the-art preference accuracy on CMI-Pref (78.20% overall) and remains competitive on Music Arena (73.43%), while general-purpose multimodal LLMs such as Gemini-3-Pro and Qwen3-Omni stagnate near 65–70%. The authors further show that

Load-bearing premise

The benchmark's ground truth is the preference labels provided by 31 human annotators, and their agreement with each other is only moderate (Krippendorff's alpha around 0.38–0.45), so the reported accuracy numbers measure agreement with a noisy reference standard.

Editorial extensions

If this is right

If CMI-RM generalizes as reported, a single ~30M-parameter reward model can replace fragmented, modality-specific evaluators in music generation pipelines, covering text-only, lyrics-only, audio-referenced, and fully compositional prompts.
The model enables practical inference-time scaling: reranking the top N of multiple candidate generations with CMI-RM produces consistent quality gains over single-shot generation, with diminishing returns beyond N=10.
The CMI-RewardBench benchmark provides a standardized way to compare future music reward models across musicality, text-music alignment, and compositional instruction alignment, exposing a clear capability gap for general multitask models.
The ablation results imply that high-quality human preference data under compositional conditions is highly transferable, improving performance even on benchmarks the model was not trained on.
The model's output can be used to aggregate Bradley-Terry-style leaderboards of music generation systems, offering an open alternative to closed commercial evaluations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next step the authors leave implicit is to use CMI-RM not only as a filter/re-ranker but as the reward signal inside RLHF or direct preference optimization for music generation models, which could close the gap to real data that their reranking experiments still show.
The paper's evidence that context (especially reference audio) improves musicality prediction suggests a broader principle: in generative-music evaluation, 'musicality' is a context-dependent judgment, not an absolute property of the audio signal—a claim that, if true, would reshape how musicality benchmarks are designed.
Because the pseudo-labels were distilled from a single model with a consistency filter, the benchmark's ceiling may be partially coupled to that model's taste; an independent pseudo-labeling source or a re-annotation study would test whether the human-correlation results are robust to distillation bias.
The finding that lyrics-heavy prompts actually hurt musicality prediction (the 'Text + Lyrics w/o Audio' subset drops by 4.00%) points to a concrete weakness in the frozen MuQ-MuLan text encoder; a trainable lyric-aware text encoder may yield further gains.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this is a substantive and well-executed resource paper. It builds the first preference dataset explicitly covering compositional text+lyrics+audio conditioning, a 4k human-annotated set plus 110k pseudo-labeled pairs, a unified benchmark over PAM/MusicEval/Music Arena, and a parameter-efficient 30M reward model that handles all those modalities in one architecture. Code, data, and weights are all released. That alone makes it a useful community asset.

The paper is also honest and careful in its presentation. It reports inter-annotator agreement (moderate, as you'd expect for subjective music preferences), runs thorough ablations on label smoothing, pseudo-data size, mapping functions, and inference duration, and uses external benchmarks to keep the CMI-Pref test set from being the only measuring stick. The position-consistency filter for pseudo-labels is a nice practical touch.

The stress-test note attached to the review does not hold up. The claim is that CMI-RM's 78.2% accuracy on CMI-Pref musicality exceeds the 72.4% human-human agreement rate, implying leakage or a flawed target. That reasoning confuses human-human agreement with human accuracy against the latent truth. If two noisy annotators each agree with the truth with probability q, their pairwise agreement is q^2 + (1-q)^2. An agreement rate of 0.724 implies q is about 0.83, so a perfect latent-truth model would agree with a single human annotator around 83% of the time. 78.2% is below that, not above it. The quantitative inconsistency is not an inconsistency.

Real soft spots are smaller. All tables are single-run point estimates with no error bars, so we can't assess whether the main accuracy gaps are significant. The headline CMI-Pref number is partly in-distribution, though the external results mitigate that. The ablation where Distill+MusicEval is worse than Distill alone on MusicEval is odd and goes unexplained. And the authors could be more explicit about how many human votes label each test pair - single vs. majority - to put the agreement question to rest.

Bottom line: this deserves a serious referee. It's not flawless, but the contribution is real, the methods are mostly sound, and the resources will likely be adopted. If I were the editor, I'd send it out with a request for error bars and a bit more transparency on test-label construction.

Referee Report

3 major / 4 minor

Summary. The paper introduces a benchmark and reward-model ecosystem for evaluating music generation under compositional multimodal instructions (text, lyrics, and reference audio). It contributes two datasets: CMI-Pref-Pseudo (110k pseudo-labeled preference pairs) and CMI-Pref (4,027 human-annotated pairs by 31 annotators), and a unified benchmark, CMI-RewardBench, combining existing resources (PAM, MusicEval, Music Arena) with the CMI-Pref test split. The authors then train CMI-RM, a ~30M-parameter two-tower reward model with frozen MuQ-MuLan encoders, using a two-stage pipeline (pseudo-pretraining followed by expert fine-tuning). The main reported results are that CMI-RM achieves state-of-the-art or competitive performance across musicality and alignment tasks—notably 78.20% accuracy on CMI-Pref musicality and 73.43% on Music Arena—and that best-of-N reranking with CMI-RM yields consistent test-time scaling gains.

Significance. If the central claims hold, this is a useful contribution: it provides a much-needed public benchmark for compositional multimodal music evaluation, a human-annotated preference dataset spanning text/lyrics/audio conditioning, and a compact reward model that can serve as a proxy for human preference across these conditions. The paper is transparent in releasing code, model weights, and datasets, and it grounds many claims on external benchmarks (PAM, MusicEval, Music Arena) in addition to its own data. The position-consistency filtering for pseudo-label generation and the human A/B validation of reranking are concrete methodological strengths. However, the benchmark's internal validity rests on how the CMI-Pref test labels were constructed, and the paper currently contains an unexplained quantitative inconsistency between the reported 78.20% accuracy and the measured human pairwise agreement ceiling of 72.4%. A second inconsistency concerns whether the final model was pre-trained on 110k or 64k pseudo-labeled pairs. Both issues are load-bearing for the paper's main claims and require a major revision.

major comments (3)

[§4.1.1, Table 2; Appendix A.2, Table 6] CMI-RM (w/ f.t.: CMI + MusicEval) is reported at 78.20% accuracy on CMI-Pref musicality, while the human pairwise agreement rate on music quality is 72.4% (Krippendorff α = 0.447). If CMI-Pref test labels are single-annotator votes—which the paper does not state otherwise—a perfect latent-truth model's expected agreement with the gold label is bounded by the human-human agreement rate under standard conditional-independence measurement error. The 5.8-point excess (roughly 3 SE at n=500) implies either that test labels are multi-annotator consensus/adjudicated (which must be documented and the ceiling re-estimated), or that the model exploits non-preference cues such as source-model identity or train/test leakage. This directly affects the central claim that CMI-RM proxies human preference and the benchmark's internal validity. Please report the exact test-label construction (single vs. a
[§3.3.2 vs Appendix C.2] Stage 1 of the training strategy states that the model is pre-trained on CMI-Pref-Pseudo, which Section 3.1 describes as containing 110k pairs. However, Appendix C.2 states: 'we adopt 64k pseudo-labeled examples as a compute-efficient operating point in subsequent experiments.' The main reported numbers (e.g., Table 2: w/ f.t. CMI-Pref 77.80%; Table 10: 64k Mus-Acc 0.778) correspond to a 64k pretraining set, and the stated 2k steps × batch size 48 ≈ 96k sample presentations is consistent with ~1.5 epochs over 64k, not over 110k. This discrepancy affects the claimed dataset scale, the training protocol description, and reproducibility. Please clarify whether the final model used 64k or 110k pseudo-labeled pairs, and align the main text with the appendix.
[All empirical tables] All accuracy, correlation, and ablation numbers are single-run point estimates with no confidence intervals, error bars, or significance tests. Given the small test sets (n=500 for CMI-Pref, n=1,340 for Music Arena) and the moderate inter-annotator agreement, the reported differences between top systems (e.g., 78.20% vs 77.80% on CMI-Pref musicality) may not be statistically reliable. At minimum, report binomial confidence intervals for accuracies and bootstrap intervals for correlation coefficients.

minor comments (4)

[§3.1.1] Music Arena labels are categorized as 'MUSICALITY' in the benchmark, but the Music Arena platform collects holistic text-to-music preference. The Appendix E.1 regression suggests musicality dominates overall preference, but this assumption should be stated and justified in the main text before using Music Arena as a musicality benchmark.
[Appendix C.2] Typo: 'Pref-Text Acc' should be 'Pref-Test Acc'. Also, the sentence 'Since directly finetuning on the 3.5k CMI-Pref yields Pref-Text Acc of 0.7215' references a number not defined in the appendix.
[Table 5] Table 5 caption spells 'MuQ-MuLan' as 'MuuQ-Mulam'. Please correct.
[References] References Xu et al. 2025a and 2025b appear to be the same Qwen2.5-Omni technical report; please consolidate and distinguish from Qwen3-Omni if applicable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CMI-RM's central claim is supported by held-out CMI-Pref evaluation plus external PAM/MusicEval/Music Arena results; no prediction reduces to a fitted input or self-citation.

full rationale

The paper's derivation chain is self-contained and non-circular. CMI-RM is trained with a standard two-stage Bradley-Terry/MSE objective on CMI-Pref-Pseudo and the training split of CMI-Pref plus MusicEval, then evaluated on a reserved 500-pair CMI-Pref test split (Section 3.2.1, Section 3.3.2, Table 1). This is ordinary supervised evaluation, not a prediction that is forced by construction: the test labels are not used in fitting. The central SOTA claim (78.20% on CMI-Pref, Section 4.1.1, Table 2) is additionally triangulated by external benchmarks: PAM (SRCC 0.6988 on the CMI-Pref-finetuned variant) and Music Arena (73.43%), where Music Arena data are not part of CMI-RM's training. No equation in the paper defines a target in terms of the fitted parameters; the Bradley-Terry loss (Eq. 5) and MSE regression (Eq. 6) are standard and do not encode the evaluation labels. Self-citations appear only in background or as baseline datasets (e.g., SongEval, YUE) and are not load-bearing; there is no imported uniqueness theorem or ansatz from the authors' prior work. The one caveat is label quality: Appendix A.2 reports only moderate inter-annotator agreement (Krippendorff's α = 0.382/0.447) and a 72.4% pairwise agreement rate for music quality. This is a validity/reliability concern about the benchmark's ground truth, and it could matter if CMI-Pref test labels are single-annotator, but it is not circularity: the reported 78.20% accuracy is a measured agreement with those reference labels, not a quantity derived from the agreement statistic. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two datasets whose human labels have only moderate reliability, and on the MuQ-MuLan frozen encoder assumption. No new physical or mathematical entities are introduced.

free parameters (3)

label smoothing epsilon = 0.2
Chosen in Appendix C.1 to combat pseudo-label over-confidence; ablations show it improves downstream accuracy.
tanh scaling parameters a, b = a=0.2, b=0 initial; learned
Equation 6 maps raw scores to [1,5] MOS; scaling parameters are learned during fine-tuning and dropped at inference.
pseudo-label dataset size = 64k (main experiments; 110k reported)
Appendix C.2 selects 64k as a compute-efficient operating point based on ablations; it is a design choice affecting the final model.

assumptions (4)

standard math Bradley-Terry model of pairwise preferences
Used in Eq. 5 for preference pretraining and fine-tuning; standard assumption that pairwise choice probability is logistic in score differences.
domain assumption MuQ-MuLan frozen encoders adequately represent text, lyrics, and audio for reward modeling
The architecture freezes MuQ-MuLan encoders (Section 3.3.1); the paper itself notes the text encoder struggles with raw lyrics (Appendix E.2.1, Table 14 shows -4% when lyrics are present), so this assumption is load-bearing and partially violated.
domain assumption Human annotations in CMI-Pref are a reliable ground truth despite moderate inter-annotator agreement
All benchmark conclusions use CMI-Pref human preferences as truth (Appendix A.2); alpha 0.382/0.447 is moderate, so noise in labels propagates to every accuracy number.
domain assumption Qwen3-Omni pseudo-labels, after position-consistency filtering, provide a useful training signal
CMI-Pref-Pseudo is built from Qwen3-Omni judgments (Section 3.1.2, Appendix B); the paper's own ablations show pseudo-pretraining helps only after human fine-tuning and label smoothing.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction." pith.science (2026). https://pith.science/paper/MRJXLS3E

@misc{pith2026260300610,
  author       = {Pith},
  title        = {Pith review of: CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MRJXLS3E}},
  note         = {Machine review of arXiv:2603.00610}
}

read the original abstract

While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind. In this paper, we bridge this critical gap by establishing a comprehensive ecosystem for music reward modeling under Compositional Multimodal Instruction (CMI), where the generated music may be conditioned on text descriptions, lyrics, and audio prompts. We first introduce CMI-Pref-Pseudo, a large-scale preference dataset comprising 110k pseudo-labeled samples, and CMI-Pref, a high-quality, human-annotated corpus tailored for fine-grained alignment tasks. To unify the evaluation landscape, we propose CMI-RewardBench, a unified benchmark that evaluates music reward models on heterogeneous samples across musicality, text-music alignment, and compositional instruction alignment. Leveraging these resources, we develop CMI reward models (CMI-RMs), a parameter-efficient reward model family capable of processing heterogeneous inputs. We evaluate their correlation with human judgment scores on musicality and alignment on CMI-Pref along with previous datasets. Further experiments demonstrate that CMI-RM not only correlates strongly with human judgments, but also enables effective inference-time scaling via top-k filtering. Code is available at GitHub (https://github.com/Haiwen-Xia/CMI-RewardBench). Model weights: CMI-RM (https://huggingface.co/HaiwenXia/CMI-RM). Datasets: CMI-Pref-Pseudo (https://huggingface.co/datasets/HaiwenXia/cmi-pref-pseudo) and CMI-Pref (https://huggingface.co/datasets/HaiwenXia/cmi-pref)

Figures

Figures reproduced from arXiv: 2603.00610 by the authors.

**Figure 1.** Reward models should act as proxies for human evaluation under compositional multimodal instructions (CMI). Human judges evaluate a generated waveform with respect to the provided prompt—text descriptions, lyrics, and/or reference audio—considering both musicality and instruction alignment. However, current reward models are typically fragmented: they either score musicality from audio alone or capture alignment for… view at source ↗

**Figure 2.** Model architecture of compositional music instruction reward model (CMI-RM). MusicGen-small and Stable-Audio-Open-small. For each of the 2,183 text prompts from the MusicCaps (Agostinelli et al., 2023) dataset eval-split, we generate 10 audio samples (10 sec each) per model. Our reward model serves as a “best-of-N” filter to select the top-performing sample, where N ∈ {1, 3, 10}. We evaluate the effectiveness via s… view at source ↗

**Figure 3.** Pairwise preference matrices for test-time scaling with RM reranking. Each cell reports the percentage of trials in which annotators preferred system A (row) over system B (column). 5. Conclusion We introduced CMI-RewardBench, a unified benchmark for evaluating music reward models under Compositional Multimodal Instruction (CMI), where the models handle optional and heterogeneous conditions (text-only, lyric-guided… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Distribution of annotators’ confidence scores for instruction following and music quality. ments between instruction following and music quality often correspond to more ambiguous or trade-off cases, where annotators are less certain about the overall preference. Ali…

**Figure 5.** Figure 5: Cross-entropy loss of checkpoints trained on CMIPseudo. Pseudo indicates metrics on the CMI-Pseudo validation set, and Real denotes results on the test set of CMI-Pref. as the average reward-head accuracy over the Musicality and Alignment heads, and (ii) Pref-Test CE,…

**Figure 6.** Figure 6: Accuracy of checkpoints trained on CMI-Pseudo. Pseudo indicates metrics on the CMI-Pseudo validation set, and Real denotes results on the test set of CMI-Pref [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Platform of human annotation [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Genre Bias or Aesthetic Perception? Identifying and Mitigating Shortcut Learning in Music Evaluation
cs.SD 2026-07 conditional novelty 6.0 of 10

SongEval's aesthetic scores are shortcut by genre (pop-centric), and a focal-loss plus group-regularized training objective measurably reduces that genre bias.
Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering
cs.SD 2026-07 conditional novelty 5.0 of 10

A unified hierarchical-LM-plus-flow-matching system reports top-tier full-song vocal generation, ranking 2–3 on an external blind leaderboard, but releases neither code nor evaluation data.

Reference graph

Works this paper leans on

13 extracted references · cited by 2 Pith papers

[1]

Forward Pass:We query the model with the sequence (A, B)to obtain preferenceLf wd
[2]

Original

Reverse Pass:We swap the positions to (B, A)and query the model again to obtain preferenceL rev. A pseudo-label is considered valid and retained only if the judgment is invariant to position—that is, the model prefers the same underlying audio clip in both the forward and reverse passes (Lf wd=L rev). Comparisons yielding con- flicting results or inconsis...
[3]

Preference Label (A/B):A forced-choice selection between two candidates
[4]

Confidence Score (1–5):A quantitative measure of the annotator’s certainty, grounded in constraint satis- faction (for alignment) or quality delta (for musicality)
[5]

fin- ished, natural, and professional

Free-text Feedback:Qualitative justifications focus- ing on fine-grained details that discrete labels cannot capture. D.1.2. GENERALPRINCIPLES • Instruction-First:Annotators must strictly evaluate theinstruction/promptbefore listening to avoid post- hoc rationalization. • Holistic and Granular Review:Each sample is evalu- ated for overall coherence as wel...
[6]

lo-fi aesthetic

Inherent Association in Human Judgment:During annotation, human experts may find it difficult to completely decouple musicality from alignment. For instance, a prompt specifying a “lo-fi aesthetic” might lead an annotator to perceive low-fidelity audio as a musical choice rather than a technical flaw. Thus, the prompt acts as a “taste anchor” that recalib...
[7]

Because of this inherent link, the prompt provides critical contextual clues about the expected acoustic features

Inference Assistance via Non-trivial Shortcuts:From the data correlation perspective, there is an undeniable, in- trinsic correlation between the prompt and the evaluated audio, as the latter is directly generated conditioned on the former from a music generation model. Because of this inherent link, the prompt provides critical contextual clues about the...
[8]

first 10: Use only the first 10 seconds of audio to infer the Musicality and Alignment scores

Show all 13 references

[9]

mean 10: Split the audio into non-overlapping 10- second chunks (hop size 10 seconds), infer scores for each chunk, and take the average as the final score
[10]

good enough

first 120: Match the training setting by extracting MuQ embeddings from the first 120 seconds (concatenating four 30-second segments), and then infer scores using the learned weights. Among these,mean 10is the most compute-intensive. With a maximum duration of 120 seconds and ...
[11]

The gap between open-weight and closed-source mod- els remains large: the top-5 models in each metric are dominated by proprietary systems
[12]

In modali- ties with abundant generations, Minimax Music and Mureka outperform several Suno variants

In pure song generation, the gap between Suno and other proprietary models is narrowing. In modali- ties with abundant generations, Minimax Music and Mureka outperform several Suno variants
[13]

You will now be shown several exam- ples of audio comparisons to help you understand how to evaluate the audios

Recent open-source models—LEVOfor song genera- tion, MAGENTAREALTIMEfor instrumental genera- tion, and ACESTEPacross both—exhibit competitive performance. Table 23.Generation counts per model by modality Instrumental Song Model w/ audio w/o audio w/ audio w/o audio suno-v5212 ...

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Forward Pass:We query the model with the sequence (A, B)to obtain preferenceLf wd

[2] [2]

Original

Reverse Pass:We swap the positions to (B, A)and query the model again to obtain preferenceL rev. A pseudo-label is considered valid and retained only if the judgment is invariant to position—that is, the model prefers the same underlying audio clip in both the forward and reverse passes (Lf wd=L rev). Comparisons yielding con- flicting results or inconsis...

[3] [3]

Preference Label (A/B):A forced-choice selection between two candidates

[4] [4]

Confidence Score (1–5):A quantitative measure of the annotator’s certainty, grounded in constraint satis- faction (for alignment) or quality delta (for musicality)

[5] [5]

fin- ished, natural, and professional

Free-text Feedback:Qualitative justifications focus- ing on fine-grained details that discrete labels cannot capture. D.1.2. GENERALPRINCIPLES • Instruction-First:Annotators must strictly evaluate theinstruction/promptbefore listening to avoid post- hoc rationalization. • Holistic and Granular Review:Each sample is evalu- ated for overall coherence as wel...

[6] [6]

lo-fi aesthetic

Inherent Association in Human Judgment:During annotation, human experts may find it difficult to completely decouple musicality from alignment. For instance, a prompt specifying a “lo-fi aesthetic” might lead an annotator to perceive low-fidelity audio as a musical choice rather than a technical flaw. Thus, the prompt acts as a “taste anchor” that recalib...

[7] [7]

Because of this inherent link, the prompt provides critical contextual clues about the expected acoustic features

Inference Assistance via Non-trivial Shortcuts:From the data correlation perspective, there is an undeniable, in- trinsic correlation between the prompt and the evaluated audio, as the latter is directly generated conditioned on the former from a music generation model. Because of this inherent link, the prompt provides critical contextual clues about the...

[8] [8]

first 10: Use only the first 10 seconds of audio to infer the Musicality and Alignment scores