Pith. sign in

REVIEW 3 major objections 4 minor 33 references

MMAC evaluates audio captions by checking, per information dimension, whether a model mentions target content and describes it correctly.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 09:56 UTC pith:QQJOIIPR

load-bearing objection Useful new diagnostic benchmark for audio captioning, but the judge validation is partly circular; fix that and release the artifacts. the 3 major comments →

arxiv 2607.27109 v2 pith:QQJOIIPR submitted 2026-07-29 cs.SD cs.AI

MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning

Weijie Wu , Junbo Li , Lin Li , Jun Fang , Qingyang Hong This is my paper
classification cs.SD cs.AI
keywords audio captioningbenchmarkmulti-dimensional evaluationcoverageprecisionaccuracyAudioLLMfree-form captioning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that free-form audio captions should be judged dimension by dimension, not with a single aggregate score. It decomposes audio content into 6 capability categories and 15 fine-grained dimensions, and for each target dimension it checks two separate things: whether the caption mentions relevant information (coverage) and whether the mention is consistent with the reference label (precision). To make this concrete, it introduces MMAC, a benchmark of 5,638 audio clips, and evaluates eight open-source and proprietary AudioLLMs. The results show that models trade off coverage against correctness—e.g., Gemini 2.5 Pro leads in accuracy and precision while Qwen3-Omni-Captioner leads in coverage—and that longer captions increase coverage but reduce precision. A sympathetic reader would care because this provides a diagnostic tool that separates omission from hallucination, showing exactly which information dimensions need improvement.

Core claim

The central claim is that the quality of an open-ended audio caption can be assessed per dimension by first detecting whether the model voluntarily mentions the target dimension and then judging whether the described content agrees with the reference label. Each sample thus yields three outcomes: omission, incorrect mention, or correct mention. Across a benchmark these aggregate into Coverage (proportion of samples where the dimension is mentioned), Precision (average score over mentioned samples), and Accuracy (average over all samples with omissions scored as zero). On MMAC, model rankings differ sharply between these metrics: Gemini 2.5 Pro reaches 46.85 Accuracy and 59.39 Precision but o

What carries the argument

The central object is the decoupled evaluation protocol: each test subset targets a small set of dimensions, and a judge LLM (Qwen3.6-27B) reads the model's free-form caption and the reference label, assigning per-dimension scores for mention and consistency. This turns open-ended caption generation into a structured check without converting it into attribute question answering. The supporting machinery is the data construction pipeline—screening public audio datasets, synthesizing missing cases for the implication and dynamic dimensions, and human verification—to ensure each sample has a clearly perceptible cue, a traceable label, and an unambiguous relation to the target dimension.

Load-bearing premise

The results assume the judge model Qwen3.6-27B can correctly map a free-form caption to a coverage/precision label per dimension; the human validation used pre-annotations generated by that same judge, so the high concordance (Kendall's W = 0.981) may partly reflect anchoring bias rather than verified ground-truth accuracy.

What would settle it

Take the 10% stratified subset, obtain human annotations from annotators who see no pre-annotations, and recompute Kendall's W and the model rankings; if W drops well below 0.9 or rankings change materially, the judge-dependent conclusions of MMAC would not be stable. A complementary check is to adversarially insert a clearly false mention into a caption and see whether the judge flags it as inconsistent in all target dimensions.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the protocol is valid, future audio captioning benchmarks can report coverage and correctness separately for each information dimension, letting developers see exactly which dimensions a model omits or gets wrong.
  • The finding that captioning fine-tuning improves coverage but not precision suggests current training recipes optimize for mentioning more rather than describing accurately; the diagnostics identify where to add targeted data or losses.
  • Since caption length raises coverage and lowers precision, length-controlled evaluation may become a standard control in audio captioning model comparisons.
  • The three-metric framework (Coverage, Precision, Accuracy) offers a template for other open-ended generation tasks where omission versus hallucination is the primary error type, such as image or video captioning.
  • Dimension-level results reveal distinct model strengths—for instance, Gemini 2.5 Pro excels on implication and dynamic dimensions while Qwen3-Omni-Captioner is stronger on content and paralinguistic coverage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to apply MMAC's decoupled scoring to additional models beyond the eight reported; the benchmark is released for exactly this, and the coverage-precision trade-off hypothesis could be checked on a broader model family.
  • The judge-based evaluation could be independently validated by collecting human labels without any pre-annotations; if the Kendall concordance drops substantially below 0.981, the reported model rankings may partly reflect judge bias rather than true caption quality.
  • The dimension decomposition points toward timestamp-based evaluation (which the paper lists as future work), where each mention could be localized in time, connecting coverage and correctness to temporal grounding.
  • If coverage and precision are governed by different mechanisms, then model improvement should be targeted separately—coverage may depend on salience-driven attention, while precision may require stronger grounding—an inference the paper does not directly claim.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents MMAC, a benchmark for free-form audio captioning that evaluates captions along 15 fine-grained dimensions grouped into 6 capability categories. Instead of an aggregate similarity score, MMAC checks per dimension whether a model-generated caption mentions the target information and whether the mention is consistent with a reference label, yielding separate Coverage, Precision, and Accuracy scores. The benchmark contains 5,638 audio clips from public datasets, text resources, and synthesized audio, with labels constructed through a multi-stage human-review pipeline. The authors evaluate eight open- and closed-source AudioLLMs and report clear differences across dimensions, coverage, and reliability. They also compare automatic scoring by four LLM judges against human evaluation on a 10% stratified subset, reporting high agreement (Kendall's W=0.981).

Significance. If the evaluation protocol is valid, MMAC would provide a valuable diagnostic tool for audio captioning, separating information omission from incorrect description—something current aggregate metrics do not offer. The benchmark construction is systematic, with explicit criteria for sample selection, label verification, and human quality inspection, and the decoupled design avoids reformulating captioning into isolated attribute QA tasks. The paper also makes a useful empirical observation that coverage and precision can trade off across models and caption lengths. However, the central validation of the automatic judge is weakened by the acknowledged anchoring bias in the human evaluation, and the absence of uncertainty estimates makes the 'clear differences' claim not yet statistically grounded. The authors are transparent about the main limitation, but the current evidence is insufficient to certify the judge as a reliable substitute for independent human assessment.

major comments (3)
  1. [Sec. 3.3 / Sec. 4] The human evaluation is anchored to the automatic judge: per Sec. 4, annotators were given pre-annotations generated by Qwen3.6-27B, the same model that produced the automatic scores in Table 2 and Fig. 3. Thus the W=0.981 concordance reflects agreement with the judge, not independent human judgment. Because all main results depend on this judge's correctness, the central claim that MMAC reliably 'distinguishes omissions, incorrect descriptions, and correct descriptions' is not yet supported. Please provide either (a) a fresh human evaluation on a held-out sample without pre-annotations, or (b) validation against captions with known gold coverage/precision labels, with per-model and per-dimension agreement statistics.
  2. [Table 2 / Sec. 3.3] No uncertainty quantification is provided for the reported scores. Several between-model differences are small (e.g., Accuracy 46.85 vs 45.62 for Gemini 2.5 Pro vs Qwen3-Omni-Captioner) and some dimension-level scores are near floor (e.g., implication coverage 0.07–0.10). Without bootstrap confidence intervals, permutation tests, or at least effective sample sizes per dimension, the claim of 'clear differences' is not statistically grounded. Please add error bars or a bootstrap analysis over samples, especially for the per-dimension comparisons in Fig. 3.
  3. [Sec. 3.2] The aggregate scores use equal category-macro averaging, giving each of the 6 capability categories equal weight despite their very different sample sizes (e.g., content 1000 vs dynamic 615). A model that excels in a small category can be advantaged in the overall ranking. The results in Table 2 should be accompanied by a sensitivity check using sample-size-weighted averaging or by reporting both macro and weighted results. If rankings change under an alternative weighting, the interpretation of the main results must be adjusted.
minor comments (4)
  1. [Fig. 3 / Table 2] The model name 'Qwen3-Omni-Captioner' appears in Table 2 while Fig. 3 uses 'Qwen3-Omni-30B-A3B-Captioner'; please standardize model names across all figures, tables, and text.
  2. [Sec. 2.3 / Fig. 2] The text states 'more than 20 data sources,' but Fig. 2 lists 14 named sources plus 'Others.' Please provide a full list of all data sources or reconcile the counts.
  3. [Sec. 3.3] Kendall's W is computed over only 8 models; with such a small number of ranked objects, the statistic is highly sensitive to one or two rating changes. Report the per-judge ranking table or a permutation-based p-value to support the stability claim.
  4. [General] The abstract and main results do not mention the anchoring limitation that is disclosed only in Sec. 4. Please add a brief caveat near the first presentation of the human-evaluation results, e.g., in Sec. 3.3, to avoid over-interpretation.

Circularity Check

1 steps flagged

Judge validation is circular: human ratings used Qwen3.6-27B pre-annotations, so judge–human agreement is partly by construction.

specific steps
  1. self definitional [Sec. 3.3 (Fig. 4, human evaluation) and Sec. 4 (limitation)]
    "One limitation is that our human evaluation used pre-annotations generated by Qwen3.6-27B, which may have introduced anchoring bias. Therefore, the close agreement between the scores produced by Qwen3.6-27B and the human ratings should not be interpreted as evidence that this judge is superior to the alternatives."

    Qwen3.6-27B is the automatic judge used to produce all MMAC scores in Table 2 and Fig. 3. Its validation in Fig. 4 relies on agreement with human ratings (W=0.981). But those human ratings were generated using pre-annotations from the same Qwen3.6-27B model, as the paper concedes. The human evaluation is therefore not independent of the system being validated, making the close judge–human agreement partly self-referential. The paper explicitly warns that the agreement should not be interpreted as evidence that Qwen3.6-27B is superior to the alternatives. Cross-judge agreement among the four LLM judges provides some stability evidence, but it does not establish correctness against unanchored ground truth.

full rationale

The benchmark construction is largely self-contained: MMAC labels come from source-dataset labels, acoustic measurements (F0, syllables/sec), and LLM-assisted draft labels that are human-reviewed and human quality-inspected, so the benchmark's target labels are not reducible to the judge used for scoring. The metric definitions (Coverage, Precision, Accuracy) are also independent of Qwen3.6-27B's outputs. However, the central diagnostic results are scored by Qwen3.6-27B, and the only human validation of this judge is compromised by the paper's own admission that the human annotators used Qwen3.6-27B pre-annotations. This makes the judge–human agreement (W=0.981) partly circular and leaves the judge's accuracy against true ground truth unverified. There is no load-bearing self-citation chain among the authors' prior works; the issue is specifically the self-referential validation of the automatic judge. Score 6 reflects a partial but substantive circularity in the evidence supporting the judge, while the benchmark labels and evaluation framework retain independent content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central claims rest on a small set of design choices: equal category weighting, a specific judge model, and an author-defined taxonomy. The most consequential premise is the reliability of the LLM judge, which is only weakly supported due to anchored human ratings.

free parameters (4)
  • Equal category-macro averaging = 1/6 per category
    Aggregate scores weight each of the 6 capability categories equally, chosen by the authors (Sec 3.1).
  • Binary/graded score normalization = [0,1]
    Each dimension maps to binary or graded scores, normalized before aggregation; the mapping is hand-defined (Sec 3.2).
  • Judge model Qwen3.6-27B = Qwen3.6-27B
    The automatic scorer is a single LLM, chosen without a principled selection; its reliability is validated via human eval (Sec 3.1, 3.3).
  • 10% stratified subset for human eval = 10% per dimension
    Human validation uses a 10% stratified sample; the size is a practical choice (Sec 3.3).
axioms (4)
  • domain assumption Target dimensions are clearly perceptible and labels are traceable
    Sec 2.3 states samples retained only if cue clearly perceptible and label traceable; this is assumed rather than demonstrated for every clip.
  • domain assumption Each audio clip can be assigned to a single test subset with a clear target dimension
    Sec 2.2 organizes data into subsets by capability category; clips with mixed cues may still be attributed to one dimension.
  • domain assumption Free-form captions can be reliably judged for mention and consistency by an LLM
    The evaluation assumes Qwen3.6-27B correctly maps natural-language captions to dimension-level labels; human validation is weakened by anchoring bias (Sec 4).
  • ad hoc to paper The 15 dimensions collectively cover the scope of detailed audio captioning
    The taxonomy of 6 categories/15 dimensions is defined by the authors without a formal completeness argument (Sec 2.2).
invented entities (1)
  • MMAC taxonomy: 6 capability categories and 15 evaluation dimensions no independent evidence
    purpose: Decompose audio captioning into separately measurable information types
    The dimensions are author-defined constructs; no external validation shows they form a complete or natural decomposition.

pith-pipeline@v1.3.0-daily-deepseek · 7721 in / 12638 out tokens · 93457 ms · 2026-08-01T09:56:16.019418+00:00 · methodology

0 comments
read the original abstract

With the development of audio large language models (AudioLLMs), audio captioning needs to move from brief descriptions toward open-ended and fine-grained free-form descriptions. Existing evaluations often focus on generation quality or task performance, making it difficult to diagnose information coverage and description reliability. We propose MMAC, a \textbf{M}assive \textbf{M}ulti-dimensional benchmark for \textbf{A}udio \textbf{C}aptioning. MMAC contains 5,638 audio clips from more than 20 data sources, covering 6 capability categories and 15 evaluation dimensions. Given a model-generated caption, MMAC checks whether it mentions relevant information in the target dimension and whether the mentioned content is consistent with the reference label. We evaluate representative open-source and proprietary AudioLLMs. Results show clear differences across evaluation dimensions, information coverage, and description reliability. We will release the MMAC benchmark and evaluation code.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 7 linked inside Pith

  1. [1]

    With the development of AudioLLMs [1, 2, 3, 4], captions are moving from brief descriptions to more open-ended and fine- grained audio understanding

    INTRODUCTION Audio captioning aims to convert audio signals into natural-language descriptions, so that models can summarize key information in audio with text. With the development of AudioLLMs [1, 2, 3, 4], captions are moving from brief descriptions to more open-ended and fine- grained audio understanding. In this setting, evaluation should not only co...

  2. [2]

    Overview MMAC is a fine-grained evaluation benchmark for free-form audio captioning

    MMAC 2.1. Overview MMAC is a fine-grained evaluation benchmark for free-form audio captioning. Given an audio clip, it evaluates the natural-language caption generated by an audio language model across multiple di- mensions. Instead of assigning only a holistic caption score, MMAC examines whether the caption covers target information and whether arXiv:26...

    Pith/arXiv arXiv 2026

  3. [3]

    Describe this audio in detail

    EXPERIMENTS 3.1. Experimental Setup We evaluate representative open-source and proprietary AudioLLMs on MMAC, including Qwen, Gemini, and other recent audio lan- guage models. All models receive the prompt“Describe this audio in detail. ”and generate free-form captions, except Qwen3-Omni- Captioner, whose inference interface does not support custom text p...

  4. [4]

    Fine-grained dimensions are first averaged within each capability category

    The three metrics reflect information coverage, the reliability of mentioned descriptions, and the joint effect of coverage and cor- rectness in captioning. Fine-grained dimensions are first averaged within each capability category. The resulting category scores are then equally averaged to obtain the aggregate score. 3.3. Results Table 2 reports the main...

  5. [5]

    MMAC comprises 6 capability categories and 15 fine-grained dimensions and evaluates the coverage and cor- rectness of target information in free-form captions

    CONCLUSION AND FUTURE WORK In this paper, we presented MMAC, a multi-dimensional benchmark for audio captioning. MMAC comprises 6 capability categories and 15 fine-grained dimensions and evaluates the coverage and cor- rectness of target information in free-form captions. Results reveal clear differences among AudioLLMs in information coverage and descrip...

  6. [6]

    Qwen2.5-omni tech- nical report,

    Jinzheng He Jin Xu, Zhifang Guo et al., “Qwen2.5-omni tech- nical report,”arXiv preprint arXiv:2503.20215, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Midashenglm: Efficient audio understanding with general audio captions,

    Heinrich Dinkel, Gang Li, Jizhong Liu, et al., “Midashenglm: Efficient audio understanding with general audio captions,” arXiv preprint arXiv:2508.03983, 2025

    arXiv 2025

  8. [8]

    Qwen3. 5-omni technical report,

    Qwen Team, “Qwen3. 5-omni technical report,”arXiv preprint arXiv:2604.15804, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Step-audio 2 technical report,

    Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al., “Step-audio 2 technical report,”arXiv preprint arXiv:2507.16632, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

    Xinhao Mei, Chutong Meng, Haohe Liu, et al., “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,”IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, vol. 32, pp. 3339–3354, 2024

    2024

  11. [11]

    Audiocaps: Generating captions for audios in the wild,

    Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, et al., “Audiocaps: Generating captions for audios in the wild,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Pa- pers), 2019, pp. 119–132

    2019

  12. [12]

    Clotho: An audio captioning dataset,

    Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen, “Clotho: An audio captioning dataset,” inICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP). IEEE, 2020, pp. 736–740

    2020

  13. [13]

    Mmau: A massive multi-task audio understanding and reasoning bench- mark,

    Sakshi Sakshi, Utkarsh Tyagi, Sonal Kumar, et al., “Mmau: A massive multi-task audio understanding and reasoning bench- mark,” inInternational Conference on Learning Representa- tions, 2025, vol. 2025, pp. 84929–84964

    2025

  14. [14]

    Mmar: A chal- lenging benchmark for deep reasoning in speech, audio, music, and their mix,

    Ziyang Ma, Yinghao Ma, Yanqiao Zhu, et al., “Mmar: A chal- lenging benchmark for deep reasoning in speech, audio, music, and their mix,”Advances in Neural Information Processing Systems, vol. 38, 2026

    2026

  15. [15]

    Omni- captioner: Data pipeline, models, and benchmark for omni de- tailed perception,

    Ziyang Ma, Ruiyang Xu, Zhenghao Xing, et al., “Omni- captioner: Data pipeline, models, and benchmark for omni de- tailed perception,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  16. [16]

    Audio flamingo next: Next-generation open audio-language models for speech, sound, and music,

    Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand, Zhifeng Kong, Siddharth Gu- rurani, Sang-gil Lee, Jaehyeon Kim, Aya Aljafari, et al., “Audio flamingo next: Next-generation open audio-language models for speech, sound, and music,”arXiv preprint arXiv:2604.10905, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    Secap: Speech emotion captioning with large language model,

    Yaoxun Xu, Hangting Chen, Jianwei Yu, et al., “Secap: Speech emotion captioning with large language model,” inProceed- ings of the AAAI Conference on Artificial Intelligence, 2024, vol. 38, pp. 19323–19331

    2024

  18. [18]

    MMSU: A massive multi-task spoken language understanding and reason- ing benchmark,

    Dingdong Wang, Junan Li, Jincenzi Wu, et al., “MMSU: A massive multi-task spoken language understanding and reason- ing benchmark,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  19. [19]

    Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens,

    Xinsheng Wang, Mingqi Jiang, and Ziyang others Ma, “Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens,”arXiv preprint arXiv:2503.01710, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    speechocean762: An open-source non-native english speech corpus for pronuncia- tion assessment,

    Junbo Zhang, Zhiwen Zhang, et al., “speechocean762: An open-source non-native english speech corpus for pronuncia- tion assessment,” inInterspeech, 2021, pp. 3710–3714

    2021

  21. [21]

    Childman- darin: A comprehensive mandarin speech dataset for young children aged 3-5,

    Jiaming Zhou, Shiyao Wang, Shiwan Zhao, et al., “Childman- darin: A comprehensive mandarin speech dataset for young children aged 3-5,” inProceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics, 2025, pp. 12524–12537

    2025

  22. [22]

    Seniortalk: A chinese conversation dataset with rich annotations for super- aged seniors,

    Hui Wang, Shiyao Wang, Junyang Chen, et al., “Seniortalk: A chinese conversation dataset with rich annotations for super- aged seniors,”Advances in Neural Information Processing Sys- tems, vol. 38, 2026

    2026

  23. [23]

    Gi- gaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    Guoguo Chen, Shuzhou Chai, and Guan-Bo others Wang, “Gi- gaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” inInterspeech, 2021, pp. 3670– 3674

    2021

  24. [24]

    Recent ad- vances in speech language models: A survey,

    Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, et al., “Recent ad- vances in speech language models: A survey,” inProceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), 2025, pp. 13943– 13970

    2025

  25. [25]

    Audio set: An on- tology and human-labeled dataset for audio events,

    Jort F Gemmeke, Daniel PW Ellis, et al., “Audio set: An on- tology and human-labeled dataset for audio events,” in2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 776–780

    2017

  26. [26]

    Esc: Dataset for environmental sound classi- fication,

    Karol J Piczak, “Esc: Dataset for environmental sound classi- fication,” inProceedings of the 23rd ACM international con- ference on Multimedia, 2015, pp. 1015–1018

    2015

  27. [27]

    Fsd50k: an open dataset of human-labeled sound events,

    Eduardo Fonseca, Xavier Favory, Jordi Pons, et al., “Fsd50k: an open dataset of human-labeled sound events,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 829–852, 2021

    2021

  28. [28]

    Kespeech: An open source speech dataset of mandarin and its eight sub- dialects,

    Zhiyuan Tang, Dong Wang, Yanguang Xu, et al., “Kespeech: An open source speech dataset of mandarin and its eight sub- dialects,” inThirty-fifth conference on neural information pro- cessing systems datasets and benchmarks track, 2021

    2021

  29. [29]

    Qwen3-omni technical report,

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al., “Qwen3-omni technical report,”arXiv preprint arXiv:2509.17765, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Coig-cqia: Qual- ity is all you need for chinese instruction fine-tuning,

    Yuelin Bai, Xeron Du, Yiming Liang, et al., “Coig-cqia: Qual- ity is all you need for chinese instruction fine-tuning,” inFind- ings of the Association for Computational Linguistics: NAACL 2025, 2025, pp. 8190–8205

    2025

  31. [31]

    Mustard: Mastering uniform synthesis of theorem and proof data,

    Yinya Huang, Xiaohan Lin, Zhengying Liu, et al., “Mustard: Mastering uniform synthesis of theorem and proof data,” in The Twelfth International Conference on Learning Represen- tations

  32. [32]

    Indextts2: A break- through in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech,

    Siyi Zhou, Yiquan Zhou, Yi He, et al., “Indextts2: A break- through in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech,” inProceedings of the AAAI Conference on Artificial Intelligence, 2026, vol. 40, pp. 35139–35148

    2026

  33. [33]

    The problem ofm rankings,

    M. G. Kendall and B. Babington Smith, “The problem ofm rankings,”Annals of Mathematical Statistics, vol. 10, no. 3, pp. 275–287, 1939

    1939