Pith. sign in

REVIEW 3 major objections 5 minor 40 references

Multi-TW: Benchmarking Multimodal Models on Traditional Chinese Question Answering in Taiwan

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multi-TW: the first Traditional Chinese benchmark for tri-modal question answering and latency.

desk verdict Useful dataset, thin evaluation: the image-text subset includes transcript-adapted items that may be solvable from text alone, and the paper lacks contamination checks and error bars. read the letter →

arxiv 2508.01274 v1 pith:EZKRRE2W submitted 2025-08-02 cs.AI cs.CL

Jui-Ming Yao , Bing-Cheng Xie , Sheng-Wei Peng , Hao-Yuan Chen , He-Rong Zheng , Bing-Jia Tan , Peter Shaojui Wang , Shun-Feng Su This is my paper
classification cs.AIcs.CL
keywords TraditionalChinesemultimodalbenchmarkany-to-anymodelsaudio-textQAimage-textinferencelatencymultiple-choicequestionansweringMandarinproficiencytest
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-TW is proposed as the first benchmark that evaluates multimodal models on Traditional Chinese using all three input modalities—image, audio, and text. The paper assembles 900 multiple-choice questions from official Mandarin proficiency exams, split evenly between image-text and audio-text items, and uses them to rank any-to-any models as well as vision-language models that receive audio via automatic speech recognition. Its central finding is that end-to-end any-to-any pipelines finish audio tasks much faster than the two-stage transcription-plus-VLM approach, while closed-source models generally score higher on accuracy than open-source ones. If this holds, Multi-TW gives the field a reusable yardstick for both correctness and speed in Traditional Chinese multimodal question answering.

What carries the argument

The central object is the Multi-TW dataset itself: 900 multiple-choice items serialized into a unified JSON schema (id, image or audio path, instruction, question, options, answer), with a five-part identifier encoding exam volume, section, level, part, and question index. The evaluation protocol constrains every model to output a single token, extracts the first A/B/C/D answer via regex, and measures latency only over data preprocessing plus model inference for open-source models. The comparison mechanism is the two-arm design: any-to-any models ingest image or audio directly, while VLMs receive the same audio task only after speech transcription, which makes the latency difference interpretable.

What would settle it

Run a contamination probe by giving each model only the text transcript of the 900 questions, without images or audio; if a model's accuracy on text-only versions is close to its full benchmark score, the original results were inflated by memorized public exam items rather than genuine multimodal understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that no prior benchmark covered text, image, and audio in Traditional Chinese, and that Multi-TW fills this gap with 900 real exam items (450 image-text, 450 audio-text). On this benchmark, the paper reports that closed-source any-to-any models outperform open-source models overall, but open-source models can excel on audio-text tasks, with one open-source any-to-any model reaching 0.8911 audio accuracy despite a much weaker image-text score. The latency measurement shows open-source any-to-any models complete the full 900-item run in 467–744 seconds, while VLM-plus-ASR pipelines take 1,187–2,131 seconds; the paper reads this as evidence that end-to-end any-to-any architectures offer a clear latency advantage over cascaded transcription pipelines for audio input.

Load-bearing premise

The evaluation gives trustworthy rankings only if the public exam questions used in Multi-TW were not already memorized by the evaluated models.

Editorial extensions

If this is right

  • Future work on Traditional Chinese multimodal models will have a public 900-item reference set with balanced image and audio coverage.
  • End-to-end any-to-any architectures become the faster choice over VLM-plus-ASR pipelines when audio latency matters, since they finished the 900-item run in 467–744 seconds versus 1,187–2,131 seconds.
  • Open-source any-to-any models trained mainly on Simplified Chinese transfer well to Traditional Chinese audio comprehension but not to image-text tasks, so fine-tuning effort should target the vision side.
  • Audio evaluation becomes more demanding: Multi-TW clips average 107.5 seconds, about twelve times longer than prior English tri-modal benchmark clips.
  • Latency is treated as a first-class metric alongside exact-match accuracy in a benchmark, not just an incidental measurement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the public origin of the exam items means the reported accuracy could partly reflect memorization, so a private held-out split is needed for durable model rankings.
  • Editorial inference: the latency comparison was run on one GPU with a fixed one-token output; batched or streaming speech recognition could shrink the gap, so the advantage is setting-specific until re-measured.
  • Editorial inference: a natural follow-up is to feed the same audio transcripts directly to a text-only model; if accuracy stays high, these tasks test text comprehension more than audio encoding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces Multi-TW, a benchmark of 900 multiple-choice questions in Traditional Chinese, split evenly between 450 image-text and 450 audio-text items, sourced from official SC-TOP proficiency exams. The authors evaluate several any-to-any multimodal models (e.g., Qwen2.5-Omni, Baichuan-Omni-1.5, UnifiedIO-2) and several vision-language models with Whisper-based audio transcription, reporting accuracy and inference latency. The main findings are that closed-source models (Gemini variants) achieve the highest accuracy, open-source models perform comparatively better on audio than on image tasks, and end-to-end any-to-any pipelines are substantially faster than cascaded VLM+ASR pipelines.

Significance. The benchmark addresses a genuine gap: no existing dataset evaluates both vision and audio understanding in Traditional Chinese, and latency is rarely considered. The use of official exam materials with external answer keys provides an objective ground truth, and the balanced 450/450 design allows direct comparison of visual and auditory performance. The audio clips are long (average 107.5s), enabling evaluation of long-form listening comprehension. If the validity issues around adapted items and training-data contamination are resolved, and if the evaluation is made statistically robust, Multi-TW would be a useful resource for the community. The latency comparison, while preliminary, is a valuable addition.

major comments (3)
  1. [Section 2.1 (Data Construction)] The construction process states that 'some questions originally coupling image and audio were adapted by extracting their ground-truth audio transcripts, which were then paired with the corresponding image as the textual component.' For typical listening-comprehension items, the answer is derivable from the transcript alone, making the image superfluous. The paper does not report the number of such adapted items, nor does it provide a text-only baseline or an image-ablation analysis. Without this information, the image-text accuracy numbers in Tables 3 and 4 cannot be interpreted as measuring visual understanding, and the central claim that Multi-TW evaluates visual, textual, and auditory modalities is not substantiated. Please report the count of adapted items, measure accuracy with and without the image, and quantify the information gain contributed by the image.
  2. [Section 3.1 and Tables 3-4] The evaluation is single-run and reports only point estimates. For a 900-item benchmark, the standard error of an accuracy near 0.65 is about 0.016, so the observed difference between Qwen2.5-Omni-7B (0.6534) and Baichuan-Omni-1.5 (0.6289) is within roughly 1.5 standard errors and is not established as significant. The random fallback for unparseable outputs described in Section 3.1 adds further noise, and the fallback rate per model is not reported. The paper's comparative claims (e.g., 'closed-source models generally outperform open-source ones' and latency advantages) require confidence intervals, significance tests, or at least a statement of variance, and the results should be compared against chance accuracy (33.3% for 3-choice items, 25% for 4-choice items).
  3. [Section 2.2 (Data Sourcing) and Section 3.1 (Zero-Shot Evaluation)] The benchmark items are drawn from official, publicly available SC-TOP proficiency exams. These materials may be present in the web-scale pretraining corpora of the evaluated models. The paper provides no contamination check or discussion. If items are memorized, the reported zero-shot accuracies are inflated and the relative model rankings are distorted. Please conduct a contamination analysis (e.g., n-gram overlap between the benchmark items and known training corpora where accessible, or evaluation on a recently created private held-out set) and discuss the potential impact on the conclusions, particularly for the top-scoring models.
minor comments (5)
  1. [Section 2.2] The text reads 'audio-based items, ' with an unmatched closing quote; the punctuation should be corrected to avoid ambiguity.
  2. [Section 3.1] The term 'exact-match accuracy' is misleading for multiple-choice tasks; consider using 'option selection accuracy' instead.
  3. [Section 1] The reference to Vicuna cites the MT-Bench paper [38] rather than the Vicuna technical report; please cite the original Vicuna source.
  4. [Section 2.1] The specification '150 dpi PNG' does not convey the actual pixel dimensions of the images; reporting resolution (e.g., width × height) would improve reproducibility.
  5. [Section 4] The statement that closed-source models 'generally exhibit higher end-to-end latency in practice for batch processing due to network factors' is speculative and not measured; it should be removed or supported with data.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Multi-TW's ground truth is external, no parameters are fitted to model outputs, and the latency comparison is direct empirical measurement.

full rationale

The paper's central artifacts are the benchmark dataset and the accuracy/latency comparison. The dataset labels come from official SC-TOP proficiency-test answer keys, which are external to the paper and fixed before any model is run; Section 2.1 describes curation and quality-control steps but no fitting of labels to model behavior. The evaluation in Section 3.1 uses all 900 questions zero-shot with exact-match accuracy, so no model output is folded back into the benchmark construction. The adapted image-plus-audio items, in which ground-truth audio transcripts are paired with images as text (Section 2.1), are a possible validity concern because some items may be solvable from the transcript alone, but that is a content-validity threat, not circularity: the answer key still comes from the external exam, and the item does not define its target result in terms of its own output. The latency comparison (Section 4) directly times the open-source any-to-any pipelines and the VLM-plus-ASR cascade under a fixed one-token generation length; the conclusion that any-to-any pipelines are faster is an empirical observation, not a quantity reconstructed from the inputs. No load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation are present; citations to prior benchmarks (OmniBench, VisTW, TMMLU+) are only comparative context. Accordingly, the paper has no circular derivation chain.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central contribution is a dataset, so no free parameters or invented entities appear. The three axioms are the unverified premises the benchmark's validity rests on.

assumptions (3)
  • domain assumption SC-TOP official answer keys are correct ground truth.
    Used to score all 900 questions; the paper does not independently validate answers beyond annotator checks.
  • domain assumption Publicly available SC-TOP items are not memorized by evaluated models.
    No contamination analysis is provided; the exams are public and may appear in training data.
  • domain assumption Single-letter output with regex extraction reflects model competence.
    Evaluation relies on the first A/B/C/D token and falls back to random choices if none is found; this may misrepresent models that respond with explanations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-TW: Benchmarking Multimodal Models on Traditional Chinese Question Answering in Taiwan." pith.science (2026). https://pith.science/paper/EZKRRE2W

@misc{pith2026250801274,
  author       = {Pith},
  title        = {Pith review of: Multi-TW: Benchmarking Multimodal Models on Traditional Chinese Question Answering in Taiwan},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZKRRE2W}},
  note         = {Machine review of arXiv:2508.01274}
}
read the original abstract

Multimodal Large Language Models (MLLMs) process visual, acoustic, and textual inputs, addressing the limitations of single-modality LLMs. However, existing benchmarks often overlook tri-modal evaluation in Traditional Chinese and do not consider inference latency. To address this, we introduce Multi-TW, the first Traditional Chinese benchmark for evaluating the performance and latency of any-to-any multimodal models. Multi-TW includes 900 multiple-choice questions (image and text, audio and text pairs) sourced from official proficiency tests developed with the Steering Committee for the Test of Proficiency-Huayu (SC-TOP). We evaluated various any-to-any models and vision-language models (VLMs) with audio transcription. Our results show that closed-source models generally outperform open-source ones across modalities, although open-source models can perform well in audio tasks. End-to-end any-to-any pipelines offer clear latency advantages compared to VLMs using separate audio transcription. Multi-TW presents a comprehensive view of model capabilities and highlights the need for Traditional Chinese fine-tuning and efficient multimodal architectures.

Figures

Figures reproduced from arXiv: 2508.01274 by the authors.

Figure 1
Figure 1. Illustration of data collection interface. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of samples from the Multi-TW dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Distribution of question types in Multi-TW. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 7 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski,...

    arXiv 2022

  2. [2]

    Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H

    Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier- Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebas- tian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujin...

    arXiv 2023

  3. [3]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical Rep...

    arXiv 2025

  5. [5]

    Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2023. AudioLM: a Language Modeling Approach to Audio Generation. arXiv:2209.03143 [cs.SD] https://arxiv.org/abs/2209.03143

    arXiv 2023

  6. [6]

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou

  7. [7]

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-Audio: Advancing Univer- sal Audio Understanding via Unified Large-Scale Audio-Language Models. arXiv:2311.07919 [eess.AS] https://arxiv.org/abs/2311.07919 Multi-TW: Benchmarking Multimodal Models on Traditional Chinese Question Answering in ...

    arXiv 2023

  8. [8]

    Chan-Jan Hsu, Chang-Le Liu, Feng-Ting Liao, Po-Chun Hsu, Yi-Chang Chen, and Da shan Shiu. 2023. Advancing the Evaluation of Traditional Chinese Language Models: Towards a Comprehensive Benchmark Suite. arXiv:2309.08448 [cs.CL] https://arxiv.org/abs/2309.08448

    work page Pith review arXiv 2023

Show all 40 references
  1. [9]

    Rafiul Biswas, Wajdi Zaghouani, Samir Brahim Belhaouari, and Zubair Shah

    Ashhadul Islam, Md. Rafiul Biswas, Wajdi Zaghouani, Samir Brahim Belhaouari, and Zubair Shah. 2023. Pushing Boundaries: Exploring Zero Shot Object Clas- sification with Large Multimodal Models. arXiv:2401.00127 [cs.CV] https: //arxiv.org/abs/2401.00127

    2023 arXiv

  2. [10]

    Schuller

    Siddique Latif, Moazzam Shoukat, Fahad Shamshad, Muhammad Usama, Yi Ren, Heriberto Cuayáhuitl, Wenwu Wang, Xulong Zhang, Roberto Togneri, Erik Cam- bria, and Björn W. Schuller. 2023. Sparks of Large Audio Models: A Survey and Outlook. arXiv:2308.12792 [cs.SD] https://arxiv.org...

    2023 arXiv

  3. [11]

    Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. What matters when building vision-language models? arXiv:2405.02246 [cs.CV] https: //arxiv.org/abs/2405.02246

    2024 arXiv

  4. [12]

    Yadong Li, Jun Liu, Tao Zhang, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, Chong Li, Yuanbo Fang, Dongdong Kuang, Mingrui Wang, Chenglin Zhu, Youwei Zhang, Hongyu Guo, Fengyu Zhang, Yuran Wang, Bowen Ding, Wei Song, Xu Li, Yuq...

    2025

  5. [13]

    Yizhi Li, Ge Zhang, Yinghao Ma, Ruibin Yuan, Kang Zhu, Hangyu Guo, Yiming Liang, Jiaheng Liu, Zekun Wang, Jian Yang, Siwei Wu, Xingwei Qu, Jinjie Shi, Xinyue Zhang, Zhenzhu Yang, Xiangzhou Wang, Zhaoxiang Zhang, Zachary Liu, Emmanouil Benetos, Wenhao Huang, and Chenghua Lin. 2...

    2025

  6. [14]

    Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. 2025. A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges. arXiv:2501.02189 [cs.CV] https://arxiv. org/abs/2501.02189

    2025 arXiv

  7. [15]

    Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. 2024. A Survey of Multimodel Large Language Models. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering (Xi’ an, China) (CAICE ’24). A...

    2024

  8. [16]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved Baselines with Visual Instruction Tuning. arXiv:2310.03744 [cs.CV] https://arxiv.org/abs/ 2310.03744

    2024 arXiv

  9. [17]

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. 2023. Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action. arXiv:2312.17172 [cs.CV] https://arxiv.org/abs/2312.17172v1

    2023 arXiv

  10. [18]

    Meta. 2024. Meta Llama 3.2-11B Vision Instruct. https://huggingface.co/meta- llama/Llama-3.2-11B-Vision-Instruct. Accessed: 2025-05-25

    2024

  11. [19]

    Jing Peng, Yucheng Wang, Yu Xi, Xu Li, Xizhuo Zhang, and Kai Yu. 2025. A Survey on Speech Large Language Models. arXiv:2410.18908 [eess.AS] https: //arxiv.org/abs/2410.18908

    2025

  12. [20]

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng, C...

    2025 arXiv

  13. [21]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  14. [22]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  15. [23]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. arXiv:2212.04356 [eess.AS] https://arxiv.org/abs/2212.04356

    2022 arXiv

  16. [24]

    Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, Hannah Muckenhirn, Dirk Padfield, James Qin, Danny Rozenberg, Tara Sainath, Johan Schalkwyk, Matt Sharif...

    2023 arXiv

  17. [25]

    Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sher- bondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sa- har Kazemzadeh, Thomas Mesnard, Ibrahim Alabdulmohsin...

    2024 arXiv

  18. [26]

    Zhi-Rui Tam, Ya-Ting Pai, Yen-Wei Lee, Jun-Da Chen, Wei-Min Chu, Sega Cheng, and Hong-Han Shuai. 2024. An Improved Traditional Chinese Evaluation Suite for Foundation Model. arXiv:2403.01858 [cs.CL] https://arxiv.org/abs/2403.01858

    2024 arXiv

  19. [27]

    Zhi Rui Tam, Ya-Ting Pai, Yen-Wei Lee, and Yun-Nung Chen. 2025. VisTW: Benchmarking Vision-Language Models for Traditional Chinese in Taiwan. arXiv:2503.10427 [cs.CL] https://arxiv.org/abs/2503.10427v2

    2025 arXiv

  20. [28]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpaca: A Strong, Replicable Instruction-Following Model. https://crfm.stanford.edu/2023/03/13/ alpaca.html

    2023

  21. [29]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  22. [30]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy...

    2023 arXiv

  23. [31]

    Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Aman- deep Kumar, Kartik Kuckreja, Mykola Maslych, Wafa Al Ghallabi, Mihail Mi- haylov, Chao Qin, Abdelrahman M Shaker, Mike Zhang, Maha...

  24. [32]

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. 2024. NExT- GPT: Any-to-Any Multimodal LLM. arXiv:2309.05519 [cs.AI] https://arxiv.org/ abs/2309.05519

    2024 arXiv

  25. [33]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025. Qwen2.5-Omni Technical Report. arXiv:2503.20215 [cs.CL] https://arxiv.org/abs/2503.20215v1

    2025 arXiv

  26. [34]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. arXiv:2010.11934 [cs.CL] https://arxiv.org/ abs/2010.11934 Preprint, August, 2025 Yao, et al

    2021 arXiv

  27. [35]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jin...

    2024 arXiv

  28. [36]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A Survey on Multimodal Large Language Models. doi:10.1093/nsr/ nwae403 arXiv:2306.13549 [cs.CV]

    2024 arXiv

  29. [37]

    Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yugang Jiang, and Xipeng Qiu. 2024. AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling. arXiv:2402.12226 ...

    2024 arXiv

  30. [38]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685 [cs.CL] https://ar...

    2023 arXiv

  31. [2024]

    arXiv:2407.10759 [eess.AS] https://arxiv

    Qwen2-Audio Technical Report. arXiv:2407.10759 [eess.AS] https://arxiv. org/abs/2407.10759

    arXiv

  32. [2025]

    arXiv:2411.16508 [cs.CV] https://arxiv.org/abs/2411.16508

    All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Lan- guages. arXiv:2411.16508 [cs.CV] https://arxiv.org/abs/2411.16508

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.