Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Where a teacher's knowledge is placed in an encoder-free audio captioning model matters as much as its presence: routing perceptual stages to the projector and semantic stages to the LLM recovers most of the lost performance without an enco

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 16:19 UTC pith:UBETZXRP

load-bearing objection Placement of teacher stages (early o projector, late o LLM) is the real result, and the ablations make it stick for encoder-free AAC. the 3 major comments →

arxiv 2607.04619 v1 pith:UBETZXRP submitted 2026-07-06 cs.SD cs.CL

CARD: Cross-component Audio Representation Distillation for Encoder-Free Audio Captioning

Ganesh Pavan Kartikeya Bharadwaj Kolluri , Yuchen Zhang , Michael Kampouridis , Ravi Shekhar This is my paper
classification cs.SD cs.CL
keywords audio captioningencoder-free modelscross-component distillationaudio-language modelsknowledge distillationCLAPLoRAHTSAT
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern audio captioning systems keep a frozen audio encoder at inference, paying its cost and limiting the language model to that encoder's fixed features. CARD removes the encoder after training by distilling a pretrained CLAP-HTSAT teacher into a small audio projector plus a LoRA-adapted LLM, then discarding the teacher. The decisive move is not merely using the teacher, but routing its early perceptual stages to supervise the projector and its later semantic stages to supervise the language model. That cross-component placement raises CIDEr-D by roughly twelve points on AudioCaps and five on Clotho over LLM-only distillation, reaching 55.4 against a 66.4 encoder-kept upper bound. A reader who cares about efficient multimodal systems learns that matching teacher hierarchy to student roles can close much of the encoder-free gap.

Core claim

The paper establishes that for encoder-free automated audio captioning, the placement of teacher representations across student components is as important as the presence of the teacher itself. Routing early CLAP-HTSAT stages into the audio projector and later stages into the LoRA-adapted LLM substantially outperforms LLM-only or mismatched projector distillation, yielding 55.4 CIDEr-D on AudioCaps and 27.5 on Clotho with no encoder retained at inference.

What carries the argument

Cross-component distillation: the four hierarchical stages of a frozen CLAP-HTSAT teacher are assigned by functional role—early perceptual stages supervise the mean-pooled projector via linear heads and cosine loss, later semantic stages supervise averaged hidden states from the LLM's lower blocks—so each student component receives the abstraction level suited to its job; teacher and heads are removed after training.

Load-bearing premise

The claim rests on treating the teacher's early layers as naturally perceptual and best for the projector, its later layers as semantic and best for the language model, and mean-pooled cosine matching as the right transfer signal for each.

What would settle it

If reversing the stage assignment (early stages to the LLM, late stages to the projector) or swapping in a teacher without a clear hierarchy produced equal or better caption metrics than CARD's routing on the same splits, the placement principle would be falsified for this setting.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Encoder-free AAC can close much of the gap to encoder-kept systems by distributing teacher knowledge rather than injecting it only into the LLM.
  • Projector-level supervision with early teacher stages is necessary; LLM-only distillation yields almost no gain over no distillation.
  • Two-phase training—distillation pretraining followed by caption-only fine-tuning—is complementary and required for strong results.
  • At deployment the system collapses to a lightweight projector plus a merged LoRA LLM, eliminating encoder inference cost and memory.
  • Matching teacher hierarchy to student roles shows that where knowledge is injected is a first-order design choice for encoder-free multimodal models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same early-to-frontend / late-to-backbone routing pattern may transfer to other encoder-free modalities that currently distill only into the language model.
  • If the perceptual-versus-semantic partition is teacher-specific, new teachers may need re-mapped stage assignments rather than a universal early/late rule.
  • Mean-pooled cosine alignment may leave spatial spectrogram structure unused; token- or patch-wise alignment could narrow the remaining gap to the encoder upper bound.
  • Unsupervised projectors appear to be a hidden bottleneck in many encoder-free multimodal LLM designs, not only in audio captioning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes CARD, an encoder-free automated audio captioning system that discards the audio encoder at inference. A lightweight (13.2M) projector maps log-Mel spectrograms into tokens for a LoRA-adapted Qwen3-4B LLM; after training, LoRA is merged and the teacher is dropped. During training a frozen CLAP-HTSAT teacher supplies hierarchical representations that are routed by role: early (perceptual) stages supervise the projector via mean-pooled cosine distillation heads, while later (semantic) stages supervise lower LLM blocks. Two-phase training (large-scale mixed distillation + caption CE, then dataset-specific caption fine-tuning) is used. On AudioCaps and Clotho, controlled ablations (Table II) show that this cross-component placement substantially outperforms LLM-only distillation and no-distillation baselines, reaching 55.4 CIDEr-D on AudioCaps versus 43.5 for LLM-only and 66.4 for an encoder-kept upper bound.

Significance. If the placement result holds, the paper supplies a concrete and useful design principle for encoder-free audio–LLM systems: teacher knowledge should be matched to student component function rather than injected only into the LLM. The contribution is primarily empirical and architectural. Strengths include a clean set of ablations that isolate placement (No Distill, LLM Distill, Proj Full, Proj Early, CARD⋄, CARD∗), complementary checks of LoRA capacity and two-phase training (Tables III–IV), and a clear inference-time architecture with no residual teacher or distillation heads. The work is timely relative to concurrent encoder-free audio efforts (e.g., AuRA) and correctly emphasizes that projector supervision has been under-studied. The remaining gap to the encoder-kept bound and the single teacher–backbone pair limit how far the principle can yet be generalized, but the reported evidence is sufficient to make the placement claim interesting and actionable for the community.

major comments (3)
  1. Table II (and the abstract’s +12.18 / +5.21 CIDEr-D claims): all metrics appear to be single-run point estimates with no multi-seed means, standard deviations, or significance tests. Because the central claim is a ranking among distillation placements, and the headline gap is large, the absence of variance estimates is load-bearing for confidence in the ordering (especially the smaller Clotho margins and the Proj Full vs Proj Early contrast). At minimum, multi-seed averages or bootstrap intervals on the key variants (No Distill, LLM Distill, Proj Early, CARD∗) should be reported.
  2. §III-B and the abstract claim that “where a teacher’s knowledge is placed matters as much as its presence.” The early/late HTSAT partition is empirically supported within this setup by Proj Full vs Proj Early, but the paper evaluates only one teacher hierarchy (CLAP-HTSAT) and one student backbone (Qwen3-4B + the described projector). Without at least one additional teacher architecture or LLM, it remains unclear whether the perceptual/semantic routing rule is a general principle or specific to this pair. A second teacher or a brief cross-backbone check would substantially strengthen the general claim.
  3. Introduction and abstract motivate encoder removal primarily by inference cost and memory, yet §IV reports no latency, peak memory, or FLOPs comparison between CARD and the encoder-kept SLAM-AAC baseline (same CLAP teacher, same LLM). The efficiency claim is therefore asserted rather than measured. A short table of inference cost (e.g., ms/clip and GPU memory on a fixed batch size) is needed to substantiate the practical advantage that justifies the encoder-free design.
minor comments (6)
  1. Figure 1 labels “Perceptual Distillation” on blocks 9–11 and “Semantic Distillation” on blocks 0–8, which appears swapped relative to the text in §III-B (early stages → projector; later stages → LLM blocks 0–11). Please align figure labels with the equations.
  2. §III-A: the projector is described as “two strided 1-D convolutional layers” projecting 64 → 512 → 2560, yet the abstract states a “13.2M projector.” A one-line parameter breakdown (conv + linear + LN) would help readers verify the count.
  3. Table I Phase-2 sample counts (AudioCaps 225,890; Clotho 95,975) are much larger than the original training sets; the text mentions “prompt-expanded” sets following SLAM-AAC, but the expansion procedure is not summarized. A brief note or citation detail would improve reproducibility.
  4. Eqs. (1)–(2): mean-pooling is used for both projector and LLM audio-token positions before the linear heads. A short justification (or a brief ablation of mean vs. other pooling) would clarify why this is appropriate for variable-length audio tokens.
  5. Related work: AuRA is correctly noted as concurrent and LLM-side only; a sentence contrasting AAC (open-vocabulary environmental events) with AuRA’s speech/ASR focus would further sharpen the positioning.
  6. Minor wording: “Blocks 0-8 / 9-11” indexing should be stated relative to the 36-block Qwen3-4B (0-indexed) so readers can reproduce the split exactly.

Circularity Check

0 steps flagged

No significant circularity: claims rest on empirical ablations and held-out CIDEr/SPIDEr metrics, not on definitions or self-fitted targets.

full rationale

CARD's central claim (placement of teacher stages matters as much as presence) is supported by controlled variants in Table II (No Distill 43.2, LLM Distill 43.5, Proj Full 40.7, Proj Early 52.5, CARD* 55.4 CIDEr-D on AudioCaps; parallel Clotho pattern) evaluated on standard held-out splits. Distillation (Eqs. 1-5: mean-pooled cosine on HTSAT stages t0-t3 routed by functional role) and caption CE (Eq. 6) are independent training objectives; the reported metrics are not algebraic rearrangements of those losses or of any fitted parameter. The early/perceptual vs late/semantic partition is a design choice that is itself ablated (Proj Full hurts relative to Proj Early), not smuggled in as a uniqueness theorem or self-citation. No self-definitional loops, no fitted-input-as-prediction, no load-bearing self-citations, and no renaming of known results. The derivation chain is ordinary empirical ML: train under different supervision placements, measure external caption quality. Score 0 is therefore the correct, non-manufactured outcome.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 1 invented entities

The central claim rests on standard ML training machinery plus a domain assumption that hierarchical CLAP stages align with student component roles, plus several hand-chosen hyperparameters (stage split, block ranges, λ, LoRA rank). No new physical entities; the ‘cross-component’ framework is a methodological construct validated by ablation rather than an unfalsifiable postulate.

free parameters (5)
  • λ_proj and λ_llm
    Fixed to 1.0 with no sweep reported; they balance caption CE against distillation and directly affect Phase-1 optimization (Eq. 7).
  • LoRA rank r and α
    Chosen as r=16, α=16 after a small grid (Table III); capacity interacts with the distillation claim.
  • LLM supervision block ranges (0–8 and 9–11)
    Hand partition of the lower 12 of 36 Qwen3 blocks for matching t2/t3; not derived from a principle beyond ‘lower blocks’.
  • Teacher stage split (t0–t1 projector, t2–t3 LLM)
    Design choice labeled perceptual vs semantic; ablated against full-stage projector but still a free architectural parameter of the method.
  • Phase-1 learning rate and schedule
    2e-4 AdamW with constant-with-warmup, batch 64, one epoch; standard but fitted to this setup.
axioms (4)
  • domain assumption CLAP-HTSAT intermediate stages form a hierarchy of increasing semantic abstraction useful for captioning.
    Invoked in Section III-B to justify routing early stages to the projector and later stages to the LLM; supported by architecture literature but not re-proven here.
  • domain assumption Mean-pooled cosine distillation (Eq. 3) is a sufficient representation-level transfer signal for both projector and LLM audio-token positions.
    Standard KD choice; alternatives (attention matching, MSE, layer-wise full sequences) are not compared.
  • domain assumption After Phase 1, discarding teacher and distillation heads and fine-tuning with caption loss alone preserves transferred acoustic knowledge.
    Core encoder-free premise (Section III-C); Table IV shows Phase 1+2 is needed, but long-term retention is assumed.
  • standard math Standard autoregressive cross-entropy on captions plus public AAC metrics (CIDEr-D, SPIDEr, SPICE, METEOR) measure caption quality.
    Community evaluation protocol used throughout Section IV.
invented entities (1)
  • Cross-component distillation framework (CARD routing) no independent evidence
    purpose: Assign teacher stages to student components by functional role rather than injecting all supervision into the LLM.
    Methodological construct introduced in Section III-B; independent evidence is the ablation table itself, not an external physical prediction.

pith-pipeline@v1.1.0-grok45 · 17087 in / 3225 out tokens · 32227 ms · 2026-07-11T16:19:13.743959+00:00 · methodology

0 comments
read the original abstract

Modern automated audio captioning systems pair a frozen audio encoder with a large language model (LLM) via a trainable projector, incurring the encoder's inference cost and bottlenecking the model through its fixed acoustic features. We present CARD, an encoder-free audio captioning model that removes the encoder at inference: a 13.2M projector feeds a frozen LLM with merged LoRA adapters, while the teacher used to train it is discarded. CARD distills a pretrained audio teacher (CLAP-HTSAT) into the model, but rather than injecting it into the LLM alone, it routes the teacher's representations across components: perceptual stages to the projector and semantic stages to the LLM. This placement improves CIDEr-D by +12.18 over an LLM-only distilled model on AudioCaps and by +5.21 on Clotho, reaching 55.4 against a 66.4 encoder-kept upper bound with no encoder at inference, showing that where a teacher's knowledge is placed matters as much as its presence.

Figures

Figures reproduced from arXiv: 2607.04619 by Ganesh Pavan Kartikeya Bharadwaj Kolluri, Michael Kampouridis, Ravi Shekhar, Yuchen Zhang.

Figure 1
Figure 1. Figure 1: Overview of CARD. During training (left), a frozen CLAP teacher supervises the two student components by role: its early, perceptual stages (0-1) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 5 canonical work pages

  1. [1]

    AudioCaps: Generating captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generating captions for audios in the wild,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V olume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association ...

    2019

  2. [2]

    Clotho: an audio captioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: an audio captioning dataset,” in2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020. IEEE, 2020, pp. 736–740. [Online]. Available: https://doi.org/10.1109/ICASSP40776.2020.9052990

    doi:10.1109/icassp40776.2020.9052990 2020

  3. [3]

    CLAP learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. A. Ismail, and H. Wang, “CLAP learning audio concepts from natural language supervision,” inIEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023. IEEE, 2023, pp. 1–5. [Online]. Available: https://doi.org/10.1109/ICASSP49357.20 23.10095889

    work page doi:10.1109/icassp49357.20 2023

  4. [4]

    Enclap: Combining neural audio codec and audio-text joint embedding for automated audio captioning,

    J. Kim, J. Jung, J. Lee, and S. H. Woo, “Enclap: Combining neural audio codec and audio-text joint embedding for automated audio captioning,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19, 2024. IEEE, 2024, pp. 6735–6739. [Online]. Available: https://doi.org/10.1109/ICASSP484...

    doi:10.1109/icassp48485.2024.10446672 2024

  5. [5]

    SLAM-AAC: enhancing audio captioning with paraphrasing augmentation and clap-refine through llms,

    W. Chen, Z. Ma, X. Li, X. Xu, Y . Liang, Z. Zheng, K. Yu, and X. Chen, “SLAM-AAC: enhancing audio captioning with paraphrasing augmentation and clap-refine through llms,” in2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6-11, 2025. IEEE, 2025, pp. 1–5. [Online]. Available: https://doi.or...

    doi:10.1109/icassp49660.2025.10889071 2025

  6. [6]

    Drcap: Decoding CLAP latents with retrieval-augmented generation for zero-shot audio captioning,

    X. Li, W. Chen, Z. Ma, X. Xu, Y . Liang, Z. Zheng, Q. Kong, and X. Chen, “Drcap: Decoding CLAP latents with retrieval-augmented generation for zero-shot audio captioning,” in2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6-11, 2025. IEEE, 2025, pp. 1–5. [Online]. Available: https://doi.o...

    doi:10.1109/icassp49660.2025.10890325 2025

  7. [7]

    Unveiling encoder-free vision-language models,

    H. Diao, Y . Cui, X. Li, Y . Wang, H. Lu, and X. Wang, “Unveiling encoder-free vision-language models,”ArXiv, vol. abs/2406.11832,

    Pith/arXiv arXiv

  8. [8]

    Available: https://api.semanticscholar.org/CorpusID: 270559398

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 270559398

  9. [9]

    Vision as lora,

    H. Wang, Y . Ye, B. Li, Y . Nie, J. Lu, J. Tang, Y . Wang, and C. Huang, “Vision as lora,”ArXiv, vol. abs/2503.20680, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID:277322798

    Pith/arXiv arXiv 2025

  10. [10]

    Aura: Internalizing audio understanding into llms as lora,

    B. Cheng, L. Shi, Z. Ma, Y . Wu, J. Xu, J. Gao, J. Hao, and R. He, “Aura: Internalizing audio understanding into llms as lora,” 2026. [Online]. Available: https://api.semanticscholar.org/CorpusID:289131922

    2026

  11. [11]

    Enhancing automated audio captioning via large language models with optimized audio encoding,

    J. Liu, G. Li, J. Zhang, H. Dinkel, Y . Wang, Z. Yan, Y . Wang, and B. Wang, “Enhancing automated audio captioning via large language models with optimized audio encoding,”ArXiv, vol. abs/2406.13275,

    Pith/arXiv arXiv

  12. [12]

    Available: https://api.semanticscholar.org/CorpusID: 270621008

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 270621008

  13. [13]

    CED: consistent ensemble distillation for audio tagging,

    H. Dinkel, Y . Wang, Z. Yan, J. Zhang, and Y . Wang, “CED: consistent ensemble distillation for audio tagging,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19, 2024. IEEE, 2024, pp. 291–295. [Online]. Available: https://doi.org/10.1109/ICASSP48485.20 24.10446348

    work page doi:10.1109/icassp48485.20 2024

  14. [14]

    Llm can read spectrogram: Encoder-free speech-language modeling,

    R. Fan, Y . Wang, Y . Hu, B. Ren, Y . Xia, X. Wang, Y . Qian, and J. Li, “Llm can read spectrogram: Encoder-free speech-language modeling,”

  15. [15]

    Available: https://api.semanticscholar.org/CorpusID: 289132537

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 289132537

  16. [16]

    Fuyu-8B: A multimodal architecture for AI agents,

    Adept AI, “Fuyu-8B: A multimodal architecture for AI agents,” https: //www.adept.ai/blog/fuyu-8b/, 2023, blog post

    2023

  17. [17]

    Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre- training,

    G. Luo, X. Yang, W. Dou, Z. Wang, J. Dai, Y . Qiao, and X. Zhu, “Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre- training,”2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24 960–24 971, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:273233458

    2025

  18. [18]

    Gemma 4 model card,

    Google DeepMind, “Gemma 4 model card,” https://developers.googleb log.com/gemma-4-12b-the-developer-guide/, 2026, online

    2026

  19. [19]

    Distilling the knowledge in a neural network,

    G. E. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”CoRR, vol. abs/1503.02531, 2015. [Online]. Available: http://arxiv.org/abs/1503.02531

    Pith/arXiv arXiv 2015

  20. [20]

    Minillm: On-policy distillation of large language models,

    Y . Gu, L. Dong, F. Wei, and M. Huang, “Minillm: On-policy distillation of large language models,” 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:259164722

    2023

  21. [21]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” inThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  23. [23]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” inIEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023. IEEE, 2023, pp. 1–5. [Online]. Available: https...

    doi:10.1109/icassp49357.2023.10095969 2023

  24. [24]

    HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection,

    K. Chen, X. Du, B. Zhu, Z. Ma, T. Berg-Kirkpatrick, and S. Dubnov, “HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23-27 May 2022. IEEE, 2022, pp. 646–650. [Online]. Available: https://doi.org/10...

    doi:10.1109/icassp43922.2022.9746312 2022

  25. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. IEEE, 2021, pp. 9992–10 002. [Online]. Available: https://doi.org/10.1109/ICCV48922.2021.00986

    doi:10.1109/iccv48922.2021.00986 2021

  26. [26]

    Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

    X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 32, pp. 3339–3354,

  27. [27]

    Available: https://doi.org/10.1109/TASLP.2024.3419446

    [Online]. Available: https://doi.org/10.1109/TASLP.2024.3419446

    doi:10.1109/taslp.2024.3419446 2024

  28. [28]

    Auto-acd: A large-scale dataset for audio-language representation learning,

    L. Sun, X. Xu, M. Wu, and W. Xie, “Auto-acd: A large-scale dataset for audio-language representation learning,” inProceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 - 1 November 2024, J. Cai, M. S. Kankanhalli, B. Prabhakaran, S. Boll, R. Subramanian, L. Zheng, V . K. Singh, P. C´esar, L...

    work page doi:10.1145/3664647.3681472 2024

  29. [29]

    Macs - multi-annotator captioned soundscapes,

    I. M. Morat ´o and A. Mesaros, “Macs - multi-annotator captioned soundscapes,” 2021. [Online]. Available: https://api.semanticscholar.or g/CorpusID:244972894

    2021

  30. [30]

    Infinity instruct: Scaling instruction selection and synthesis to enhance language models,

    J. Li, L. Du, H. Zhao, B. wen Zhang, L. Wang, B. Gao, G. Liu, and Y . Lin, “Infinity instruct: Scaling instruction selection and synthesis to enhance language models,”ArXiv, vol. abs/2506.11116, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID:279391576

    Pith/arXiv arXiv 2025

  31. [31]

    Cider: Consensus-based image description evaluation,

    R. Vedantam, C. L. Zitnick, and D. Parikh, “Cider: Consensus-based image description evaluation,” inIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. IEEE Computer Society, 2015, pp. 4566–4575. [Online]. Available: https://doi.org/10.1109/CVPR.2015.7299087

    doi:10.1109/cvpr.2015.7299087 2015

  32. [32]

    Improved image captioning via policy gradient optimization of spider,

    S. Liu, Z. Zhu, N. Ye, S. Guadarrama, and K. Murphy, “Improved image captioning via policy gradient optimization of spider,” inIEEE International Conference on Computer Vision, ICCV 2017, V enice, Italy, October 22-29, 2017. IEEE Computer Society, 2017, pp. 873–881. [Online]. Available: https://doi.org/10.1109/ICCV.2017.100

    work page doi:10.1109/iccv.2017.100 2017

  33. [33]

    SPICE: semantic propositional image caption evaluation,

    P. Anderson, B. Fernando, M. Johnson, and S. Gould, “SPICE: semantic propositional image caption evaluation,” inComputer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V, ser. Lecture Notes in Computer Science, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., vol. 9909. Springer, 2016, p...

    work page doi:10.1007/978-3-319-46454-1 2016

  34. [34]

    METEOR: an automatic metric for MT evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: an automatic metric for MT evaluation with improved correlation with human judgments,” in Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, Ann Arbor , Michigan, USA, June 29, 2005, J. Goldstein, A. Lavie, C. Lin, and C. R. V oss, Eds. Associ...

    2005