Pith. sign in

REVIEW 2 major objections 4 minor 58 references

StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos

T0 review · 2 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes the first end-to-end streaming approach to audio description generation for full-length videos, together with a benchmark of 33 manually verified videos across five genres.

desk verdict Genuinely useful benchmark and a sensible streaming reformulation, but fix the Algorithm 1 pseudocode bug and tighten the annotation-reliability story before building on it. read the letter →

arxiv 2608.12549 v1 pith:GZIUOG3B submitted 2026-08-12 cs.CV

Julian Spravil , Sebastian Houben , Sven Behnke This is my paper · ORCID
classification cs.CV
keywords audiodescriptiongenerationstreamingdensevideocaptioninglong-formunderstandingvision-languagemodelsmultimodalLLMtemporallocalizationaccessibilitybenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that audio description (AD) generation — the narration that makes video accessible to blind and low-vision viewers — should be reframed as a streaming task on full-length videos, not as clip captioning with pre-given timestamps. To make that reframing testable, it introduces StrAD, a benchmark of 33 full-length videos (22.3 hours) spanning movies, documentaries, short films, video games, and performances, with manually verified AD transcripts and timestamps. It also presents StrAD-FT, a fine-tuned multimodal model that sweeps a sliding window over the video and inserts timestamped AD sentences into speech gaps without any ground-truth boundaries. On segment-level benchmarks the paper reports 36.3 CIDEr on CMD-AD (a 10.0-point gain over the previous best training-free method), 51.0 CIDEr on StrAD, and 24.9 CIDEr on MAD-Eval; on the streaming task it reports 2.4 SODA versus 1.1 for a zero-shot baseline, at faster-than-real-time speed with under 5 billion parameters. The purpose of the benchmark is to make full-video AD generation measurable, so that accessibility tooling can scale beyond the small fraction of content that is currently described by hand.

What carries the argument

The load-bearing mechanism is the sliding-window streaming loop defined in Algorithm 1. A cursor advances through the video in four-second strides; whenever the transcript contains a speech gap of at least half a second, the model receives the preceding eight seconds of video plus the last 120 seconds of transcript and generated ADs, and either emits an AD with a start and end timestamp or returns a special WAIT token. Two design choices make this viable in practice: streaming training samples are created by moving a window with random stride over each video and labeling a sample WAIT when the window contains only dialogue, no AD, or a partial AD event; and 25 percent of the training budget is deliberately allocated to WAIT signals so the model learns when not to speak. The fine-tuned models are Phi-4-mm and Qwen-3.5, both under 5 billion parameters, adapted with LoRA on the language layers and full training of the vision and audio projection layers.

What would settle it

Have two professional describers independently annotate a random subset of StrAD videos and measure agreement on AD event boundaries and wording (e.g., tIoU of intervals and BERTScore/CIDEr between their annotations). If agreement is low, or if the benchmark's pronoun heuristic is shown to misclassify third-person narration in documentaries as dialogue, then the reference numbers and the claimed streaming improvement would not be trustworthy.

Watch

Extended reading notes

Core claim

The central claim is that AD generation can be executed as streaming dense video captioning: rather than receiving a clip boundary and producing one sentence, the model receives the video, its transcript, and previously generated ADs through a sliding window, and must decide both when to speak and what to say. The paper's fine-tuned model StrAD-FT is trained on this formulation using only CMD-AD data and, the authors report, matches or exceeds prior methods on the standard segment-level benchmarks while also producing coherent full-video AD without ground-truth timestamps. A zero-shot companion, StrAD-Zero, shows the same streaming interface works with prompted vision-language models. Ablations in the paper support a specific division of labor: video is the dominant signal for what to describe, while audio mainly tells the model when to narrate, reducing overlap with speech and cutting repetition. Together with the StrAD benchmark, these results are offered as the first end-to-end streaming solution to full-video AD generation.

Load-bearing premise

The load-bearing premise is that StrAD's ground truth — AD transcripts and timestamps produced by Whisper transcription, Silero VAD, pyannote diarization, and a personal-pronoun heuristic, then corrected by one annotator per video with spot-checking by a second — is accurate enough to serve as an evaluation reference; systematic errors there would shift every StrAD score and the streaming conclusion.

Editorial extensions

If this is right

  • AD generation no longer depends on ground-truth timestamps or character databases: a single streaming pass over a full video can place descriptions into natural speech gaps.
  • The benchmark gives the field a shared, automatically scored target on full-length videos with manually verified annotations, reducing the need for expensive per-video user studies.
  • The reported segment-level gains on CMD-AD (+10.0 CIDEr over Shot-by-shot) suggest that a small model trained on the streaming objective transfers well to the standard clip-based evaluation.
  • Audio input changes streaming behavior measurably: in the ablations, removing audio raises speech overlap from 3.9% to 5.3% and repetition from 11.5% to 16.3%, marking audio as a first-class input for the streaming task.
  • The large gap between segment-level performance (51.0 CIDEr) and streaming performance (2.4 SODA) defines temporal localization and narrative coherence as the main open problems the benchmark is designed to track.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper reports no inter-annotator agreement on StrAD, a natural next check is an independent re-annotation study on a random subset; if agreement is low, the benchmark reference scores and the streaming comparison would need to be re-interpreted.
  • The streaming formulation could transfer to live events and interactive media such as video games, where transcripts arrive incrementally; the reported real-time factor below 0.6 on a single A100 suggests the latency budget is already feasible.
  • The WAIT decision — learning when speaking would cover dialogue or intentional sound — is a reusable control task that could improve automated dubbing, commentary, or narration systems beyond accessibility.
  • Per-genre results show large variation (e.g., documentary CIDEr 65.0 vs game 40.3 for the Qwen-3.5 variant), hinting that genre-specific AD conventions such as describing game UI may need dedicated fine-tuning or evaluation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces StrAD, a benchmark for full-video audio description (AD) generation spanning diverse genres and long-form videos, along with three tasks: segment-level, streaming, and document-level. The authors propose two baselines: StrAD-Zero, a zero-shot pipeline extending AutoAD-Zero, and StrAD-FT, a fine-tuned multimodal model trained on CMD-AD. On segment-level benchmarks, StrAD-FT (Qwen-3.5) reports state-of-the-art CIDEr on CMD-AD (36.3) and strong results on StrAD (51.0), while remaining competitive on MAD-Eval (24.9). On the streaming task, StrAD-FT outperforms StrAD-Zero in SODA, CIDEr, localization, and overlap metrics, with real-time factor below 1 and fewer than 5B parameters. The paper also provides detailed reproducibility analyses, including validation against published predictions on an incomplete CMD-AD test set and a unified evaluation suite.

Significance. If the reported results hold, this is a valuable contribution: it is the first streaming, end-to-end AD generation method for full-length videos and the first benchmark of that kind with full-length, manually verified annotations. The paper strengthens credibility through careful metric validation: it checks its evaluation implementation against published predictions (Table 5), quantifies the effect of the incomplete CMD-AD test set, and reports a diagnostic breakdown (REP, OL, RTF) that honestly exposes remaining limitations. The segment-level SOTA on CMD-AD is supported by consistent CIDEr numbers and careful comparisons. However, the central streaming contribution is undermined by an internal inconsistency in Algorithm 1, and the benchmark's annotation reliability lacks inter-annotator agreement evidence.

major comments (2)
  1. [Algorithm 1, line 8 vs. Section 4.3] Algorithm 1's cursor update, t←max(t, t−wv+prediction.t_e), directly contradicts the prose rule in Section 4.3: "if the predicted end time t_e exceeds the current t, we set t=t_e." Under the pseudocode, after t is incremented by δ and an AD with t_e≈t+2s is emitted, the cursor jumps to roughly 2t (e.g., from t=100 to ≈194) because wv=8s, skipping most of the video. This would make the reported streaming Recall of 59.5 and SODA of 2.4 in Table 2 numerically impossible. The prose rule is consistent with the reported results and likely reflects the actual implementation, but the printed algorithm is not. The pseudocode must be corrected, and the released code must be verified against the stated rule. This is load-bearing because the streaming method is the paper's primary contribution.
  2. [Section A (StrAD annotation process)] The benchmark's ground truth is produced by automated Whisper/Silero/pyannote extraction followed by manual correction, but each video is reviewed by only one annotator with spot-checking by a second annotator, and no inter-annotator agreement is reported. Since all StrAD reference numbers and the streaming conclusions depend on this ground truth, systematic annotation errors cannot be ruled out. The authors should report IAA on a subset of videos (e.g., AD event boundary agreement and transcript accuracy) or otherwise validate the reliability of the manual correction step.
minor comments (4)
  1. [Section 5.2, random-padding variant] The sentence "Although this model achieves state-of-the-art results for CMD-AD and MAD-Eval" is not accurate: the random-padding variant reaches 31.1 CIDEr on CMD-AD, which is below the reported StrAD-FT (Qwen-3.5) score of 36.3; it is state-of-the-art only on MAD-Eval (31.9 CIDEr).
  2. [Abstract/Introduction] There is a grammatical error: "The main contributions of this paper is the first end-to-end streaming..." should be "are" or the sentence should be restructured.
  3. [Table 1 formatting] Several rows in Table 1, such as the AutoAD-II row showing "2.08 | 19.5 51.3", lack clear column separators for the LLM-AD-Eval and MAD-Eval columns, making the table hard to read; the table should use explicit column boundaries or be split.
  4. [Section D.2.1] ActionScore is listed as a streaming quality metric, but it is not reported anywhere in the streaming evaluation (Table 2); the authors should either report it or clarify why it is omitted from the streaming results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: benchmark and baselines are evaluated against external AD tracks and prior public datasets; the only self-reference is a preprocessing tool.

full rationale

StrAD's evaluation design is self-contained. StrAD-FT and StrAD-Zero are trained (or prompted) on CMD-AD and off-the-shelf models, then evaluated on the author-constructed StrAD benchmark, MAD-Eval, and CMD-AD test. The benchmark ground truth is derived from external paired YouTube AD audio tracks, transcribed and manually corrected, and is not a function of the model outputs; no parameter is fitted to the StrAD reference numbers. The only author-adjacent artifact is the describealign alignment tool used to construct the benchmark, but this is a preprocessing utility rather than a load-bearing scientific claim, and the AD content itself comes from external sources. The reported streaming and segment-level numbers are therefore measured results, not predictions forced by construction. The discrepancy between Algorithm 1's cursor update and Section 4.3's prose is an internal consistency issue relevant to correctness, but it does not make the derivation circular.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard domain assumptions about AD placement and transcript availability, plus the accuracy of the annotation pipeline. No mathematical axioms are needed; the free parameters are hand-chosen system constants that affect performance but were not fitted to StrAD results.

free parameters (7)
  • streaming stride δ = 4 s
    Hand-chosen advance step for the sliding window; affects how often ADs can be emitted (Algorithm 1, Section 4.3).
  • visual window w_v = 8 s (StrAD-FT); equals stride for StrAD-Zero
    Hand-chosen length of the backward-facing visual context; not ablated.
  • text context window w_c = 120 s
    Hand-chosen window over transcript and prior ADs; not ablated.
  • segment padding δ_p = 2 s (CMD-AD, StrAD); 0 s (MAD-Eval)
    Padded segment context that lets the model infer AD duration; authors note a random-padding variant changes the MAD-Eval trade-off.
  • presence penalty = 1.5
    Decoding hyperparameter to reduce repetitions; chosen without reported sweep.
  • assumed speech rate = 150 words/min
    Used by StrAD-Zero to estimate AD duration from word count and center it in the window.
  • training sample mix = 25% WAIT, 37.5% segment, 37.5% streaming
    Hand-set allocation of the training sample budget.
assumptions (4)
  • domain assumption ADs are placed in speech gaps; the system only emits when a transcript gap ≥0.5 s is available.
    Encoded in Algorithm 1's HASGAP test; reflects accessibility practice but is a modeling choice.
  • domain assumption A transcript of the speech is available as input, and using the verified transcript isolates AD from ASR error.
    Section 5.1; real-world ASR errors are explicitly out of scope.
  • domain assumption The annotation pipeline (Whisper transcription, Silero VAD, pyannote diarization, personal-pronoun heuristic, manual correction) yields valid AD events and timestamps.
    Section A; 73.8% of events were corrected by annotators, and no inter-annotator agreement is reported.
  • domain assumption Paired YouTube videos with AD tracks can be reliably aligned to the original via audio cross-correlation or transcript matching.
    Section A.1; alignment is the basis of all ground-truth timestamps.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos." pith.science (2026). https://pith.science/paper/GZIUOG3B

@misc{pith2026260812549,
  author       = {Pith},
  title        = {Pith review of: StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GZIUOG3B}},
  note         = {Machine review of arXiv:2608.12549}
}
read the original abstract

Visual content is the dominant medium of communication, yet without audio descriptions (ADs), it remains inaccessible to blind and low-vision people. ADs narrate context-relevant visual events during natural audio pauses. Manually creating ADs is expensive, limiting coverage to a small fraction of available content. Most existing automatic AD generation methods frame the task as video clip captioning, requiring ground-truth timestamps and additional context cues such as character databases. Current benchmarks reinforce this framing, consisting of short video segments paired with automatic or task-mismatched annotations. We introduce StrAD, a benchmark for long-form AD generation on full-length videos spanning diverse genres such as movies, documentaries, short films, performances, and video games. We reformulate AD generation as streaming dense video captioning. Our approach processes full-length videos with a sliding window, inserting ADs into existing transcripts without ground-truth timestamps, and supports both fine-tuned models and zero-shot prompting of vision-language models. On the segment-level task with given timestamps, our fine-tuned StrAD-FT sets the state of the art on CMD-AD with 36.3 CIDEr (+10.0 over Shot-by-shot), establishes a reference point on StrAD (51.0 CIDEr), and remains competitive on MAD-Eval at 24.9 CIDEr. On the full-video streaming task, StrAD-FT reaches a SODA score of 2.4 against 1.1 for our zero-shot baseline StrAD-Zero, though both exhibit limitations in temporal localization and narrative coherence. While prior work has tackled full-video AD generation in an offline, multi-stage fashion, ours is the first streaming approach, generating ADs on the fly without ground-truth timestamps. StrAD makes progress on full-video AD generation measurable, a prerequisite for scaling accessibility.

Figures

Figures reproduced from arXiv: 2608.12549 by the authors.

Figure 1
Figure 1. Statistics of StrAD. Relative to the MAD-Eval [38, 12] and CMD-AD test split [3, 14], StrAD (a) spans a wider range of video durations and AD densities; (b) covers diverse formats including documentaries (Doc.), short films (S. Film), video games, and performances (Perf.); and (c) provides more speech gaps with AD, more AD events per gap, and less speech overlap. Audio Video Context AD ts te Model Full video (a) Seg… view at source ↗
Figure 2
Figure 2. Tasks on StrAD. The three tasks differ in input context and the granularity at which ADs are produced: (a) generate an AD for one segment given ground-truth timestamps ts, te and curated context (e.g., characters, screenplay); (b) generate the next AD conditioned on video, audio, transcript, and previously generated ADs up to cursor t advanced by stride δ; (c) generate the complete AD transcript for the full video i… view at source ↗
Figure 3
Figure 3. Example from StrAD. An example sequence from Elden Ring—Gameplay Preview (youtu.be/JldMvQMO_5U) and its audio-described version (youtu.be/neQPEIAqpQ8). temporal grounding. Zhou et al. [58] processes videos in a streaming fashion by building a memory via online clustering of frame embeddings, generating captions at decoding points. In contrast, AD events need to be story-relevant, are limited in the number of words, … view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: StrAD construction pipeline. (a) We source paired YouTube videos with original and AD-augmented audio tracks and align them via cross-correlation with describealign, comparing three alignment strategies. (b) We transcribe the AD track with Whisper [32] and separate AD …
Figure 5
Figure 5. Figure 5: Video captioning prompt. This is the exact prompt used by AutoAD-Zero [51]. C Detailed Experimental Setup We provide hyperparameters and other details required for reproduction of StrAD-FT in the [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Decision prompt. Inserted before AD generation to predict whether the current window should produce an AD. ric is not fully specified with respect to averaging strategy and ordering, which leads to discrepancies across implementations (see Section D.3). 16 [PITH_FULL_…
Figure 7
Figure 7. Figure 7: AD generation prompt. This is an extension for streaming of the the AD generation prompt of AutoAD-Zero [51] CRITIC [14] calculates the IoU between character identities in predictions and references extracted with a co-referencing model and the ground-truth character l…
Figure 8
Figure 8. Figure 8: Learning rate sweep. CMD-AD validation loss across 10 Optuna trials per model [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Full streaming output overview. AD events for the ground truth, our fine-tuned Qwen-3.5 and Phi-4-mm StrAD-FT variants, and the zero-shot StrAD-Zero baseline, placed on the full video timeline. Each bar marks the start and duration of an emitted AD. 25 [PITH_FULL_IMAG…
Figure 10
Figure 10. Figure 10: Side-by-side qualitative comparison on The Sense of Sight. Sampled frames, the transcribed dialogue, and the AD stream emitted by the ground truth, both StrAD-FT variants, and the StrAD-Zero baseline. Original video: youtu.be/Oph8kL_Z4c4; audio-described version: yout…
Figure 11
Figure 11. Figure 11: Side-by-side qualitative comparison on Elden Ring—Gameplay Preview (780–840 s). Sampled frames, the transcribed dialogue, and the AD stream emitted by the ground truth, both StrAD-FT variants, and the StrAD-Zero baseline. Original video: youtu.be/JldMvQMO_5U; audio￾de…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 42 canonical work pages

  1. [1]

    Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

    Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. CoRR, abs/2503.01743, 2025

    arXiv 2025

  2. [2]

    Optuna: A next-generation hyperparameter optimization framework

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. InKDD, pages 2623–2631, 2019

    work page 2019

  3. [3]

    Condensed Movies: Story Based Retrieval with Contextual Embeddings

    Max Bain, Arsha Nagrani, Andrew Brown, and Andrew Zisserman. Condensed Movies: Story Based Retrieval with Contextual Embeddings. InACCV, pages 460–479, 2020

    work page 2020

  4. [4]

    METEOR: An automatic metric for MT evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. InProceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL, pages 65–72, 2005

    work page 2005

  5. [5]

    Evaluation of AI-generated audio description for factual TV/media genres

    Sabine Braun, Shenbin Qian, Yuan Zou, and Constantin Orasan. Evaluation of AI-generated audio description for factual TV/media genres. Research report, Royal National Institute of Blind People (RNIB) and University of Surrey, 2025. URL https://www.rnib.org.uk/ documents/3024/AI-Generated_Audio_Description_Report_August_2025.pdf. Ac- cessed: 2026-04-18

    work page 2025

  6. [6]

    MCAD: Multimodal Context-Aware Audio Description Generation for Soccer

    Lipisha Chaudhary, Trisha Mittal, Subhadra Gopalakrishnan, Ifeoma Nwogu, and Jaclyn Pytlarz. MCAD: Multimodal Context-Aware Audio Description Generation for Soccer. InInternational Symposium on Multimedia (ISM), pages 280–287, 2025

    work page 2025

  7. [7]

    VideoLLaMA 2: Advancing Spatial- Temporal Modeling and Audio Understanding in Video-LLMs.CoRR, abs/2406.07476, 2024

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. VideoLLaMA 2: Advancing Spatial- Temporal Modeling and Audio Understanding in Video-LLMs.CoRR, abs/2406.07476, 2024

    arXiv 2024

  8. [8]

    LLM-AD: Large Language Model based Audio Description System.CoRR, abs/2405.00983, 2024

    Peng Chu, Jiang Wang, and Andre Abrantes. LLM-AD: Large Language Model based Audio Description System.CoRR, abs/2405.00983, 2024

    arXiv 2024

Show all 58 references
  1. [9]

    Bo Fang, Wenhao Wu, Qiangqiang Wu, Yuxin Song, and Antoni B. Chan. DistinctAD: Distinctive Audio Description Generation in Contexts. InCVPR, pages 13571–13581, 2025

    2025

  2. [10]

    Soda: Story oriented dense video captioning evaluation framework

    Soichiro Fujita, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. Soda: Story oriented dense video captioning evaluation framework. InECCV, pages 517–531, 2020

    2020

  3. [11]

    Accelerate: Training and inference at scale made simple, efficient and adaptable.https://github.com/huggingface/accelerate, 2022

    Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable.https://github.com/huggingface/accelerate, 2022

    2022

  4. [12]

    AutoAD: Movie Description in Context

    Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. AutoAD: Movie Description in Context. InCVPR, pages 18930–18940, 2023

    2023

  5. [13]

    AutoAD II: The Sequel - Who, When, and What in Movie Audio Description

    Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. AutoAD II: The Sequel - Who, When, and What in Movie Audio Description. InICCV, pages 13599– 13609, 2023

    2023

  6. [14]

    AutoAD III: The Prequel - Back to the Pixels

    Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. AutoAD III: The Prequel - Back to the Pixels. InCVPR, pages 18164–18174, 2024. 10

    2024

  7. [15]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR, 2022

    2022

  8. [16]

    What You See is What You Ask: Evaluating Audio Descriptions

    Divy Kala, Eshika Khandelwal, and Makarand Tapaswi. What You See is What You Ask: Evaluating Audio Descriptions. InEMNLP, pages 23496–23518, 2025

    2025

  9. [17]

    Varshney, Caiming Xiong, and Richard Socher

    Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: A Conditional Transformer Language Model for Controllable Generation.CoRR, abs/1909.05858, 2019

    1909 arXiv

  10. [18]

    More than a Moment: Towards Coherent Sequences of Audio Descriptions.CoRR, abs/2510.25440, 2025

    Eshika Khandelwal, Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Andrew Zisserman, Gül Varol, and Makarand Tapaswi. More than a Moment: Towards Coherent Sequences of Audio Descriptions.CoRR, abs/2510.25440, 2025

    2025

  11. [19]

    Dense- captioning events in videos

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense- captioning events in videos. InICCV, pages 706–715, 2017

    2017

  12. [20]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems ...

    2023

  13. [21]

    The semi-automatic generation of audio description from screenplays.Dept

    James Lakritz and Andrew Salway. The semi-automatic generation of audio description from screenplays.Dept. of Computing Technical Report CS-06-05, University of Surrey, 2006

    2006

  14. [22]

    Audio description in video games? Persons with visual disabilities weigh in.Univers

    María Eugenia Larreina-Morales and Carme Mangiron. Audio description in video games? Persons with visual disabilities weigh in.Univers. Access Inf. Soc., pages 577–588, 2024

    2024

  15. [23]

    Now you see me: Context-aware automatic audio description

    Seon-Ho Lee, Jue Wang, David Fan, Zhikang Zhang, Linda Liu, Xiang Hao, Vimal Bhat, and Xinyu Li. Now you see me: Context-aware automatic audio description. InWACV, pages 5530–5539, 2025

    2025

  16. [24]

    ROUGE: A Package for Automatic Evaluation of Summaries

    Chin-Yew Lin. ROUGE: A Package for Automatic Evaluation of Summaries. InText Summa- rization Branches Out, pages 74–81, 2004

    2004

  17. [25]

    Learning Video Context as Interleaved Multimodal Sequences

    Kevin Qinghong Lin, Pengchuan Zhang, Difei Gao, Xide Xia, Joya Chen, Ziteng Gao, Jinheng Xie, Xuhong Xiao, and Mike Zheng Shou. Learning Video Context as Interleaved Multimodal Sequences. InECCV, pages 375–396, 2024

    2024

  18. [26]

    SGDR: Stochastic Gradient Descent with Warm Restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic Gradient Descent with Warm Restarts. In ICLR, 2017

    2017

  19. [27]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2019

    2019

  20. [28]

    Ka, and Dongsu Han

    Jaehyeong Park, Juncheol Ye, Seungkook Lee, Hyun W. Ka, and Dongsu Han. NarrAD: Automatic Generation of Audio Descriptions for Movies with Rich Narrative Context. InWACV, pages 409–419, 2025

    2025

  21. [29]

    Pytorch: An imperative style, high-performance deep learning library.NeurIPS, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.NeurIPS, 2019

    2019

  22. [30]

    Powerset multi-class cross entropy loss for neural speaker diarization

    Alexis Plaquet and Hervé Bredin. Powerset multi-class cross entropy loss for neural speaker diarization. InInterspeech, pages 3222–3226, 2023

    2023

  23. [31]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5

    2026

  24. [32]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  25. [33]

    MatchTime: Towards Automatic Soccer Game Commentary Generation

    Jiayuan Rao, Haoning Wu, Chang Liu, Yanfeng Wang, and Weidi Xie. MatchTime: Towards Automatic Soccer Game Commentary Generation. InEMNLP, pages 1671–1685, 2024. 11

    2024

  26. [34]

    EUT Edizioni Università di Trieste, 2015

    Aline Remael, Nina Reviers, and Gert Vercauteren.Pictures painted in words: ADLAB audio description guidelines. EUT Edizioni Università di Trieste, 2015. ISBN 978-88-8303-675-0

    2015

  27. [35]

    TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding. InCVPR, pages 14313– 14323, 2024

    2024

  28. [36]

    A dataset for Movie Description

    Anna Rohrbach, Marcus Rohrbach, Niket Tandon, and Bernt Schiele. A dataset for Movie Description. InCVPR, pages 3202–3212, 2015

    2015

  29. [37]

    Courville, and Bernt Schiele

    Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Joseph Pal, Hugo Larochelle, Aaron C. Courville, and Bernt Schiele. Movie description.IJCV, 123(1):94–120, 2017

    2017

  30. [38]

    MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions

    Mattia Soldan, Alejandro Pardo, Juan León Alcázar, Fabian Caba Heilbron, Chen Zhao, Silvio Giancola, and Bernard Ghanem. MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions. InCVPR, pages 5016–5025, 2022

    2022

  31. [39]

    MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, and Gaoang Wang. MovieChat: From Dense Token to Sparse Memory for Long Video Understanding. InCVPR, pages 18221–18232, 2024

    2024

  32. [40]

    Contrastive Search Is What You Need For Neural Text Generation

    Yixuan Su and Nigel Collier. Contrastive Search Is What You Need For Neural Text Generation. Trans. Mach. Learn. Res., 2023

    2023

  33. [41]

    Llama 2: Open Foundation and Fine-Tuned Chat Models.CoRR, abs/2307.09288, 2023

    Llama 2 Team. Llama 2: Open Foundation and Fine-Tuned Chat Models.CoRR, abs/2307.09288, 2023

    2023 arXiv

  34. [42]

    The Llama 3 Herd of Models.CoRR, abs/2407.21783, 2024

    Llama 3 Team. The Llama 3 Herd of Models.CoRR, abs/2407.21783, 2024

    2024 arXiv

  35. [43]

    Silero V AD: pre-trained enterprise-grade V oice Activity Detector (V AD), Number Detector and Language Classifier.https://github.com/snakers4/silero-vad, 2024

    Silero Team. Silero V AD: pre-trained enterprise-grade V oice Activity Detector (V AD), Number Detector and Language Classifier.https://github.com/snakers4/silero-vad, 2024

    2024

  36. [44]

    Pal, Hugo Larochelle, and Aaron C

    Atousa Torabi, Christopher J. Pal, Hugo Larochelle, and Aaron C. Courville. Using Descrip- tive Video Services to Create a Large Data Source for Video Annotation Research.CoRR, abs/1503.01070, 2015. URLhttp://arxiv.org/abs/1503.01070

    2015 arXiv

  37. [45]

    CIDEr: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. CIDEr: Consensus-based image description evaluation. InCVPR, pages 4566–4575, 2015

    2015

  38. [46]

    Contextual AD Narration with Interleaved Multimodal Sequence

    Hanlin Wang, Zhan Tong, Kecheng Zheng, Yujun Shen, and Limin Wang. Contextual AD Narration with Interleaved Multimodal Sequence. InCVPR, pages 8372–8383, 2025

    2025

  39. [47]

    Toward Automatic Audio Description Generation for Accessible Videos

    Yujia Wang, Wei Liang, Haikun Huang, Yongqi Zhang, Dingzeyu Li, and Lap-Fai Yu. Toward Automatic Audio Description Generation for Accessible Videos. InCHI Conference on Human Factors in Computing Systems, pages 277:1–277:12, 2021

    2021

  40. [48]

    Transformers: State-of- the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of- the-art natural language processing. InEMNLP, pages 38–45, 2020

    2020

  41. [49]

    World Health Organization, 2019

    World Health Organization.World report on vision. World Health Organization, 2019

    2019

  42. [50]

    A simple yet effective knowledge guided method for entity-aware video captioning on a basketball benchmark.Neurocomputing, 619:129177, 2025

    Zeyu Xi, Ge Shi, Xuefen Li, Junchi Yan, Zun Li, Lifang Wu, Zilin Liu, and Liang Wang. A simple yet effective knowledge guided method for entity-aware video captioning on a basketball benchmark.Neurocomputing, 619:129177, 2025

    2025

  43. [51]

    AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description

    Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisser- man. AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description. InACCV, pages 81–97, 2024

    2024

  44. [52]

    Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation

    Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Eshika Khandelwal, Gül Varol, Weidi Xie, and Andrew Zisserman. Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation. InICCV, 2025. 12

    2025

  45. [53]

    Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning

    Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning. InCVPR, pages 10714–10726, 2023

    2023

  46. [54]

    Story-context-aware audio description generation via hierarchical narrative modeling

    Yuanzhe Yang and Xianzhong Liu. Story-context-aware audio description generation via hierarchical narrative modeling. InInternational Conference on Virtual Reality and Visualization (ICVRV), pages 240–245, 2025

    2025

  47. [55]

    MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning

    Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning. InCVPR, pages 13647–13657, 2024

    2024

  48. [56]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. BERTScore: Evaluating Text Generation with BERT. InICLR, 2020

    2020

  49. [57]

    Character- Centric Understanding of Animated Movies

    Zhongrui Gui and Junyu Xie and Tengda Han and Weidi Xie and Andrew Zisserman. Character- Centric Understanding of Animated Movies. InACM MM, pages 3300–3309, 2025

    2025

  50. [58]

    " " Video c a p t i o n i n g from AutoAD - Zero

    Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Nagrani, and Cordelia Schmid. Streaming dense video captioning. InCVPR, pages 18243– 18252, 2024. 13 AD videoAnnotateMask AD audioAD audioOriginal video A) Only AD video: B) AD & original vide...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.