Pith. sign in

REVIEW 4 major objections 5 minor 76 references

BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dataset-level visual and textual bias holds text-video retrieval back, and BiMa's scene-element guidance plus content/bias disentanglement removes enough of it to reach best results on five benchmarks.

desk verdict The BiMa framework is a clean engineering combination, but its reported gains—especially OOD—are undermined by a taxonomy dictionary built from the same test sets the model is evaluated on. read the letter →

arxiv 2506.03589 v3 pith:OB2YA6VL submitted 2025-06-04 cs.CV cs.AIcs.CL

Huy Le , Nhat Chung , Tung Kieu , Anh Nguyen , Ngan Le This is my paper
classification cs.CVcs.AIcs.CL
keywords text-videoretrievalbiasmitigationvisual-linguisticsceneelementsrepresentationdisentanglementout-of-distributionvision-languagemodelscontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-video retrieval models inherit two systematic biases: they attend to visually dominant regions while missing small actors and objects, and they absorb the subjective wording of whichever annotator wrote the caption. BiMa is built to counter both. A taxonomy dictionary of noun phrases (entities) and verb phrases (activities) tags each video with its most relevant scene elements, and those elements are fused into the video embedding so fine-grained content is emphasized. On the text side, each caption is decomposed into a content component, which is what gets matched to video, and a bias component modeled as Gaussian noise, with a reconstruction objective ensuring the two pieces together still reproduce the caption. If the approach works as claimed, retrieval models become less tied to dataset-specific patterns without needing any new human annotations, and the reported gains — best results on MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo, plus large improvements in cross-dataset retrieval — are presented as evidence.

What carries the argument

Three mechanisms carry the argument. Scene Element Construction builds a taxonomy dictionary of noun phrases (scene entities) and verb phrases (scene activities) distilled from the benchmark captions, and a frozen CLIP model selects the top-κ elements most similar to each video; an aggregation coefficient g then weights the activities against their closest entities before the two are summed into one element embedding. Visual Scene Debias fuses that element embedding into the video embedding with a cross-attention layer, then attaches a captioning head that autoregressively predicts caption tokens from the augmented embedding, so the matching feature must retain small but salient details. Textual Content Debias splits each caption into a content embedding and a bias embedding, the bias drawn from a Gaussian with the combination reconstructed through a VAE-style ELBO objective, so that contrastive matching uses only the content part. The overall objective combines an InfoNCE loss with Weighted Token Interaction token-level matching together with the captioning, reconstruction, and KL terms.

What would settle it

Rebuild the taxonomy dictionary using only the MSR-VTT training captions, retrain BiMa with that dictionary, and rerun the three OOD benchmarks; if the OOD R@1 numbers fall back to roughly the prior-method level while in-distribution results stay high, the reported OOD advantage came from dictionary leakage rather than bias mitigation.

Watch

Extended reading notes

Core claim

The central claim is that visual-linguistic bias, not just encoder capacity, is what limits text-video retrieval, and that this bias can be neutralized with self-supervised machinery. Visual bias is addressed by augmenting the video embedding with the top-κ scene elements selected from a large taxonomy of entities and activities: a cross-attention layer fuses the element embeddings into the video feature, and a captioning head forces the fused feature to support generative reconstruction of the caption, pushing it to carry fine-grained details such as small actors and their actions. Textual bias is addressed by splitting each caption into a content component and a bias component, with only the content component contrastively aligned against the augmented video embedding; the bias component is modeled as a draw from a Gaussian, and the two parts are recombined and reconstructed under a VAE-style objective, isolating the bias as residual information. The paper reports the best retrieval numbers on all five benchmarks and substantially improved out-of-distribution retrieval, which it presents as evidence that the scene-element guidance and the disentanglement, not the backbone alone, are what drive the gains.

Load-bearing premise

The load-bearing premise is that the scene-element dictionary does not leak information about target distributions, yet it is built from the same five datasets used for training and for out-of-distribution testing, so the OOD improvements could come from exposure to target-distribution captions rather than from genuine debiasing.

Editorial extensions

If this is right

  • Debiasing can be done without extra annotations, because the scene elements come from existing captions plus a frozen CLIP matcher, and the text disentanglement is self-supervised.
  • Fusing scene entities and activities into video embeddings redirects attention toward small objects and actions; the ablations attribute most of the gain over the CLIP4Clip baseline to this module.
  • Isolating the bias component is decisive: matching with content features alone scores R@1 53.5 on MSR-VTT, while matching with bias features alone collapses to 1.3.
  • Bias mitigation transfers: cross-dataset retrieval from MSR-VTT to LSMDC, ActivityNet, and DiDeMo improves sharply over prior methods, which the paper reads as generalization beyond training-specific biases.
  • The cost is modest: the top-κ scene-element selection is performed once per video, and the full model adds roughly five seconds of inference time over the baseline at equal memory use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the taxonomy dictionary is built from the same five datasets that later serve as training and out-of-distribution targets, the reported OOD gains may partly reflect exposure to target-distribution captions; rebuilding the dictionary from training-domain captions only would isolate this.
  • The bias component is defined by exclusion — whatever the content alignment leaves over — so it may still carry transferable semantic signal, and the paper's own t-SNE shows partial entanglement between the two components.
  • The captioning head adds a generative pressure that could improve retrieval on its own; adding it to a non-debiased baseline would separate its contribution from the debiasing story.
  • A direct test of the bias claim: if the mechanism works, retrieval gains should concentrate on queries about small, non-salient objects in cluttered scenes, so measuring per-query recall by object size would make the bias effect visible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes BiMa, a framework for text-video retrieval that targets visual and textual biases. It has three modules: Scene Element Construction selects top-k noun/verb phrases from a taxonomy dictionary using frozen CLIP; Visual Scene Debias fuses these scene elements into video embeddings via cross-attention and a captioning head; Textual Content Debias uses a beta-VAE-style objective to split text embeddings into content and bias components. Experiments on MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo report state-of-the-art T2V results and strong cross-dataset OOD results. The paper's central claim is that the scene-element guidance and content/bias disentanglement are the causes of the observed gains.

Significance. If the claims were supported, BiMa would be a notable contribution: it proposes a self-supervised bias-mitigation pipeline that requires no additional human annotations, and its OOD results would suggest strong generalization. The architecture is coherent and the ablation structure is informative. However, the main experimental evidence is currently undermined by a dictionary-construction protocol that can leak test and target-dataset vocabulary into the video embeddings at inference time, so the significance of the reported SOTA and OOD improvements is contingent on re-evaluation with a leakage-free protocol.

major comments (4)
  1. [Appendix A, Sec. 3.1, Table .3] The taxonomy dictionary is built from 'all captions available in existing TVR datasets', and Table .3 explicitly combines MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo with no indication of a train/test split. At inference, Sec. 3.1 selects top-k scene elements for each test video from this dictionary using the same frozen CLIP text encoder, and Sec. 3.2 fuses the selected elements into the video embedding via cross-attention. For any test video whose ground-truth caption contributed phrases to the dictionary, the selected scene elements can include words from the query itself, making the augmented video embedding artificially closer to the query in CLIP space. This is a direct leakage channel that inflates the in-distribution results in Tables 1 and 2, and it also affects the OOD results in Table 3 because the target datasets are included in the dictionary. The monotonic improvement in Table .3 as target datasets are added is exactly the signature of vocabulary leakage; the paper's interpretation that larger dictionaries reduce bias is not supported by the current experiment.
  2. [Table 3, Appendix A] The OOD experiment is presented as evidence that BiMa generalizes beyond training-specific biases, but the evaluation is self-referential. The model is trained only on MSR-VTT, yet the dictionary used at test time includes captions from LSMDC, ActivityNet, and DiDeMo (Appendix A, Table .3). Since the text encoder is frozen CLIP, top-k scene elements drawn from target-dataset captions can directly encode target vocabulary into the video embedding. To support the debiasing claim, the authors need to rerun the OOD evaluation with a dictionary built exclusively from MSR-VTT training captions, or with split-specific dictionaries, and report whether the advantage over baselines survives.
  3. [Section 2, Eqs. (2)-(3)] The formal bias definitions are not sufficiently specified to support the paper's claim of 'explicitly identifying and defining visual-linguistic biases.' The quantities P(v|V), P(v|T), P(v|t), P(v|ˆt), and P(v|˜t) are never defined in terms of the model or the data distribution, and Eq. (3) uses an ambiguous addition ˆt + ˜t of embeddings. Moreover, the content/bias decomposition has no external ground truth: Table 5 only shows that the learned 'bias' component has low retrieval signal, which is expected by construction because it is optimized as residual noise. A quantitative validation of the bias component—for example, correlation with annotator subjectivity or a behavioral bias test—is needed to support the debiasing interpretation.
  4. [Tables 1-3] All retrieval results are reported as single numbers without multiple seeds, error bars, or significance tests. The claimed improvements over runners-up are often small (e.g., 1.2 R@1 over TextProxy in Table 1, 2.4 R@1 over TextProxy in Table 2 on ActivityNet), so the state-of-the-art claim cannot be reliably assessed. Please report means and standard deviations over at least three seeds for the main tables, or provide paired bootstrap confidence intervals.
minor comments (5)
  1. [Section 3, intro] The sentence 'Figure 2 shows the the framework overview' contains a duplicated article; it should read 'the framework overview'.
  2. [Eq. (4)] The notation max_j [s(ˆa^{(i)}_l, ˆe^{(i)}_j) κ_{j=1}] is confusing; please define the set over which the maximum is taken, e.g., max_{1≤j≤κ} s(ˆa^{(i)}_l, ˆe^{(i)}_j).
  3. [Table .3] The column header 'T2V' over R@1/R@5/R@10 with checkmarks is unclear; please restructure the table so that the dataset-source rows and the metric columns are unambiguous.
  4. [Reference [11]] Reference [11] has a typo in the arXiv identifier: 'arXiv:706.02677' should be 'arXiv:1706.02677'.
  5. [Appendix B] The claim that integrating scene activities gives 'an increase of over 2% on R@1 metrics' compared to scene entities is not supported by Table .1, where the largest difference is 0.2 R@1 (49.4 vs. 49.2 at κ=20); please correct or reword this statement.

Circularity Check

2 steps flagged · score 6.0 of 10

OOD and in-domain gains are partly built from a taxonomy dictionary that includes test and OOD target captions, so the claimed bias-mitigation validation is not self-contained.

  1. fitted input called prediction [Appendix A (dictionary construction); Sec. 3.1; Sec. 4.2, Table 3]
    "For phrasal extraction, we query the LLM QWEN2.5-7B using all captions available in existing TVR datasets with the prompt illustrated below. ... we calculate the similarity between ¯v(i) with scene entities dictionary e and scene activities dictionary a. After that, we select the top-κ relevant scene entities ... and top-κ scene activities ... with the highest similarity scores as 'scene elements' of V(i)."

    The OOD protocol says the target is 'unseen during training,' but the taxonomy dictionary is built from 'all captions available in existing TVR datasets,' including LSMDC, ActivityNet, and DiDeMo. At inference, Sec. 3.1 selects top-κ dictionary phrases by CLIP similarity to each test video and Sec. 3.2 fuses them into the video embedding. For a target video whose ground-truth caption contributed phrases, those words can be selected as scene elements; the query is embedded in the same frozen CLIP text space, so the augmented video embedding is artificially closer to the query. Table .3 shows monotonic R@1 gains (49.7→51.5) as target datasets are added to the dictionary, exactly the signature of vocabulary leakage.

  2. fitted input called prediction [Appendix B, Table .3; Sec. 4.1, Table 1]
    "As shown in Table .3, increasing the number of datasets used to construct the dictionary yields consistent performance gains, further underscoring the importance of diverse and fine-grained video descriptions for reducing bias."

    MSR-VTT test captions are included in the dictionary because it is built from 'all captions available in existing TVR datasets,' with no train/test separation. At inference, test videos are augmented with top-κ scene elements selected from this dictionary; a test video's own ground-truth caption phrases can be among them, making the augmented video embedding closer to the query independently of learned bias mitigation. Table .3 quantifies this: adding LSMDC, ActivityNet, and DiDeMo captions to the dictionary improves MSR-VTT R@1 from 49.7 to 51.5 in the same framework. Some of the SOTA gain in Table 1 is therefore attributable to the dictionary containing test-set vocabulary rather than to the proposed modules.

full rationale

The VAE loss (Eq. 7) and captioning loss (Eq. 6) are standard, and the uses of external prior work (β-VAE, CoCa, CLIP, WTI) are not circular. The self-citations to [53,58] in Related Work are not load-bearing. The central problem is the taxonomy dictionary: Appendix A builds it from all captions of all five benchmarks, including the MSR-VTT test split and the three OOD target datasets, and Sec. 3.1–3.2 route this dictionary into the video embedding at inference. Because queries are embedded in the same frozen CLIP space, a test video whose caption contributed phrases to the dictionary can have those phrases selected as 'scene elements' and fused into its embedding, artificially boosting similarity to the query. Table .3 shows R@1 rising monotonically as target datasets are added to the dictionary (49.7→51.5), confirming the mechanism is measurable. Both the in-domain SOTA (Table 1) and the OOD generalization claim (Table 3) therefore reduce in part to the dictionary containing evaluation captions; no control (e.g., a dictionary built only from training captions) is provided. This is a substantial partial circularity/leakage, but it does not make the whole framework vacuous: the learned fusion, captioning, and VAE losses are independent and could still contribute.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the dataset-derived taxonomy dictionary, which creates a leakage risk for in-distribution and especially OOD evaluation, and on the untested Gaussian model for textual bias.

free parameters (4)
  • lambda_Cap = 0.3
    Weight for captioning loss in Eq. 13; selected via hyperparameter sweep on MSR-VTT (Fig. 3 left).
  • lambda_Rec = 0.5
    Weight for reconstruction loss; selected via sweep on MSR-VTT (Fig. 3 middle).
  • lambda_KL = 1e-4
    Weight for KL divergence; chosen to balance T2V and V2T performance (Fig. 3 right).
  • kappa (number of scene elements) = 20
    Top-k scene elements selected per video; best in ablation study (Appendix Table .1).
assumptions (3)
  • domain assumption Textual bias can be modeled as Gaussian noise in the latent space (Eq. 9).
    The paper assumes bias variations follow a multivariate Gaussian; no empirical justification is given.
  • domain assumption Scene elements extracted from text captions and matched to videos via frozen CLIP capture the salient content needed for debiasing.
    The Scene Element Construction relies on CLIP's zero-shot matching; this is an unproved assumption about the sufficiency of the taxonomy dictionary.
  • ad hoc to paper The taxonomy dictionary built from all benchmark datasets does not introduce leakage, i.e., it is neutral with respect to train/test splits.
    This is necessary for the OOD generalization claims and is not tested; in fact, Appendix A shows the dictionary includes OOD target captions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance." pith.science (2026). https://pith.science/paper/OB2YA6VL

@misc{pith2026250603589,
  author       = {Pith},
  title        = {Pith review of: BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OB2YA6VL}},
  note         = {Machine review of arXiv:2506.03589}
}
read the original abstract

Text-video retrieval (TVR) systems often suffer from visual-linguistic biases present in datasets, which cause pre-trained vision-language models to overlook key details. To address this, we propose BiMa, a novel framework designed to mitigate biases in both visual and textual representations. Our approach begins by generating scene elements that characterize each video by identifying relevant entities/objects and activities. For visual debiasing, we integrate these scene elements into the video embeddings, enhancing them to emphasize fine-grained and salient details. For textual debiasing, we introduce a mechanism to disentangle text features into content and bias components, enabling the model to focus on meaningful content while separately handling biased information. Extensive experiments and ablation studies across five major TVR benchmarks (i.e., MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo) demonstrate the competitive performance of BiMa. Additionally, the model's bias mitigation capability is consistently validated by its strong results on out-of-distribution retrieval tasks.

Figures

Figures reproduced from arXiv: 2506.03589 by the authors.

Figure 1
Figure 1. Illustration of Visual-Linguistic Bias in TVR. (a) Visual Bias: For each example, video (left of the 1st row), the associated visual feature map (right of the 1st row) extracted by pre-trained CLIP [31] and its corresponding Textual query (2nd row). This highlights a bias toward visually dominant regions while overlooking main actors/objects and activities when they occupy smaller regions in the scene. (b) Textual B… view at source ↗
Figure 2
Figure 2. Overview of BiMa. The dashed line represents the model flow used exclusively during training, while the solid line indicates usage in both training and inference. 3. Methodology [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Hyperparameter sensitivity study on MSR-VTT. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: t-SNE [51] visualiza￾tion between content and bias embeddings on MSR-VTT test￾set. 4.3. Qualitative Results [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: TVR qualitative Results. Videos in green boxes are true top-1 videos retrieved from BiMa, and in red boxes are false top-1 videos retrieved from CLIP4Clip. We also provide attention visualizations and the top 3 relevant scene elements for each video to show where BiMa …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

76 extracted references · 73 canonical work pages

  1. [1]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G¨ul Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 1708–1718, 2021. 6

    work page 2021

  2. [2]

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings.Annual Conference on Neural Information Processing Systems (NeurIPS), pages 4349–4357,

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings.Annual Conference on Neural Information Processing Systems (NeurIPS), pages 4349–4357,

    work page

  3. [3]

    Chen and William B

    David L. Chen and William B. Dolan. Collecting highly parallel data for paraphrase evaluation. InAnnual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL), pages 190–200, 2011. 1, 5

    work page 2011

  4. [4]

    Fine-grained video-text retrieval with hierarchical graph reasoning

    Shizhe Chen, Yida Zhao, Qin Jin, and Qi Wu. Fine-grained video-text retrieval with hierarchical graph reasoning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10635– 10644, 2020. 7

    work page 2020

  5. [5]

    ”factual” or ”emotional”: Stylized image captioning with adaptive learning and attention

    Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, and Jiebo Luo. ”factual” or ”emotional”: Stylized image captioning with adaptive learning and attention. In European Conference on Computer Vision (ECCV), pages 527–543,

    work page

  6. [6]

    Tagging before alignment: Integrating multi-modal tags for video-text retrieval

    Yizhen Chen, Jie Wang, Lijian Lin, Zhongang Qi, Jin Ma, and Ying Shan. Tagging before alignment: Integrating multi-modal tags for video-text retrieval. InAAAI Conference on Artificial Intelligence (AAAI), pages 396–404, 2023. 6, 8

    work page 2023

  7. [7]

    UATVR: uncertainty-adaptive text-video retrieval

    Bo Fang, Wenhao Wu, Chang Liu, Yu Zhou, Yuxin Song, Weiping Wang, Xiangbo Shu, Xiangyang Ji, and Jingdong Wang. UATVR: uncertainty-adaptive text-video retrieval. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 13677–13687, 2023. 6, 8

    work page 2023

  8. [8]

    Multi-modal transformer for video retrieval

    Valentin Gabeur, Chen Sun, Karteek Alahari, and Cordelia Schmid. Multi-modal transformer for video retrieval. InEuropean Conference on Computer Vision (ECCV), pages 214–229, 2020. 5

    work page 2020

Show all 76 references
  1. [9]

    Word embeddings quantify 100 years of gender and ethnic stereotypes

    Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):3635– 3644, 2018. 8

    2018

  2. [10]

    X-pool: Cross- modal language-video attention for text-video retrieval

    Satya Krishna Gorti, No ¨el V ouitsis, Junwei Ma, Keyvan Golestan, Maksims V olkovs, Animesh Garg, and Guangwei Yu. X-pool: Cross- modal language-video attention for text-video retrieval. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4996–5005, 2022. 6

    2022

  3. [11]

    Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaim- ing He

    Priya Goyal, Piotr Doll´ar, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaim- ing He. Accurate, large minibatch SGD: training imagenet in 1 hour. arXiv preprint arXiv:706.02677, 2017. 5

    2017

  4. [12]

    Mscap: Multi-style image captioning with unpaired stylized text

    Longteng Guo, Jing Liu, Peng Yao, Jiangwei Li, and Hanqing Lu. Mscap: Multi-style image captioning with unpaired stylized text. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4204–4213, 2019. 9

    2019

  5. [13]

    Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan C. Russell. Localizing moments in video with natural language. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 5804–5813, 2017. 1, 5

    2017

  6. [14]

    Burgess, Xavier Glorot, Matthew M

    Irina Higgins, Lo¨ıc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Ler- chner. beta-V AE: Learning basic visual concepts with a constrained variational framework. InInternational Conference on Learning Representat...

    2017

  7. [15]

    Reducing sentiment bias in language models via counterfactual evalu- ation

    Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. Reducing sentiment bias in language models via counterfactual evalu- ation. InFindings of Conference on Empirical Methods in Natural Language Proce...

    2020

  8. [16]

    Narrating the video: Boosting text-video retrieval via comprehensive utilization of frame- level captions

    Chan Hur, Jeong hun Hong, Dong hun Lee, Dabin Kang, Semin Myeong, Sang hyo Park, and Hyeyoung Park. Narrating the video: Boosting text-video retrieval via comprehensive utilization of frame- level captions. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR...

    2025

  9. [17]

    Imagenet-x: Un- derstanding model mistakes with factor of variation annotations

    Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Un- derstanding model mistakes with factor of variation annotations. In International Confe...

    2023

  10. [18]

    Clifton, and Jie Chen

    Peng Jin, Jinfa Huang, Fenglin Liu, Xian Wu, Shen Ge, Guoli Song, David A. Clifton, and Jie Chen. Expectation-maximization contrastive learning for compact video-and-language representations. InAnnual Conference on Neural Information Processing Systems (NeurIPS), pages 30291–3...

    2022

  11. [19]

    Video-text as game players: Hi- erarchical banzhaf interaction for cross-modal representation learning

    Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, and Jie Chen. Video-text as game players: Hi- erarchical banzhaf interaction for cross-modal representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVP...

    2023

  12. [20]

    Text-video retrieval with disentangled conceptualization and set-to-set alignment

    Peng Jin, Hao Li, Zesen Cheng, Jinfa Huang, Zhennan Wang, Li Yuan, Chang Liu, and Jie Chen. Text-video retrieval with disentangled conceptualization and set-to-set alignment. InInternational Joint Conference on Artificial Intelligence (IJCAI), pages 938–946, 2023. 6

    2023

  13. [21]

    DiffusionRet: Generative text-video retrieval with diffusion model

    Peng Jin, Hao Li, Zesen Cheng, Kehan Li, Xiangyang Ji, Chang Liu, Li Yuan, and Jie Chen. DiffusionRet: Generative text-video retrieval with diffusion model. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 2470–2481, 2023. 6, 8

    2023

  14. [22]

    Disentangled representation learning for non-parallel text style trans- fer

    Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. Disentangled representation learning for non-parallel text style trans- fer. InAnnual Meeting of the Association for Computational Linguis- tics: Human Language Technologies (ACL), pages 424–434, 2019. 9

    2019

  15. [23]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInternational Conference on Learning Representation (ICLR), 2015. 5

    2015

  16. [24]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. InInternational Conference on Learning Representation (ICLR), 2014. 4

    2014

  17. [25]

    Dense-captioning events in videos

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. InIEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 706–715, 2017. 1, 5

    2017

  18. [26]

    3db: A framework for debugging com- puter vision models

    Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Lo- gan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, et al. 3db: A framework for debugging com- puter vision models. InAnnual Conference on Neural Information Processing Systems (N...

    2022

  19. [27]

    Berg, Mohit Bansal, and Jingjing Liu

    Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, and Jingjing Liu. Less is more: Clipbert for video-and- language learning via sparse sampling. InIEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), pages 7331–7341, 2021. 6

    2021

  20. [28]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024. 13

    2024 arXiv

  21. [29]

    Prototype-based aleatoric uncertainty quantification for cross-modal retrieval

    Hao Li, Jingkuan Song, Lianli Gao, Xiaosu Zhu, and Hengtao Shen. Prototype-based aleatoric uncertainty quantification for cross-modal retrieval. InAnnual Conference on Neural Information Processing Systems (NeurIPS), pages 24564–2458, 2023. 8

    2023

  22. [30]

    Blip: Boot- strapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Boot- strapping language-image pre-training for unified vision-language understanding and generation. InInternational Conference on Ma- chine Learning (ICML), pages 12888–12900, 2022. 1, 8 10

    2022

  23. [31]

    Clip-event: Connecting text and images with event structures

    Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, and Shih-Fu Chang. Clip-event: Connecting text and images with event structures. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16420–16429, 2022. 1

    2022

  24. [32]

    Towards debiasing sentence representations

    Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Rus- lan Salakhutdinov, and Louis-Philippe Morency. Towards debiasing sentence representations. InAnnual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL),

  25. [33]

    Text-adaptive multiple visual pro- totype matching for video-text retrieval.Annual Conference on Neu- ral Information Processing Systems (NeurIPS), pages 38655–38666,

    Chengzhi Lin, Ancong Wu, Junwei Liang, Jun Zhang, Wenhang Ge, Wei-Shi Zheng, and Chunhua Shen. Text-adaptive multiple visual pro- totype matching for video-text retrieval.Annual Conference on Neu- ral Information Processing Systems (NeurIPS), pages 38655–38666,

  26. [34]

    Disentangled multimodal representation learning for recommendation.IEEE Transactions on Multimedia, 25:7149– 7159, 2022

    Fan Liu, Huilin Chen, Zhiyong Cheng, Anan Liu, Liqiang Nie, and Mohan Kankanhalli. Disentangled multimodal representation learning for recommendation.IEEE Transactions on Multimedia, 25:7149– 7159, 2022. 5

    2022

  27. [35]

    Ts2-net: Token shift and selection transformer for text-video retrieval

    Yuqi Liu, Pengfei Xiong, Luhui Xu, Shengming Cao, and Qin Jin. Ts2-net: Token shift and selection transformer for text-video retrieval. InEuropean Conference on Computer Vision (ECCV), pages 319–335,

  28. [36]

    A decade’s battle on dataset bias: Are we there yet? InICLR, 2025

    Zhuang Liu and Kaiming He. A decade’s battle on dataset bias: Are we there yet? InICLR, 2025. 1, 2, 8, 9

    2025

  29. [37]

    Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning.Neurocomputing, 508:293–304,

    Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning.Neurocomputing, 508:293–304,

  30. [38]

    Hauptmann, Jo ˜ao F

    Mandela Patrick, Po-Yao Huang, Yuki Markus Asano, Florian Metze, Alexander G. Hauptmann, Jo ˜ao F. Henriques, and Andrea Vedaldi. Support-set bottlenecks for video-text representation learning. In International Conference on Learning Representations (ICLR), 2021. 6

    2021

  31. [39]

    Find- ing and fixing spurious patterns with explanations.Transactions on Machine Learning Research, 2022

    Gregory Plumb, Marco T ´ulio Ribeiro, and Ameet Talwalkar. Find- ing and fixing spurious patterns with explanations.Transactions on Machine Learning Research, 2022. 9

    2022

  32. [40]

    Learning transferable visual mod- els from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual mod- els from natural language supervision. InInternational Conference on Machine Learning (I...

    2021

  33. [41]

    Movie description.International Journal of Computer Vision, 123: 94–120, 2017

    Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movie description.International Journal of Computer Vision, 123: 94–120, 2017. 1, 5

    2017

  34. [42]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakr- ishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 618–626, 2017. 15

    2017

  35. [43]

    Tempme: Video temporal token merging for efficient text-video retrieval

    Leqi Shen, Tianxiang Hao, Tao He, Sicheng Zhao, Yifeng Zhang, pengzhang liu, Yongjun Bao, and Guiguang Ding. Tempme: Video temporal token merging for efficient text-video retrieval. InICLR,

  36. [44]

    In-style: Bridging text and uncurated videos with style transfer for text-video retrieval

    Nina Shvetsova, Anna Kukleva, Bernt Schiele, and Hilde Kuehne. In-style: Bridging text and uncurated videos with style transfer for text-video retrieval. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21981–21992, 2023. 1, 6

    2023

  37. [45]

    Unbiasing through textual descriptions: Mitigat- ing representation bias in video benchmarks

    Nina Shvetsova, Arsha Nagrani, Bernt Schiele, Hilde Kuehne, and Christian Rupprecht. Unbiasing through textual descriptions: Mitigat- ing representation bias in video benchmarks. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2025. 1, 2, 8, 9

    2025

  38. [46]

    Belding, Kai-Wei Chang, and William Yang Wang

    Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth M. Belding, Kai-Wei Chang, and William Yang Wang. Mitigating gender bias in natural language processing: Literature review. InAnnual Meeting of the Association for Computational ...

    2019

  39. [47]

    Detach and attach: Stylized image captioning without paired stylized dataset

    Yutong Tan, Zheng Lin, Peng Fu, Mingyu Zheng, Lanrui Wang, Yanan Cao, and Weiping Wang. Detach and attach: Stylized image captioning without paired stylized dataset. InACM International Conference on Multimedia (MM), pages 4733–4741, 2022. 9

    2022

  40. [48]

    Qwen2.5: A party of foundation models, 2024

    Qwen Team. Qwen2.5: A party of foundation models, 2024. 13

    2024

  41. [49]

    Holistic features are almost sufficient for text-to-video retrieval

    Kaibin Tian, Ruixiang Zhao, Zijie Xin, Bangxiang Lan, and Xirong Li. Holistic features are almost sufficient for text-to-video retrieval. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17138–17147, 2024. 1, 6, 8

    2024

  42. [50]

    Representa- tion learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    A¨aron van den Oord, Yazhe Li, and Oriol Vinyals. Representa- tion learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 5

    2018 arXiv

  43. [51]

    Visualizing data using t-SNE.Journal of Machine Learning Research, 9:2579–2605, 2008

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE.Journal of Machine Learning Research, 9:2579–2605, 2008. 8

    2008

  44. [52]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Atten- tion is all you need. InAnnual Conference on Neural Information Processing Systems (NeurIPS), pages 5998–6008, 2017. 4

    2017

  45. [53]

    Aoe-net: Entities interactions modeling with adaptive attention mechanism for temporal action proposals gener- ation.International Journal of Computer Vision, 131(1):302–323,

    Khoa V o, Sang Truong, Kashu Yamazaki, Bhiksha Raj, Minh-Triet Tran, and Ngan Le. Aoe-net: Entities interactions modeling with adaptive attention mechanism for temporal action proposals gener- ation.International Journal of Computer Vision, 131(1):302–323,

  46. [54]

    Dianat, Majid Rabbani, Raghuveer Rao, and Zhiqiang Tao

    Jiamian Wang, Pichao Wang, Dongfang Liu, Qiang Guan, Sohail A. Dianat, Majid Rabbani, Raghuveer Rao, and Zhiqiang Tao. Diffusion- inspired truncated sampler for text-video retrieval. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Informatio...

    2024

  47. [55]

    Towards fairness in visual recognition: Effective strategies for bias mitigation

    Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8916...

    2020

  48. [56]

    Text proxy: De- composing retrieval from a 1-to-n relationship into n 1-to-1 relation- ships for text-video retrieval

    Jian Xiao, Zhenzhen Hu, Jia Li, and Richang Hong. Text proxy: De- composing retrieval from a 1-to-n relationship into n 1-to-1 relation- ships for text-video retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, 2025. 1, 5, 6

    2025

  49. [57]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5288–5296, 2016. 1, 5

    2016

  50. [58]

    Vltint: visual-linguistic transformer-in-transformer for coherent video paragraph captioning

    Kashu Yamazaki, Khoa V o, Quang Sang Truong, Bhiksha Raj, and Ngan Le. Vltint: visual-linguistic transformer-in-transformer for coherent video paragraph captioning. InAAAI Conference on Artificial Intelligence (AAAI), pages 3081–3090, 2023. 9

    2023

  51. [59]

    Video-text pre-training with learned regions for retrieval

    Rui Yan, Mike Zheng Shou, Yixiao Ge, Jinpeng Wang, Xudong Lin, Guanyu Cai, and Jinhui Tang. Video-text pre-training with learned regions for retrieval. InAAAI Conference on Artificial Intelligence (AAAI), pages 3100–3108, 2023. 6, 8

    2023

  52. [60]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guant- ing Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzhe...

    2024 arXiv

  53. [61]

    Dgl: Dynamic global-local prompt tuning for text-video retrieval

    Xiangpeng Yang, Linchao Zhu, Xiaohan Wang, and Yi Yang. Dgl: Dynamic global-local prompt tuning for text-video retrieval. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 6540–6548, 2024. 6

    2024

  54. [62]

    Coca: Contrastive captioners are image- text foundation models.Transactions on Machine Learning Research,

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyed- hosseini, and Yonghui Wu. Coca: Contrastive captioners are image- text foundation models.Transactions on Machine Learning Research,

  55. [63]

    Gender bias in contextualized word embeddings

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. Gender bias in contextualized word embeddings. InConference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (NAACL-HLT), pag...

    2019

  56. [64]

    slicing mango

    Shuai Zhao, Linchao Zhu, Xiaohan Wang, and Yi Yang. Centerclip: Token clustering for efficient text-video retrieval. InACM SIGIR In- ternational Conference on Research and Development in Information Retrieva (SIGIR), pages 970–981, 2022. 6 12 Appendix Table .1. Effect of the n...

    2022

  57. [65]

    A bulldozer removes dirt MSR-VTT 2) An infomercial with a pharmeceutical company talking about an epilepsy drug pending approval from the FDA (∼43 characters3) Extreme violence scenes with people fighting with each other in a text) 4) A woman is in front of a whiteboard talkin...

  58. [66]

    A man is playing an instrument

  59. [67]

    A lady is pouring raw strawberry juice into a bowl MSVD 2) A man is slicing the crust into a potato (∼31 characters3) A man lifts three sunflowers in a text) 4) A man is putting a pan into an oven

  60. [68]

    A boy rides around in circles on a tricycle

  61. [69]

    LSMDC 2) He slaps SOMEONE again

    The dish is covered in saffron and spices. LSMDC 2) He slaps SOMEONE again. (∼46 characters3) He and SOMEONE join forces to grab the cube, which’s connected with several more wire. in a text) 4) In the race, a rider falls

  62. [70]

    The cup spins across the floor

    SOMEONE dashes to a clothes closet and ducks inside. The cup spins across the floor

  63. [71]

    ActivityNet 2) A woman is seen kneeling down next to a man

    A man is seen speaking to the camera and pans out into more men standing behind him. ActivityNet 2) A woman is seen kneeling down next to a man. (∼67 characters3) Several people are shown doing long boarding stunts in the parking lots of a college. in a text) 4) People are sit...

  64. [72]

    Several shots are shown of a man speaking to groups of people and leads into people wearing wet suits and walking. DiDeMo

  65. [73]

    They first start crouching and hitting the ground with the sticks in a text)

    First time we see the dancers go down on one leg the men hit the ground with their sticks. They first start crouching and hitting the ground with the sticks in a text)

  66. [74]

    The guitarist is looking straight up

    When the man puts his head down the guitar player is looking up. The guitarist is looking straight up. A man plays the guitar while looking up. The guitarist is looking straight up as he plays. (∼147 characters3) Red phone booth is visible a red phone booth is in the scene. A ...

  67. [75]

    White square exits frame left the camera pans back the way it came

    The camera moves back to the left to the tree. White square exits frame left the camera pans back the way it came. Square area lines with stones comes into view the fence comes into view

  68. [76]

    a woman spins around several times very fast

    Fog moves in toward the ice skater. a woman spins around several times very fast. A woman pirouettes as she comes near the camera. Woman spins more than 5 times in a row. Table D.5. Five random examples of text descriptions in different datasets. For each dataset, we also repo...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.