Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning

Liu Qin; Qiang Chen; Qing Gu; Shiping Ge; Yafeng Yin; Zhiwei Jiang; Ziyao Chen

REVIEW 3 major objections 6 minor 42 references

Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that complementary masking lets a captioning model learn event locations implicitly from video-level captions alone, removing the need for temporal annotations and complex proposal pipelines.

desk verdict A solid, well-evaluated WSDVC method with a genuinely new masking trick, though the localization mechanism is less clean than the title implies. read the letter →

arxiv 2412.12791 v2 pith:DICSPAHQ submitted 2024-12-17 cs.CV cs.AIcs.MM

Shiping Ge , Qiang Chen , Zhiwei Jiang , Yafeng Yin , Liu Qin , Ziyao Chen , Qing Gu This is my paper · ORCID

classification cs.CVcs.AIcs.MM

keywords Weakly-superviseddensevideocaptioningComplementarymaskingImplicitlocation-captionalignmentGaussianmaskEventlocalizationTemporalgroundingDual-mode

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dense video captioning normally requires knowing where each event starts and ends in a video; this paper claims those temporal boundaries can be learned implicitly from caption text alone. The trick is complementary masking: for each event the model predicts a soft Gaussian mask, then must write that event's caption from the masked video and all other captions from the inverse mask, so the two half-descriptions have to fit together into the full video description. That pairing pushes each mask onto the region whose visual content actually supports its caption. On ActivityNet Captions the method beats every weakly supervised baseline and lands close to fully supervised systems, and the authors report the first weakly supervised results on YouCook2 and ViTT. If the claim holds, the elaborate event-proposal machinery of prior weakly supervised methods — predefined proposals, external detectors, heavy inference-time sampling — is unnecessary.

What carries the argument

Complementary masking with differentiable Gaussian masks is the load-bearing mechanism. For each event, a Transformer decoder over learnable event embeddings and temporally encoded CLIP frame embeddings predicts a center $\mu_i$ and width $\sigma_i$, which define a Gaussian mask $M_i(t) = \exp\big(-(t/N_v - \mu_i)^2 / (2(\sigma_i/\tau)^2)\big)$ over the frame dimension. The positive captioning loss $L(\hat{v}_b, \hat{r})$ requires that the caption written from the masked video be exactly that event's caption, the negative loss $L(\check{v}_b, \check{r})$ requires the inverse mask to yield all other captions, and a cosine-similarity diversity loss keeps the masks apart; the two losses sum so that writing the full description forces each mask to sit on the region that visually supports its caption. The dual-mode captioning module supplies the supporting bookkeeping: a full-video mode with a '[FULL] N events' prompt fixes the event count, and masked modes with '[MASK] 1 event' / '[MASK] N−1 events' prompts structure the complementary pairs. This mechanism replaces the explicit proposal-and-matching loop of earlier weakly supervised systems.

What would settle it

Retrain the model on ActivityNet Captions with each video's sentence annotations randomly permuted across events: if predicted masks still localize the true events about as well as in the intact training run, the captions cannot be supplying the localization signal. As a cheaper check, measure the CIDEr gap between captions generated from a predicted mask and from a random mask of the same width on the validation set — a near-zero gap would mean the masks are not aligned to caption content.

Watch

Extended reading notes

Core claim

The central discovery is that an event's location and its caption can be aligned with no temporal supervision at all, using only the requirement that captions generated from a masked video and from its inverse mask be complementary. A mask generation module predicts a Gaussian center and width for each event from a learned event embedding; positive masking keeps only that region and must reproduce the event's caption, while negative masking keeps everything else and must reproduce the remaining captions. Because caption accuracy depends on which region the mask reveals, gradient flows from the two captioning losses reshape the masks toward the true event segments. With CLIP features the full model reaches a SODA score of 6.08 and CIDEr of 33.42 on ActivityNet Captions, surpassing all weakly supervised comparators and a fully supervised Vid2Seq baseline; temporal localization recall (53.72) trails fully supervised PDVC (55.42) by a small margin, suggesting implicit alignment recovers most of what explicit boundary supervision provides.

Load-bearing premise

The mask predictor is trained only through captioning losses, with nothing ever comparing a predicted mask to a true event segment, so the entire method rests on the bet that caption accuracy is sensitive enough to which region of the video the mask reveals to push each Gaussian onto the real event.

Editorial extensions

If this is right

Event localization in WSDVC can be trained without any proposal generator: the mask module is learned purely from captioning losses, so inference needs only a single pass through the mask predictor rather than sampling many candidate segments.
The event count is recovered from the full-video captioning mode at inference, letting the pipeline handle videos of unknown content without a separate counting or proposal stage.
The Gaussian (soft) mask is essential to the mechanism: replacing it with a hard binary mask collapses CIDEr from 33.42 to 16.96, because the hard mask blocks gradients from reaching the predicted location.
Both masked captioning directions matter — dropping the positive loss drops CIDEr to 20.22, while dropping the negative loss costs about 3 points — so complementarity, not just masking, is what drives alignment.
The same recipe transfers to other datasets: the model reports the first weakly supervised results on YouCook2 and ViTT and outperforms the reimplemented baselines there.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same complementary-masking loss should apply to other weakly supervised grounding tasks — temporal sentence grounding, moment retrieval, or weakly supervised action localization — wherever a captioning head can be made sensitive to which region it sees; the paper only demonstrates it for dense video captioning.
The paper's inference still uses a refinement stage that re-generates captions from the predicted masks; an untested consequence of the training scheme is that the masks alone might be precise enough to serve as final timestamps, making inference a single forward pass.
Because nothing ties the mask to a true boundary, the mechanism's precision is bounded by the caption model's sensitivity to content: on datasets where captions are predictable from global video cues, the masks should drift to coarse or arbitrary regions while caption scores stay high — a failure mode the overlap analysis only partially probes.
The explicit complementary structure suggests a stronger test than the paper runs: if captions are randomly permuted across events during training and localization accuracy survives, then the alignment signal is not coming from the caption–region pairing.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the complementary masking training paradigm: predict Gaussian masks for events, then train positive and negative masked captioning losses so that captions generated from the masked and inverse-masked video complement each other. That is a real departure from the reconstruction and distillation approaches in the cited weakly supervised work, and it removes the proposal machinery entirely. The experiments back it up: three datasets, multiple metrics, ablations for every component, re-run baselines, a model-size comparison, overlap analysis, and released code. The gains over prior weakly supervised methods are consistent, and the method also lands close to fully supervised baselines. This is careful empirical work.

The soft spot is the localization mechanism. In training, the full target captions are already present in the prompt text, so a GPT-2 decoder can largely copy from the prompt; the gradient reaching the mask generator through the visual prefix is concentrated in the first few tokens and may not strongly constrain mask boundaries. The paper does not include a shuffled-caption control (e.g., corrupting or permuting the prompt captions) to confirm that the masks actually respond to visual-caption alignment rather than a combination of event-embedding order, the diversity loss, and the temporal encoder's inductive bias. The overlap analysis in Table 8 is a partial response, and the ablations show the positive masked captioning loss is essential, so the mechanism is not empty. But it remains an empirical bet.

Minor issues: no variance estimates, and no theoretical guarantee for the alignment. These are minor and fairly standard for this subfield.

The stress-test concern does not sink the paper. It points to a missing control that should be easy to run and would strengthen the causal story. The central empirical claim—that complementary masking outperforms existing weakly supervised methods without proposals—holds up. This paper is for researchers working on dense video captioning and weakly supervised temporal grounding. It deserves a serious referee and, in my view, acceptance with a request for the shuffled-caption control and variance reporting in revision.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes a weakly-supervised dense video captioning (WSDVC) method that learns temporal event localization through complementary masking. A dual-mode captioning module generates full-video captions (providing the number of events) and masked-video captions, while a mask generation module produces differentiable Gaussian masks. Positive and negative masked-captioning losses and a diversity loss train the mask generator without temporal annotations. Experiments on ActivityNet Captions, YouCook2, and ViTT report consistent improvements over weakly-supervised baselines and results competitive with fully-supervised methods. The paper includes ablations for each component, an analysis of mask types, model size, hyperparameters, and event overlap robustness, and it releases code.

Significance. If the claimed mechanism holds, the work offers a practical simplification of WSDVC: it removes external proposal detectors and random proposal sampling while achieving state-of-the-art weakly-supervised performance and competitive fully-supervised results. The manuscript is unusually transparent: it reruns baseline methods on new datasets, ablates each loss and mask type, and provides code. The main weakness is that the core mechanism—that complementary captioning losses are what drive accurate localization—is not directly verified; this is the key risk to the paper's central claim and should be addressed before publication.

major comments (3)

The central claim that the complementary masked-captioning losses produce implicit location-caption alignment is not isolated from alternative training signals. Table 3 ablates the positive and negative captioning losses and the diversity loss against composite captioning metrics, but the localization metrics from Table 7 are not reported for these ablations. Consequently, the strong localization performance in Table 7 could be driven by the diversity loss, the learned event queries, or the temporal encoder's inductive bias rather than by the complementary captioning objective. A control that breaks the correspondence between masks and captions (e.g., training with a random permutation of the captions assigned to the masks, or with captions from a different video) would test the mechanism directly. This is a load-bearing point for the paper's stated paradigm and should be addressed with additional experiments or with localization-metric ablations.
The event embeddings e_i are described only as a 'randomly initialized learnable embedding' for each caption proposal. It is not specified whether these embeddings are globally shared queries, per-video queries, or initialized from caption text. If they are globally shared and ordered, the positive captioning loss may simply align each query to the i-th event in the fixed temporal order of the ground-truth captions, meaning the reported alignment is partly a consequence of the order rather than of learned semantic correspondence. The authors should clarify the initialization and discuss the implications for videos whose caption order does not follow the temporal order.
The prompts for both positive and negative masked captioning contain the full target captions. Because the decoder is a pretrained language model, a large fraction of the caption tokens can be predicted from the text prefix, and the gradient signal reaching the mask generator through the masked video features is concentrated in the first token(s). The paper does not quantify this effect. The authors should either report an analysis of gradient magnitudes or run a control in which the caption is removed from the prompt (using only '[MASK] 1 event:') to show that the mask can still be trained. This would strengthen the claim that the visual masking signal, rather than the language prior, drives localization.

minor comments (6)

There are minor grammatical errors, e.g., 'the temporal location of event' and 'an dual-mode'. These should be corrected.
The notation T_{n,i} appears in the parameter list but not in the summation, and the denominator N is not explicitly defined in the equation. Please clarify the token-counting conventions.
The main text states that scores are averaged over tIoU thresholds 0.3, 0.5, 0.7, and 0.9, but Table 1's caption does not mention this. Add this information to the table caption for clarity.
The model sizes reported for GPT2 are inconsistent: Table 4 lists 'GPT2-Base' as 104.62M, while Supplementary Table 6 lists 'GPT2' as 163.89M. Please reconcile the numbers and clarify which configuration is used in each table.
The paper does not report standard deviations or significance tests for any of the main results. For a method claiming consistent improvements, it would be helpful to report variance across runs or at least state that differences are stable across seeds.
The reimplementation of WSDEC and PWS-DVC on YouCook2 and ViTT lacks implementation details. Please provide hyperparameters, number of epochs, and any preprocessing shared with the proposed method to support reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: localization is trained end-to-end from captioning losses and evaluated on held-out annotations, so it does not reduce to its inputs by construction.

full rationale

The paper's claimed derivation chain—using complementary positive/negative masked captioning to train an implicit event localizer—does not equate any prediction to its training input by construction. The mask generator (Eq. 6-8) predicts centers and widths from a Transformer decoder over random event embeddings and frame embeddings; these predicted masks are free variables, not copies of temporal annotations or of fitted localization outputs. The objective (Eq. 16) combines the positive masked captioning loss (Eq. 12), the negative masked captioning loss (Eq. 15), and a diversity regularizer (Eq. 9), where the supervised targets are the provided captions themselves. The masks are optimized only through gradients from caption generation, and the evaluation (Tables 1, 2, and 7) is performed on held-out ActivityNet Captions, YouCook2, and ViTT sets against temporal annotations not used in training. No parameter is fit to a localization output and then renamed as a prediction; the hyperparameters tau and gamma are tuned on validation, which is standard practice. The only potentially self-referential element is the citation of prior work (Zheng et al. 2022a,b) for Gaussian-mask design, but the paper explicitly notes that 'other functions generating soft masks can be used as alternatives to the Gaussian mask,' so that citation is not load-bearing. The skeptic's concern that the captioning gradient may be too weak or insufficiently constraining to localize events is an empirical risk about optimization and supervision strength, not a circularity: it does not show that the claimed result is assumed rather than derived.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper's method is purely a neural architecture and loss design. It does not introduce new physical entities or quantities. The only tunable parameters are hyperparameters tau and gamma, and standard model parameters.

free parameters (2)

tau = 2.0
Controls the steepness of the Gaussian mask. Tuned on validation set; scores increase with tau up to 2.0 then decrease (Figure 4a-b).
gamma = 0.8
Overlap threshold in diversity loss. Tuned on validation set; optimal around 0.8 (Figure 4c-d).

assumptions (3)

domain assumption Complementary masked captioning losses are a sufficient supervisory signal for learning event temporal masks without boundary annotations.
The mask predictor in Section 3.3 is trained only through losses L(hat v_b, hat r) and L(check v_b, check r) (Eq. 12 and 15). No temporal supervision is used. The paper implies this is sufficient for event localization.
domain assumption Gaussian masks parameterized by center and width can represent the temporal location of events.
Eq. 8 defines the Gaussian mask with predicted center mu_i and width sigma_i. The choice is motivated by differentiability and ablations favoring it over sigmoid and Cauchy masks.
domain assumption A pretrained CLIP image encoder provides feature representations suitable for temporal localization.
The spatial encoder Ea is a frozen CLIP ViT-L/14 (Section 3.2). The paper relies on this to capture semantics for captioning and localization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning." pith.science (2026). https://pith.science/paper/DICSPAHQ

@misc{pith2026241212791,
  author       = {Pith},
  title        = {Pith review of: Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DICSPAHQ}},
  note         = {Machine review of arXiv:2412.12791}
}

read the original abstract

Weakly-Supervised Dense Video Captioning (WSDVC) aims to localize and describe all events of interest in a video without requiring annotations of event boundaries. This setting poses a great challenge in accurately locating the temporal location of event, as the relevant supervision is unavailable. Existing methods rely on explicit alignment constraints between event locations and captions, which involve complex event proposal procedures during both training and inference. To tackle this problem, we propose a novel implicit location-caption alignment paradigm by complementary masking, which simplifies the complex event proposal and localization process while maintaining effectiveness. Specifically, our model comprises two components: a dual-mode video captioning module and a mask generation module. The dual-mode video captioning module captures global event information and generates descriptive captions, while the mask generation module generates differentiable positive and negative masks for localizing the events. These masks enable the implicit alignment of event locations and captions by ensuring that captions generated from positively and negatively masked videos are complementary, thereby forming a complete video description. In this way, even under weak supervision, the event location and event caption can be aligned implicitly. Extensive experiments on the public datasets demonstrate that our method outperforms existing weakly-supervised methods and achieves competitive results compared to fully-supervised methods.

Figures

Figures reproduced from arXiv: 2412.12791 by the authors.

**Figure 1.** Comparison of our complementary masking paradigm with previous paradigms for event localization. task, which only relies on video-level captions and does not require extensive temporal location annotations for training, has been proposed and appears to be more feasible for practical applications (Duan et al. 2018). However, the lack of supervision on event localization poses a great challenge in accurately locating… view at source ↗

**Figure 2.** Illustration of our proposed framework, which consists of two main components: a Dense Video Captioning model [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** Impact of τ in the Gaussian mask construction (a-b) and impact of γ in the diversity loss (c-d). result can be attributed to the fact that the hard binary mask is non-differentiable, which prevents the optimization of the event proposal generation process. Replacing the Gaussian mask with a Sigmoid mask (Duan et al. 2018) or a Cauchy mask (based on Cauchy Probability density function) also leads to a decrease in per… view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: A Qualitative Example from Activity Caption. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Comparison of different masks with µ = 0.5 and σ = 0.3. Dataset Train Val Test Dur (mins) Sents ActivityNet 10,024 4, 926 5,044 2.0 3.7 ViTT 4915 2487 2471 4.2 7.1 YouCook2 1333 457 210 5.3 7.7 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: Impact of the number of video frames. temporally annotated sentence descriptions, covering various themes. The statistics are shown in [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 19 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Banerjee, S.; and Lavie, A. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65--72

2005
[4]

Buch, S.; Escorcia, V.; Shen, C.; Ghanem, B.; and Carlos Niebles, J. 2017. Sst: Single-stream temporal action proposals. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2911--2920

work page 2017
[5]

Chen, S.; and Jiang, Y.-G. 2021. Towards bridging event captioner and sentence localizer for weakly supervised dense event captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8425--8435

work page 2021
[6]

Chen, S.; Song, Y.; Zhao, Y.; Jin, Q.; Zeng, Z.; Liu, B.; Fu, J.; and Hauptmann, A. 2019. Activitynet 2019 task 3: Exploring contexts for dense captioning events in videos. arXiv preprint arXiv:1907.05092

arXiv 2019
[7]

Chidume, C. 1987. Iterative approximation of fixed points of Lipschitzian strictly pseudocontractive mappings. Proceedings of the American Mathematical Society, 99(2): 283--288

work page 1987
[8]

Choi, W.; Chen, J.; and Yoon, J. 2023. PWS-DVC: Enhancing Weakly Supervised Dense Video Captioning With Pretraining Approach. IEEE Access, 11: 128162--128174

work page 2023

Show all 42 references

[9]

Duan, X.; Huang, W.; Gan, C.; Wang, J.; Zhu, W.; and Huang, J. 2018. Weakly supervised dense event captioning in videos. Advances in Neural Information Processing Systems, 31

2018
[10]

Fujita, S.; Hirao, T.; Kamigaito, H.; Okumura, M.; and Nagata, M. 2020. SODA: Story oriented dense video captioning evaluation framework. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part VI 16, 517--531. Springer

2020
[11]

Gao, L.; Guo, Z.; Zhang, H.; Xu, X.; and Shen, H. T. 2017. Video captioning with attention-based LSTM and semantic consistency. IEEE Transactions on Multimedia, 19(9): 2045--2055

2017
[12]

Huang, G.; Pang, B.; Zhu, Z.; Rivera, C.; and Soricut, R. 2020. Multimodal pretraining for dense video captioning. arXiv preprint arXiv:2011.11760

2020 arXiv
[13]

Iashin, V.; and Rahtu, E. 2020. Multi-modal dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 958--959

2020
[14]

Kim, S.; Cho, J.; Yu, J.; Yoo, Y.; and Choi, J. Y. 2024. Gaussian Mixture Proposals with Pull-Push Learning Scheme to Capture Diverse Events for Weakly Supervised Temporal Video Grounding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2795--2803

2024
[15]

Krishna, R.; Hata, K.; Ren, F.; Fei-Fei, L.; and Carlos Niebles, J. 2017. Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, 706--715

2017
[16]

Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597

2023 arXiv
[17]

Li, Y.; Yao, T.; Pan, Y.; Chao, H.; and Mei, T. 2018. Jointly localizing and describing events for dense video captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7492--7500

2018
[18]

Lin, C.-Y. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74--81

2004
[19]

Lin, T.; Zhao, X.; Su, H.; Wang, C.; and Yang, M. 2018. Bsn: Boundary sensitive network for temporal action proposal generation. In Proceedings of the European conference on computer vision (ECCV), 3--19

2018
[20]

Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024. Visual instruction tuning. Advances in neural information processing systems, 36

2024
[21]

Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

2017 arXiv
[22]

Mun, J.; Yang, L.; Ren, Z.; Xu, N.; and Han, B. 2019. Streamlined dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6588--6597

2019
[23]

Nie, L.; Qu, L.; Meng, D.; Zhang, M.; Tian, Q.; and Bimbo, A. D. 2022. Search-oriented micro-video captioning. In Proceedings of the 30th ACM international conference on multimedia, 3234--3243

2022
[24]

Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311--318

2002
[25]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

2021
[26]

Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8): 9

2019
[27]

Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 1--67

2020
[28]

H.; Nagrani, A.; Arnab, A.; and Schmid, C

Seo, P. H.; Nagrani, A.; Arnab, A.; and Schmid, C. 2022. End-to-end generative pretraining for multimodal video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17959--17968

2022
[29]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[30]

Vedantam, R.; Lawrence Zitnick, C.; and Parikh, D. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4566--4575

2015
[31]

Wang, J.; Jiang, W.; Ma, L.; Liu, W.; and Xu, Y. 2018. Bidirectional attentive fusion with context gating for dense video captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7190--7198

2018
[32]

Wang, T.; Zhang, R.; Lu, Z.; Zheng, F.; Cheng, R.; and Luo, P. 2021. End-to-end dense video captioning with parallel decoding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6847--6857

2021
[33]

Wu, B.; Niu, G.; Yu, J.; Xiao, X.; Zhang, J.; and Wu, H. 2021. Weakly supervised dense video captioning via jointly usage of knowledge distillation and cross-modal matching. arXiv preprint arXiv:2105.08252

2021 arXiv
[34]

Xu, J.; Mei, T.; Yao, T.; and Rui, Y. 2016. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5288--5296

2016
[35]

H.; Miech, A.; Pont-Tuset, J.; Laptev, I.; Sivic, J.; and Schmid, C

Yang, A.; Nagrani, A.; Seo, P. H.; Miech, A.; Pont-Tuset, J.; Laptev, I.; Sivic, J.; and Schmid, C. 2023. Vid2seq: Large-scale pretraining of a visual language model for dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

2023
[36]

Zeng, R.; Huang, W.; Tan, M.; Rong, Y.; Zhao, P.; Huang, J.; and Gan, C. 2019. Graph convolutional networks for temporal action localization. In Proceedings of the IEEE/CVF international conference on computer vision, 7094--7103

2019
[37]

Zhao, Y.; Zhang, H.; Gao, Z.; Guan, W.; Wang, M.; and Chen, S. 2024. A Snippets Relation and Hard-Snippets Mask Network for Weakly-Supervised Temporal Action Localization. IEEE Transactions on Circuits and Systems for Video Technology

2024
[38]

Zheng, M.; Huang, Y.; Chen, Q.; and Liu, Y. 2022 a . Weakly supervised video moment localization with contrastive negative sample mining. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 3517--3525

2022
[39]

Zheng, M.; Huang, Y.; Chen, Q.; Peng, Y.; and Liu, Y. 2022 b . Weakly supervised temporal sentence grounding with gaussian-based contrastive proposal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15555--15564

2022
[40]

Zhou, L.; Xu, C.; and Corso, J. 2018. Towards automatic learning of procedures from web instructional videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32

2018
[41]

Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

2023 arXiv
[42]

V.; Wang, W

Zhu, W.; Pang, B.; Thapliyal, A. V.; Wang, W. Y.; and Soricut, R. 2022. End-to-end dense video captioning as sequence generation. arXiv preprint arXiv:2204.08121

2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Banerjee, S.; and Lavie, A. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65--72

2005

[4] [4]

Buch, S.; Escorcia, V.; Shen, C.; Ghanem, B.; and Carlos Niebles, J. 2017. Sst: Single-stream temporal action proposals. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2911--2920

work page 2017

[5] [5]

Chen, S.; and Jiang, Y.-G. 2021. Towards bridging event captioner and sentence localizer for weakly supervised dense event captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8425--8435

work page 2021

[6] [6]

Chen, S.; Song, Y.; Zhao, Y.; Jin, Q.; Zeng, Z.; Liu, B.; Fu, J.; and Hauptmann, A. 2019. Activitynet 2019 task 3: Exploring contexts for dense captioning events in videos. arXiv preprint arXiv:1907.05092

arXiv 2019

[7] [7]

Chidume, C. 1987. Iterative approximation of fixed points of Lipschitzian strictly pseudocontractive mappings. Proceedings of the American Mathematical Society, 99(2): 283--288

work page 1987

[8] [8]

Choi, W.; Chen, J.; and Yoon, J. 2023. PWS-DVC: Enhancing Weakly Supervised Dense Video Captioning With Pretraining Approach. IEEE Access, 11: 128162--128174

work page 2023