Pith. sign in

REVIEW 2 major objections 5 minor 55 references

Enhancing Video Memorability Prediction with Text-Motion Cross-modal Contrastive Loss and Its Application in Video Summarization

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Text-description similarity, used as a contrastive training signal for motion features, sets new state-of-the-art results in video memorability prediction and improves video summarization.

desk verdict Eq. 14 as written is inverted — minimizing it repels positives — so the reported gains can't be attributed to the stated method; the underlying idea is worth a careful revision. read the letter →

arxiv 2506.08649 v1 pith:3S7M3Z4A submitted 2025-06-10 cs.CV

Zhiyi Zhu , Xiaoyu Wu , Youwei Lu This is my paper
classification cs.CV
keywords videomemorabilitypredictioncontrastivelearningcross-modalmotionfeaturestext-motionalignmentI3Dfine-tuningsummarizationSpearmanrankcorrelation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that motion, an underused cue in video memorability prediction, can be made much more useful by borrowing structure from accompanying text descriptions. The proposed TMCCL loss builds positive and negative motion samples from text-description similarity during fine-tuning of an I3D motion encoder, letting semantically related motion content be pulled together without extra human labels. With this addition the full multimodal model reaches Spearman rank correlations of 0.692 on Memento10k and 0.614 (short-term) and 0.324 (long-term) on VideoMem, surpassing prior published results on both datasets. The same memorability predictions are then used to correct the importance scores of a video summarization model (MWCVS), improving F1 scores on SumMe and TVSum.

What carries the argument

The central object is TMCCL, a cross-modal contrastive loss applied during fine-tuning of the I3D motion encoder, a 3D convolutional network pre-trained for action recognition. Given a target video's motion feature $f_m$, positives are $K$ samples whose BERT text features are most similar to the target's among the top $2K$, negatives come from a MoCo-style queue of 1024 samples, and the loss is $\mathcal{L}_{TMCCL} = \log \frac{\sum_{f_m^+ \in P_i^m} sim(f_m, f_m^+)}{\sum_{f_m^+ \in P_i^m} sim(f_m, f_m^+) + \sum_{f_m^- \in N_i^m} sim(f_m, f_m^-)}$ with $sim(x,y)=\exp(x\cdot y/\tau)$ and $\tau=0.07$. It is added to the memorability MSE with weight $\lambda=0.5$. The second object, MWCVS, is the corrected summary score $\tilde Y_t = \hat Y_i + \mu \hat S_i$, where $\hat Y_i$ is the MSVA importance score, $\hat S_i$ the TMCCL-predicted memorability, and $\mu=0.5$.

What would settle it

Measure on Memento10k whether the top-$2K$ text-similar pairs selected by Algorithm 1 actually have higher motion-feature similarity than random pairs; if they do not, the positive pairs are noisy and TMCCL's reported gains cannot come from aligning semantically related motion. A direct ablation that replaces text-selected pairs with random pairs and observes the same memorability improvement would also falsify the claim that text cues are the effective signal.

Watch

Extended reading notes

Core claim

The central claim is that motion features fine-tuned with TMCCL carry complementary information from text cues and therefore predict memorability better than motion features trained on memorability labels alone. For each target video, the method selects as positive motion samples the videos whose BERT text embeddings are most similar to the target's, draws negatives from a dynamic queue of other videos, and adds the contrastive term to the MSE loss used to fine-tune I3D. The ablation results are presented as direct evidence: on Memento10k the motion-only ST rank correlation rises from 0.551 to 0.576 with TMCCL, and on VideoMem from 0.394 to 0.417 (ST) and 0.181 to 0.212 (LT); the full multimodal model improves to 0.692 and 0.614/0.324 respectively.

Load-bearing premise

The method assumes that videos with similar text descriptions also have similar motion content, so that pulling their motion features together during contrastive training improves memorability prediction; if text similarity and motion similarity come apart, the constructed pairs become noisy and the added loss can push motion features in the wrong direction.

Editorial extensions

If this is right

  • Motion-only memorability prediction improves on both datasets when TMCCL is added, so text-derived contrastive pairs can substitute for missing motion labels during fine-tuning.
  • The multimodal model sets a new best Spearman rank correlation on Memento10k (0.692) and on VideoMem short- and long-term (0.614 and 0.324), indicating the text-motion alignment helps on both datasets.
  • The same frozen memorability module improves MSVA's F1 on SumMe (56.6) and TVSum (61.9) when added as a weighted correction, so memorability predictions can serve as a perceptual prior for summary score selection.
  • Because MWCVS requires only a weighted sum of two scores, the correction can be applied to any summary model that outputs clip-level importance scores without retraining that model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The pairing scheme is effectively a proxy-label mechanism that assumes text-space proximity teaches motion-space semantics; a comparison against pairs built from CLIP appearance features or random pairs would show whether text is uniquely informative for motion or simply one available alignment signal.
  • Part of the improvement may come from contrastive regularization that prevents the small memorability dataset from overfitting the I3D encoder, and a random-pair contrastive baseline would separate this effect from the semantic pairing effect.
  • MWCVS suggests a broader pattern: video memorability, once predicted reliably, can act as a perceptual prior for other content-selection tasks such as highlights, thumbnails, or storytelling, not just summarization.
  • The largest reported gain is on VideoMem long-term memorability, which hints that LT memorability depends more heavily on semantic-motion structure; stratifying the analysis by caption-motion agreement could test where TMCCL's benefit concentrates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes TMCCL, a method that adds a text-motion cross-modal contrastive loss to the I3D motion feature extractor to improve video memorability prediction, and MWCVS, a post-processing correction for video summarization that uses memorability scores to reweight importance scores. The authors report state-of-the-art Spearman rank correlation on Memento10k (0.692) and VideoMem (0.614/0.324), and improved F1 over a reproduced MSVA baseline on SumMe (56.6) and TVSum (61.9).

Significance. The motivation is clear and the application of memorability to summarization is interesting. The paper provides concrete quantitative claims and an ablation that isolates the TMCCL contribution, which is good experimental practice. However, the core loss function as written in Eq. (14) is mathematically inconsistent with the stated objective, so the reported improvements cannot be attributed to the proposed method as described. No code is provided, so the actual implementation cannot be checked. The secondary summarization results do not establish state-of-the-art performance.

major comments (2)
  1. [Sec. 3.1.3, Eq. (14) and Eq. (17)] The TMCCL loss is written as log(A/(A+B)) with A the sum of positive similarities and B the sum of negative similarities. Since L_overall = L_MSE + λ·L_TMCCL is minimized, the partial derivatives ∂L/∂A = B/(A(A+B)) > 0 and ∂L/∂B = -1/(A+B) < 0 imply that the optimizer decreases positive-pair similarity and increases negative-pair similarity. This is the opposite of the text-motion alignment described in Section 3.1.3 and Algorithm 1. The paper never states that the loss is to be maximized, so the method as written would degrade the motion representation. Consequently, the improvements in Tables 1-3 must come from a different loss than the one defined, and the central claim is unsupported.
  2. [Sec. 4.4, Table 4] The summarization experiments compare MWCVS only against the authors' own reproduction of MSVA. On TVSum, the reproduced baseline drops from the official 62.8 to 60.1, and MWCVS recovers to 61.9, still below the official MSVA number. The claims of 'better performance on two video summarization datasets' and 'state-of-the-art' are therefore overstated; the contribution is a relative improvement over a weakened baseline, not a new state of the art.
minor comments (5)
  1. [Eq. (25)] The Spearman rank correlation formula is missing the squared difference in the numerator; as written, it does not define the rank correlation coefficient.
  2. [Eq. (15)] The similarity function sim(f_m, f+_m) = exp(f_m, f+_m / τ) is ambiguous: it should explicitly specify an inner product, cosine similarity, or other distance before the exponential.
  3. [Algorithm 1, line 5] The line defining P_i^m contains a typo ('Ramdom') and the set-builder notation is unclear; it should be rewritten to specify how the K positive samples are chosen from the top-2K candidates.
  4. [Sec. 3.1.4] The sentence 'Let θv, θt, θm be the score weights of Ŝv, Ŝv and Ŝv' repeats Ŝv three times; the last one should be Ŝm.
  5. [Sec. 4.4.2] The ablation discussion of μ is garbled ('When the importance score on results is weakened. When the μ is too small, μ is excessively large...') and should be rewritten for readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TMCCL is isolated by the Table 3 ablation and evaluated against external human-annotation benchmarks, so the central claim is not equivalent to its inputs.

full rationale

The paper's central derivation is self-contained rather than circular. TMCCL is an auxiliary contrastive loss that uses BERT text similarities only to define positive and negative motion pairs during I3D fine-tuning (Algorithm 1, Eq. 14); the memorability prediction is then evaluated by Spearman rank correlation against human ground-truth scores (Eq. 25), not against the text-similarity signal used to construct the pairs. The ablation in Table 3 removes the TMCCL term while keeping the text-fusion branch, so the reported motion-feature gain is not forced by construction. No load-bearing self-citation occurs: reference [33] (Lu and Wu, including two of the present authors) appears only as a general pointer to video storytelling and is not used to justify TMCCL or MWCVS. The MWCVS application tunes the scalar mu on the evaluation datasets (Table 5), which is a potential data-leakage or correctness concern, but it is not a circular reduction because the F1 scores are computed against external summary labels. I also note the printed Eq. 14 has a sign such that minimizing L_TMCCL would repel positive samples and attract negative samples, which contradicts the stated purpose; this is an internal correctness inconsistency, not a circularity, so it does not raise the circularity score.

Assumptions & free parameters 8 free parameters · 3 assumptions · 0 invented entities

The method relies on a set of hand-chosen hyperparameters and two domain assumptions about text-motion correspondence and cross-dataset transfer of memorability. No new physical or conceptual entities are introduced.

free parameters (8)
  • K (number of positive samples) = 8
    Chosen by hand; controls the size of the positive set in TMCCL.
  • Queue size Q = 1024
    MoCo queue size for negative samples, set without sensitivity analysis.
  • temperature tau = 0.07
    Standard contrastive temperature, set by convention.
  • lambda (TMCCL weight) = 0.5
    Balances MSE and contrastive loss; no ablation shown.
  • mu (memorability weight in summarization) = 0.5
    Tuned on the evaluation sets via Table 5, a form of test-set fitting.
  • c (weight step size) = 0.05
    Step for the self-adaptive fusion weights, chosen by hand.
  • l (number of attention segments) = 9
    Divides the visual appearance feature into 9 segments; no sensitivity analysis.
  • n (number of sampled frames) = 8
    Uniform frame sampling rate, set empirically.
assumptions (3)
  • domain assumption Pre-trained CLIP, BERT, and I3D features are meaningful for video memorability.
    The whole architecture relies on the transferability of these external features (Section 3.1).
  • ad hoc to paper Textual similarity between video descriptions implies semantic similarity of motion content.
    This is the core modeling assumption behind constructing positive/negative motion samples from text (Algorithm 1, Eq. 14). It is plausible but not independently verified.
  • domain assumption Video memorability scores are intrinsic and transferable across datasets for summarization.
    MWCVS applies a model trained on Memento10k to clips from SumMe/TVSum, assuming memorability rankings generalize (Section 3.2.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Video Memorability Prediction with Text-Motion Cross-modal Contrastive Loss and Its Application in Video Summarization." pith.science (2026). https://pith.science/paper/3S7M3Z4A

@misc{pith2026250608649,
  author       = {Pith},
  title        = {Pith review of: Enhancing Video Memorability Prediction with Text-Motion Cross-modal Contrastive Loss and Its Application in Video Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3S7M3Z4A}},
  note         = {Machine review of arXiv:2506.08649}
}
read the original abstract

Video memorability refers to the ability of videos to be recalled after viewing, playing a crucial role in creating content that remains memorable. Existing models typically focus on extracting multimodal features to predict video memorability scores but often fail to fully utilize motion cues. The representation of motion features is compromised during the fine-tuning phase of the motion feature extractor due to a lack of labeled data. In this paper, we introduce the Text-Motion Cross-modal Contrastive Loss (TMCCL), a multimodal video memorability prediction model designed to enhance the representation of motion features. We tackle the challenge of improving motion feature representation by leveraging text description similarities across videos to establish positive and negative motion sample sets for a given target. This enhancement allows the model to learn similar feature representations for semantically related motion content, resulting in more accurate memorability predictions. Our model achieves state-of-the-art performance on two video memorability prediction datasets. Moreover, the potential applications of video memorability prediction have been underexplored. To address this gap, we present Memorability Weighted Correction for Video Summarization (MWCVS), using video memorability prediction to reduce subjectivity in video summarization labels. Experimental results on two video summarization datasets demonstrate the effectiveness of MWCVS, showcasing the promising applications of video memorability prediction.

Figures

Figures reproduced from arXiv: 2506.08649 by the authors.

Figure 1
Figure 1. A diagram is presented, illustrating optical flow images from two videos, along with their corresponding text descriptions. Both images depict individuals surfing on a woven surface. The motion and text semantics in both instances align. However, the limited availability of labeled data during the fine-tuning phase of the motion feature extractor results in weak motion features. As a result, the motion features are … view at source ↗
Figure 2
Figure 2. Overview of TMCCL multimodal video memorability prediction model [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. The example of constructing positive and negative samples for a target sample based on text space similarity. The target sample has a high similarity to the positive sample in text space, while the target sample has a low similarity to the negative sample in text space. First Author et al.: Preprint submitted to Elsevier Page 11 of 9 [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Overview of the Memorability Weighted Correction for Video Summarization model [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Protocol to collect ground-truth ST scores [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of MWCVS First Author et al.: Preprint submitted to Elsevier Page 12 of 9 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 50 canonical work pages

  1. [1]

    Beit: Bert pre-training of image transformers, in: International Conference on Learning Representations, pp

    Bao, H., Dong, L., Piao, S., Wei, F., 2021. Beit: Bert pre-training of image transformers, in: International Conference on Learning Representations, pp. 1–8

    work page 2021

  2. [2]

    Emerging properties in self-supervised vision transformers, in: Proceedings of the IEEE/CVF international confer- ence on computer vision, pp

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A., 2021. Emerging properties in self-supervised vision transformers, in: Proceedings of the IEEE/CVF international confer- ence on computer vision, pp. 9650–9660

    work page 2021

  3. [3]

    Quo vadis, action recognition? a new model and the kinetics dataset, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Carreira, J., Zisserman, A., 2017. Quo vadis, action recognition? a new model and the kinetics dataset, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299– 6308

    work page 2017

  4. [4]

    An empirical study of training self-supervisedvisiontransformers,in:ProceedingsoftheIEEE/CVF International Conference on Computer Vision, pp

    Chen, X., Xie, S., He, K., 2021. An empirical study of training self-supervisedvisiontransformers,in:ProceedingsoftheIEEE/CVF International Conference on Computer Vision, pp. 9640–9649

    work page 2021

  5. [5]

    Videomem:Constructing,analyzing,predictingshort-termandlong- term video memorability, in: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp

    Cohendet, R., Demarty, C.H., Duong, N.Q., Engilberge, M., 2019. Videomem:Constructing,analyzing,predictingshort-termandlong- term video memorability, in: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 2531–2540

    work page 2019

  6. [6]

    Annotating, understanding, and predicting long-term video memora- bility, in: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp

    Cohendet, R., Yadati, K., Duong, N.Q.K., Demarty, C.H., 2018. Annotating, understanding, and predicting long-term video memora- bility, in: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 178–186

    work page 2018

  7. [7]

    Constantin, M.G., Ionescu, B., 2021. Using vision transformers and memorable moments for the prediction of video memorability, in: Working Notes Proceedings of the MediaEval 2021 Workshop (CEUR Workshop Proceedings), pp. 217–219

    work page 2021

  8. [8]

    Aimultimedialab at mediaeval 2022:Predictingmediamemorabilityusingvideovisiontransformers and augmented memorable moments , 12–16

    Constantin, M.G., Ionescu, B., 2022. Aimultimedialab at mediaeval 2022:Predictingmediamemorabilityusingvideovisiontransformers and augmented memorable moments , 12–16

    work page 2022

Show all 55 references
  1. [9]

    Dong, J., Li, X., Xu, C., Yang, X., Yang, G., Wang, X., Wang, M.,

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale, in: International Conference on Learning Representations, pp

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al., 2020. An image is worth 16x16 words: Transformers for image recognition at scale, in: International Conference on Learning Repres...

    2020

  3. [11]

    Modular memorability: Tieredrepresentationsforvideomemorabilityprediction,in:Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Dumont, T., Hevia, J.S., Fosco, C.L., 2023. Modular memorability: Tieredrepresentationsforvideomemorabilityprediction,in:Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10751–10760

    2023

  4. [12]

    Memory: A contribution to experimental psychology

    Ebbinghaus, H., 2013. Memory: A contribution to experimental psychology. Annals of neurosciences 20, 155

    2013

  5. [13]

    Amnet: Memorability estimation with attention, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Fajtl, J., Argyriou, V., Monekosso, D., Remagnino, P., 2018. Amnet: Memorability estimation with attention, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6363– 6372

    2018

  6. [14]

    Supervised video summarization via multiple feature sets with parallel attention, in: 2021 IEEE International Conference on Multimedia and Expo, pp

    Ghauri, J.A., Hakimov, S., Ewerth, R., 2021. Supervised video summarization via multiple feature sets with parallel attention, in: 2021 IEEE International Conference on Multimedia and Expo, pp. 1–6s

    2021

  7. [15]

    Creating summaries from user videos, in: Computer Vision–ECCV 2014: 13th European Conference, pp

    Gygli, M., Grabner, H., Riemenschneider, H., Van Gool, L., 2014. Creating summaries from user videos, in: Computer Vision–ECCV 2014: 13th European Conference, pp. 505–520

    2014

  8. [16]

    Learning computational models of video memorability from fmri brain imag- ing

    Han, J., Chen, C., Shao, L., Hu, X., Han, J., Liu, T., 2014. Learning computational models of video memorability from fmri brain imag- ing. IEEE transactions on cybernetics 45, 1692–1703

    2014

  9. [17]

    Self-supervised co-training for video representation learning

    Han, T., Xie, W., Zisserman, A., 2020. Self-supervised co-training for video representation learning. Advances in Neural Information Processing Systems 33, 5679–5690. First Author et al.:Preprint submitted to ElsevierPage 8 of 9 Short Title of the Article

    2020

  10. [18]

    Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Hara, K., Kataoka, H., Satoh, Y., 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6546–6555

    2018

  11. [19]

    Momentum contrast for unsupervised visual representation learning, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    He, K., Fan, H., Wu, Y., Xie, S., Girshick, R., 2020. Momentum contrast for unsupervised visual representation learning, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738

    2020

  12. [20]

    Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

    2016

  13. [21]

    Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., 2017. Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708

    2017

  14. [22]

    Whatmakesanimage memorable?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Isola,P.,Xiao,J.,Torralba,A.,Oliva,A.,2011. Whatmakesanimage memorable?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 145–152

    2011

  15. [23]

    Kenton, J.D.M.W.C., Toutanova, L.K., 2019. Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. ...

    2019

  16. [24]

    Understanding and predicting image memorability at a large scale, in: Proceedings oftheIEEEInternationalConferenceonComputerVision,pp.2390– 2398

    Khosla, A., Raju, A.S., Torralba, A., Oliva, A., 2015. Understanding and predicting image memorability at a large scale, in: Proceedings oftheIEEEInternationalConferenceonComputerVision,pp.2390– 2398

    2015

  17. [25]

    Topic-oriented text features can match visual deep models of video memorability

    Kleinlein, R., Luna-Jiménez, C., Arias-Cuadrado, D., Ferreiros, J., Fernández-Martínez, F., 2021a. Topic-oriented text features can match visual deep models of video memorability. Applied Sciences 11, 7406

  18. [26]

    Kleinlein, R., Luna-Jiménez, C., Fernández-Martínez, F., 2021b. Thau-upmatmediaeval2021:Fromvideosemanticstomemorability using pretrained transformers, in: Working Notes Proceedings of the MediaEval2021Workshop(CEURWorkshopProceedings),pp.124– 126

  19. [27]

    Scene memory is more detailed than you think: The role of categories in visual long-term memory

    Konkle, T., Brady, T.F., Alvarez, G.A., Oliva, A., 2010. Scene memory is more detailed than you think: The role of categories in visual long-term memory. Psychological Science 21, 1551–1556

    2010

  20. [28]

    Multimodal deep features fusion for video memorability prediction, in: Working Notes Proceedings of the MediaEval 2019 Workshop (CEUR Workshop Proceedings), pp

    Leyva, R., Doctor, F., Garcia Seco De Herrera, A., Sahab, S., 2019. Multimodal deep features fusion for video memorability prediction, in: Working Notes Proceedings of the MediaEval 2019 Workshop (CEUR Workshop Proceedings), pp. 115–117

    2019

  21. [29]

    Adaptive multi- modalensemblenetworkforvideomemorabilityprediction

    Li, J., Guo, X., Yue, F., Xue, F., Sun, J., 2022. Adaptive multi- modalensemblenetworkforvideomemorabilityprediction. Applied Sciences 12, 8599

    2022

  22. [30]

    Deephierarchicallstmnetworks with attention for video summarization

    Lin,J.,Zhong,S.h.,Fares,A.,2022. Deephierarchicallstmnetworks with attention for video summarization. Computers & Electrical Engineering 97, 107618

    2022

  23. [31]

    Liu, T., Meng, Q., Huang, J.J., Vlontzos, A., Rueckert, D., Kainz, B.,

  24. [32]

    Liu,Z.,Lin,Y.,Cao,Y.,Hu,H.,Wei,Y.,Zhang,Z.,Lin,S.,Guo,B.,

  25. [33]

    Video storytelling based on gated video memorability filtering

    Lu, Y., Wu, X., 2022. Video storytelling based on gated video memorability filtering. Electronics Letters 58, 576–578

    2022

  26. [34]

    Audio-visual instance discrimination with cross-modal agreement, in: Proceedings of the IEEE/CVFConferenceonComputerVisionandPatternRecognition, pp

    Morgado, P., Vasconcelos, N., Misra, I., 2021. Audio-visual instance discrimination with cross-modal agreement, in: Proceedings of the IEEE/CVFConferenceonComputerVisionandPatternRecognition, pp. 12475–12486

    2021

  27. [35]

    10012–10022

    Swintransformer:Hierarchicalvisiontransformerusingshifted windows,in:ProceedingsoftheIEEE/CVFInternationalConference on Computer Vision, pp. 10012–10022

  28. [36]

    Multimodal memorability: Modeling effects of semantics anddecayonvideomemorability,in:ComputerVision–ECCV2020: 16th European Conference, pp

    Newman, A., Fosco, C., Casser, V., Lee, A., McNamara, B., Oliva, A., 2020. Multimodal memorability: Modeling effects of semantics anddecayonvideomemorability,in:ComputerVision–ECCV2020: 16th European Conference, pp. 223–240

    2020

  29. [37]

    Spatiotemporal contrastive video representation learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Qian, R., Meng, T., Gong, B., Yang, M.H., Wang, H., Belongie, S., Cui, Y., 2021. Spatiotemporal contrastive video representation learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6964–6974

    2021

  30. [38]

    Does video summarization require videos? quantifying the effectiveness of language in video summarization.arXiv:2309.09405

    Nam,Y.,Lehavi,A.,Yang,D.,Bose,D.,Swayamdipta,S.,Narayanan, S., 2023. Does video summarization require videos? quantifying the effectiveness of language in video summarization.arXiv:2309.09405

    2023 arXiv

  31. [39]

    Self-supervised video transformer, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Ranasinghe, K., Naseer, M., Khan, S., Khan, F.S., Ryoo, M.S., 2022. Self-supervised video transformer, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2874– 2884

    2022

  32. [40]

    Ex- ploringmultimodality,perplexityandexplainabilityformemorability prediction, in: Working Notes Proceedings of the MediaEval 2021 Workshop (CEUR Workshop Proceedings), pp

    Reboud, A., Harrando, I., Laaksonen, J., Troncy, R., et al., 2021. Ex- ploringmultimodality,perplexityandexplainabilityformemorability prediction, in: Working Notes Proceedings of the MediaEval 2021 Workshop (CEUR Workshop Proceedings), pp. 157–159

    2021

  33. [41]

    Learning transferable visual models from natural language supervision, in: International Conference on Machine Learning, pp

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S.,Sastry,G.,Askell,A.,Mishkin,P.,Clark,J.,etal.,2021. Learning transferable visual models from natural language supervision, in: International Conference on Machine Learning, pp. 8748–8763

    2021

  34. [42]

    Squalli-Houssaini, H., Duong, N.Q., Gwenaëlle, M., Demarty, C.H.,

  35. [43]

    A network linking scene perception and spatial memory systems in posterior cerebral cortex

    Steel, A., Billings, M.M., Silson, E.H., Robertson, C.E., 2021. A network linking scene perception and spatial memory systems in posterior cerebral cortex. Nature Communications 12, 2632

    2021

  36. [44]

    Tvsum: Summarizing web videos using titles, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp

    Song, Y., Vallmitjana, J., Stent, A., Jaimes, A., 2015. Tvsum: Summarizing web videos using titles, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5179– 5187

    2015

  37. [45]

    Predicting media memorability:Comparingvisual,textualandauditoryfeatures,103– 105

    Sweeney, L., Healy, G., Smeaton, A.F., 2021. Predicting media memorability:Comparingvisual,textualandauditoryfeatures,103– 105

    2021

  38. [46]

    Diffusing surrogate dreamsofvideoscenestopredictvideomemorability

    Sweeney, L., Healy, G., Smeaton, A.F., 2022. Diffusing surrogate dreamsofvideoscenestopredictvideomemorability. arXivpreprint arXiv:2212.09308

    2022 arXiv

  39. [47]

    Siamese image modeling for self-supervised vision representationlearning,in:ProceedingsoftheIEEE/CVFConference on Computer Vision and Pattern Recognition, pp

    Tao,C.,Zhu,X.,Su,W.,Huang,G.,Li,B.,Zhou,J.,Qiao,Y.,Wang, X., Dai, J., 2023. Siamese image modeling for self-supervised vision representationlearning,in:ProceedingsoftheIEEE/CVFConference on Computer Vision and Pattern Recognition, pp. 2132–2141

    2023

  40. [48]

    Recurrent unit augmented memory network for video summarisation

    Su, M., Ma, R., Zhang, B., Li, K., 2023. Recurrent unit augmented memory network for video summarisation. IET Computer Vision

    2023

  41. [49]

    Modelling of video memorability using ensemble learning and transformers , 7–11

    Usmani, M.M.A., Zahid, S., Tahir, M.A., 2022. Modelling of video memorability using ensemble learning and transformers , 7–11

    2022

  42. [50]

    Reconstructivesequence-graph network for video summarization

    Zhao,B.,Li,H.,Lu,X.,Li,X.,2021. Reconstructivesequence-graph network for video summarization. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 2793–2801

    2021

  43. [51]

    Learning multiscale hierarchical attention for video summarization

    Zhu, W., Lu, J., Han, Y., Zhou, J., 2022. Learning multiscale hierarchical attention for video summarization. Pattern Recognition 122, 108312. CRediT authorship contribution statement Zhiyi Zhu:Conceptualization, Methodology, Software, Investigation, Formal Analysis, Writing -...

    2022

  44. [52]

    Training data-efficient image transformers & distillation throughattention,in:InternationalConferenceonMachineLearning, PMLR

    Touvron,H.,Cord,M.,Douze,M.,Massa,F.,Sablayrolles,A.,Jégou, H., 2021. Training data-efficient image transformers & distillation throughattention,in:InternationalConferenceonMachineLearning, PMLR. pp. 10347–10357

    2021

  45. [2018]

    2371–2375

    Deep learning for predicting image memorability, in: Proceed- ings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2371–2375

  46. [2021]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4065–4080

    Dual encoding for video retrieval by text. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4065–4080

  47. [2022]

    IEEE Transactions on Image Processing 31, 1573–1586

    Video summarization through reinforcement learning with a 3d spatio-temporal u-net. IEEE Transactions on Image Processing 31, 1573–1586

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.