Pith. sign in

REVIEW 2 major objections 4 minor 99 references

Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization

T0 review · 2 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VideoGraph claims that video summarization improves when frames are modeled as language-guided graphs over detected objects, reaching state-of-the-art F-scores of 58.1 on SumMe and 69.5 on TVSum.

desk verdict Competent incremental architecture with real benchmark gains, but the headline margin is fragile because N and W are tuned on the test splits and no variance is reported. read the letter →

arxiv 2509.05604 v1 pith:4X26SCXD submitted 2025-09-06 cs.CV cs.AI

Jungin Park , Jiyoung Lee , Kwanghoon Sohn This is my paper · ORCID
classification cs.CVcs.AI
keywords videosummarizationlanguage-guidedquery-focusedspatiotemporalgraphconvolutionalnetworkrecursiverefinementkeyframeselectionobject-levelrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that video summarization should be modeled at the level of objects and their semantic relations, not just whole frames. It proposes VideoGraph, a spatiotemporal graph network in which detected objects are nodes of per-frame spatial graphs, frames are nodes of a temporal graph, and a language query derived from the video is injected into both graphs to guide node representations. The graphs are then refined recursively by estimating residual adjacency matrices, so keyframes end up connected by story-level semantic affinity rather than visual similarity. On the SumMe and TVSum benchmarks the model reports state-of-the-art F-scores (58.1 and 69.5 with word-level queries in the supervised standard setting), and on the QFVS dataset it improves both generic and query-focused summarization. A sympathetic reader would care because this offers a concrete recipe for distinguishing keyframes from visually similar background frames, and it works in supervised and unsupervised regimes.

What carries the argument

The load-bearing machinery is a pair of nested graph networks with recursive residual refinement: a spatial graph whose nodes are detected objects in each frame, and a temporal graph whose nodes are averaged object representations with positional encodings. The language query representation is used as the query in multi-head cross-attention at both graph levels, so each node's features are rewritten to be semantic rather than purely visual. At every iteration, the temporal adjacency matrix is refined with a residual estimated from spatial graph outputs, and the spatial adjacency matrix is refined with a residual estimated from temporal graph outputs, so the graphs gradually converge to story-level connectivity rather than visual-similarity connectivity. Graph convolution then aggregates nodes along these refined edges.

What would settle it

On TVSum, set the language query representation $q$ to zero in both cross-attention modules while keeping the object graph, recursive refinement, and all losses identical; if the F-score stays near the reported 69.5 instead of falling toward the no-language 65.2, then language guidance is not what is carrying the result.

Watch

Extended reading notes

Core claim

The central claim is that formulating video summarization as a language-guided spatiotemporal graph over detected objects, with recursively refined adjacency matrices, outperforms prior frame-level and language-guided approaches. In VideoGraph, each frame is a spatial graph whose nodes are object regions; those nodes are averaged and position-encoded into frame nodes of a temporal graph. A language query---either captions from a dense video captioning model or the most frequent detected object class names---acts as the query in multi-head cross-attention, rewriting object and frame features so that graph edges encode semantic affinity instead of raw visual similarity. The spatial and temporal graphs refine each other: the temporal graph estimates residuals that update the spatial adjacency matrix, and the spatial graph estimates residuals that update the temporal adjacency matrix, repeated for five iterations. At convergence the temporal graph links keyframes with high affinity, giving the reported 58.1 F-score on SumMe, 69.5 on TVSum, and 57.95 average F-score on query-focused QFVS.

Load-bearing premise

The load-bearing premise is that the frozen object detector finds the entities that carry the video's story; if its top proposals miss or mislabel those entities, the spatial graph, the temporal graph, and the word-level language queries all degrade together because they share that single visual source.

Editorial extensions

If this is right

  • Word-level queries (frequent detected class names) are enough to reach the best scores on short videos: 58.1 on SumMe and 69.5 on TVSum, beating the same model without language by 3.4 and 4.3 points respectively.
  • Sentence-level queries become more valuable on long multi-event videos: on QFVS generic summarization the sentence variant scores 71.29 versus 68.75 for word-level, and with ground-truth captions 77.83.
  • Recursive refinement is not decorative: the unrefined initial graph gives 49.8 on SumMe and 59.6 on TVSum, while five iterations give 58.1 and 69.5, with convergence in three to five iterations.
  • Language guidance can be added without caption-generation latency: the word-level model skips the roughly 250-second captioning step and still gets a higher F-score than the sentence-level variant trained on generated captions.
  • The unsupervised variant, trained only with sparsity, diversity, and reconstruction losses, reaches 55.0 on SumMe and 66.8 on TVSum with word-level queries, narrowing the gap between supervised and unsupervised video summarization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own future-work paragraph concedes that pretrained-model biases can propagate into the model; because the same frozen detector feeds both node features and word queries, swapping in an open-vocabulary detector (one that can recognize objects beyond its training classes) is the most direct stress test of where the gains come from.
  • The optimal number of query words shifts with video length (8 words on SumMe and TVSum, 25 words on QFVS), suggesting an adaptive rule that chooses word count per video from detection statistics or scene boundaries rather than fixing it per dataset.
  • The recursive residual coupling between spatial and temporal graphs is a general way to impose story-level connectivity, so it could plausibly transfer to neighbouring video-language tasks such as query-focused moment retrieval or temporal grounding.
  • The dominance scores read off the final spatial edge weights imply a weakly supervised object-importance signal, one that could rank objects or select queries without any object-level annotations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes VideoGraph, a method for generic and query-focused video summarization that formulates the task as language-guided spatiotemporal graph modeling. Detected objects in each frame are treated as nodes of a per-frame spatial graph, and averaged object representations with positional encodings form the nodes of a temporal graph over frames. Both graphs use language-query-conditioned multi-head cross-attention, and the adjacency matrices are refined recursively by residual updates. The authors compare sentence-level and word-level language queries and evaluate supervised and unsupervised settings on SumMe, TVSum, and QFVS, reporting state-of-the-art F-scores, for example 58.1 on SumMe and 69.5 on TVSum in the supervised standard setting.

Significance. If the reported results hold, this is a meaningful advance: object-level spatial and temporal graph reasoning with recursive adjacency refinement, combined with low-cost word-level language queries, would outperform prior frame-level and transformer-based baselines on standard benchmarks. The architecture is specified precisely (Eqs. 4-17, Table 2), the ablations isolate the contribution of each component, the code is released, and the runtime/parameter comparison in Table 11 is a useful contribution. The main caveat is empirical rather than architectural: the two headline hyperparameters are selected directly on the test benchmarks, and the reported margins over the strongest baseline are comparable in size to the effect of that selection. The missing variance reporting further prevents assessing whether the claimed state-of-the-art margins are statistically meaningful.

major comments (2)
  1. [Sections 5.2.2, 5.5.3, 5.5.4; Tables 9 and 10] The two headline configuration choices are selected directly on the test benchmarks. Section 5.2.2 states that N=16 is used 'based on the ablation study', and Table 9 shows that N=16 is simply the peak of the TVSum F-score curve (60.2 at N=1 rising to 69.5 at N=16 and then falling). Similarly, Table 10 selects W=8 on SumMe and TVSum and W=25 on QFVS at the peak of each curve. The standard data configuration in Section 5.1.3 defines only an 80/20 train/test split with no validation subset, so these choices appear to be made on the same test splits that produce the headline numbers in Tables 3 and 5. The magnitude of the selection effect is comparable to the claimed improvement: the supervised standard margins over [37] are 3.9 on SumMe and 3.2 on TVSum, while the W-ablation spans 54.6-58.1 on SumMe and 65.4-69.5 on TVSum, and the N-ablation spans 60.2-69.5 on TVSum. Please either select N and W on held-out validation splits, fix them a priori, or report the full sensitivity analysis as the basis for a more cautious claim. As written, the state-of-the-art claim at the reported magnitude is not established.
  2. [Section 5.2.3 and Table 3] Section 5.2.3 states that experiments are run five times on five random splits and reported as averages, but no standard deviations, per-split scores, or significance tests are provided anywhere in the paper. For the supervised standard setting, the margin over [37] is only 3.9 F-score on SumMe and 3.2 on TVSum; with only five random splits, these averages could easily overlap under typical split-level variance in this benchmark. Please report per-split results and standard deviations, and ideally run paired comparisons against the strongest baseline on identical splits so the reader can judge whether the claimed improvements are statistically distinguishable from noise.
minor comments (4)
  1. [Figure 8] The per-iteration F-score labels in Figure 8(a)-(c) are identical (0.368, 0.423, 0.466, 0.487) for SumGraph, VideoGraph without language, and VideoGraph with word-level queries, even though the final F-scores differ (50.3, 62.1, 70.2). Please correct the labels, or state explicitly that these intermediate values are not per-method and explain why they are repeated.
  2. [Section 5.4 and Table 11] The text says VideoGraph uses '0.8M fewer parameters' than [44], but Table 11 shows total parameters of 49.3M for VideoGraph versus 12.1M for [44]; the 0.8M reduction holds only for learnable parameters. Please disambiguate 'parameters' as 'learnable parameters' in that sentence.
  3. [Sections 4.2.2 and 5.2.2] The word-level language queries are the class labels of the same detected objects that provide the visual node features, so the language signal is not independent of the visual detector. A short discussion of how much of the word-query gain is genuinely semantic rather than a re-encoding of detector output would strengthen the comparison, and the future-work paragraph already partially acknowledges this concern.
  4. [Section 5.1.3] For the QFVS query-focused protocol, the paper says two videos are used for training, one for validation, and one for testing, but Table 6 reports results for four videos. Please state explicitly whether the 2/1/1 split is rotated over all four videos and whether Table 10's word-count selection on QFVS uses the validation video or the same test video used in the main table.

Circularity Check

2 steps flagged · score 2.0 of 10

No derivation-level circularity: the recursive graph refinement is learned from data and evaluated on external benchmarks, so the core model does not reduce to its inputs; the two mild concerns are that word-level queries are detector class labels and that key hyperparameters (N, W) are selected on the same test splits used for the headline F-scores.

  1. other [Sec. 4.2.1-4.2.2, Eqs. (4) and (6)]
    "As an alternative to the sentence-level language query, we directly use the object classes detected from the object detector (e.g. car, tire, and person in Fig. 4) as word-level language queries. We select W object classes based on the detected frequency over the whole video. ... we extract a set of object feature representations O_t for the t-th frame using an off-the-shelf object detector."

    The word-level language query is not an independent language signal: its tokens are the class labels emitted by the same frozen Faster R-CNN that produces the object node features in Eq. (4). The language-guided attention in Eq. (7) therefore conditions the visual node representations on a text-encoded re-statement of their own detector outputs, so the claimed semantic guidance is partly a self-conditioning loop through the detector. CLIP text embeddings add external knowledge, so this is not a full equivalence, but the 'language' input is partially determined by the very visual features it is meant to disambiguate.

  2. fitted input called prediction [Sec. 5.1.3, Sec. 5.5.3, Sec. 5.5.4]
    "The standard data setting selects training and testing videos from the same dataset by splitting 80% of the videos for training and the remainder for testing. ... we evaluated our model primarily using the top 16 object feature representations based on the ablation study. ... In the main experiments, we thus use 8 words on SumMe and TVSum, and 25 words on QFVS, respectively."

    Tables 9 and 10 choose N=16 and W=8/25 by the F-score achieved on the TVSum, SumMe, and QFVS benchmarks themselves, and the data configuration has no validation subset. The same test labels are therefore used both to select these configuration hyperparameters and to produce the headline numbers in Tables 3 and 6. The N sweep spans 60.2-69.5 F-score and the W sweep moves about one point, which is comparable to the 3.2-3.9 point margins over the strongest baselines, so part of the reported 'state-of-the-art' performance is selected rather than predicted. This is an evaluation-protocol overfitting issue rather than a circularity of the graph derivation itself.

full rationale

The core VideoGraph derivation is self-contained: the spatial and temporal adjacency matrices are computed from object/frame features and language-conditioned attention (Eqs. 7-12), and the recursive refinement (Eqs. 14-16) learns residual adjacency matrices from intermediate representations trained with classification, sparsity, reconstruction, and diversity losses (Eqs. 18-25). Nothing in these equations is analytically forced to equal the ground-truth keyframe labels, and the ablation study shows that removing the spatial graph, temporal graph, language queries, or loss terms changes the F-score, so the method has independent empirical content. The two flagged issues are real but limited. First, the word-level query is derived from the same object detector that supplies the visual node features; this weakens the claim of genuinely external semantic guidance but does not invalidate the learned graph refinement. Second, N and W are selected on the test splits used for the reported F-scores, which inflates the headline margins and should be reported with a validation split or per-split variance; however, this is a statistical evaluation concern, not a by-construction equivalence between the model output and its inputs. Self-citation of the authors' prior SumGraph work [44] is transparent and used as a baseline and architectural predecessor, not as a load-bearing uniqueness theorem, so it does not contribute to circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The model is a trained neural architecture evaluated on external benchmarks, so its empirical claim rests on hyperparameters, frozen pretrained components, and per-dataset design choices. The most consequential choices are the object count N and the word count W, both selected by ablating on the test benchmarks (Tables 9 and 10). The frozen Faster R-CNN detector underlies both the visual features and the word-level queries, which couples the two modalities. Loss weights (alpha, beta, gamma, rho) and edge scaling factors (lambda_o, lambda_f) are set by hand without sensitivity analysis. No new physical or formal entities are introduced.

free parameters (6)
  • Edge scaling factor lambda_o (Eq. 8) = 1.6
    Scaling factor for spatial adjacency weights, set by hand in Sec 5.2.3 and used in all experiments without sensitivity analysis.
  • Edge scaling factor lambda_f (Eq. 12) = 30
    Scaling factor for temporal adjacency weights, set by hand in Sec 5.2.3 and used in all experiments without sensitivity analysis.
  • Number of objects per frame N = 16
    Selected from the TVSum ablation (Table 9), where N=16 gives the best F-score of 69.5; N=1 and N=36 give 60.2 and 66.2.
  • Number of word-level queries W = 8 for SumMe and TVSum, 25 for QFVS
    Selected from ablations summarized in Table 10, run on the respective test benchmarks; the optimal W differs by dataset.
  • Maximum refinement iterations K = 5
    Set in Sec 5.2.3; Fig. 12 shows F-scores still rising at K=5, so the stopping point is a hand choice.
  • Loss weights alpha, beta, gamma, rho = alpha=1e-4 (supervised), 1e-3 (unsupervised); beta=gamma=0.1 (supervised), 10 (unsupervised); rho=5
    Set in Sec 5.2.3 (Eqs. 21, 24, 25); no ablation of these weights is reported.
assumptions (6)
  • standard math GCN aggregation per Kipf and Welling (Eq. 1) yields useful node updates in the spatial and temporal graphs.
    Standard result from the graph neural network literature, used throughout Sec 4 without modification.
  • domain assumption The frozen Faster R-CNN detector pretrained on Visual Genome provides object features that carry the information needed to separate keyframes from backgrounds.
    Invoked in Sec 4.2.1 and 5.2.2; Table 9 shows performance collapses when the number of objects is 1, so the claim depends on this detector being informative.
  • ad hoc to paper The most frequently detected object classes are the most informative word-level language queries.
    Sec 4.2.2 selects W words by detection frequency; the optimal W is chosen per dataset from test-benchmark ablations (Table 10), so the assumption is tuned to the benchmarks.
  • domain assumption Recursive residual refinement of both adjacency matrices converges to a semantically meaningful graph after K iterations.
    Empirically supported by Fig. 12 and Fig. 14, but the semantic interpretation of refined edges is only argued qualitatively (Figs. 7, 10).
  • domain assumption The keyshot-based F-score evaluation (KTS segmentation plus knapsack, Sec 5.1.4) is a valid measure of summary quality.
    Standard protocol from [5, 13], but it is known to be sensitive to segmentation and to the summary-length budget.
  • domain assumption CLIP text embeddings of object class names and generated captions provide useful language representations for the queries.
    Sec 4.2.2 and 5.2.2 use frozen CLIP encoders; the authors note in the future-work section that pretrained model biases can propagate into the model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization." pith.science (2026). https://pith.science/paper/4X26SCXD

@misc{pith2026250905604,
  author       = {Pith},
  title        = {Pith review of: Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4X26SCXD}},
  note         = {Machine review of arXiv:2509.05604}
}
read the original abstract

Video summarization aims to select keyframes that are visually diverse and can represent the whole story of a given video. Previous approaches have focused on global interlinkability between frames in a video by temporal modeling. However, fine-grained visual entities, such as objects, are also highly related to the main content of the video. Moreover, language-guided video summarization, which has recently been studied, requires a comprehensive linguistic understanding of complex real-world videos. To consider how all the objects are semantically related to each other, this paper regards video summarization as a language-guided spatiotemporal graph modeling problem. We present recursive spatiotemporal graph networks, called VideoGraph, which formulate the objects and frames as nodes of the spatial and temporal graphs, respectively. The nodes in each graph are connected and aggregated with graph edges, representing the semantic relationships between the nodes. To prevent the edges from being configured with visual similarity, we incorporate language queries derived from the video into the graph node representations, enabling them to contain semantic knowledge. In addition, we adopt a recursive strategy to refine initial graphs and correctly classify each frame node as a keyframe. In our experiments, VideoGraph achieves state-of-the-art performance on several benchmarks for generic and query-focused video summarization in both supervised and unsupervised manners. The code is available at https://github.com/park-jungin/videograph.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 78 canonical work pages

  1. [37]

    Narasimhan, M., Rohrbach, A., Darrell, T.: Clip-it! language-guided video summariza- tion. Adv. Neural Inform. Process. Syst., 13988–14000 (2021)

    work page 2021

  2. [1]

    IEEE Trans

    Zhang, X., Zhu, Z., Zhao, Y., Chang, D., Liu, J.: Seeing all from a few:ℓ1-norm-induced dis- criminative prototype selection. IEEE Trans. Neural Net. and Learn. Syst.30(7), 1954– 1966 (2019)

    2019

  3. [2]

    IEEE Trans

    Wu, L., Wang, Y., Shao, L., Wang, M.: 3-d personvlad: Learning deep global representa- tions for video-based person reidentification. IEEE Trans. Neural Net. and Learn. Syst. 30(11), 3347–3359 (2019)

    2019

  4. [3]

    IEEE Conf

    Zhang, K., Chao, W.-L., Sha, F., Grauman, K.: Summary transfer: Exemplar-based sub- set selection for video summarization. IEEE Conf. Comput. Vis. Pattern Recog., 1059– 1067 (2016)

    2016

  5. [4]

    IEEE Conf

    Mahasseni, B., Lam, M., Todorovic, S.: Unsu- pervised video summarization with adversar- ial lstm networks. IEEE Conf. Comput. Vis. Pattern Recog., 202–211 (2017)

    2017

  6. [5]

    Zhang, K., Chao, W.-L., Sha, F., Grau- man, K.: Video summarization with long short-term memory. Eur. Conf. Comput. Vis. (2016)

    2016

  7. [6]

    IEEE Conf

    Sharghi, A., Laurel, J.S., Gong, B.: Query- focused video summarization: Dataset, eval- uation, and a memory network based approach. IEEE Conf. Comput. Vis. Pattern Recog., 4788–4797 (2017)

    2017

  8. [7]

    He, X., Hua, Y., Song, T., Zhang, Z., Xue, Z., Ma, R., Robertson, N., Guan, H.: Unsu- pervised video summarization with atten- tive conditional generative adversarial net- works. ACM Int. Conf. Multimedia, 2296– 2304 (2019)

    2019

Show all 99 references
  1. [8]

    AAAI Conf

    Yuan, L., Tay, F.E., Li, P., Zhou, L., Feng, J.: Cycle-sum: Cycle-consistent adversarial lstm networks for unsupervised video summa- rization. AAAI Conf. Art. Intell., 9143–9150 (2019)

    2019

  2. [9]

    IEEE Conf

    Zhao, B., Li, X., Lu, X.: Hsa-rnn: Hierarchi- cal structure-adaptive rnn for video summa- rization. IEEE Conf. Comput. Vis. Pattern Recog., 7405–7414 (2020)

    2020

  3. [10]

    IEEE Trans

    Ji, Z., Zhao, Y., Pang, Y., Li, X., Han, J.: Deep attentive video summarization with dis- tribution consistency learning. IEEE Trans. Neural Net. and Learn. Syst.32(4), 1765– 1775 (2021)

    2021

  4. [11]

    IEEE Conf

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. IEEE Conf. Comput. Vis. Pattern Recog., 770–778 (2016)

    2016

  5. [12]

    IEEE Conf

    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. IEEE Conf. Com- put. Vis. Pattern Recog., 7794–7803 (2018)

    2018

  6. [13]

    Rochan, M., Ye, L., Wang, Y.: Video sum- marization using fully convolutional sequence networks. Eur. Conf. Comput. Vis., 347–363 (2018)

    2018

  7. [14]

    IEEE Conf

    Rochan, M., Wang, Y.: Video summariza- tion by learning from unpaired data. IEEE Conf. Comput. Vis. Pattern Recog., 7902– 7911 (2019)

    2019

  8. [15]

    IEEE Int

    Park, J., Lee, J., Jeon, S., Sohn, K.: Video summarization by learning relation- ships between action and scene. IEEE Int. Conf. Comput. Vis. Worksh. (2019)

    2019

  9. [16]

    IEEE Trans

    Zhu, W., Lu, J., Li, J., Zhou, J.: Dsnet: A flex- ible detect-to-summarize network for video summarization. IEEE Trans. Image Process. 30, 948–962 (2020) 24

    2020

  10. [17]

    Vaswani, A., Shazeer, N., Parmar, N., Uszko- reit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Adv. Neural Inform. Process. Syst., 5998– 6008 (2017)

    2017

  11. [18]

    IEEE Trans

    Li, H., Ke, Q., Gong, M., Zhang, R.: Video joint modelling based on hierarchical trans- former for co-summarization. IEEE Trans. Pattern Anal. Mach. Intell., 1–14 (2022)

    2022

  12. [19]

    Kr¨ ahenb¨ uhl, P., Koltun, V.: Efficient infer- ence in fully connected crfs with gaussian edge potentials. Adv. Neural Inform. Process. Syst., 109–117 (2011)

    2011

  13. [20]

    IEEE Trans

    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)

    2018

  14. [21]

    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional net- works. Int. Conf. Learn. Represent. (2017)

    2017

  15. [22]

    IEEE Int

    Zeng, R., Huang, W., Tan, M., Rong, Y., Zhao, P., Huang, J., Gan, C.: Graph convo- lutional networks for temporal action local- ization. IEEE Int. Conf. Comput. Vis., 7094– 7103 (2019)

    2019

  16. [23]

    IEEE Conf

    Jain, A., Zamir, A.R., Savarese, S., Saxena, A.: Structural-rnn: Deep learning on spatio- temporal graphs. IEEE Conf. Comput. Vis. Pattern Recog., 5308–5317 (2016)

    2016

  17. [24]

    IEEE Int

    Park, J., Lee, J., Jeon, S., Kim, S., Sohn, K.: Graph regularization network with seman- tic affinity for weakly-supervised temporal action localization. IEEE Int. Conf. Image Process. (2019)

    2019

  18. [25]

    IEEE Int

    Yuan, Y., Liang, X., Wang, X., Yeung, D.-Y., Gupta, A.: Temporal dynamic graph lstm for action-driven video object detection. IEEE Int. Conf. Comput. Vis., 1801–1810 (2017)

    2017

  19. [26]

    Nassar, A.S., D’Aronco, S., Lef` evre, S., Weg- ner, J.D.: Geograph: Graph-based multi-view object detection with geometric cues end- to-end. Eur. Conf. Comput. Vis., 488–504 (2020)

    2020

  20. [27]

    IEEE Conf

    Shi, W., Rajkumar, R.R.: Point-gnn: Graph neural network for 3d object detection in a point cloud. IEEE Conf. Comput. Vis. Pattern Recog., 1711–1719 (2020)

    2020

  21. [28]

    Wang, X., Gupta, A.: Videos as space-time region graphs. Eur. Conf. Comput. Vis., 413– 431 (2018)

    2018

  22. [29]

    IEEE Conf

    Shi, L., Zhang, Y., Cheng, J., Lu, H.: Skeleton-based action recognition with directed graph neural networks. IEEE Conf. Comput. Vis. Pattern Recog., 7912–7921 (2019)

    2019

  23. [30]

    IEEE Conf

    Gao, J., Zhang, T., Xu, C.: Graph convo- lutional tracking. IEEE Conf. Comput. Vis. Pattern Recog., 4649–4659 (2019)

    2019

  24. [31]

    Winter Applications Comput

    Li, J., Gao, X., Jiang, T.: Graph networks for multiple object tracking. Winter Applications Comput. Vis., 719–728 (2020)

    2020

  25. [32]

    IEEE Trans

    Zeng, R., Huang, W., Tan, M., Rong, Y., Zhao, P., Huang, J., Gan, C.: Graph convolu- tional module for temporal action localization in videos. IEEE Trans. Pattern Anal. Mach. Intell.44(10), 6209–6223 (2021)

    2021

  26. [33]

    IEEE Conf

    Jiang, B., Zhang, Z., Lin, D., Tang, J., Luo, B.: Semi-supervised learning with graph learning-convolutional networks. IEEE Conf. Comput. Vis. Pattern Recog., 11313–11320 (2019)

    2019

  27. [34]

    IEEE Int

    Teng, Y., Wang, L., Li, Z., Wu, G.: Target adaptive context aggregation for video scene graph generation. IEEE Int. Conf. Comput. Vis., 13688–13697 (2021)

    2021

  28. [35]

    Wang, S., Gao, L., Lyu, X., Guo, Y., Zeng, P., Song, J.: Dynamic scene graph generation via temporal prior inference. ACM Int. Conf. Multimedia, 5793–5801 (2022)

    2022

  29. [36]

    IEEE Conf

    Gao, K., Chen, L., Niu, Y., Shao, J., Xiao, J.: Classification-then-grounding: Reformulating video scene graphs as temporal bipartite 25 graphs. IEEE Conf. Comput. Vis. Pattern Recog., 19497–19506 (2022)

    2022

  30. [38]

    arXiv preprint arXiv:2207.01814 (2022)

    Park, J., Kwoun, K., Lee, C., Lim, H.: Multimodal frame-scoring transformer for video summarization. arXiv preprint arXiv:2207.01814 (2022)

    2022 arXiv

  31. [39]

    Winter Appli- cations Comput

    Li, H., Ke, Q., Gong, M., Drummond, T.: Progressive video summarization via multi- modal self-supervised learning. Winter Appli- cations Comput. Vis., 5584–5593 (2023)

    2023

  32. [40]

    Iashin, V., Rahtu, E.: A better use of audio- visual cues: Dense video captioning with bi- modal transformer. Brit. Mach. Vis. Conf. (2020)

    2020

  33. [41]

    IEEE Conf

    Liu, C., Mao, Z., Zhang, T., Xie, H., Wang, B., Zhang, Y.: Graph structured network for image-text matching. IEEE Conf. Comput. Vis. Pattern Recog., 10921–10930 (2020)

    2020

  34. [42]

    IEEE Conf

    Park, J., Lee, J., Sohn, K.: Bridge to answer: Structure-aware graph interaction network for video question answering. IEEE Conf. Comput. Vis. Pattern Recog., 15526–15535 (2021)

    2021

  35. [43]

    IEEE Int

    Zhu, Z., Tang, W., Wang, L., Zheng, N., Hua, G.: Enriching local and global contexts for temporal action localization. IEEE Int. Conf. Comput. Vis., 13516–13525 (2021)

    2021

  36. [44]

    Park, J., Lee, J., Kim, I.-J., Sohn, K.: Sum- graph: Video summarization via recursive graph modeling. Eur. Conf. Comput. Vis., 647–663 (2020)

    2020

  37. [45]

    IEEE Conf

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Van- houcke, V., Rabinovich, A.: Going deeper with convolutions. IEEE Conf. Comput. Vis. Pattern Recog. (2015)

    2015

  38. [46]

    Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. Adv. Neural Inform. Process. Syst., 91–99 (2015)

    2015

  39. [47]

    Gygli, M., Grabner, H., Riemenschneider, H., Gool, L.V.: Creating summaries from user videos. Eur. Conf. Comput. Vis. (2014)

    2014

  40. [48]

    IEEE Conf

    Song, Y., Vallmitjana, J., Stent, A., Jaimes, A.: Tvsum: Summarizing web videos using titles. IEEE Conf. Comput. Vis. Pattern Recog., 5179–5187 (2015)

    2015

  41. [49]

    IEEE Int

    Pritch, Y., Rav-Acha, A., Gutman, A., Peleg, S.: Webcam synopsis: Peeking around the world. IEEE Int. Conf. Comput. Vis. (2007)

    2007

  42. [50]

    ACM Trans

    Joshi, N., Kienzle, W., Toelle, M., Uytten- daele, M., Cohen, M.F.: Real-time hyperlapse creation via optimal frame selection. ACM Trans. on Graphics34(4), 1–9 (2015)

    2015

  43. [51]

    IEEE Conf

    Poleg, Y., Halperin, T., Arora, C., Peleg, S.: Egosampling: Fast-forward and stereo for egocentric videos. IEEE Conf. Comput. Vis. Pattern Recog., 4768–4776 (2015)

    2015

  44. [52]

    IEEE Conf

    Kang, H.-W., Matsushita, Y., Tang, X., Chen, X.-Q.: Space-time video montage. IEEE Conf. Comput. Vis. Pattern Recog., 1331–1338 (2006)

    2006

  45. [53]

    Sun, M., Farhadi, A., Taskar, B., Seitz, S.: Salient montages from unconstrained videos. Eur. Conf. Comput. Vis., 472–488 (2014)

    2014

  46. [54]

    Gong, B., Chao, W.-L., Grauman, K., Sha, F.: Diverse sequential subset selection for supervised video summarization. Adv. Neural Inform. Process. Syst., 2069–2077 (2014)

    2014

  47. [55]

    IEEE Conf

    Gygli, M., Grabner, H., Gool, L.V.: Video summarization by learning submodular mix- tures of objectives. IEEE Conf. Comput. Vis. Pattern Recog., 3090–3098 (2015)

    2015

  48. [56]

    IEEE Conf

    Lee, Y.J., Ghosh, J., Grauman, K.: Dis- covering important people and objects for egocentric video summarization. IEEE Conf. Comput. Vis. Pattern Recog. (2012)

    2012

  49. [57]

    Liu, T., Kender, J.R.: Optimization algo- rithms for the selection of key frame 26 sequences of variable length. Eur. Conf. Com- put. Vis., 403–417 (2002)

    2002

  50. [58]

    IEEE Int

    Ngo, C.-W., Ma, Y.-F., Zhang, H.-J.: Auto- matic video summarization by graph model- ing. IEEE Int. Conf. Comput. Vis. (2003)

    2003

  51. [59]

    IEEE Conf

    Lu, Z., Grauman, K.: Story-driven summa- rization for egocentric video. IEEE Conf. Comput. Vis. Pattern Recog. (2013)

    2013

  52. [60]

    Potapov, D., Douze, M., Harchaoui, Z., Schmid, C.: Category-specific video summa- rization. Eur. Conf. Comput. Vis., 540–555 (2014)

    2014

  53. [61]

    Zhang, K., Grauman, K., Sha, F.: Retrospec- tive encoders for video summarization. Eur. Conf. Comput. Vis., 391–408 (2018)

    2018

  54. [62]

    ICASSP, 6519–6527 (2022)

    Kaseris, M., Mademlis, I., Pitas, I.: Exploit- ing caption diversity for unsupervised video summarization. ICASSP, 6519–6527 (2022)

    2022

  55. [63]

    Wu, G., Lin, J., Silva, C.T.: Era: Entity- relationship aware video summarization with wasserstein gan. Brit. Mach. Vis. Conf., 1–14 (2021)

    2021

  56. [64]

    IEEE Conf

    Wu, G., Lin, J., Silva, C.T.: Intentvizor: Towards generic query guided interactive video summarization. IEEE Conf. Comput. Vis. Pattern Recog., 10503–10512 (2022)

    2022

  57. [65]

    IEEE Conf

    Carreira, J., Agrawal, P., Fragkiadaki, K., Malik, J.: Human pose estimation with itera- tive error feedback. IEEE Conf. Comput. Vis. Pattern Recog., 4733–4742 (2016)

    2016

  58. [66]

    Proceedings of Machine Learning Research89, 313–322 (2019)

    Mukhoty, B., Gopakumar, G., Jain, P., Kar, P.: Globally-convergent iteratively reweighted least squares for robust regression problems. Proceedings of Machine Learning Research89, 313–322 (2019)

    2019

  59. [67]

    Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. Int. Conf. Mach. Learn., 282–289 (2001)

    2001

  60. [68]

    Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: A deep learn- ing framework for traffic forecasting. Int. Joint. Conf. Art. Intell., 3634–3640 (2018)

    2018

  61. [69]

    Qian, X., Zhuang, Y., Li, Y., Xiao, S., Pu, S., Xiao, J.: Video relation detection with spatio- temporal graph. ACM Int. Conf. Multimedia, 84–93 (2020)

    2020

  62. [70]

    IEEE Conf

    Pan, B., Cai, H., Huang, D.-A., Lee, K.-H., Gaidon, A., Adeli, E., Niebles, J.C.: Spatio- temporal graph for video captioning with knowledge distillation. IEEE Conf. Comput. Vis. Pattern Recog., 10870–10879 (2020)

    2020

  63. [71]

    IEEE Conf

    Zhang, Z., Shi, Y., Yuan, C., Li, B., Wang, P., Hu, W., Zha, Z.: Object relational graph with teacher-recommended learning for video captioning. IEEE Conf. Comput. Vis. Pattern Recog., 13278–13288 (2020)

    2020

  64. [72]

    IEEE Conf

    Zhang, Z., Zhao, Z., Zhao, Y., Wang, Q., Liu, H., Gao, L.: Where does it exist: Spatio- temporal video grounding for multi-form sen- tences. IEEE Conf. Comput. Vis. Pattern Recog., 10668–10677 (2020)

    2020

  65. [73]

    IEEE Conf

    Li, M., Wang, H., Zhang, W., Miao, J., Zhao, Z., Zhang, S., Ji, W., Wu, F.: Win- ner: Weakly-supervised hierarchical decom- position and alignment for spatio-temporal video grounding. IEEE Conf. Comput. Vis. Pattern Recog., 23090–23099 (2023)

    2023

  66. [74]

    AAAI Conf

    Bhattacharya, U., Mittal, T., Chandra, R., Randhavane, T., Bera, A., Manocha, D.: Step: Spatial temporal graph convolutional networks for emotion perception from gaits. AAAI Conf. Art. Intell., 1342–1350 (2020)

    2020

  67. [75]

    IEEE Conf

    Yang, J., Zheng, W.-S., Yang, Q., Chen, Y., Tian, Q.: Spatial-temporal graph con- volutional network for video-based person re-identification. IEEE Conf. Comput. Vis. Pattern Recog., 3289–3299 (2020)

    2020

  68. [76]

    AAAI Conf

    Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton- based action recognition. AAAI Conf. Art. Intell., 7444–7452 (2018) 27

    2018

  69. [77]

    IEEE Conf

    Shi, L., Zhang, Y., Cheng, J., Lu, H.: Two- stream adaptive graph convolutional net- works for skeleton-based action recognition. IEEE Conf. Comput. Vis. Pattern Recog., 12026–12035 (2019)

    2019

  70. [78]

    Ye, F., Pu, S., Zhong, Q., Li, C., Xie, D., Tang, H.: Dynamic gcn: Context-enriched topology learning for skeleton-based action recognition. ACM Int. Conf. Multimedia, 55– 63 (2020)

    2020

  71. [79]

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. Int. Conf. Mach. Learn., 8748–8763 (2021)

    2021

  72. [80]

    IEEE Trans

    Zhao, B., Li, H., Lu, X., Li, X.: Recon- structive sequence-graph network for video summarization. IEEE Trans. Pattern Anal. Mach. Intell.44(5), 2793–2801 (2021)

    2021

  73. [81]

    Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. Adv. Neu- ral Inform. Process. Syst., 529–536 (2004)

    2004

  74. [82]

    Zhao, B., Li, X., Lu, X.: Hierarchical recur- rent neural network for video summarization. ACM Int. Conf. Multimedia, 863–871 (2017)

    2017

  75. [83]

    De Avila, S.E.F., Lopes, A.P.B., Luz Jr, A., Albuquerque Ara´ ujo, A.: Vsumm: A mecha- nism designed to produce static video sum- maries and a novel evaluation method. Patt. Rec. Letters32(1), 56–68 (2011)

    2011

  76. [84]

    https://open-video.org/

    Open video project. https://open-video.org/

  77. [85]

    Biometrika33(3), 239–251 (1945)

    Kendall, M.G.: The treatment of ties in ranking problems. Biometrika33(3), 239–251 (1945)

    1945

  78. [86]

    CRC Press (1999)

    Zwillinger, D., Kokoska, S.: Crc standard probability and statistics tables and formu- lae. CRC Press (1999)

    1999

  79. [87]

    IEEE Conf

    Otani, M., Nakashima, Y., Rahtu, E., Heikkil¨ a, J.: Rethinking the evaluation of video summaries. IEEE Conf. Comput. Vis. Pattern Recog., 7596–7604 (2019)

    2019

  80. [88]

    IEEE Conf

    Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., Zhang, L.: Bottom- up and top-down attention for image cap- tioning and visual question answering. IEEE Conf. Comput. Vis. Pattern Recog., 6077– 6086 (2018)

    2018

  81. [89]

    Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalan- tidis, Y., Li, L.-J., Shamma, D.A., Bernstein, M.S., Fei-Fei, L.: Visual genome: Connect- ing language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis., 32–73 (2017)

    2017

  82. [90]

    IEEE Conf

    Hosang, J., Benenson, R., Schiele, B.: Learn- ing non-maximum suppression. IEEE Conf. Comput. Vis. Pattern Recog., 4507–4515 (2017)

    2017

  83. [91]

    AAAI Conf

    Zhou, K., Qiao, Y., Xiang, T.: Deep reinforce- ment learning for unsupervised video sum- marization with diversity-representativeness reward. AAAI Conf. Art. Intell., 7582–7589 (2018)

    2018

  84. [92]

    IEEE Trans

    Ji, Z., Xiong, K., Pang, Y., Li, X.: Video summarization with attention-based encoder- decoder networks. IEEE Trans. Circuit Syst. Vid. Tech.30(6), 1709–1717 (2020)

    2020

  85. [93]

    IEEE Conf

    Jiang, H., Mu, Y.: Joint video summarization and moment localization by cross-task sample transfer. IEEE Conf. Comput. Vis. Pattern Recog., 16388–16398 (2022)

    2022

  86. [94]

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. Int. Conf. Learn. Represent. (2015)

    2015

  87. [95]

    IEEE Conf

    Zhao, B., Xing, E.P.: Quasi real-time sum- marization for consumer videos. IEEE Conf. Comput. Vis. Pattern Recog. (2014)

    2014

  88. [96]

    Sharghi, A., Gong, B., Shah, M.: Query- focused extractive video summarization. Eur. Conf. Comput. Vis., 3–19 (2016)

    2016

  89. [97]

    arXiv preprint arXiv:1406.5824 (2014) 28

    Yeung, S., Fathi, A., Fei-Fei, L.: Videoset: Video summary evaluation through text. arXiv preprint arXiv:1406.5824 (2014) 28

    2014 arXiv

  90. [98]

    NeurIPS Workshop (2021)

    Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., Komatsuzaki, A.: Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. NeurIPS Workshop (2021)

    2021

  91. [99]

    ACL Conf

    Xu, H., Ghosh, G., Huang, P.-Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettlemoyer, L., Feichtenhofer, C.: Videoclip: Contrastive pre-training for zero-shot video-text under- standing. ACL Conf. Emp. Meth. Natural Lang. Process., 6787–6800 (2021) 29

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.