Pith. sign in

REVIEW 4 major objections 4 minor 58 references

THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read THYME combines hierarchical spatial aggregation with cyclic temporal attention to improve video scene graph generation on both ground and aerial benchmarks.

desk verdict A plausible incremental combination of the authors' own modules with a useful dataset extension, but the temporal refinement step silently assumes DETR slots are stable object identities, and the experimental reporting needs tightening. read the letter →

arxiv 2507.09200 v1 pith:TOXMCC7X submitted 2025-07-12 cs.CV

Trong-Thuan Nguyen , Pha Nguyen , Jackson Cothren , Alper Yilmaz , Minh-Triet Tran , Khoa Luu This is my paper
classification cs.CV
keywords videoscenegraphgenerationhierarchicalfeatureaggregationcyclictemporalattentionaerialunderstandinginteractivityannotationdronefootagetransformerrecallandmean
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that video scene graph generation does not have to choose between spatial detail and temporal consistency, and that the two can be combined by stacking a hierarchical feature-aggregation stage on top of a cyclic temporal-attention stage. The proposed THYME model first aggregates per-frame object features across multiple abstraction levels, then runs attention over each object's sequence of refined features with the sequence wrapped into a cycle, so the last frame can attend to the first. The paper also introduces AeroEye-v1.0, a drone-captured dataset with annotations for five interactivity types (appearance, situation, position, interaction, relation), and reports that THYME improves recall and mean recall by 2 to 3 percent over previous methods on both ASPIRe and AeroEye-v1.0. A sympathetic reader would care because accurate and temporally stable scene graphs matter for applications like autonomous driving, surveillance, and sports analytics.

What carries the argument

Two mechanisms carry the argument. Hierarchical feature aggregation updates each object's feature by attention-weighted sums over all objects in the same frame, for several levels, so single-frame graphs are built at multiple spatial scales. Cyclic temporal attention then joins each object's sequence of high-level features into a ring by indexing time with modulo arithmetic, allowing every time step, including the last, to attend to every other step; this is the mechanism the paper credits for long-range coherence and for surviving occlusions and re-entry. The five interactivity types in the AeroEye-v1.0 dataset also function as machinery, because the paper uses them to define the predicate vocabulary that both anchors and evaluates the method.

What would settle it

Take a video, shuffle the order of the detected object queries in a single frame while leaving all visual content unchanged, and rerun THYME; if recall or mean recall changes materially, the temporal stage is responding to slot identity rather than object identity, which would put the claimed temporal-coherence benefit in question.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that integrating hierarchical feature aggregation with cyclic temporal refinement yields video scene graphs that are simultaneously more spatially detailed and more temporally coherent than either component alone. The model extracts DETR object-query embeddings per frame, refines them through multiple levels of attention-based aggregation over all objects in the frame, and then, for each object Si, builds the temporal sequence X_t'(Si) = $F^{{Lh}}$_t'(Si) and applies cyclic attention in which the attention weight for time offset τ uses indices (t'+τ) mod T'. The refined features feed a gated relation decoder that predicts predicates for every object pair, and focal loss is summed over hierarchical levels. The paper reports this design outperforms HIG, CYCLO, STTran, TEMPURA, and other baselines on ASPIRe and AeroEye-v1.0, with the largest gains on double-actor predicates such as position, interaction, and relation.

Load-bearing premise

The temporal stage assumes that the same detector query slot keeps referring to the same physical object across frames, but the paper does not describe any tracking, matching, or identity-preservation step to guarantee that.

Editorial extensions

If this is right

  • If THYME is correct, video scene graph models can combine fine-grained per-frame spatial reasoning with long-range temporal coherence in one architecture, removing the frame-level versus video-level tradeoff.
  • Cyclic attention would reduce predicate flicker and keep relationships stable when objects disappear and reappear, since the sequence wraps and preserves historical context.
  • AeroEye-v1.0 gives the field a drone-captured benchmark with all five interactivity types, so aerial scene understanding can be measured separately from ground-view performance.
  • The reported 2 to 3 percent recall and mean recall gains on ASPIRe and AeroEye-v1.0 imply the combined mechanism transfers across viewpoints and predicate vocabularies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not say this, but the cyclic temporal stage is only as good as the slot-to-object correspondence: because no tracking or association step is described, an explicit re-identification module inserted before cyclic attention is a natural extension that could strengthen the same reported gains.
  • A testable extension would be to apply the same hierarchy-plus-cycle recipe to other sequence tasks, such as action recognition or trajectory prediction, where short-lived interactions and long-range context also compete.
  • The dataset's long-tail predicate statistics suggest the model's mean-recall improvements may be driven by better handling of rare predicates; an ablation that re-weights or removes rare predicates would show whether the gain is specifically a long-tail effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces THYME, a video scene graph generation method that combines hierarchical intra-frame feature aggregation with cyclic temporal attention, and simultaneously introduces AeroEye-v1.0, a new aerial video dataset annotated with five interactivity types (appearance, situation, position, interaction, relation). The authors evaluate THYME on ASPIRe and AeroEye-v1.0, reporting state-of-the-art results with claimed 2-3% improvements in recall and mean recall over prior methods, and they provide ablations on hierarchical depth, attention type, and temporal window size.

Significance. If the results hold, the paper offers a practically relevant architecture that directly combines two previously separate ideas, and the AeroEye-v1.0 dataset could fill a real gap in aerial-view video scene graph benchmarks. The manuscript is explicit in its model equations, includes comparative tables with several baselines, and reports ablations that help localize the contribution of hierarchical aggregation and cyclic attention. However, the central temporal-coherence claim depends on an unstated cross-frame identity assumption, one experimental table directly contradicts its accompanying text, and the stated 2-3% improvement is not supported for mean recall. These issues need to be resolved before the performance claims can be accepted.

major comments (4)
  1. [Section IV-C, Eqs. (3)-(5)] The temporal refinement stage constructs, for each "tracked object" S_i, a sequence {X_t'(S_i)} whose elements are F^(L_h)_t'(S_i), but no cross-frame association, tracking, or re-identification step is described that would make the DETR query index i correspond to the same physical object over time. DETR queries are per-frame detection slots and are assigned by a per-frame matching process; they are not tracklets, and the paper's own qualitative examples (Figs. 5 and 6) show objects leaving and re-entering. Without an explicit association mechanism, or at least a clear statement of the temporal pooling that defines T' and handles missing frames, the sequences fed to cyclic attention may mix features from different objects, and the reported temporal gains cannot be attributed to coherent object trajectories. Please specify this association step or provide evidence that query indices are identity-stable.
  2. [Section VI-B-3, Table IV] The text and Table IV are inconsistent: the text says the 3/4 window gives Appearance 16.22 R@20 / 0.67 mR@20 and Position 15.22 / 1.03, while the full window gives 16.52 / 0.68 and 15.52 / 1.05; Table IV instead lists 3/4 as 16.52 / 0.68 and 15.52 / 1.05 and Full as 16.22 / 0.67 and 15.22 / 1.03. The text also refers to the smallest window as "1" although the table labels it "1/2". Please correct this mismatch and state explicitly which window size is used in the main comparisons, since the current contradiction undermines the temporal-window ablation.
  3. [Abstract and contribution list] The claimed "consistent improvements of 2 to 3% in recall and mean recall" is not supported by the reported mR values. In Table VI (AeroEye-v1.0), the mR@20 differences between THYME and CYCLO are 0.30 (Position), 0.02 (Interaction), and 0.12 (Relation); Table V shows similarly small mR margins. The 2-3% margin holds approximately for some R@20 values on double-actor predicates, but not for mean recall. Please rephrase the claim to be specific about the metric and the predicate types, or report the mean-recall margins accurately.
  4. [Sections V and VI-C-2] AeroEye-v1.0 is introduced and evaluated by the same authors in this paper, and the main SOTA claim rests partly on it, yet Section V provides only aggregate statistics and no train/validation/test split, annotation agreement, or release plan. Please describe the dataset splits, annotation protocol, and availability so that Table VI can be reproduced and so that the use of an in-house benchmark does not make the comparison circular.
minor comments (4)
  1. [Figures 5 and 6] The proposed method is labeled "HIERCY" in both qualitative figures, while the paper consistently calls it THYME; please correct the figure labels.
  2. [Table II and Section VI-B-1] The table header "factor size" and the section title "Number of Hierarchical Levels" are inconsistent; please define what the 1/4, 1/2, 3/4, and Full entries mean in terms of L_h.
  3. [Section VI-B-3] Please clarify whether the temporal window size is a fraction of the video length or a number of frames, and how the pooled time steps T' in Eq. (3) are obtained from the original frame count T.
  4. [Captions of Figures 5 and 6] The caption text "red edges is incorrect predictions" should be corrected to "red edges denote incorrect predictions," and the qualitative claims about tracking should be made consistent with the actual mechanism described in Section IV.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: THYME integrates two self-cited modules but specifies them fully with equations and ablations; the self-authored benchmark is an external-validity concern, not a derivation-level circularity.

full rationale

The paper does not contain a prediction that is equivalent to its inputs by construction. The two core components, hierarchical feature aggregation and cyclic temporal attention, are adopted from the authors' prior work [32, 33], but the present paper states the full equations (Eqs. 1-5) and tests them in ablations (Tables II-IV), so the self-citations are not the only support for the mechanism. No uniqueness theorem is imported, and no ansatz is smuggled solely through citation. The AeroEye-v1.0 benchmark is self-authored and self-evaluated, which limits independent verification of the 'state-of-the-art' claim, but a self-created dataset is a contribution rather than a circular derivation. The temporal-refinement stage (Sec. IV-C) assumes that DETR query index i corresponds to the same physical object across frames without describing tracking or association; this is an unverified modeling assumption that threatens attribution of the temporal gains, but it is not a circularity of the kind where the output is defined in terms of the input. Because the central empirical claim rests on measurements rather than on a chain of definitions, the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central contribution is an architecture built from two previously published modules plus a re-annotated version of the authors' own dataset. The unstated loads are: (1) DETR query indices are treated as stable object identities across frames with no association step, which is load-bearing for the temporal sequence construction in Eq. 4; (2) the hidden scale choices Lh and T' measurably change results in Tables II and IV yet are never reported in absolute terms; (3) the dataset's value assumes the five interactivity types are separable and consistently annotated, but no annotation-quality metrics are provided; (4) the focal loss coefficients from [32] that control the long-tail behavior (visible in mR values near 0.1 to 1.0) are not stated. The only invented entity is AeroEye-v1.0 itself, which has no falsifiable handle outside the paper because no release mechanism is given.

free parameters (3)
  • hierarchical aggregation depth Lh = not reported (ablated as 1/4, 1/2, 3/4, Full in Table II)
    Table II shows R@20 varies from 14.12 to 16.52 with depth; the actual number of levels is never stated, so the main architectural knob is unreported.
  • temporal window size T' = not reported (ablated as 1/2, 3/4, Full in Table IV)
    Table IV shows window size changes results by up to about one point; the window length in frames is never given and temporal pooling is undefined.
  • focal loss coefficients alpha_t, gamma = not reported (adopted from HIG [32])
    Eq. 7 follows [32] but the balancing factor and focusing parameter are not stated; mR@20 values are extremely sensitive (0.14 to 0.16) and no tuning details are given.
assumptions (4)
  • domain assumption DETR query embeddings preserve object identity across frames
    Section IV-C builds a temporal sequence X_t'(Si) per object directly from DETR query embeddings with no matching or tracking step; DETR queries are detection slots, not tracklets, and the paper itself notes objects leave and re-enter scenes.
  • domain assumption The five interactivity types are separable and consistently annotatable
    Section V-B defines appearance, situation, position, interaction, and relation as distinct layers; the value of AeroEye-v1.0 depends on annotators agreeing on the relation/interaction distinction (e.g., towing vs assisting), but no inter-annotator agreement or quality protocol is reported.
  • domain assumption Fixed temporal windows are sufficient for interaction durations
    Section III-A criticizes HIG for fixed temporal segmentation, yet THYME's cyclic attention also operates on a fixed window T' (Eq. 4); the paper does not state how the window size was chosen across videos of varying length.
  • standard math Standard dot-product attention and transformer blocks are correct
    Eqs. 1-5 use standard softmax attention and transformer encoder blocks; no nonstandard mathematics is introduced.
invented entities (1)
  • AeroEye-v1.0 dataset
    purpose: aerial video benchmark with five interactivity annotation types for VidSGG
    No dataset URL, release, or documentation is provided in the paper; it is built on the authors' own AeroEye dataset and can only be obtained, if at all, through the authors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage." pith.science (2026). https://pith.science/paper/TOXMCC7X

@misc{pith2026250709200,
  author       = {Pith},
  title        = {Pith review of: THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TOXMCC7X}},
  note         = {Machine review of arXiv:2507.09200}
}
read the original abstract

The rapid proliferation of video in applications such as autonomous driving, surveillance, and sports analytics necessitates robust methods for dynamic scene understanding. Despite advances in static scene graph generation and early attempts at video scene graph generation, previous methods often suffer from fragmented representations, failing to capture fine-grained spatial details and long-range temporal dependencies simultaneously. To address these limitations, we introduce the Temporal Hierarchical Cyclic Scene Graph (THYME) approach, which synergistically integrates hierarchical feature aggregation with cyclic temporal refinement to address these limitations. In particular, THYME effectively models multi-scale spatial context and enforces temporal consistency across frames, yielding more accurate and coherent scene graphs. In addition, we present AeroEye-v1.0, a novel aerial video dataset enriched with five types of interactivity that overcome the constraints of existing datasets and provide a comprehensive benchmark for dynamic scene graph generation. Empirically, extensive experiments on ASPIRe and AeroEye-v1.0 demonstrate that the proposed THYME approach outperforms state-of-the-art methods, offering improved scene understanding in ground-view and aerial scenarios.

Figures

Figures reproduced from arXiv: 2507.09200 by the authors.

Figure 1
Figure 1. An exemplar video from the proposed AeroEye-v1.0 dataset, captured by a drone, is annotated with five interactivity types: appearance, situation, position, interaction, and relation. Specifically, the proposed THYME approach then generates frame-by-frame scene graphs, hierarchically aggregating per-frame object features and refining them through cyclic temporal attention. Therefore, our proposed THYME approach accur… view at source ↗
Figure 2
Figure 2. (a) HIG [32] stacks single-frame progressively coarser layers; however, its fixed temporal windows and acyclic flow limit the modeling of recurrence and frame-to-frame coherence. (b) CYCLO [33] employs a ring-shaped spatio-temporal transformer whose cyclic attention links the clip’s tail back to its head, preserving long-range context and preventing relation drift, yet operates at a single spatial scale and omits ex… view at source ↗
Figure 3
Figure 3. Overview of the Hierarchical Cyclic Scene Graph approach for Video Scene Graph Generation (VidSGG). Our approach begins by extracting per-frame [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Example from AeroEye-v1.0. The scene graph is annotated with nodes denoting objects (bounding boxes), each with appearance and situation annotations. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison on the ASPIRe dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison on the AeroEye-v1.0 dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 55 canonical work pages

  1. [1]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision , pages 213–229. Springer, 2020. 6

    work page 2020

  2. [2]

    Unifying short and long-term tracking with graph hierarchies

    Orcun Cetintas, Guillem Brasó, and Laura Leal-Taixé. Unifying short and long-term tracking with graph hierarchies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22877–22887, 2023. 2

    work page 2023

  3. [3]

    More knowledge, less bias: Unbiasing scene graph generation with explicit ontological adjustment

    Zhanwen Chen, Saed Rezayi, and Sheng Li. More knowledge, less bias: Unbiasing scene graph generation with explicit ontological adjustment. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4023–4032, 2023. 3

    work page 2023

  4. [4]

    Spatial-temporal transformer for dynamic scene graph generation

    Yuren Cong, Wentong Liao, Hanno Ackermann, Bodo Rosenhahn, and Michael Ying Yang. Spatial-temporal transformer for dynamic scene graph generation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 16372–16382, 2021. 3

    work page 2021

  5. [5]

    Reltr: Relation transformer for scene graph generation

    Yuren Cong, Michael Ying Yang, and Bodo Rosenhahn. Reltr: Relation transformer for scene graph generation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(9):11169–11183, 2023. 2, 4

    work page 2023

  6. [6]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. International Journal of Computer Vision, pages 1–23, 2022. 3

    work page 2022

  7. [7]

    Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answering

    Long Hoang Dang, Thao Minh Le, Vuong Le, and Truyen Tran. Hierarchical object-oriented spatio-temporal reasoning for video question answering. arXiv preprint arXiv:2106.13432 , 2021. 2

    work page Pith review arXiv 2021

  8. [8]

    Skateformer: skeletal-temporal transformer for human action recognition

    Jeonghyeok Do and Munchurl Kim. Skateformer: skeletal-temporal transformer for human action recognition. In European Conference on Computer Vision, pages 401–420. Springer, 2024. 2

    work page 2024

Show all 58 references
  1. [9]

    Exploiting long-term dependencies for generating dynamic scene graphs

    Shengyu Feng, Hesham Mostafa, Marcel Nassar, Somdeb Majumdar, and Subarna Tripathi. Exploiting long-term dependencies for generating dynamic scene graphs. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 5130–5139, 2023. 2, 3, 4

    2023

  2. [10]

    Mist: Multi-modal iterative spatial-temporal transformer for long- form video question answering

    Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, and Mike Zheng Shou. Mist: Multi-modal iterative spatial-temporal transformer for long- form video question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14773– 1478...

    2023

  3. [11]

    Graphdreamer: Compositional 3d scene synthesis from scene graphs

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Schölkopf. Graphdreamer: Compositional 3d scene synthesis from scene graphs. arXiv preprint arXiv:2312.00093 , 2023. 3

    2023 arXiv

  4. [12]

    Dsgg: Dense relation transformer for an end-to-end scene graph generation

    Zeeshan Hayder and Xuming He. Dsgg: Dense relation transformer for an end-to-end scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 28317– 28326, 2024. 2, 3, 4

    2024

  5. [13]

    Toward a unified transformer-based framework for scene graph generation and human- object interaction detection

    Tao He, Lianli Gao, Jingkuan Song, and Yuan-Fang Li. Toward a unified transformer-based framework for scene graph generation and human- object interaction detection. IEEE Transactions on Image Processing , 32:6274–6288, 2023. 2, 4

    2023

  6. [14]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709, 2019. 3

    2019

  7. [15]

    Hierarchical spatio–temporal graph convolutional networks and transformer network for traffic flow forecasting

    Guangyu Huo, Yong Zhang, Boyue Wang, Junbin Gao, Yongli Hu, and Baocai Yin. Hierarchical spatio–temporal graph convolutional networks and transformer network for traffic flow forecasting. IEEE Transactions 13 on Intelligent Transportation Systems , 24(4):3855–3867, 2023. 2

    2023

  8. [16]

    Egtr: Extracting graph from transformer for scene graph generation

    Jinbae Im, JeongYeon Nam, Nokyung Park, Hyungmin Lee, and Seunghyun Park. Egtr: Extracting graph from transformer for scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24229–24238, 2024. 2, 3, 4, 5, 7

    2024

  9. [17]

    Action genome: Actions as compositions of spatio-temporal scene graphs

    Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. Action genome: Actions as compositions of spatio-temporal scene graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10236–10247, 2020. 2, 3

    2020

  10. [18]

    Hdgt: Heterogeneous driving graph transformer for multi-agent trajectory prediction via scene encoding

    Xiaosong Jia, Penghao Wu, Li Chen, Yu Liu, Hongyang Li, and Junchi Yan. Hdgt: Heterogeneous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE transactions on pattern analysis and machine intelligence , 45(11):13860–13875, 2023. 2

    2023

  11. [19]

    Fast contextual scene graph generation with unbiased context augmentation

    Tianlei Jin, Fangtai Guo, Qiwei Meng, Shiqiang Zhu, Xiangming Xi, Wen Wang, Zonghao Mu, and Wei Song. Fast contextual scene graph generation with unbiased context augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6302–63...

    2023

  12. [20]

    Hierarchical cross-modal graph consistency learning for video-text retrieval

    Weike Jin, Zhou Zhao, Pengcheng Zhang, Jieming Zhu, Xiuqiang He, and Yueting Zhuang. Hierarchical cross-modal graph consistency learning for video-text retrieval. In Proceedings of the 44th International ACM SIGIR Conference on research and development in information retrieval...

    2021

  13. [21]

    Segmentation-grounded scene graph generation

    Siddhesh Khandelwal, Mohammed Suhail, and Leonid Sigal. Segmentation-grounded scene graph generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15879– 15889, 2021. 3

    2021

  14. [22]

    Llm4sgg: Large language model for weakly supervised scene graph generation

    Kibum Kim, Kanghoon Yoon, Jaehyeong Jeon, Yeonjun In, Jinyoung Moon, Donghyun Kim, and Chanyoung Park. Llm4sgg: Large language model for weakly supervised scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,

  15. [23]

    Llm4sgg: Large language models for weakly supervised scene graph generation

    Kibum Kim, Kanghoon Yoon, Jaehyeong Jeon, Yeonjun In, Jinyoung Moon, Donghyun Kim, and Chanyoung Park. Llm4sgg: Large language models for weakly supervised scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2830...

    2024

  16. [24]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer ...

    2017

  17. [25]

    Is-ggt: Iterative scene graph generation with generative transformers

    Sanjoy Kundu and Sathyanarayanan N Aakur. Is-ggt: Iterative scene graph generation with generative transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6292–6301, 2023. 2, 3, 4

    2023

  18. [26]

    Hero: Hierarchical encoder for video+ language omni-representation pre-training

    Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, and Jingjing Liu. Hero: Hierarchical encoder for video+ language omni-representation pre-training. arXiv preprint arXiv:2005.00200 , 2020. 2

    2005 arXiv

  19. [27]

    Sgtr: End-to-end scene graph generation with transformer

    Rongjie Li, Songyang Zhang, and Xuming He. Sgtr: End-to-end scene graph generation with transformer. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19486– 19496, 2022. 2, 3, 4, 5

    2022

  20. [28]

    Vrr-vg: Refocusing visually-relevant relationships

    Yuanzhi Liang, Yalong Bai, Wei Zhang, Xueming Qian, Li Zhu, and Tao Mei. Vrr-vg: Refocusing visually-relevant relationships. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10403–10412, 2019. 3

    2019

  21. [29]

    Social graph transformer networks for pedestrian trajectory prediction in complex social scenarios

    Yao Liu, Lina Yao, Binghao Li, Xianzhi Wang, and Claude Sammut. Social graph transformer networks for pedestrian trajectory prediction in complex social scenarios. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 1339–1349,

  22. [30]

    Graph transformer net- work with temporal kernel attention for skeleton-based action recognition

    Yanan Liu, Hao Zhang, Dan Xu, and Kangjian He. Graph transformer net- work with temporal kernel attention for skeleton-based action recognition. Knowledge-Based Systems, 240:108146, 2022. 2

    2022

  23. [31]

    Unbiased scene graph generation in videos

    Sayak Nag, Kyle Min, Subarna Tripathi, and Amit K Roy-Chowdhury. Unbiased scene graph generation in videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22803–22813, 2023. 3, 4

    2023

  24. [32]

    Hig: Hierarchical interlacement graph approach to scene graph generation in video understanding

    Trong-Thuan Nguyen, Pha Nguyen, and Khoa Luu. Hig: Hierarchical interlacement graph approach to scene graph generation in video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024. 2, 3, 4, 5, 7, 8

    2024

  25. [33]

    CYCLO: Cyclic graph transformer approach to multi-object relationship modeling in aerial videos

    Trong-Thuan Nguyen, Pha Nguyen, Li Xin, Cothren Jackson, Yilmaz Alper, and Khoa Luu. CYCLO: Cyclic graph transformer approach to multi-object relationship modeling in aerial videos. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. 2, 3, 4...

    2024

  26. [34]

    Learning hierarchical relationships for object-goal navigation

    Anwesan Pal, Yiding Qiu, and Henrik Christensen. Learning hierarchical relationships for object-goal navigation. In Conference on Robot Learning, pages 517–528. PMLR, 2021. 2

    2021

  27. [35]

    Igformer: Interaction graph transformer for skeleton-based human interaction recognition

    Yunsheng Pang, Qiuhong Ke, Hossein Rahmani, James Bailey, and Jun Liu. Igformer: Interaction graph transformer for skeleton-based human interaction recognition. In European Conference on Computer Vision , pages 605–622. Springer, 2022. 2

    2022

  28. [36]

    Video relation detection with spatio-temporal graph

    Xufeng Qian, Yueting Zhuang, Yimeng Li, Shaoning Xiao, Shiliang Pu, and Jun Xiao. Video relation detection with spatio-temporal graph. In Proceedings of the 27th ACM international conference on multimedia , pages 84–93, 2019. 3, 4

    2019

  29. [37]

    Annotating objects and relations in user-generated videos

    Xindi Shang, Donglin Di, Junbin Xiao, Yu Cao, Xun Yang, and Tat-Seng Chua. Annotating objects and relations in user-generated videos. In Proceedings of the 2019 on International Conference on Multimedia Retrieval, pages 279–287, 2019. 3

    2019

  30. [38]

    Video visual relation detection via iterative inference

    Xindi Shang, Yicong Li, Junbin Xiao, Wei Ji, and Tat-Seng Chua. Video visual relation detection via iterative inference. In Proceedings of the 29th ACM international conference on Multimedia , pages 3654–3663,

  31. [39]

    Video visual relation detection

    Xindi Shang, Tongwei Ren, Jingfan Guo, Hanwang Zhang, and Tat-Seng Chua. Video visual relation detection. In Proceedings of the 25th ACM international conference on Multimedia , pages 1300–1308, 2017. 3

    2017

  32. [40]

    Progress-aware online action segmentation for egocentric procedural task videos

    Yuhan Shen and Ehsan Elhamifar. Progress-aware online action segmentation for egocentric procedural task videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18186–18197, 2024. 2

    2024

  33. [41]

    Target adaptive context aggregation for video scene graph generation

    Yao Teng, Limin Wang, Zhifeng Li, and Gangshan Wu. Target adaptive context aggregation for video scene graph generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13688–13697, 2021. 2, 3, 4, 5

    2021

  34. [42]

    Video relationship reasoning using gated spatio-temporal energy graph

    Yao-Hung Hubert Tsai, Santosh Divvala, Louis-Philippe Morency, Ruslan Salakhutdinov, and Ali Farhadi. Video relationship reasoning using gated spatio-temporal energy graph. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10424– 1043...

    2019

  35. [43]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30,

  36. [44]

    Oed: Towards one-stage end-to-end dynamic scene graph generation

    Guan Wang, Zhimin Li, Qingchao Chen, and Yang Liu. Oed: Towards one-stage end-to-end dynamic scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 27938–27947, 2024. 2, 4, 5

    2024

  37. [45]

    Pair then relation: Pair-net for panoptic scene graph generation

    Jinghao Wang, Zhengyu Wen, Xiangtai Li, Zujin Guo, Jingkang Yang, and Ziwei Liu. Pair then relation: Pair-net for panoptic scene graph generation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024. 3

    2024

  38. [46]

    Sportshhi: A dataset for human-human interaction detection in sports videos

    Tao Wu, Runyu He, Gangshan Wu, and Limin Wang. Sportshhi: A dataset for human-human interaction detection in sports videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,

  39. [47]

    Video as conditional graph hierarchy for multi-granular question answering

    Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, and Tat- Seng Chua. Video as conditional graph hierarchy for multi-granular question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2804–2812, 2022. 2

    2022

  40. [48]

    Contrastive video question answering via video graph transformer

    Junbin Xiao, Pan Zhou, Angela Yao, Yicong Li, Richang Hong, Shuicheng Yan, and Tat-Seng Chua. Contrastive video question answering via video graph transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):13265–13280, 2023. 2

    2023

  41. [49]

    Scene graph generation by iterative message passing

    Danfei Xu, Yuke Zhu, Christopher B Choy, and Li Fei-Fei. Scene graph generation by iterative message passing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5410–5419,

  42. [50]

    A multi-task learning network with a collision- aware graph transformer for traffic-agents trajectory prediction

    Biao Yang, Fucheng Fan, Rongrong Ni, Hai Wang, Ammar Jafaripourn- imchahi, and Hongyu Hu. A multi-task learning network with a collision- aware graph transformer for traffic-agents trajectory prediction. IEEE Transactions on Intelligent Transportation Systems , 25(7):6677–6690,

  43. [51]

    Panoptic scene graph generation

    Jingkang Yang, Yi Zhe Ang, Zujin Guo, Kaiyang Zhou, Wayne Zhang, and Ziwei Liu. Panoptic scene graph generation. In European Conference on Computer Vision , pages 178–196. Springer, 2022. 1, 3

    2022

  44. [52]

    Panoptic video scene graph generation

    Jingkang Yang, Wenxuan Peng, Xiangtai Li, Zujin Guo, Liangyu Chen, Bo Li, Zheng Ma, Kaiyang Zhou, Wayne Zhang, Chen Change Loy, et al. Panoptic video scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18675–1868...

    2023

  45. [53]

    Trajgat: A graph-based long-term dependency modeling approach for trajectory similarity computation

    Di Yao, Haonan Hu, Lun Du, Gao Cong, Shi Han, and Jingping Bi. Trajgat: A graph-based long-term dependency modeling approach for trajectory similarity computation. In Proceedings of the 28th ACM 14 SIGKDD conference on knowledge discovery and data mining , pages 2275–2285, 2022. 2

    2022

  46. [54]

    Graph transformer networks

    Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph transformer networks. Advances in neural information processing systems , 32, 2019. 2

    2019

  47. [55]

    Hiker-sgg: Hierarchical knowledge enhanced robust scene graph generation

    Ce Zhang, Simon Stepputtis, Joseph Campbell, Katia Sycara, and Yaqi Xie. Hiker-sgg: Hierarchical knowledge enhanced robust scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 28233–28243, 2024. 2

    2024

  48. [56]

    Where does it exist: Spatio-temporal video grounding for multi-form sentences

    Zhu Zhang, Zhou Zhao, Yang Zhao, Qi Wang, Huasheng Liu, and Lianli Gao. Where does it exist: Spatio-temporal video grounding for multi-form sentences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10668–10677, 2020. 3

    2020

  49. [57]

    Textpsg: Panoptic scene graph generation from textual descriptions

    Chengyang Zhao, Yikang Shen, Zhenfang Chen, Mingyu Ding, and Chuang Gan. Textpsg: Panoptic scene graph generation from textual descriptions. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2839–2850, 2023. 3

    2023

  50. [58]

    Hi- erarchical planning for long-horizon manipulation with geometric and symbolic scene graphs

    Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, and Yuke Zhu. Hi- erarchical planning for long-horizon manipulation with geometric and symbolic scene graphs. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 6541–6548. Ieee, 2021. 2

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.