Pith. sign in

REVIEW 4 major objections 5 minor 65 references

Temporally Consistent Dynamic Scene Graphs: An End-to-End Approach for Action Tracklet Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TCDSG claims that a transformer video scene-graph detector trained with a temporal matching penalty and LSTM-conditioned queries yields identity-stable action tracklets end-to-end, lifting Action Genome tR@50 from 18.6% to 39.1%.

desk verdict A genuinely new end-to-end temporal consistency mechanism for video scene graphs with honest ablations, but the headline AG numbers rest on IoU pseudo-labels and the abstract overclaims. read the letter →

arxiv 2412.02808 v2 pith:KQZAEKT3 submitted 2024-12-03 cs.CV cs.LG

Raphael Ruschel , Md Awsafur Rahman , Hardik Prajapati , Suya You , B. S. Manjuanth This is my paper · ORCID
classification cs.CVcs.LG
keywords dynamicscenegraphsactiontrackletstemporalrecallKsequence-levelbipartitematchingtemporallyconditionedqueriesvideounderstandingLSTMfeedbackpersistentobjectIDs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that action tracklets can be produced end-to-end by making the detector itself temporally consistent, instead of stitching per-frame scene graphs together afterward. The two mechanisms claimed to do the work are a sequence-level matching objective that locks each ground-truth interaction to a single decoder slot across frames, and decoder queries that are updated with a recurrent memory of previous frames' outputs. The headline evidence is temporal Recall@50 rising from 18.6% for a framewise baseline to 39.1% on Action Genome, with gains also reported on OpenPVSG and a newly re-annotated MEVA subset. If the claim is right, continuous video tasks such as surveillance and autonomous navigation could read stable subject-object-relation tracklets directly from one transformer pipeline.

What carries the argument

The load-bearing mechanism is Sequence-Level Bipartite Matching (SLBM): a cache-and-penalize extension of Hungarian matching, the optimal one-to-one assignment between predicted slots and ground-truth objects. A hash map records which decoder query first won each ground-truth triplet $\langle s, o, r\rangle$; in every later frame, any cost-matrix entry that would assign that triplet to a different query receives a large penalty, so the matching is biased to keep identities fixed across time. The second mechanism is Temporally Conditioned Queries (TCQ): decoder queries are formed by cross-attending the learnable query embeddings to an LSTM state computed from the previous frame's decoder outputs, giving the decoder memory, context, and continuity. Together these make query index a stable proxy for object-triplet identity, which is exactly what lets tracklet construction be a simple grouping step.

What would settle it

Re-measure Action Genome tR@50 using independently verified tracklet identities, for example human annotations or a separate strong tracker, instead of the paper's IoU-overlap pseudo-labels; if the gap between TCDSG and the DDS baseline narrows sharply, part of the reported gain is an artifact of shared label noise. On the re-annotated MEVA subset, the analogous check is to recompute tR@K while comparing predicted query groups against the provided persistent IDs rather than against groups formed by the same pseudo-linking rule.

Watch

Extended reading notes

Core claim

The central claim is that temporal coherence can be trained into a DETR-style scene-graph detector instead of being added as post-processing. TCDSG's Sequence-Level Bipartite Matching objective caches each ground-truth triplet the first time it appears, binds it to the decoder query it was matched to, and adds a large penalty $\lambda_{\mathrm{co}}$ to any later Hungarian assignment that would move the triplet to another query. Alongside it, Temporally Conditioned Queries update the decoder's input queries as $Q_t = f(Q_{\mathrm{learn}}, \mathrm{LSTM}(D_{t-1}))$, giving the decoder a learned memory of prior frames. At inference the predictions are grouped by query index, and because the training objective has suppressed query-to-triplet reassignment, those groups form action tracklets without IoU linking or re-identification. On Action Genome the authors report tR@50 improving from 18.6% for the DDS baseline to 39.1%, while frame-level R@50 stays competitive at 52.5%.

Load-bearing premise

The load-bearing premise is that the IoU-overlap pseudo-labels on Action Genome give correct ground-truth tracklet identities; if same-class objects re-enter or occlude each other, the pseudo-IDs merge or swap, and because the same labels drive both training and evaluation the reported tR@K gain can reward stable-but-wrong identities.

Editorial extensions

If this is right

  • Tracklet construction becomes a grouping step: predictions that share a query index and triplet label across consecutive frames are concatenated directly, with no IoU linking or re-identification post-processing.
  • Temporal consistency is bought at a measurable cost to single-frame accuracy: removing temporal locking raises R@50 from 52.5% to 58.4% on Action Genome while tR@50 drops from 39.1% to 27.1%.
  • Long, densely sampled videos expose transient query switches: on the re-annotated MEVA subset, keeping every 20th frame lifts tR@50 from 7.8% at full frame rate to 31.2%.
  • The re-annotated MEVA subset, with 141 training and 47 test videos across multiple cameras, provides a benchmark for long-horizon tracklet generation in real surveillance footage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the first-match lock is permanent: if a triplet is assigned to the wrong query in an early frame, SLBM will keep that wrong assignment alive for the whole visible tracklet, and a confidence-gated commitment or delayed locking would be a testable remedy.
  • Because Action Genome's pseudo-IDs are shared by training and evaluation, the tR@K gain partly measures consistency toward the approximation; a metric that also requires the tracked identity to match an independently verified ID would separate consistency from correctness.
  • The matching-level lock is independent of the detector's class vocabulary, so the same assignment mechanism plausibly transfers to open-vocabulary or language-prompted video scene understanding, with queries conditioned on text embeddings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TCDSG, an end-to-end transformer-based framework for dynamic scene graph generation and action tracklet construction. Two core mechanisms are introduced: a Sequence-Level Bipartite Matching (SLBM) objective that caches ground-truth triplet-to-query assignments and adds a penalty for identity switches, and Temporally Conditioned Decoder Queries (TCQ) that feed LSTM-summarized decoder outputs from the previous frame into the current frame's query embeddings. The authors evaluate on Action Genome, OpenPVSG, and a newly re-annotated MEVA subset, reporting large temporal Recall@K gains, notably tR@50 improving from 18.6% (DDS) to 39.1% on Action Genome. The paper also proposes new persistent object-ID annotations for a subset of MEVA.

Significance. If the results are reliable, the paper addresses a real limitation of frame-wise dynamic scene graph methods: tracklet fragmentation caused by query reassignment between frames. The SLBM idea is simple and plausible, and the ablations in Table 4 show a consistent trend that each proposed component contributes to temporal consistency. The paper is also honest in disclosing the pseudo-labeling strategy on Action Genome and the fact that MEVA results are on a newly curated subset with non-standard metrics. However, the headline evidence is weakened by the shared pseudo-label approximation between training and evaluation on Action Genome, the absence of the full TCDSG model on OpenPVSG, and the lack of any external baseline on MEVA. These issues are load-bearing because the paper claims state-of-the-art temporal consistency on three datasets.

major comments (4)
  1. [Sec. 4.1, Sec. 3.4, Sec. 3.6] The Action Genome ground-truth tracklet identities are pseudo-labeled by bounding-box IoU overlap (Sec. 4.1), and these same pseudo-IDs are used both as training targets for the SLBM lock (Eq. 3) and as ground truth for the tR@K metric (Sec. 3.6). This shared approximation can inflate temporal consistency: if the IoU chaining merges or swaps distinct same-class objects, a model that learns to reproduce the chaining heuristic will score highly on tR@K without achieving true identity-stable tracking. The manual verification mentioned in Sec. 4.1 is not quantified, and no independent labels or error bars are provided. I request a quantitative analysis of pseudo-label noise (e.g., agreement statistics on a manually labeled subset) or an evaluation on a dataset with true persistent IDs, such as the full TCDSG model on OpenPVSG.
  2. [Table 2, Sec. 4.2] The OpenPVSG results report only TCDSG*, which Table 1 defines as the variant with Temporal Hungarian Matching removed. The full TCDSG model is never evaluated on OpenPVSG, so the paper does not actually demonstrate the proposed method's temporal consistency gains on this dataset. The authors should report the full model's R@K and tR@K on OpenPVSG and, if possible, include external baselines that report the same tR@K metric.
  3. [Sec. 5.1, Table 3] The MEVA evaluation contains no external baseline; Table 3 compares only subsampling factors of the authors' own model on their re-annotated subset. As the paper acknowledges, prior MEVA work uses different metrics, but this means the claim of strong temporal consistency on MEVA is supported only by an internal analysis. A comparison with at least one existing method under a common metric, or a clear statement that the MEVA contribution is a dataset/annotation resource rather than a performance claim, is needed.
  4. [Abstract and Sec. 5.1] The abstract claims 'over 60% improvement in temporal recall@k on the Action Genome, OpenPVSG, and MEVA datasets,' whereas Sec. 5.1 reports '+110% rel.' for the Action Genome tR@50 improvement (18.6 to 39.1) and no comparable baseline improvement is shown for OpenPVSG or MEVA. These statements are inconsistent and the abstract overstates what the tables support. The authors should reconcile the numbers and restrict the improvement claim to the comparisons actually reported.
minor comments (5)
  1. [Eq. (1)] The description of the cross-attention operation f(a, b) is ambiguous: 'using a as input queries and b as query and value' is unclear about the roles of Q, K, and V, and b is described as an LSTM state rather than a sequence of keys/values. Please specify the exact tensor shapes and attention formulation.
  2. [Sec. 3.6] The tR@K metric is described only verbally as requiring 'sufficient temporal Intersection over Union with ground truth.' The definition should be formalized: how is temporal IoU computed, what threshold is used, and how are tracklet boundaries aligned?
  3. [Throughout] There are frequent typos and spacing errors, including 'MEV A' instead of 'MEVA', 'reducing improving prediction stability' in Sec. 1, and inconsistent capitalization in 'The Reference point loss'. A careful proofread is needed.
  4. [Tables 1 and 4] The TCDSG* rows are defined only in the caption of Table 1 and in the ablation text; it would help to state explicitly in each table caption that TCDSG* denotes removal of the temporal matching component.
  5. [Sec. 5.1] No error bars, standard deviations, or multiple-seed results are reported anywhere. Given the reliance on pseudo-labels and the small MEVA test subset, at least a variance estimate over seeds or annotation subsets would materially strengthen the temporal consistency claims.

Circularity Check

1 steps flagged · score 3.0 of 10

Headline tR@50 gain on Action Genome rests on IoU-pseudo-labeled object IDs shared by training and evaluation; OpenPVSG provides independent support.

  1. self definitional [Sec. 4.1 (Action Genome pseudo-labeling), applied in Sec. 3.4 (Eq. 3) and Sec. 3.6 (tR@K)]
    "To enable tracking, we implemented a pseudo-labeling strategy to assign consistent object identifiers across frames based on bounding box overlap. ... we cache each ground-truth triplet ⟨s, o, rs,o⟩ the first time it appears and bind it to the decoder query to which it was matched. ... tR@K requires predictions to have both spatial overlap (IoU ≥ 0.5) and sufficient temporal Intersection over Union with ground truth."

    On Action Genome, the same IoU-overlap pseudo-labeling generates both the SLBM training targets (the ground-truth triplets locked to queries in Eq. 3) and the tR@K evaluation ground truth (Sec. 3.6). The reported temporal consistency is therefore consistency with a label-generation heuristic that is itself a training input. A model that learns to reproduce the IoU-chaining rule can inflate tR@K even if it is not tracking physical identities through occlusion or re-entry. The paper's manual-verification caveat in Sec. 4.1 mitigates but does not remove the shared-approximation issue.

full rationale

TCDSG's core derivation—the SLBM temporal penalty (Eq. 3), the TCQ feedback loop (Eq. 1), and inference-time grouping by query index—is not circular by itself: the temporal penalty is a training objective and tR@K is a distinct evaluation function. The load-bearing concern is confined to Action Genome. Sec. 4.1 states that AG provides no persistent object IDs, so the authors assign training and evaluation identities by IoU pseudo-labeling. The same pseudo-identities are then used as the ground-truth triplets locked to queries by SLBM (Sec. 3.4) and as the ground truth for tR@K (Sec. 3.6). Thus the headline 18.6→39.1 tR@50 improvement measures consistency with a heuristic that is itself a training input; identity merges or swaps under occlusion or re-entry could inflate the apparent gain. The manual-verification note is the only mitigation and provides no independent labels or error bars. This is partial self-reference, not a full collapse: DDS is a same-group citation but is a published, externally peer-reviewed baseline, and the OpenPVSG tR@K comparisons (Table 2) plus frame-level R@K against many external methods provide independent content. No uniqueness theorem, ansatz-via-citation, or renaming-of-known-result pattern is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest mainly on pseudo-labeled identities for Action Genome, on the assumption that stable query indices imply stable tracklet identities, and on hand-chosen matching and loss weights whose values are not reported. No new physical or model entities are introduced; the re-annotated MEVA subset is a new annotation artifact, not an entity.

free parameters (3)
  • lambda_co (identity-switch penalty in SLBM) = not reported
    Eq 3; large positive scalar chosen by hand, no value given, directly controls the strength of temporal binding that produces the reported tR@K gains.
  • loss weights lambda_spat, lambda_const, lambda_ref, lambda_label = not reported
    Eq 4; balancing weights for spatial, consistency, reference-point, and label losses; no values or search procedure reported.
  • LSTM hidden size / query dimension = not reported
    TCQ uses an LSTM over decoder outputs; architecture size not specified, affecting temporal context capacity.
assumptions (3)
  • domain assumption IoU overlap between consecutive frames identifies the same object instance in Action Genome
    Sec 4.1; Action Genome lacks persistent IDs, so pseudo-labels are generated from bounding-box overlap; re-entering objects can produce wrong identity links, and all AG tR@K numbers inherit this assumption.
  • domain assumption Query index identity implies tracklet identity at inference
    Sec 4.4; tracklets are formed by grouping predictions with the same query index and triplet label; this holds only if the learned query assignment is stable, which is exactly what SLBM enforces but not guaranteed for occlusions or re-appearances.
  • standard math DETR-style set prediction and deformable attention as base model
    Sec 3.2; architecture builds on Deformable DETR and DDS, adopting their assumptions about learnable queries and bipartite matching.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Temporally Consistent Dynamic Scene Graphs: An End-to-End Approach for Action Tracklet Generation." pith.science (2026). https://pith.science/paper/KQZAEKT3

@misc{pith2026241202808,
  author       = {Pith},
  title        = {Pith review of: Temporally Consistent Dynamic Scene Graphs: An End-to-End Approach for Action Tracklet Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KQZAEKT3}},
  note         = {Machine review of arXiv:2412.02808}
}
read the original abstract

Understanding video content is pivotal for advancing real-world applications like activity recognition, autonomous systems, and human-computer interaction. While scene graphs are adept at capturing spatial relationships between objects in individual frames, extending these representations to capture dynamic interactions across video sequences remains a significant challenge. To address this, we present TCDSG, Temporally Consistent Dynamic Scene Graphs, an innovative end-to-end framework that detects, tracks, and links subject-object relationships across time, generating action tracklets, temporally consistent sequences of entities and their interactions. Our approach leverages a novel bipartite matching mechanism, enhanced by adaptive decoder queries and feedback loops, ensuring temporal coherence and robust tracking over extended sequences. This method not only establishes a new benchmark by achieving over 60% improvement in temporal recall@k on the Action Genome, OpenPVSG, and MEVA datasets but also pioneers the augmentation of MEVA with persistent object ID annotations for comprehensive tracklet generation. By seamlessly integrating spatial and temporal dynamics, our work sets a new standard in multi-frame video analysis, opening new avenues for high-impact applications in surveillance, autonomous navigation, and beyond.

Figures

Figures reproduced from arXiv: 2412.02808 by the authors.

Figure 1
Figure 1. An example of the desired output for a video sequence. At the top, we show a few selected frames and their corresponding [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the TCDSG architecture. A shared CNN backbone extracts features that are fed into two parallel transformer [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparative visualization of action tracklet continuity between TCDSG (our method) and TCDSG*, which represents our module [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 55 canonical work pages

  1. [1]

    Vision-based navigation and guid- ance for agricultural autonomous vehicles and robots: A review

    Yuhao Bai, Baohua Zhang, Naimin Xu, Jun Zhou, Jiayou Shi, and Zhihua Diao. Vision-based navigation and guid- ance for agricultural autonomous vehicles and robots: A review. Computers and Electronics in Agriculture , 205: 107584, 2023. 1

    work page 2023

  2. [2]

    Simple online and realtime tracking

    Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP) , pages 3464–3468. IEEE, 2016. 3

    work page 2016

  3. [3]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 3, 4, 5, 6

    work page 2020

  4. [4]

    Counterfactual critic multi- agent training for scene graph generation

    Long Chen, Hanwang Zhang, Jun Xiao, Xiangnan He, Shil- iang Pu, and Shih-Fu Chang. Counterfactual critic multi- agent training for scene graph generation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 4613–4623, 2019. 2

    work page 2019

  5. [5]

    Toward general cross-modal signal reconstruction for robotic teleop- eration

    Yanan Chen, Ang Li, Dan Wu, and Liang Zhou. Toward general cross-modal signal reconstruction for robotic teleop- eration. IEEE Transactions on Multimedia, 2023. 1

    work page 2023

  6. [6]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 7

    work page 2022

  7. [7]

    Recovering the unbiased scene graphs from the biased ones

    Meng-Jiun Chiou, Henghui Ding, Hanshu Yan, Changhu Wang, Roger Zimmermann, and Jiashi Feng. Recovering the unbiased scene graphs from the biased ones. In Proceedings of the 29th ACM International Conference on Multimedia , pages 1581–1590, 2021. 2

    2021

  8. [8]

    Nodis: Neural ordinary differential scene understanding

    Yuren Cong, Hanno Ackermann, Wentong Liao, Michael Ying Yang, and Bodo Rosenhahn. Nodis: Neural ordinary differential scene understanding. In European Conference on Computer Vision , pages 636–653. Springer,

    work page

Show all 65 references
  1. [9]

    Spatial-temporal trans- former for dynamic scene graph generation

    Yuren Cong, Wentong Liao, Hanno Ackermann, Bodo Rosenhahn, and Michael Ying Yang. Spatial-temporal trans- former for dynamic scene graph generation. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 16372–16382, 2021. 2

    2021

  2. [10]

    Meva: A large-scale multiview, multimodal video dataset for activity detection

    Kellie Corona, Katie Osterdahl, Roderic Collins, and An- thony Hoogs. Meva: A large-scale multiview, multimodal video dataset for activity detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1060–1068, 2021. 6

    2021

  3. [11]

    Scaling egocentric vision: The epic- kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Da- vide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Scaling egocentric vision: The epic- kitchens dataset. In European Conference on Computer Vi...

    2018

  4. [12]

    Gabriellav2: Towards better generalization in surveillance videos for ac- tion detection

    Ishan Dave, Zacchaeus Scheffer, Akash Kumar, Sarah Shi- raz, Yogesh Singh Rawat, and Mubarak Shah. Gabriellav2: Towards better generalization in surveillance videos for ac- tion detection. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision, p...

  5. [13]

    Exploiting long-term de- pendencies for generating dynamic scene graphs

    Shengyu Feng, Hesham Mostafa, Marcel Nassar, Somdeb Majumdar, and Subarna Tripathi. Exploiting long-term de- pendencies for generating dynamic scene graphs. In Pro- ceedings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pages 5130–5139, 2023. 6

    2023

  6. [14]

    Drg: Dual relation graph for human-object interaction detection

    Chen Gao, Jiarui Xu, Yuliang Zou, and Jia-Bin Huang. Drg: Dual relation graph for human-object interaction detection. In Proc. European Conference on Computer Vision (ECCV),

  7. [15]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InPro- ceedings of the IEEE/CVF Conference on Computer Vision...

    2022

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 3, 6

    2016

  9. [17]

    Detecting human-object interaction via fab- ricated compositional learning

    Zhi Hou, Baosheng Yu, Yu Qiao, Xiaojiang Peng, and Dacheng Tao. Detecting human-object interaction via fab- ricated compositional learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14646–14655, 2021. 1

    2021

  10. [18]

    Discovering human-object interaction concepts via self-compositional learning

    Zhi Hou, Baosheng Yu, and Dacheng Tao. Discovering human-object interaction concepts via self-compositional learning. In ECCV, 2022. 1

    2022

  11. [19]

    Dds: Decoupled dynamic scene-graph generation network

    ASM Iftekhar, Raphael Ruschel, Satish Kumar, Suya You, and BS Manjunath. Dds: Decoupled dynamic scene-graph generation network. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , pages 9670–

    2025

  12. [20]

    Decoupled weight decay regularization

    Loshchilov Ilya, Hutter Frank, et al. Decoupled weight decay regularization. Proceedings of ICLR, 2019. 6

    2019

  13. [21]

    Action genome: Actions as compositions of spatio- temporal scene graphs

    Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. Action genome: Actions as compositions of spatio- temporal scene graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10236–10247, 2020. 5

    2020

  14. [22]

    Detect- ing human-object relationships in videos

    Jingwei Ji, Rishi Desai, and Juan Carlos Niebles. Detect- ing human-object relationships in videos. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 8106–8116, 2021. 2

    2021

  15. [23]

    Image retrieval using scene graphs

    Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs. InProceedings of the IEEE con- ference on computer vision and pattern recognition , pages 3668–3678, 2015. 2

    2015

  16. [24]

    Iterative scene graph generation

    Siddhesh Khandelwal and Leonid Sigal. Iterative scene graph generation. Advances in Neural Information Process- ing Systems, 35:24295–24308, 2022. 6

    2022

  17. [25]

    Learning by tracking: Siamese cnn for robust tar- get association

    Laura Leal-Taix ´e, Cristian Canton-Ferrer, and Konrad Schindler. Learning by tracking: Siamese cnn for robust tar- get association. In Proceedings of the IEEE conference on 9 computer vision and pattern recognition workshops , pages 33–40, 2016. 3

    2016

  18. [26]

    Video k-net: A simple, strong, and unified baseline for video segmentation

    Xiangtai Li, Wenwei Zhang, Jiangmiao Pang, Kai Chen, Guangliang Cheng, Yunhai Tong, and Chen Change Loy. Video k-net: A simple, strong, and unified baseline for video segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1884...

    2022

  19. [27]

    Dynamic scene graph generation via anticipatory pre-training

    Yiming Li, Xiaoshan Yang, and Changsheng Xu. Dynamic scene graph generation via anticipatory pre-training. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13874–13883, 2022. 6

    2022

  20. [28]

    Transferable interactiveness knowledge for human-object interaction de- tection

    Yong-Lu Li, Siyuan Zhou, Xijie Huang, Liang Xu, Ze Ma, Hao-Shu Fang, Yanfeng Wang, and Cewu Lu. Transferable interactiveness knowledge for human-object interaction de- tection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3585–3594, 2019. 2

    2019

  21. [29]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014. 6

    2014

  22. [30]

    Td2-net: Toward denoising and debiasing for dynamic scene graph generation

    Xin Lin, Chong Shi, Yibing Zhan, Zuopeng Yang, Yaqi Wu, and Dacheng Tao. Td2-net: Toward denoising and debiasing for dynamic scene graph generation. ArXiv, abs/2401.12479,

    arXiv

  23. [31]

    Xiaochen Liu, Pradipta Ghosh, Oytun Ulutan, B. S. Manju- nath, Kevin Chan, and Ramesh Govindan. Caesar: cross- camera complex activity recognition. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems , page 232–244, New York, NY , USA, 2019. Association ...

    2019

  24. [32]

    Ampli- fying key cues for human-object-interaction detection

    Yang Liu, Qingchao Chen, and Andrew Zisserman. Ampli- fying key cues for human-object-interaction detection. In European Conference on Computer Vision, pages 248–265. Springer, 2020. 2

    2020

  25. [33]

    Prior knowledge-driven dy- namic scene graph generation with causal inference

    Jiale Lu, Lianggangxu Chen, Youqi Song, Shaohui Lin, Changbo Wang, and Gaoqi He. Prior knowledge-driven dy- namic scene graph generation with causal inference. In Pro- ceedings of the 31st ACM International Conference on Mul- timedia, page 4877–4885, New York, NY , USA, 2023. ...

    2023

  26. [34]

    Context-aware scene graph generation with seq2seq transformers

    Yichao Lu, Himanshu Rai, Jason Chang, Boris Knyazev, Guangwei Yu, Shashank Shekhar, Graham W Taylor, and Maksims V olkovs. Context-aware scene graph generation with seq2seq transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15931– 1...

    2021

  27. [35]

    Trackformer: Multi-object track- ing with transformers

    Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Trackformer: Multi-object track- ing with transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8844–8854, 2022. 3

    2022

  28. [36]

    Roy- Chowdhury

    Sayak Nag, Kyle Min, Subarna Tripathi, and Amit K. Roy- Chowdhury. Unbiased scene graph generation in videos. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 22803–22813, 2023. 1, 2, 6

    2023

  29. [37]

    Motion-aware contrastive learning for temporal panoptic scene graph gen- eration

    Thong Thanh Nguyen, Xiaobao Wu, Yi Bin, Cong-Duy T Nguyen, See-Kiong Ng, and Anh Tuan Luu. Motion-aware contrastive learning for temporal panoptic scene graph gen- eration. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6218–6226, 2025. 7

    2025

  30. [38]

    Computer vision-based hand gesture recognition for human-robot in- teraction: a review

    Jing Qi, Li Ma, Zhenchao Cui, and Yushu Yu. Computer vision-based hand gesture recognition for human-robot in- teraction: a review. Complex & Intelligent Systems , 10(1): 1581–1606, 2024. 1

    2024

  31. [39]

    Raphael Ruschel, A. S. M. Iftekhar, B. S. Manjunath, and Suya You. Bload: Enhancing neural network train- ing with efficient sequential data handling. arXiv preprint arXiv:2310.10879, 2023. 6

    2023 arXiv

  32. [40]

    Deep network flow for multi-object track- ing

    Samuel Schulter, Paul Vernaza, Wongun Choi, and Manmo- han Chandraker. Deep network flow for multi-object track- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6951–6960, 2017. 3

    2017

  33. [41]

    Annotating objects and relations in user- generated videos

    Xindi Shang, Donglin Di, Junbin Xiao, Yu Cao, Xun Yang, and Tat-Seng Chua. Annotating objects and relations in user- generated videos. In Proceedings of the 2019 on Interna- tional Conference on Multimedia Retrieval, pages 279–287. ACM, 2019. 5

    2019

  34. [42]

    Eye tracking assisted extraction of attentionally important objects from videos

    Karthikeyan Shanmuga Vadivel, Thuyen Ngo, Miguel Eck- stein, and BS Manjunath. Eye tracking assisted extraction of attentionally important objects from videos. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3241–3250, 2015. 4

    2015

  35. [43]

    A simple baseline for weakly-supervised scene graph gener- ation

    Jing Shi, Yiwu Zhong, Ning Xu, Yin Li, and Chenliang Xu. A simple baseline for weakly-supervised scene graph gener- ation. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 16393–16402, 2021. 2

    2021

  36. [44]

    Hollywood in homes: Crowdsourcing data collection for activity under- standing

    Gunnar A Sigurdsson, G ¨ul Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity under- standing. In European Conference on Computer Vision , pages 510–526. Springer, 2016. 5

    2016

  37. [45]

    Transtrack: Multiple object tracking with transformer

    Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zehuan Yuan, Changhu Wang, and Ping Luo. Transtrack: Multiple object tracking with transformer. arXiv preprint arXiv:2012.15460, 2020. 3

    2012 arXiv

  38. [46]

    Learning to compose dynamic tree structures for visual contexts

    Kaihua Tang, Hanwang Zhang, Baoyuan Wu, Wenhan Luo, and Wei Liu. Learning to compose dynamic tree structures for visual contexts. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6619–6628, 2019. 2

    2019

  39. [47]

    Tar- get adaptive context aggregation for video scene graph gen- eration

    Yao Teng, Limin Wang, Zhifeng Li, and Gangshan Wu. Tar- get adaptive context aggregation for video scene graph gen- eration. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 13688–13697, 2021. 2

    2021

  40. [48]

    Vsgnet: Spatial attention network for detecting human ob- ject interactions using graph convolutions

    Oytun Ulutan, ASM Iftekhar, and Bangalore S Manjunath. Vsgnet: Spatial attention network for detecting human ob- ject interactions using graph convolutions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13617–13626, 2020. 2

    2020

  41. [49]

    Pose-aware multi-level feature network for human ob- ject interaction detection

    Bo Wan, Desen Zhou, Yongfei Liu, Rongjie Li, and Xuming He. Pose-aware multi-level feature network for human ob- ject interaction detection. In Proceedings of the IEEE Inter- 10 national Conference on Computer Vision, pages 9469–9478,

  42. [50]

    Oed: Towards one-stage end-to-end dynamic scene graph genera- tion

    Guan Wang, Zhimin Li, Qingchao Chen, and Yang Liu. Oed: Towards one-stage end-to-end dynamic scene graph genera- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 27938–27947,

  43. [51]

    Learning transferable human-object interaction detector with natural language su- pervision

    Suchen Wang, Yueqi Duan, Henghui Ding, Yap-Peng Tan, Kim-Hui Yap, and Junsong Yuan. Learning transferable human-object interaction detector with natural language su- pervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 939–948,

  44. [52]

    Dynamic scene graph generation via temporal prior inference

    Shuang Wang, Lianli Gao, Xinyu Lyu, Yuyu Guo, Pengpeng Zeng, and Jingkuan Song. Dynamic scene graph generation via temporal prior inference. InProceedings of the 30th ACM International Conference on Multimedia , page 5793–5801, New York, NY , USA, 2022. Association for Computi...

    2022

  45. [53]

    Deep contextual attention for human- object interaction detection

    Tiancai Wang, Rao Muhammad Anwer, Muhammad Haris Khan, Fahad Shahbaz Khan, Yanwei Pang, Ling Shao, and Jorma Laaksonen. Deep contextual attention for human- object interaction detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 5694– 5702, 2019. 2

    2019

  46. [54]

    Exploring context and visual pattern of relationship for scene graph generation

    Wenbin Wang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Exploring context and visual pattern of relationship for scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8188–8197, 2019. 2

    2019

  47. [55]

    Topic scene graph generation by attention distillation from caption

    Wenbin Wang, Ruiping Wang, and Xilin Chen. Topic scene graph generation by attention distillation from caption. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15900–15910, 2021. 2

    2021

  48. [56]

    Do different track- ing tasks require different appearance models? Advances in Neural Information Processing Systems, 34:726–738, 2021

    Zhongdao Wang, Hengshuang Zhao, Ya-Li Li, Shengjin Wang, Philip Torr, and Luca Bertinetto. Do different track- ing tasks require different appearance models? Advances in Neural Information Processing Systems, 34:726–738, 2021. 7

    2021

  49. [57]

    An introduction to the kalman filter

    G Welch. An introduction to the kalman filter. 1995. 3

    1995

  50. [58]

    Simple online and realtime tracking with a deep association metric

    Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP), pages 3645–3649. IEEE, 2017. 3

    2017

  51. [59]

    Panoptic video scene graph generation

    Jingkang Yang, Wenxuan Peng, Xiangtai Li, Zujin Guo, Liangyu Chen, Bo Li, Zheng Ma, Kaiyang Zhou, Wayne Zhang, Chen Change Loy, and Ziwei Liu. Panoptic video scene graph generation. In CVPR, 2023. 1, 2, 5, 7

    2023

  52. [60]

    Motr: End-to-end multiple- object tracking with transformer

    Fangao Zeng, Bin Dong, Yuang Zhang, Tiancai Wang, Xi- angyu Zhang, and Yichen Wei. Motr: End-to-end multiple- object tracking with transformer. In European Conference on Computer Vision, pages 659–675. Springer, 2022. 3

    2022

  53. [61]

    Graphical contrastive losses for scene graph parsing

    Ji Zhang, Kevin J Shih, Ahmed Elgammal, Andrew Tao, and Bryan Catanzaro. Graphical contrastive losses for scene graph parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11535– 11543, 2019. 2

    2019

  54. [62]

    End-to-end video scene graph gener- ation with temporal propagation transformer

    Yong Zhang, Yingwei Pan, Ting Yao, Rui Huang, Tao Mei, and Chang-Wen Chen. End-to-end video scene graph gener- ation with temporal propagation transformer. IEEE Transac- tions on Multimedia, 26:1613–1625, 2023. 6

    2023

  55. [63]

    Object track- ing using sift features and mean shift

    Huiyu Zhou, Yuan Yuan, and Chunmei Shi. Object track- ing using sift features and mean shift. Computer vision and image understanding, 113(3):345–352, 2009. 3

    2009

  56. [64]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 3 11

    2010 arXiv

  57. [9680]

    1, 2, 3, 6

    IEEE, 2025. 1, 2, 3, 6

    2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.