Pith. sign in

REVIEW 3 major objections 6 minor 56 references

SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SceneLLM claims that an LLM can act as a scene analyzer for dynamic scene graph generation by converting video frames into discrete 'scene token' sequences and decoding the LLM's hidden reasoning into semantic triplets, achieving…

desk verdict SceneLLM reuses a known VQ-OT-LoRA recipe for a new task, but the central LLM-input pathway is internally inconsistent as written (512-d codebook vs 5120-d LLaMA-13B), so the SOTA claims cannot be checked. read the letter →

arxiv 2412.11026 v2 pith:PZQIJYAB submitted 2024-12-15 cs.CV cs.AI

Hang Zhang , Zhuoling Li , Jun Liu This is my paper · ORCID
classification cs.CVcs.AI
keywords dynamicscenegraphgenerationlargelanguagemodelreasoningvideo-to-languagemappingimplicitlinguisticsignalVQ-VAEdiscretizationoptimaltransportcodebookLoRAfine-tuningActionGenomebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a large language model can serve as a scene analyzer for dynamic scene graph generation, not by asking the LLM to output text, but by turning video frames into a discrete token sequence that the LLM reasons over internally. The payoff would be a single framework that uses the LLM's pretrained world knowledge to infer subject-predicate-object triplets in video, improving recall over prior spatio-temporal models. On the Action Genome benchmark the authors report consistent state-of-the-art results across predicate classification, scene graph classification, and scene graph detection under both constraint settings. The ablations attribute the gain to the LLM itself, to discrete feature quantization, to the optimal-transport temporal codebook, and to LoRA fine-tuning. If the central claim is right, the next generation of video scene understanding can be built on implicit language reasoning rather than bespoke spatio-temporal architectures.

What carries the argument

The load-bearing mechanism is the Video-to-Language (V2L) mapping that turns continuous video features into discrete, language-like tokens. It has three parts: a VQ-VAE codebook whose vectors are meant to match the LLM's word-token dimension; a Spatial Information Aggregation (SIA) scheme, inspired by how Chinese characters combine radicals, that uses MLP position embedding, hierarchical clustering, and a GCN to produce one frame-level token per frame; and an optimal-transport update that merges frame-level tokens into an updated codebook with temporal semantics. These tokens, plus a short text prompt, are passed to a LoRA-fine-tuned LLaMA-13B, and the LLM's hidden states rather than its text output are decoded by a transformer SGG predictor. The OT codebook update and LoRA tuning are what let the pretrained LLM treat visual tokens as a 'scene sentence'.

What would settle it

Run the published SceneLLM code on a single video and inspect the tensor shapes at the boundary between the V2L module and LLaMA-13B: if the codebook vectors enter the model without passing through a projection that changes their dimension from 512 to 5120, the forward pass will fail and the claimed architecture is not instantiated. A second check is to replace LLaMA-13B with a randomly initialized transformer of the same size and measure SGCLS recall: if recall does not drop sharply, the LLM's pretrained knowledge is not doing the causal work attributed to it.

Watch

Extended reading notes

Core claim

The central discovery claimed is that LLMs can reason about dynamic visual scenes when the video is converted into an implicit linguistic signal. SceneLLM's Video-to-Language mapping first quantizes object ROI features into codebook vectors with a VQ-VAE, aggregates them via a graph convolution based on hierarchical spatial clustering, and then uses optimal transport to merge frame-level tokens into a temporally informed codebook. The resulting token sequence is fed into a frozen LLaMA-13B fine-tuned with LoRA, and the LLM's final hidden features are decoded by a transformer-based SGG predictor into semantic triplets. The paper reports state-of-the-art recall on Action Genome and concludes that the LLM's implicit knowledge is what enables this, with ablations showing that removing the LLM, replacing it with T5, removing discretization, removing the OT scheme, or removing LoRA all degrade SGCLS performance.

Load-bearing premise

The pipeline depends on feeding 512-dimensional codebook vectors into a model whose word-token embeddings are 5120-dimensional, with no described projection to bridge them; if that mismatch is real, the central scene-token-to-LLM pathway cannot run as written.

Editorial extensions

If this is right

  • Dynamic scene graph generation can be recast as an implicit language-reasoning problem, so advances in LLM architecture and pretraining transfer directly to video scene understanding.
  • Discrete visual tokens that mimic word tokens are a viable interface between vision encoders and LLMs, suggesting that explicit text or caption supervision is not required for LLM-based scene reasoning.
  • The optimal-transport codebook update provides a concrete way to compress frame-level spatial tokens into a temporally coherent sequence, which could replace hand-designed temporal modules in other video tasks.
  • Because SceneLLM reports state-of-the-art results on all three Action Genome tasks and settings, it sets a new numerical baseline that future dynamic SGG methods will need to beat on recall@K.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the same video-to-language interface could be applied to other video-understanding tasks, such as action recognition, video captioning, or open-vocabulary predicate prediction, by keeping the frozen LLM and swapping only the decoding head.
  • Our inference: the Chinese-character-inspired SIA gives each frame a compositional, graph-structured token, so probing the LLM's hidden states might reveal which object 'radicals' drive each predicted predicate, making the reasoning more inspectable than monolithic video features.
  • Our caution: because the stated codebook dimension (512) does not match LLaMA-13B's token-embedding dimension (5120), reproducing the method as written likely requires an unmentioned projection layer, and readers should check the released code before trusting the end-to-end pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. SceneLLM is a dynamic scene graph generation framework that first quantizes object-level ROI features with a VQ-VAE, then aggregates them into frame-level tokens via a Spatial Information Aggregation (SIA) scheme inspired by Chinese character composition, and finally uses Optimal Transport (OT) to form a video-level implicit linguistic signal. This signal is fed into a frozen LLaMA-13B fine-tuned with LoRA, and the LLM's final hidden features are decoded by a transformer-based SGG predictor into semantic triplets. The paper reports state-of-the-art results on Action Genome for PREDCLS, SGCLS, and SGDET under both constraint settings, along with ablations showing the contribution of the LLM, feature discretization, OT, and LoRA.

Significance. If the reported results were reproducible, the paper would be a useful demonstration that LLMs can serve as implicit scene analyzers for dynamic scene graph generation, an interesting and timely direction. The entropy derivation in Eq. (4) appears internally consistent, and the ablations (w/o LLM, w/ T5, w/o OT, w/o LoRA) are genuine comparative tests, not fitted quantities relabeled as predictions. However, the central empirical claims are undermined by a dimensional inconsistency in the LLM input pathway and by a state-of-the-art claim that Table 1 contradicts. The lack of code and variance reporting further limits verification. These issues must be resolved before the contribution can be accepted.

major comments (3)
  1. [Section 3.2 and Section 4.2] The manuscript claims in Section 3.2 that the VQ-VAE codebook dimension l is the same as the LLM's word-token dimension, but Section 4.2 sets l = 512 and uses frozen LLaMA-13B, whose token-embedding dimension is 5120. No projection or alignment layer is described between the codebook output and the LLM input, and Eq. (6) writes LLM(S_LLM) as if the scene tokens were directly consumable. Consequently, the central V2L-to-LLM pathway cannot be instantiated as described. The authors must either specify the projection/alignment mechanism, correct the dimension claim, or provide the actual implementation details that reconcile this inconsistency.
  2. [Section 4.3 and Table 1] The text claims that SceneLLM 'consistently achieves state-of-the-art results across all tasks and metrics.' This is contradicted by Table 1 in the With Constraint setting: for SGDET R@10, DDS achieves 36.2 while SceneLLM achieves 34.9. Thus the state-of-the-art claim is not supported for at least one metric. The authors should correct the claim and discuss the comparison with DDS, which is listed in the table but omitted from the comparison text.
  3. [Section 4.2 and Tables 1-3] No code is provided, and all results in Tables 1, 2, and 3 are reported without variance or the number of seeds. As an empirical systems paper whose central claim is that a specific novel architecture achieves state-of-the-art performance, the absence of reproducibility details is a significant omission. At a minimum, the authors should report mean and standard deviation over multiple runs and make the code available.
minor comments (6)
  1. [Abstract] The abstract contains a typo: 'UA Vs' should be 'UAVs'.
  2. [Algorithm 1] Algorithm 1 has a typo ('Flag=Flase') and the termination criterion is not guaranteed to halt; specify a maximum iteration or another early-stopping condition.
  3. [Eq. (2)] Eq. (2) does not specify the output dimension of the MLP that embeds the position information; please clarify how this dimension interacts with the codebook dimension.
  4. [Section 3.2 and Section 4.2] The phrase in Section 3.2 that l is the same as the LLM word-token dimension conflicts with the implementation detail in Section 4.2; this must be reconciled even after a projection layer is added.
  5. [Figures 2-4] Figures 2-4 would benefit from more detailed labels and captions; for example, the OT block in Fig. 4 is not described in the caption.
  6. [Contribution claim] The claim of being 'the first to consider LLM as a scene analyzer through implicit language reasoning' should be supported by a more thorough discussion of recent LLM-based scene understanding systems.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SceneLLM's claims are empirical and its ablations are genuine comparative tests.

full rationale

SceneLLM is an empirical systems paper whose central claim, state-of-the-art dynamic scene graph generation on Action Genome, is established by benchmark comparisons in Tables 1 and 2 and by ablations in Tables 3 through 6. These ablations remove or replace components (w/o LLM, w/ T5, w/o discretization, w/o OT, TC, Clustering, w/o LoRA), so they compare alternative architectures rather than relabeling fitted quantities as predictions. The grid-searched weighting factor alpha and LoRA fine-tuning are training choices evaluated against held-out metrics, not fitted parameters disguised as predictions. The paper cites its own prior work [23] and the same authors' related work [42] as motivation or as implementation precedents, but neither citation is invoked as a uniqueness theorem or as the sole justification for the main result; external references and the paper's own experiments carry the argument. The dimensional mismatch between the VQ-VAE latent dimension (512) and LLaMA-13B's token embedding dimension noted by reviewers is a reproducibility and correctness concern, not a circularity step, because no equation in the paper defines the target result in terms of the input. Accordingly, the derivation chain is not circular and no specific reduction of a claimed prediction to an input by construction can be exhibited.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The framework is an empirical model stack; it introduces no derivation from first principles. Every component is learned or tuned on Action Genome, and several design choices (codebook size, dimension, loss weights, LoRA configuration) are free parameters not determined by theory. The central premise that discrete visual tokens can be treated as language tokens for LLaMA is an unverified domain assumption, especially given the stated dimension mismatch. No genuinely new physical or conceptual entities are introduced; 'scene tokens' and 'implicit linguistic signal' are learned representations, not independent postulates.

free parameters (7)
  • Codebook size m = 512
    Number of discrete tokens in VQ-VAE codebook, chosen in Section 4.2 and central to the V2L mapping.
  • Latent feature dimension l = 512
    Set in Section 4.2; claimed in Section 3.2 to match the LLM word-token dimension, but LLaMA-13B uses a larger dimension and no projection is described.
  • Commitment loss weight lambda = 0.02
    VQ-VAE training hyperparameter given in Section 4.2.
  • SGG loss weight alpha = 0.5
    Grid-searched in Section 4.5 (Figs. 6 and 7) and set to 0.5.
  • OT codebook increment Delta_s = not specified
    The paper says a fixed size increment is used (Section 3.2, Algorithm 1) but does not give its value.
  • LoRA rank and configuration = not specified
    LoRA is mentioned but rank, alpha, and target modules are not reported in Section 4.2.
  • Frame count T and sampling = not specified
    The method processes T frames per video, but the paper does not state how frames are sampled or how T is chosen.
assumptions (5)
  • domain assumption Object detector outputs (bounding boxes, categories, ROI features) are accurate enough to support scene graph training and evaluation.
    The entire pipeline starts from detector outputs; no detector error analysis is provided.
  • domain assumption Discrete codebook vectors can serve as LLM-compatible token embeddings after LoRA fine-tuning.
    This is the core V2L premise; Section 3.2 assumes dimensional compatibility with LLM word tokens, which is contradicted by Section 4.2.
  • domain assumption The OT codebook update produces a temporally meaningful implicit linguistic signal.
    Section 3.2 relies on Sinkhorn optimization over codebook units to encode temporal consistency; the paper only validates this via one ablation.
  • domain assumption Recall@K on Action Genome is a sufficient proxy for dynamic scene graph quality.
    All conclusions rely on this benchmark metric; no user study or downstream task evaluation is included.
  • domain assumption LLM hidden features can be decoded by the STTran-based predictor into semantic triplets.
    Equation (6) assumes the transformer SGG predictor can map F_implicit to scene graph Y; no alignment or projection is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation." pith.science (2026). https://pith.science/paper/PZQIJYAB

@misc{pith2026241211026,
  author       = {Pith},
  title        = {Pith review of: SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZQIJYAB}},
  note         = {Machine review of arXiv:2412.11026}
}
read the original abstract

Dynamic scenes contain intricate spatio-temporal information, crucial for mobile robots, UAVs, and autonomous driving systems to make informed decisions. Parsing these scenes into semantic triplets <Subject-Predicate-Object> for accurate Scene Graph Generation (SGG) is highly challenging due to the fluctuating spatio-temporal complexity. Inspired by the reasoning capabilities of Large Language Models (LLMs), we propose SceneLLM, a novel framework that leverages LLMs as powerful scene analyzers for dynamic SGG. Our framework introduces a Video-to-Language (V2L) mapping module that transforms video frames into linguistic signals (scene tokens), making the input more comprehensible for LLMs. To better encode spatial information, we devise a Spatial Information Aggregation (SIA) scheme, inspired by the structure of Chinese characters, which encodes spatial data into tokens. Using Optimal Transport (OT), we generate an implicit language signal from the frame-level token sequence that captures the video's spatio-temporal information. To further improve the LLM's ability to process this implicit linguistic input, we apply Low-Rank Adaptation (LoRA) to fine-tune the model. Finally, we use a transformer-based SGG predictor to decode the LLM's reasoning and predict semantic triplets. Our method achieves state-of-the-art results on the Action Genome (AG) benchmark, and extensive experiments show the effectiveness of SceneLLM in understanding and generating accurate dynamic scene graphs.

Figures

Figures reproduced from arXiv: 2412.11026 by the authors.

Figure 1
Figure 1. Overview of our proposed SceneLLM framework. In our framework, given an input video signal, we [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Video-to-Language (V2L) Mapping Process: (a) Features of the regions of interest (ROIs) in video [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Spatial Information Aggregation for Frame-level Token Generation: (a) Illustration of Spatial Rep [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Update for an optimal codebook C + via an optimal transport scheme so that dynamic information can be embedded into implicit linguistic signal. contain different semantic information, our first step is to find out which units should be gathered to make the units in the…
Figure 5
Figure 5. Figure 5: Visualization results of our method. All results are given under the SGDET setup. (Zoom in for the [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Impact of weight factor α (With Constraint) [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Impact of weight factor α (Without Constraint) [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 44 canonical work pages

  1. [1]

    D. Xu, Y . Zhu, C. Choy, L. Fei-Fei, Scene graph generation by iterative message passing, in: Computer Vision and Pattern Recognition (CVPR), 2017. 22

    work page 2017

  2. [2]

    J. Wang, J. Huang, C. Zhang, Z. Deng, Cross-modality time-variant relation learn- ing for generating dynamic scene graphs, in: Proceedings of the IEEE Interna- tional Conference on Robotics and Automation (ICRA), 2023

    work page 2023

  3. [3]

    T. Pu, T. Chen, H. Wu, Y . Lu, L. Lin, Spatial-temporal knowledge-embedded transformer for video scene graph generation, IEEE Transactions on Image Pro- cessing 33 (2024) 556–568. doi:10.1109/TIP.2023.3345652

    arXiv 2024

  4. [4]

    Amiri, K

    S. Amiri, K. Chandan, S. Zhang, Reasoning with scene graphs for robot planning under partial observability, IEEE Robotics and Automation Letters 7 (2) (2022) 5560–5567

    work page 2022

  5. [5]

    Z. Jiao, Y . Niu, Z. Zhang, S.-C. Zhu, Y . Zhu, H. Liu, Sequential manipulation planning on scene graph, in: 2022 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), IEEE, 2022, pp. 8203–8210

    work page 2022

  6. [6]

    Y . Teng, L. Wang, Z. Li, G. Wu, Target adaptive context aggregation for video scene graph generation, in: Proceedings of the IEEE /CVF International Confer- ence on Computer Vision, 2021, pp. 13688–13697

    work page 2021

  7. [7]

    X. Lin, C. Ding, J. Zeng, D. Tao, Gps-net: Graph property sensing network for scene graph generation, in: Proceedings of the IEEE /CVF Conference on Com- puter Vision and Pattern Recognition, 2020, pp. 3746–3753

    work page 2020

  8. [8]

    Y . Cong, W. Liao, H. Ackermann, B. Rosenhahn, M. Y . Yang, Spatial- temporal transformer for dynamic scene graph generation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 16372–16382

    work page 2021

Show all 56 references
  1. [9]

    Y . Li, X. Yang, C. Xu, Dynamic scene graph generation via anticipatory pre- training, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition, 2022, pp. 13874–13883

    2022

  2. [10]

    X. Lin, C. Shi, Y . Zhan, Z. Yang, Y . Wu, D. Tao, Td²-net: Toward denoising and debiasing for video scene graph generation, Proceedings of the AAAI Confer- ence on Artificial Intelligence 38 (4) (2024) 3495–3503. doi:10.1609/aaai. 23 v38i4.28137. URL https://ojs.aaai.org/inde...

    2024 doi

  3. [11]

    S. Nag, K. Min, S. Tripathi, A. K. Roy-Chowdhury, Unbiased scene graph gen- eration in videos, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22803–22813

    2023

  4. [12]

    W. Wang, K. Gao, Y . Luo, T. Jiang, F. Gao, J. Shao, J. Sun, J. Xiao, Triple correlations-guided label supplementation for unbiased video scene graph gener- ation, in: Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 5153–5163

    2023

  5. [13]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., Gpt-4 technical report, arXiv preprint arXiv:2303.08774 (2023)

    2023 arXiv

  6. [14]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and e fficient foundation language models, arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  7. [15]

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y . Lin, et al., A survey on large language model based autonomous agents, Fron- tiers of Computer Science 18 (6) (2024) 1–26

    2024

  8. [16]

    W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P. Luo, T. Lu, J. Zhou, Y . Qiao, et al., Visionllm: Large language model is also an open-ended decoder for vision-centric tasks, Advances in Neural Information Processing Systems 36 (2024)

    2024

  9. [17]

    Minaee, T

    S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, J. Gao, Large language models: A survey, arXiv preprint arXiv:2402.06196 (2024)

    2024 arXiv

  10. [18]

    M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili, et al., A survey on large language models: Applications, chal- lenges, limitations, and practical usage, Authorea Preprints (2023). 24

    2023

  11. [19]

    Sharma, T

    P. Sharma, T. R. Shaham, M. Baradad, S. Fu, A. Rodriguez-Munoz, S. Dug- gal, P. Isola, A. Torralba, A vision check-up for language models, arXiv preprint arXiv:2401.01862 (2024)

    2024 arXiv

  12. [20]

    Ghanimifard, S

    M. Ghanimifard, S. Dobnik, What a neural language model tells us about spa- tial relations, in: Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP), 2019, pp. 71–81

    2019

  13. [21]

    Huang, K

    J. Huang, K. C.-C. Chang, Towards reasoning in large language models: A sur- vey, in: Findings of the Association for Computational Linguistics: ACL 2023, Association for Computational Linguistics, Toronto, Canada, 2023, pp. 1049–

    2023

  14. [22]

    W. Feng, W. Zhu, T.-j. Fu, V . Jampani, A. Akula, X. He, S. Basu, X. E. Wang, W. Y . Wang, Layoutgpt: Compositional visual planning and generation with large language models, Advances in Neural Information Processing Systems 36 (2024)

    2024

  15. [23]

    Zhang, W

    H. Zhang, W. Zhang, H. Qu, J. Liu, Enhancing human-centered dynamic scene understanding via multiple llms collaborated reasoning, arXiv preprint arXiv:2403.10107 (2024)

    2024 arXiv

  16. [24]

    X. Wang, Y . Xiong, H. Niu, J. Yue, Y . Zhu, P. S. Yu, Improving chinese character representation with formation graph attention network, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 1999–2009

    2021

  17. [25]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  18. [26]

    J. Ji, R. Krishna, L. Fei-Fei, J. C. Niebles, Action genome: Actions as composi- tions of spatio-temporal scene graphs, in: Proceedings of the IEEE /CVF Confer- ence on Computer Vision and Pattern Recognition, 2020, pp. 10236–10247. 25

    2020

  19. [27]

    Johnson, R

    J. Johnson, R. Krishna, M. Stark, L.-J. Li, D. Shamma, M. Bernstein, L. Fei-Fei, Image retrieval using scene graphs, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3668–3678

    2015

  20. [28]

    T. Qian, J. Chen, S. Chen, B. Wu, Y .-G. Jiang, Scene graph refinement network for visual question answering, IEEE Transactions on Multimedia (2022)

    2022

  21. [29]

    G. Zhai, E. P. ¨Ornek, S.-C. Wu, Y . Di, F. Tombari, N. Navab, B. Busam, Common- scenes: Generating commonsense 3d indoor scenes with scene graphs, Advances in Neural Information Processing Systems 36 (2024)

    2024

  22. [30]

    Y . Cong, J. Yi, B. Rosenhahn, M. Y . Yang, Ssgvs: Semantic scene graph-to-video synthesis, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2554–2564

    2023

  23. [31]

    H. Tang, Z. Li, D. Zhang, S. He, J. Tang, Divide-and-conquer: Confluent triple- flow network for rgb-t salient object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 47 (3) (2025) 1958–1974. doi:10.1109/ TPAMI.2024.3511621

    2025

  24. [32]

    H. Tang, C. Yuan, Z. Li, J. Tang, Learning attention-guided pyramidal features for few-shot fine-grained recognition, Pattern Recognition 130 (2022) 108792. doi:https://doi.org/10.1016/j.patcog.2022.108792

    2022

  25. [33]

    H. Tang, J. Liu, S. Yan, R. Yan, Z. Li, J. Tang, M3net: Multi-view encoding, matching, and fusion for few-shot fine-grained action recognition, in: Proceed- ings of the 31st ACM International Conference on Multimedia, MM ’23, Asso- ciation for Computing Machinery, New York, NY...

    2023

  26. [34]

    Z. Zeng, Z. Zhou, Z. Sui, O. C. Jenkins, Semantic robot programming for goal- directed manipulation in cluttered scenes, in: 2018 IEEE international conference on robotics and automation (ICRA), IEEE, 2018, pp. 7462–7469. 26

    2018

  27. [35]

    Zhang, K

    J. Zhang, K. J. Shih, A. Elgammal, A. Tao, B. Catanzaro, Graphical contrastive losses for scene graph parsing, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11535–11543

    2019

  28. [36]

    Zellers, M

    R. Zellers, M. Yatskar, S. Thomson, Y . Choi, Neural motifs: Scene graph parsing with global context, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5831–5840

    2018

  29. [37]

    Y . Cong, M. Y . Yang, B. Rosenhahn, Reltr: Relation transformer for scene graph generation, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)

    2023

  30. [38]

    Kundu, S

    S. Kundu, S. N. Aakur, Is-ggt: Iterative scene graph generation with generative transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6292–6301

    2023

  31. [39]

    F. Zhu, W. Lei, C. Wang, J. Zheng, S. Poria, T.-S. Chua, Retrieving and read- ing: A comprehensive survey on open-domain question answering, arXiv preprint arXiv:2101.00774 (2021)

    2021 arXiv

  32. [40]

    Z. He, T. Liang, W. Jiao, Z. Zhang, Y . Yang, R. Wang, Z. Tu, S. Shi, X. Wang, Ex- ploring human-like translation strategy with large language models, Transactions of the Association for Computational Linguistics 12 (2024) 229–246

    2024

  33. [41]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., Training language models to follow instruc- tions with human feedback, Advances in neural information processing systems 35 (2022) 27730–27744

    2022

  34. [42]

    J. Gong, L. G. Foo, Y . He, H. Rahmani, J. Liu, Llms are good sign language translators, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18362–18372

    2024

  35. [43]

    Kevian, U

    D. Kevian, U. Syed, X. Guo, A. Havens, G. Dullerud, P. Seiler, L. Qin, B. Hu, Capabilities of large language models in control engineering: A benchmark study 27 on gpt-4, claude 3 opus, and gemini 1.0 ultra, arXiv preprint arXiv:2404.03647 (2024)

    2024 arXiv

  36. [44]

    Van Den Oord, O

    A. Van Den Oord, O. Vinyals, et al., Neural discrete representation learning, Ad- vances in neural information processing systems 30 (2017)

    2017

  37. [45]

    J. Xu, H. Zhou, C. Gan, Z. Zheng, L. Li, V ocabulary learning via optimal transport for neural machine translation, in: Proceedings of ACL 2021, 2021

    2021

  38. [46]

    L. Chen, Y . Zhang, R. Zhang, C. Tao, Z. Gan, H. Zhang, B. Li, D. Shen, C. Chen, L. Carin, Improving sequence-to-sequence learning via optimal transport, in: In- ternational Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=S1xtAjR5tX

    2019

  39. [47]

    A. Nag, B. Samanta, A. Mukherjee, N. Ganguly, S. Chakrabarti, Entropy-guided vocabulary augmentation of multilingual language models for low-resource tasks, in: Findings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 8619–8629

    2023

  40. [48]

    Peyr ´e, M

    G. Peyr ´e, M. Cuturi, et al., Computational optimal transport: With applications to data science, Foundations and Trends ® in Machine Learning 11 (5-6) (2019) 355–607

    2019

  41. [49]

    Q. C. Guan Wang, Zhimin Li, Y . Liu, Oed: Towards one-stage end-to-end dy- namic scene graph generation, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 27938–27947

    2024

  42. [50]

    K. Tang, H. Zhang, B. Wu, W. Luo, W. Liu, Learning to compose dynamic tree structures for visual contexts, in: Conference on Computer Vision and Pattern Recognition, 2019

    2019

  43. [51]

    A. Khandelwal, Flocode: Unbiased dynamic scene graph generation with tem- poral consistency and correlation debiasing, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 2516–2526. 28

    2024

  44. [52]

    Iftekhar, R

    A. Iftekhar, R. Ruschel, S. Kumar, S. You, B. Manjunath, Dds: Decoupled dy- namic scene-graph generation network, in: 2025 IEEE /CVF Winter Conference on Applications of Computer Vision (W ACV), IEEE, 2025, pp. 9670–9680

    2025

  45. [53]

    Zhuang, B

    W. Zhuang, B. Dong, Z. Zhu, Z. Li, J. Liu, Y . Wang, X. Hong, X. Li, W. Zuo, Spatial-temporal saliency guided unbiased contrastive learning for video scene graph generation, IEEE Transactions on Multimedia (2025)

    2025

  46. [54]

    M. Chen, L. Li, W. Wang, Y . Yang, Diffvsgg: Diffusion-driven online video scene graph generation, in: CVPR, 2025

    2025

  47. [55]

    Ra ffel, N

    C. Ra ffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text trans- former, Journal of machine learning research 21 (140) (2020) 1–67. 29

    2020

  48. [1065]

    URL https://aclanthology.org/2023.findings-acl.67

    doi:10.18653/v1/2023.findings-acl.67. URL https://aclanthology.org/2023.findings-acl.67

    2023 doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.