Pith. sign in

REVIEW 3 major objections 5 minor 42 references

Advancing 3D Scene Understanding with MV-ScanQA Multi-View Reasoning Evaluation and TripAlign Pre-training Dataset

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that existing 3D vision-language benchmarks are mostly single-view solvable, and answers with MV-ScanQA, a 68% multi-view benchmark, the TripAlign pretraining corpus, and the LEGO model that tops those benchmarks.

desk verdict Useful new 3D VLM benchmark and pretraining corpus, but the headline multi-view statistics rest on an unvalidated projection overlap metric that should be fixed before the numbers are cited. read the letter →

arxiv 2508.11058 v1 pith:NWHPQEQQ submitted 2025-08-14 cs.CV cs.MM

Wentao Mo , Qingchao Chen , Yuxin Peng , Siyuan Huang , Yang Liu This is my paper · ORCID
classification cs.CVcs.MM
keywords 3Dvision-languagelearningmulti-viewreasoningquestionansweringdatasetconstruction2D-3Dpretrainingmultimodalalignmentsceneunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing 3D vision-language benchmarks are far easier than they appear: under the paper's solvability analysis, 94% of ScanQA, 96% of ScanRefer, and 93% of Nr3D instructions can be answered from a single egocentric view. To remove that single-view bottleneck, the paper builds MV-ScanQA, a question-answering benchmark in which 68% of questions require integrating at least two views and 13% require three or more, synthesized by an LLM from pairs of questions that share an anchor object but need complementary information. It also contributes TripAlign, a fully automatic one-million-triplet corpus of (2D view, set of visible 3D objects, text), whose grouped-object supervision is denser than existing single-object annotations. Trained on TripAlign, the LEGO model built atop a frozen 2D vision-language model reaches state-of-the-art results on ScanQA, ScanRefer, and Nr3D, and improves markedly on MV-ScanQA when given multiple views. The value of the package is a harder, more honest 3D evaluation and a scalable way to train for it.

What carries the argument

The load-bearing mechanism is the Intersection-over-Smaller-Area (IoSA) visibility metric, $\mathrm{IoSA}(A,B)=\frac{\mathrm{Area}(A\cap B)}{\min(\mathrm{Area}(A),\mathrm{Area}(B))}$, applied to a 3D object's projected 2D box against an egocentric view. IoSA with threshold 0.5 does three jobs: it defines which objects are witnessed in the solvability analysis that yields the 94/96/93% single-view figures for existing benchmarks; the same analysis marks MV-ScanQA's 68% multi-view share; and LEGO's view-dependent multi-object alignment uses the same projection and threshold to filter 3D object proposals down to those actually visible in the paired view. Around this metric, the other machinery is data construction: LLM-based compositional question synthesis for MV-ScanQA, and the TripAlign dual pipeline that produces 1M triplets by generating captions from egocentric views with image captioners, filtering them by image-text retrieval, and attaching informative views to existing QA and captioning samples.

What would settle it

Re-run the solvability analysis on ScanQA, ScanRefer, Nr3D, and MV-ScanQA with the visibility threshold set to 0.3 and to 0.7, keeping the same camera projections; if MV-ScanQA's 68% multi-view share drops toward the single digits at either setting, the multi-view bottleneck is an artifact of the threshold rather than a property of the questions. Complementarily, if human annotators answered MV-ScanQA questions correctly while seeing only the best single view, the questions would not truly need multiple views.

Watch

Extended reading notes

Core claim

The paper's central discovery is a measurement plus a remedy. Measured with an Intersection-over-Smaller-Area visibility rule, existing 3D vision-language benchmarks are single-view solvable almost all the time, so their scores do not certify genuine multi-view 3D understanding. MV-ScanQA fixes the measurement by composing pairs of existing single-view questions into new ones that share an anchor object but require complementary information, pushing the multi-view requirement to 68% of questions (13% needing three or more views); manual checking of 100 samples finds 94% correctly formed. TripAlign fixes the data side by generating, from egocentric views of ScanNet scenes, over one million triplets that couple a 2D view, the set of 3D objects visible in it, and a caption, using image captioners and retrieval filters, plus an extension that attaches informative views to existing QA and captioning samples. LEGO then shows the remedy works: built on a frozen 2D vision-language model with a 3D detector and LoRA, using the same visibility projection to mask out-of-view objects, it achieves state-of-the-art scores on ScanQA, ScanRefer, and Nr3D, and on MV-ScanQA its multi-view variant scores 34.1 exact-match versus 30.0 for single-view, with the largest gain on questions needing four or more views.

Load-bearing premise

Everything rests on the 0.5 visibility threshold in the IoSA metric, the rule that an object counts as seen only if its projected box covers more than half of the view, because the same threshold sets the 68% multi-view statistic and decides which objects LEGO aligns with text.

Editorial extensions

If this is right

  • MV-ScanQA resets the bar for 3D QA: a model given one informative view scores 30.0 exact-match, while the same model with four spliced views scores 34.1, with the largest gap on questions needing four or more views (26.5 to 30.9).
  • TripAlign's two data pipelines each add measurable value: view extension alone improves ScanRefer C@0.25 by +4.4 and ScanQA exact-match by +2.1, and adding generated triplets yields cumulative gains of +10.5, +8.4, and +3.3 on ScanRefer, Nr3D, and ScanQA.
  • The view-dependent multi-object alignment mask is beneficial: removing it lowers ScanRefer C@0.25 by +3.6, Nr3D C@0.5 by +0.4, and ScanQA exact-match by +0.5, showing that filtering out-of-view objects improves 2D-to-3D transfer.
  • Because LEGO reaches state-of-the-art results across ScanQA, ScanRefer, and Nr3D in addition to MV-ScanQA, multi-object, view-grounded pretraining transfers to tasks whose annotations remain single-object.
  • The LLM-guided composition recipe, which pairs questions with a shared anchor and non-nested object sets, is a general procedure that can convert other single-view QA datasets into multi-view evaluations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not calibrate the IoSA threshold of 0.5 against human visibility judgments; if that threshold drifts, both the 68% difficulty statistic and LEGO's object filtering would move, so the benchmark's difficulty gradient is best read as conditional on that choice.
  • The same triplet structure could be reused for tasks that also need viewpoint-dependent object grouping, such as referring expression segmentation or embodied instruction following, since it already provides the alignment between text, visible objects, and a specific view.
  • A direct control would compare TripAlign's view-grouped triplets against random object groups of the same size; if random groups transfer nearly as well, the value may come from scale and caption quality rather than from view-based grouping.
  • Because TripAlign is generated automatically from ScanNet, it could be extended to other RGB-D scan collections or to synthetic scenes, which would test whether the approach depends on egocentric human-captured views or works with any view distribution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper makes three contributions: MV-ScanQA, a 3D question-answering benchmark built by LLM-based composition of ScanQA questions, with 68% of questions claimed to require multiple egocentric views; TripAlign, an automatically generated corpus of 1M <2D view, set of 3D objects, text> triplets; and LEGO, a 2D-3D large vision-language model built on Fuyu that uses informative 2D views and a view-dependent object filter. The authors report a solvability analysis in which 94%, 96%, and 93% of ScanQA, ScanRefer, and Nr3D instructions are solvable from a single view, and report state-of-the-art results for LEGO on ScanRefer, Nr3D, ScanQA, and MV-ScanQA. The empirical improvements are consistent and the datasets/code are promised for release, but the central multi-view statistics and a key training-signal filter both depend on an unvalidated IoSA visibility proxy.

Significance. If the multi-view requirement and downstream gains hold, the paper supplies a more demanding benchmark, a scalable multi-object pre-training signal, and a strong baseline; the release of datasets and code is a concrete community asset. The solvability analysis is a useful diagnostic that goes beyond prior benchmark papers. However, the two headline claims—the 68% multi-view statistic and the value of TripAlign's object filtering—rest on a single unvalidated visibility proxy, so the significance cannot be fully assessed until that proxy is checked.

major comments (3)
  1. [Section 3.1 and Section 5] The IoSA visibility criterion (IoSA > 0.5 in Section 3.1) is load-bearing for both central claims: it produces the 94%/96%/93% single-view solvability rates and the 68% multi-view rate for MV-ScanQA, and it is reused in Section 5 as the in-view object filter for TripAlign and LEGO. The threshold is introduced without calibration or ablation, the tau used for the Section 5 filter is not stated, and the projection-overlap test ignores occlusion by walls or furniture and camera pose error. Because the same proxy drives the benchmark statistic and the training-signal filter, any bias in it moves both claims together. Please (i) report tau in every use, (ii) sweep tau and show the sensitivity of the solvability percentages and of Table 6, and (iii) validate 'witnessed' labels against depth-based occlusion tests or human annotation on a stratified sample.
  2. [Tables 2–6] All main result tables report single point estimates with no standard deviations, confidence intervals, or significance tests. Several reported advantages are small (e.g., Table 4, ScanQA test EM 28.4 vs 27.0; Table 3, ScanRefer C@0.25 84.66 vs 81.94), and without variance over seeds the 'state-of-the-art on all 9 metrics' claim is not yet supported. Please provide at least three seeds with mean and standard deviation for the main comparisons, or a paired significance test for the ablated variants.
  3. [Section 3.2] The dataset quality statement rests on 'Manual verification on 100 images shows 94% questions are synthesized correctly,' but the manuscript gives no sampling protocol, annotator instructions, inter-annotator agreement, or error taxonomy. This check validates question-answer text, not the visibility labels behind the 68% statistic. Please report a detailed verification protocol, and consider verifying a larger stratified sample of question-view assignments.
minor comments (5)
  1. [Section 7] The concluding paragraph is grammatically incomplete ('more than 10×Second'); please fix the sentence.
  2. [Section 6.3] The text uses 'CiDER' and 'CiDER metrics'; the standard name is CIDEr, and the metric should be defined consistently.
  3. [Table 6] The sentence 'we investigate the effectiveness two data pipelines of of TripAlign' contains a typo and an incomplete phrase; please rewrite.
  4. [Section 6.3] The claim of state-of-the-art on 'all 9 metrics' is unclear because Tables 3 and 4 together report 8 metrics (3 in Table 3, 5 in Table 4); please clarify the ninth metric or correct the count.
  5. [Table 4] The columns 'w/ obj' and 'w/o obj' are used without definition in the text; please state what these conditions mean.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the 68% multi-view statistic is an explicit geometric operationalization, and LEGO is evaluated against held-out external benchmarks.

full rationale

The paper's central quantitative claims are computed from an explicitly stated visibility rule (IoSA with threshold 0.5) rather than imported from the outcome being predicted. The solvability analysis in Section 3.1 defines what counts as a witnessed object and then measures existing benchmarks against that definition; the resulting 94%/96%/93% figures are an empirical application of the metric, not a hidden reuse of the target labels. MV-ScanQA's 68% multi-view figure is likewise a direct statistic over the synthesized questions, and the question synthesis itself is driven by LLM composition and semantic object-set criteria, not by the IoSA threshold. TripAlign and LEGO use the same IoSA idea to select informative views and filter in-view objects, but this is a consistent modeling choice rather than a circular reduction: the model is trained on text and objects and tested on held-out ScanNet splits of ScanQA, ScanRefer, Nr3D, and the newly created MV-ScanQA, where answers are not encoded by the training filter. The self-citations in the paper (BridgeQA baseline, 3D-VLP synthetic data, reannotation work) are related-work and baseline references, not load-bearing justifications for the main claims. The unvalidated nature of the 0.5 IoSA threshold and its occlusion blindness is a legitimate validity and robustness concern, but it is not a case where a prediction is equivalent to its input by construction. Therefore, no significant circularity is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on automated dataset construction pipelines whose quality controls are thin. A single IoSA threshold drives the benchmark statistics and the model's object masking, captions are generated and filtered by pre-trained 2D models, and object features come from an off-the-shelf detector. These are domain assumptions rather than derived results, and each could affect the reported outcomes.

free parameters (3)
  • IoSA visibility threshold tau = 0.5
    Used to define whether an object is witnessed in a view (Section 3.1) and to filter 3D objects in TripAlign and LEGO (Section 5). The 68% multi-view statistic and the training-time object selection both depend directly on this threshold.
  • Number of input views in multi-view LEGO = 4
    In Section 6.2, LEGO Multi-View splices four egocentric views in a 2x2 grid. The choice of four views is not justified experimentally.
  • Retrieval models for view selection and caption filtering = BLIP and CLIP (pre-trained)
    Section 4 uses BLIP [19] to select the most relevant view for QA extensions and CLIP [29] to filter generated captions. These model choices are made without comparison to alternatives.
assumptions (4)
  • domain assumption ScanNet egocentric views provide sufficient visual evidence about the 3D scene for solving the tasks.
    Section 3.1 defines solvability using these views. If the views are too sparse, distorted, or misaligned with camera poses, the multi-view statistics and the training signal from TripAlign would be unreliable.
  • domain assumption The LLM (claude-3.5-sonnet) composes questions that are valid, unambiguous, and correctly answered by the composed answer.
    Section 3.2 relies on LLM-based question composition with only 100 manually verified examples (94% correct). The benchmark's quality depends on this verification being representative.
  • domain assumption Pre-trained 2D captioners and retrieval models produce captions and view-text pairs that are semantically aligned with groups of objects.
    TripAlign construction in Section 4 uses captions from [22,26] and filtering from [29]. The downstream improvements rely on these pre-trained models being accurate enough to generate useful training signal.
  • domain assumption The 3D object detector Vote2Cap-DETR provides accurate object proposals for the scenes.
    LEGO in Section 5 uses E3D [6] to extract object features and boxes. Errors in detection propagate into the multi-object alignment and the final response.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing 3D Scene Understanding with MV-ScanQA Multi-View Reasoning Evaluation and TripAlign Pre-training Dataset." pith.science (2026). https://pith.science/paper/NWHPQEQQ

@misc{pith2026250811058,
  author       = {Pith},
  title        = {Pith review of: Advancing 3D Scene Understanding with MV-ScanQA Multi-View Reasoning Evaluation and TripAlign Pre-training Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NWHPQEQQ}},
  note         = {Machine review of arXiv:2508.11058}
}
read the original abstract

The advancement of 3D vision-language (3D VL) learning is hindered by several limitations in existing 3D VL datasets: they rarely necessitate reasoning beyond a close range of objects in single viewpoint, and annotations often link instructions to single objects, missing richer contextual alignments between multiple objects. This significantly curtails the development of models capable of deep, multi-view 3D scene understanding over distant objects. To address these challenges, we introduce MV-ScanQA, a novel 3D question answering dataset where 68% of questions explicitly require integrating information from multiple views (compared to less than 7% in existing datasets), thereby rigorously testing multi-view compositional reasoning. To facilitate the training of models for such demanding scenarios, we present TripAlign dataset, a large-scale and low-cost 2D-3D-language pre-training corpus containing 1M <2D view, set of 3D objects, text> triplets that explicitly aligns groups of contextually related objects with text, providing richer, view-grounded multi-object multimodal alignment signals than previous single-object annotations. We further develop LEGO, a baseline method for the multi-view reasoning challenge in MV-ScanQA, transferring knowledge from pre-trained 2D LVLMs to 3D domain with TripAlign. Empirically, LEGO pre-trained on TripAlign achieves state-of-the-art performance not only on the proposed MV-ScanQA, but also on existing benchmarks for 3D dense captioning and question answering. Datasets and code are available at https://matthewdm0816.github.io/tripalign-mvscanqa.

Figures

Figures reproduced from arXiv: 2508.11058 by the authors.

Figure 1
Figure 1. (a). MV-ScanQA tests multi-view reasoning, with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MV-ScanQA evaluation dataset analyses. (a). Ques [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Constructing TripAlign: (a). We sample 2D views described by an image captioner, and filtered by an image-text [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: LEGO Overview. (a). LEGO builds upon a 2D LVLM and accepts 3D scene input. (b). Our proposed [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 26 canonical work pages

  1. [1]

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. 2020. ReferIt3D: Neural Listeners for Fine-Grained 3D Ob- ject Identification in Real-World Scenes.16th European Conference on Computer Vision (ECCV)(2020)

    work page 2020

  2. [2]

    Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. 2022. ScanQA: 3D question answering for spatial scene understanding. Inproceedings of the IEEE/CVF conference on computer vision and pattern recognition. 19129–19139

    work page 2022

  3. [3]

    Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sağnak Taşırlar. 2023. Introducing our Multimodal Models. https://www.adept.ai/blog/fuyu-8b

    work page 2023

  4. [4]

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. 2020. ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language.16th European Conference on Computer Vision (ECCV)(2020)

    work page 2020

  5. [5]

    Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, and Tao Chen. 2024. LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 26428–26438

    work page 2024

  6. [6]

    Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Gang Yu, and Tao Chen. 2023. End-to-end 3d dense captioning with vote2cap-detr. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11124–11133

    work page 2023

  7. [7]

    Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, and Tao Chen. 2024. Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning.IEEE Trans. Pattern Anal. Mach. Intell.46, 11 (Nov. 2024), 7331–7347. doi:10.1109/TPAMI.2024.3387838

    arXiv 2024

  8. [8]

    Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang. 2021. Scan2Cap: Context-aware Dense Captioning in RGB-D Scans. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3193– 3203

    work page 2021

Show all 42 references
  1. [9]

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. 2017. Scannet: Richly-annotated 3d reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition. 5828–5839

    2017

  2. [10]

    Rao Fu, Jingyu Liu, Xilun Chen, Yixin Nie, and Wenhan Xiong. 2025. Scene-LLM: Extending Language Model for 3D Visual Reasoning. InProceedings of the Winter Conference on Applications of Computer Vision (W ACV). 2195–2206

    2025

  3. [11]

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 2023. 3D-LLM: Injecting the 3D World into Large Language Models.NeurIPS(2023)

    2023

  4. [12]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InInternational Conference on Learning Representations. https: //openreview.net/forum?id=nZeVKeeFYf9

    2022

  5. [13]

    Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, et al . 2024. Chat-scene: Bridging 3d scene and large language models with object identifiers.Proceedings of the Advances in Neural Information Process...

    2024

  6. [14]

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. 2024. An Embodied Generalist Agent in 3D World. (2024)

    2024

  7. [15]

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. 2024. Sceneverse: Scaling 3d vision-language learning for grounded scene understanding. InEuropean Conference on Computer Vision (ECCV)

    2024

  8. [16]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023 arXiv

  9. [17]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne...

    2024 arXiv

  10. [18]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Boot- strapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. InProceedings of the 40th International Conference on Ma- chine Learning (Proceedings of Machine Learning Rese...

    2023

  11. [19]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. InICML

    2022

  12. [20]

    Xiaoqi Li, Jiaming Liu, Nuowei Han, Liang Heng, Yandong Guo, Hao Dong, and Yang Liu. 2025. 3D Weakly Supervised Visual Grounding at Category and Instance Levels. InProceedings of the International Conference on Robotics and Automation

    2025

  13. [21]

    Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. 2024. LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6517–6526

    2024

  14. [22]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved Baselines with Visual Instruction Tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 26296–26306

    2024

  15. [23]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual In- struction Tuning. InAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 34892–34916. https://p...

    2023

  16. [24]

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. 2023. SQA3D: Situated Question Answering in 3D Scenes. In International Conference on Learning Representations. https://openreview.net/ forum?id=IDJx97BC38

    2023

  17. [25]

    Wentao Mo and Yang Liu. 2024. Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA.Proceedings of the AAAI Conference on Artificial Intelligence38, 5 (Mar. 2024), 4261–4268. doi:10.1609/aaai. v38i5.28222

    2024 doi

  18. [26]

    OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]

    2023 arXiv

  19. [27]

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. 2023. DreamFusion: Text-to-3D using 2D Diffusion. InICLR

    2023

  20. [28]

    Yiming Qin, Zhu Xu, and Yang Liu. 2025. Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 18521–18530

    2025

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings ...

    2021

  22. [30]

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. 2023. Mask3D: Mask Transformer for 3D Semantic Instance Segmentation. InInternational Conference on Robotics and Automation (ICRA)

    2023

  23. [31]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  24. [32]

    Dejie Yang, Zhu Xu, Wentao Mo, Qingchao Chen, Siyuan Huang, and Yang Liu

  25. [33]

    Zhaoda Ye, Yang Liu, and Yuxin Peng. 2024. MAAN: Memory-Augmented Auto- regressive Network for Text-driven 3D Indoor Scene Generation.IEEE Transac- tions on Multimedia(2024), 1–14. doi:10.1109/TMM.2024.3443657

    2024

  26. [34]

    Zhaoda Ye, Xinhan Zheng, Yang Liu, and Yuxin Peng. 2024. RelScene: A Bench- mark and baseline for Spatial Relations in text-driven 3D Scene Generation. In Proceedings of the 32nd ACM International Conference on Multimedia(Melbourne VIC, Australia)(MM ’24). Association for Comp...

    2024

  27. [35]

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. 2024. GaussianDreamer: Fast Gener- ation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models. In CVPR

    2024

  28. [36]

    Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang, Lu Sheng, Lei Bai, et al. 2024. Lamm: Language- assisted multi-modal instruction-tuning dataset, framework, and benchmark. Advances in Neural Information Processing Systems...

    2024

  29. [37]

    Taolin Zhang, Sunan He, Tao Dai, Zhi Wang, Bin Chen, and Shu-Tao Xia. 2024. Vision-language pre-training with object contrastive learning for 3D scene MM ’25, October 27–31, 2025, Dublin, Ireland Wentao Mo, Qingchao Chen, Yuxin Peng, Siyuan Huang, & Yang Liu understanding. InP...

    2024

  30. [38]

    Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. 2024. Towards Learning a Generalist Model for Embodied Navigation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13624– 13634

    2024

  31. [39]

    Shengli Zhou, Yang Liu, and Feng Zheng. 2025. Learn 3D VQA Better with Active Selection and Reannotation. arXiv:2507.04630 [cs.CV] https://arxiv.org/abs/2507. 04630

    2025 arXiv

  32. [40]

    Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li

  33. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2911–2921

  34. [2024]

    In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelli- gence, IJCAI-24, Kate Larson (Ed.)

    3D Vision and Language Pretraining with Large-Scale Synthetic Data. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelli- gence, IJCAI-24, Kate Larson (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1552–1560. ...

    doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.