Pith. sign in

REVIEW 5 major objections 8 minor 30 references

AdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning

T0 review · 5 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 3D multimodal model can discard most of its spatial tokens per layer and still answer 3D scene questions, with 21% faster inference and 63% fewer FLOPs.

desk verdict Incremental but real token-pruning gains for 3D LMMs; the headline redundancy claim is unvalidated and the speedup numbers are inconsistent. read the letter →

arxiv 2505.12782 v1 pith:2ZFTPYC2 submitted 2025-05-19 cs.GR cs.CVcs.IRcs.ITmath.IT

Kai Zhang , Xingyu Chen , Xiaofeng Zhang This is my paper · ORCID
classification cs.GRcs.CVcs.IRcs.ITmath.IT
keywords 3DlargemultimodalmodelsspatialtokenpruningattentioncontributionanalysisinferenceaccelerationLLaVA-3DredundancysceneunderstandingFLOPsreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AdaToken-3D claims that a 3D large multimodal model can discard most of its spatial tokens during inference without losing task accuracy, because the model's own attention patterns reveal which tokens matter. The method scores each spatial token in every layer by an information-contribution value that combines intra-modal attention, cross-modal attention, and a smoothed historical flow, then schedules layer-wise token retention by fitting an exponential decay curve to that signal. On LLaVA-3D-7B the authors report 21% faster inference and 63% fewer FLOPs while keeping accuracy on 3D question answering, visual grounding, and dense captioning benchmarks at the level of the unpruned model. They also report that more than 60% of spatial tokens contribute less than 5% to final predictions, and that the same pruning schedule can shorten training time.

What carries the argument

The mechanism is the Information Contribution score $INF(i)$, built from a weighted sum of inter-modal attention scores between spatial tokens and prompt/system tokens, plus the Spatial Flow Value $F^i_{3D} = \sigma\,S^{\mathrm{self},i}_{3D} + \gamma\,F^{i-1}_{3D}$, a damped running total of intra-modal spatial attention. Each layer's retention ratio comes from fitting an exponential decay curve $O_{\mathrm{pre}}(i)=\alpha\exp(-\beta(i-L))+M$ under a derivative-smoothing loss, optimized with SLSQP under a global token-reduction constraint. Individual spatial tokens are ranked by $V_j=Q_t(K^s)^T$, the similarity between the last instruction token's query and each spatial token's key state. The pruning schedule is therefore adaptive to the architecture rather than a fixed ratio.

What would settle it

Run a causal test on a sample of SQA3D questions: prune the tokens ranked lowest by $INF(i)$ and compare answer accuracy against pruning a random subset and against pruning the highest-ranked tokens. If removing the lowest-ranked tokens degrades accuracy as much as removing the highest-ranked tokens, the contribution measure is not causal; if the lowest-ranked tokens can be removed with little effect, the redundancy claim is confirmed. A complementary check is attention patching: replace each spatial token's key and value states with zero vectors and measure the resulting change in the generated answer.

Watch

Extended reading notes

Core claim

The central claim is that spatial-token redundancy in 3D large multimodal models is strong and layer-dependent: shallow layers carry the spatial information the reasoning depends on, while deeper layers can shed a large share of tokens. AdaToken-3D measures each spatial token's information contribution from attention patterns, separates intra-modal attention (spatial tokens attending to spatial tokens) from inter-modal attention (spatial tokens interacting with prompt and system tokens), and uses a fitted exponential layer-wise retention schedule to prune tokens at the end of each stage. The paper reports that this preserves accuracy on ScanQA, SQA3D, ScanRefer, and Scan2Cap while cutting 63% of FLOPs and about 21% of inference time, and that over 60% of spatial tokens contribute under 5% to the final predictions. The contribution is therefore an efficiency method plus an empirical claim about where redundancy lives in 3D multimodal reasoning.

Load-bearing premise

The load-bearing premise is that the attention-derived Information Contribution $INF(i)$ measures a spatial token's true causal contribution to the model's answer; if that proxy is wrong, both the token ranking and the claim that over 60% of tokens contribute less than 5% collapse.

Editorial extensions

If this is right

  • 3D large multimodal models can be served with substantially less GPU compute during inference, which makes real-time 3D question answering and embodied-agent workloads more practical.
  • The pruning schedule can be reused during training, and the paper reports that it reduces GPU hours by roughly 37%.
  • Applied to 2D tasks through the same backbone, the method matches dedicated 2D pruning methods on VQA and multimodal benchmarks, so the redundancy analysis transfers across modalities.
  • The layer-wise contribution curves give a quantitative target for architectural changes, such as concentrating spatial computation in shallow layers instead of spreading it uniformly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the attention-derived contribution measure truly tracks causal importance, the same scoring could guide where to add tokens, not only where to remove them; one testable extension is increasing spatial resolution only in shallow layers, where the paper locates most contribution.
  • The claim that over 60% of spatial tokens contribute under 5% predicts that a 3D LMM trained from scratch with only a fraction of the spatial tokens should reach near-baseline accuracy; the paper does not run this training-time test.
  • Because the retention curve is fitted to the same attention-derived signal that defines INF, confirming the redundancy claim would require an independent causal intervention, such as attention patching, which the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. AdaToken-3D proposes an adaptive spatial-token pruning framework for 3D large multimodal models. The method defines an attention-based 'Information Contribution' INF(i) that combines intra-modal and inter-modal attention statistics with a spatial flow recurrence, fits an exponential per-layer retention schedule O_pre(i) to INF(i) via a derivative-constrained optimization, and ranks individual spatial tokens by an attention similarity V_j = Q_t (K_s)^T. On LLaVA-3D-7B, the paper reports accuracy preservation on SQA3D, ScanQA, ScanRefer, and Scan2Cap while claiming large FLOPs and latency reductions, and it interprets the results as evidence that over 60% of spatial tokens contribute minimally (<5%) to final predictions, presenting this as a theoretical foundation for efficient 3D multimodal learning.

Significance. If the reported efficiency-accuracy trade-off is reproducible, AdaToken-3D would be a practically valuable plug-and-play sparsification method for 3D LMMs, extending 2D pruning insights to the 3D setting and comparing against PyramidDrop and FastV on a range of benchmarks, including 2D transfer. The paper's analytical contribution—quantifying spatial-token contribution via attention patterns—is potentially useful but is not yet causally validated; as written, the redundancy conclusion is an interpretation of the method's own proxy rather than an independent empirical finding. The efficiency claim is significant but requires corrected, consistent numbers to be credible. The experimental breadth, especially the multiple benchmarks and baselines, is a strength, though the lack of error bars and the unspecified hyperparameters limit the strength of the parity claims.

major comments (5)
  1. [§III-B, Eqs. (2)–(5)] Several constants in the central definitions are never given values: epsilon in Eq. (4), sigma and gamma in Eq. (2), a1 and a2 in Eq. (3), alpha_i in Eq. (4), and M in Eq. (5). The implementation details mention only alpha=0.8 and the text constrains alpha to [0.5,1.2], but no values are reported for the other parameters. Because Eq. (7) fits O_pre(i) to INF(i), different choices of these constants change the retention schedule and therefore the efficiency results. Please report all parameter values and a sensitivity analysis, or the method is not reproducible from the paper as written.
  2. [§III-B, §V] The claim that over 60% of spatial tokens contribute minimally (<5%) to final predictions is a threshold applied to the attention-derived scores INF(i) and V_j, not a measured property of the model's predictions. The paper never validates these proxies against causal token importance, such as the change in answer likelihood or task metrics when a token is removed. Since the retention schedule is itself fit to INF(i) via Eq. (7), the benchmark results show only that one attention-derived pruning schedule preserves accuracy; they do not establish that the pruned tokens are causally minimal contributors. I recommend either removing the theoretical redundancy claim or adding a rank-correlation comparison against a causal oracle on a sample of SQA3D or ScanRefer examples.
  3. [Abstract, §IV-C, Table III] The headline efficiency numbers are mutually inconsistent. The abstract reports 21% faster inference and 63% FLOPs reduction; Section IV-C reports 72% latency reduction and 60% FLOPs reduction; Table III for Scene0011-00 implies (5.957-4.439)/5.957 = 25.5% time reduction and 4.57/11.46 = 60.1% FLOPs reduction; Section III-C states a 23% increase in token utilization and a 63% FLOPs reduction. The training-time reductions also differ (37% in the introduction versus about 45% in Section IV-B). Please specify exactly which setting each number refers to and ensure the abstract matches the principal experimental measurement.
  4. [Table III] Table III is internally difficult to interpret. The header row "Total-time(s) FLOPs(T) Total-time(s) Latency" does not indicate which columns are absolute measurements and which are percentage reductions. The LLaVA-3D row lists 72% in the "Latency" column even though it is the baseline row, and the PyramidDrop row lists 16.6%, while the AdaToken-3D row has a dash; presumably these are reductions relative to LLaVA-3D, but the table should state this explicitly and give the corresponding AdaToken-3D reduction. Please clarify the table structure, units, and what the percentages are relative to.
  5. [§IV-B] The paper states that AdaToken-3D "can prune up to 90% of the spatial tokens without degrading the performance of LLaVA-3D," but Table II reports only 10%, 20%, and 40% drop ratios. No experimental evidence for the 90% claim is presented. Please add the corresponding experiment or remove/qualify this claim.
minor comments (8)
  1. [Eqs. (1) and (3)] The symbol n is used both as the number of layers and as the summation index over tokens; rename one of these to avoid ambiguity.
  2. [Eqs. (4) and (5)] The notation alpha_i in Eq. (4) conflicts with alpha in Eq. (5), and the later statement "default setting of alpha=0.8" does not clarify which alpha is intended.
  3. [§III-C] The sentence "the the token utilization rate increased by 23%" contains a typo and should read "the token utilization rate increased by 23%."
  4. [References] References [6] and [31] appear to be the same paper ("An Embodied Generalist Agent in 3D World"); please merge them.
  5. [Fig. 1] The figure caption includes "3x 3.5x" and training-time values without axes or units, leaving unclear what these multipliers compare; please clarify.
  6. [Tables II and IV] No error bars or multiple-seed statistics are reported; since parity claims rest on differences of one to two points, variance estimates would strengthen the conclusions.
  7. [§IV-B] The sentence "less spatial tokens also mean less useless tokens" is grammatically awkward and should be rewritten.
  8. [§III-C] The statement that SLSQP "iteratively approximates the Hessian matrix" is imprecise; SLSQP is a quasi-Newton method, but the sentence should be reworded for accuracy.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: the 60%/<5% token-contribution claim is a threshold on the paper's attention-defined INF metric, and the layer-wise retention schedule is fit to that same metric; benchmarked efficiency gains remain independent.

  1. self definitional [Abstract; Section III-B, Eqs. (1)-(4)]
    "To this end, we introduced Intra-Modal value Sself,i_I and Inter-Modal value Scross,i_I to dynamically evaluate the contribution of each modal in layer i. ... INF(i) = exp(Scross,i_3D/epsilon) + alpha_i*F_i_3D + log(1+Sself,i_3D). ... Our findings reveal that over 60% of spatial tokens contribute minimally (<5%) to the final predictions"

    The quantity the paper calls 'contribution' is assembled entirely from attention-score sums (Eqs. 1-4); it is never tied to a causal intervention on the decoded answer. The abstract's finding that 'over 60% of spatial tokens contribute minimally (<5%)' is therefore a threshold applied to this attention-defined metric, not an independent measurement of each token's effect on final predictions. The redundancy conclusion is encoded in the definition of INF, so the claimed 'theoretical foundations' restate the chosen proxy rather than derive it.

  2. fitted input called prediction [Section III-C, Eqs. (5)-(7)]
    "Opre(i) = α· exp(−β(i−L)) +M ... Inorm = f(INF), L = Σ (Opre−Inorm)^2 + λ·Σ(dO/di − dI/di)^2. ... The resulting parameter is optimized to minimize L while maintaining the desired token reduction ratio Oratio,i across all layers."

    The layer-wise retention schedule O_pre is fit by least squares to Inorm = f(INF), the same attention-derived signal used for the redundancy claim. The optimized O_pre is then presented as the 'theoretical retention ratio' Oratio,i and used to compute the reported FLOPs and latency reductions. The schedule is thus a smoothed copy of the input metric, not a prediction derived from independent principles. The accuracy results on 3D benchmarks are external, so this is partial rather than total circularity.

full rationale

The paper's measured efficiency and accuracy results (Tables II-IV) are evaluated against LLaVA-3D on external 3D and 2D benchmarks, so the engineering claim that aggressive spatial-token pruning preserves task accuracy has independent grounding and is not circular. The circular component is in the paper's stated scientific finding: 'contribution' is defined by the attention-based INF metric (Eqs. 1-4), and the 60%/<5% redundancy statement is a threshold on that definition rather than a causal analysis of final-prediction sensitivity. In addition, the per-layer retention schedule O_pre is fitted to the same INF signal via Eq. 7, so the 'theoretical retention ratio' is a smoothed restatement of the metric. The authors' self-citations (Refs. [1] and [11]) motivate the information-flow framing but are not load-bearing uniqueness claims, so they do not add circularity. Overall, the central redundancy/theoretical claim reduces by construction to the chosen attention proxy, while the benchmarked efficiency gains remain independent; score 4.

Assumptions & free parameters 8 free parameters · 3 assumptions · 0 invented entities

The method introduces several tunable constants and a hand-chosen decay schedule. The central efficiency result depends on these choices, and the paper does not state values for most of them. There are no invented physical entities.

free parameters (8)
  • alpha (attenuation amplitude) = default 0.8
    Eq. 5; controls magnitude of layer-wise decay; empirically constrained to [0.5, 1.2].
  • beta (attenuation rate) = learned, value not reported
    Eq. 5; determines decay steepness; learned via back-propagation with L2 regularization.
  • L (attenuation center offset) = not reported
    Eq. 5; phase shift of decay initiation; tuned to align with critical layers.
  • M (base retention constant) = not specified
    Appears in Eq. 5 but never defined; likely a base retention level.
  • epsilon (inter-modal scaling) = not specified
    Used in Eq. 4 as exp(S_cross / epsilon); controls scaling of inter-modal contribution.
  • gamma (temporal persistence coefficient) = not specified
    Eq. 2; weights historical information in the Spatial Flow Value F.
  • sigma (spatial attenuation factor) = not specified
    Eq. 2; controls current layer's contribution to F; stated to be in [0,1].
  • lambda (regularization weight) = not specified
    Eq. 7; balances accuracy and smoothness in the loss for fitting O_pre.
assumptions (3)
  • domain assumption Attention scores are a valid proxy for token contribution to the final prediction.
    The pruning rule and the redundancy finding rely on this; the paper does not test causal alternatives.
  • domain assumption Spatial token redundancy in 3D LMMs can be removed without retraining.
    The method prunes at inference time and assumes LLaVA-3D is robust to token removal; supported empirically in the tables but not theoretically.
  • ad hoc to paper The parametric exponential decay form O_pre(i) in Eq. 5 adequately captures the ideal per-layer retention schedule.
    This functional form is assumed without derivation; the fitted parameters are then used as the pruning schedule.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning." pith.science (2026). https://pith.science/paper/2ZFTPYC2

@misc{pith2026250512782,
  author       = {Pith},
  title        = {Pith review of: AdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2ZFTPYC2}},
  note         = {Machine review of arXiv:2505.12782}
}
abstract

Large Multimodal Models (LMMs) have become a pivotal research focus in deep learning, demonstrating remarkable capabilities in 3D scene understanding. However, current 3D LMMs employing thousands of spatial tokens for multimodal reasoning suffer from critical inefficiencies: excessive computational overhead and redundant information flows. Unlike 2D VLMs processing single images, 3D LMMs exhibit inherent architectural redundancy due to the heterogeneous mechanisms between spatial tokens and visual tokens. To address this challenge, we propose AdaToken-3D, an adaptive spatial token optimization framework that dynamically prunes redundant tokens through spatial contribution analysis. Our method automatically tailors pruning strategies to different 3D LMM architectures by quantifying token-level information flows via attention pattern mining. Extensive experiments on LLaVA-3D (a 7B parameter 3D-LMM) demonstrate that AdaToken-3D achieves 21\% faster inference speed and 63\% FLOPs reduction while maintaining original task accuracy. Beyond efficiency gains, this work systematically investigates redundancy patterns in multimodal spatial information flows through quantitative token interaction analysis. Our findings reveal that over 60\% of spatial tokens contribute minimally ($<$5\%) to the final predictions, establishing theoretical foundations for efficient 3D multimodal learning.

Figures

Figures reproduced from arXiv: 2505.12782 by the authors.

Figure 1
Figure 1. Overview of Spatial tokens in 3D LMMs. Block (A) demonstrates that LLaVA-3D is injected with 3D patches to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Observations about Spatial Information Flow in 3D [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. AdaToken-3D Pruning Strategy. We extract information in various forms using modal characteristics, and layer by [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 13 canonical work pages

  1. [1]

    From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks

    Zhang, Xiaofeng et al. “From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks.” (2024)

    work page 2024

  2. [2]

    An Image is Worth 1/2 Tokens After Layer 2: Plug- and-Play Inference Acceleration for Large Vision-Language Models

    Chen, Liang et al. “An Image is Worth 1/2 Tokens After Layer 2: Plug- and-Play Inference Acceleration for Large Vision-Language Models.” European Conference on Computer Vision (2024)

    work page 2024

  3. [3]

    Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning

    Fu, Rao et al. “Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning.” ArXiv abs/2403.11401 (2024): n. pag

    arXiv 2024

  4. [4]

    MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices

    Chu, Xiangxiang et al. “MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices.” ArXiv abs/2312.16886 (2023): n. pag

    arXiv 2023

  5. [5]

    Chat-Scene: Bridging 3D Scene and Large Lan- guage Models with Object Identifiers

    Huang, Haifeng et al. “Chat-Scene: Bridging 3D Scene and Large Lan- guage Models with Object Identifiers.” Neural Information Processing Systems (2023)

    work page 2023

  6. [7]

    DeepSeek-VL: Towards Real-World Vision-Language Understanding

    Lu, Haoyu et al. “DeepSeek-VL: Towards Real-World Vision-Language Understanding.” ArXiv abs/2403.05525 (2024): n. pag

    arXiv 2024

  7. [8]

    GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

    Qi, Zhangyang et al. “GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models.” ArXiv abs/2501.01428 (2025): n. pag

    arXiv 2025

  8. [9]

    LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning

    Chen, Sijin et al. “LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning.” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023): 26418-26428

    work page 2023

Show all 30 references
  1. [10]

    PARIS3D: Reasoning-based 3D Part Segmenta- tion Using Large Multimodal Model

    Kareem, Amrin et al. “PARIS3D: Reasoning-based 3D Part Segmenta- tion Using Large Multimodal Model.” ArXiv abs/2404.03836 (2024): n. pag

    2024 arXiv

  2. [11]

    Seeing Clearly by Layer Two: Enhanc- ing Attention Heads to Alleviate Hallucination in LVLMs

    Zhang, Xiaofeng et al. “Seeing Clearly by Layer Two: Enhanc- ing Attention Heads to Alleviate Hallucination in LVLMs.” ArXiv abs/2411.09968 (2024): n. pag

    2024 arXiv

  3. [12]

    PyramidDrop: Accelerating Your Large Vision- Language Models via Pyramid Visual Redundancy Reduction

    Xing, Long et al. “PyramidDrop: Accelerating Your Large Vision- Language Models via Pyramid Visual Redundancy Reduction.” ArXiv abs/2410.17247 (2024): n. pag

    2024 arXiv

  4. [13]

    LLaV A-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models

    Li, Feng et al. “LLaV A-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models.” ArXiv abs/2407.07895 (2024): n. pag

    2024 arXiv

  5. [14]

    DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification

    Rao, Yongming et al. “DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification.” ArXiv abs/2106.02034 (2021): n. pag

    2021 arXiv

  6. [15]

    AdaViT: Adaptive Vision Transformers for Ef- ficient Image Recognition

    Meng, Lingchen et al. “AdaViT: Adaptive Vision Transformers for Ef- ficient Image Recognition.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 12299-12308

    2021

  7. [16]

    LLaV A-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness

    Zhu, Chenming et al. “LLaV A-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness.” ArXiv abs/2409.18125 (2024): n. pag

    2024 arXiv

  8. [17]

    Visual Instruction Tuning

    Liu, Haotian et al. “Visual Instruction Tuning.” ArXiv abs/2304.08485 (2023): n. pag

    2023 arXiv

  9. [18]

    SQA3D: Situated Question Answering in 3D Scenes

    Ma, Xiaojian et al. “SQA3D: Situated Question Answering in 3D Scenes.” ArXiv abs/2210.07474 (2022): n. pag

    2022 arXiv

  10. [19]

    ScanQA: 3D Question Answering for Spatial Scene Understanding

    Azuma, Daich et al. “ScanQA: 3D Question Answering for Spatial Scene Understanding.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 19107-19117

    2021

  11. [20]

    Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

    Chen, Zhe et al. “Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling.” ArXiv abs/2412.05271 (2024): n. pag

    2024 arXiv

  12. [21]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    Bai, Jinze et al. “Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond.” (2023)

    2023

  13. [22]

    ScanRefer: 3D Object Localization in RGB- D Scans using Natural Language

    Chen, Dave Zhenyu et al. “ScanRefer: 3D Object Localization in RGB- D Scans using Natural Language.” ArXiv abs/1912.08830 (2019): n. pag

    2019 arXiv

  14. [23]

    Scan2Cap: Context-aware Dense Captioning in RGB-D Scans

    Chen, Dave Zhenyu et al. “Scan2Cap: Context-aware Dense Captioning in RGB-D Scans.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020): 3192-3202

    2020

  15. [24]

    MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities

    Yu, Weihao et al. “MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities.” ArXiv abs/2308.02490 (2023): n. pag

    2023 arXiv

  16. [25]

    MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

    Fu, Chaoyou et al. “MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models.” ArXiv abs/2306.13394 (2023): n. pag

    2023 arXiv

  17. [26]

    MMBench: Is Your Multi-modal Model an All-around Player?

    Liu, Yuanzhan et al. “MMBench: Is Your Multi-modal Model an All-around Player?” European Conference on Computer Vision (2023)

    2023

  18. [27]

    Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

    Goyal, Yash et al. “Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering.” International Journal of Computer Vision 127 (2016): 398 - 414

    2016

  19. [28]

    Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs

    Kassem, Aly M. et al. “Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs.” ArXiv abs/2403.04801 (2024): n. pag

    2024 arXiv

  20. [29]

    ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

    Dai, Angela et al. “ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017): 2432-2443

    2017

  21. [30]

    GPT-4 Technical Report

    Achiam, OpenAI Josh et al. “GPT-4 Technical Report.” (2023)

    2023

  22. [31]

    An Embodied Generalist Agent in 3D World

    Huang, Jiangyong et al. “An Embodied Generalist Agent in 3D World.” ArXiv abs/2311.12871 (2023): n. pag

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.