Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a dynamic-resolution voxelization scheme, guided by seven hand-designed complexity metrics, cuts point-cloud feature-extraction time by over a third while improving reconstruction accuracy, and that a token-level…

desk verdict Competent incremental system paper whose speedup claim is plausible and honestly reported, but whose accuracy gains rest partly on circular voxel metrics and missing variance. read the letter →

arxiv 2507.20110 v1 pith:J4Z633WN submitted 2025-07-27 cs.CV cs.AIcs.LG

Shiyu Liu , Lianlei Shan This is my paper
classification cs.CVcs.AIcs.LG
keywords 3DperceptionpointcloudvoxelizationdynamicresolutionNeuralRadianceFieldsmeta-embeddingattentionpoolinglanguage-alignedunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NeuroVoxel-LM is a framework for language-aligned 3D perception that targets the inefficiency of processing sparse, large-scale point clouds. Its first proposal, DR-MSV, starts with a fixed voxel grid, flags each voxel as complex or non-complex using seven geometric and structural metrics, and iteratively merges 2x2x2 blocks of non-complex voxels into a multiresolution pyramid; on ShapeNet this cuts total training time from 19.87 to 12.24 hours while improving Chamfer distance from 0.195 to 0.150 and voxel IoU from 0.45 to 0.56. Its second proposal, TAP-LME, replaces max-pooling over NeRF-weight tokens with an attention-weighted sum fused to max-pooling through a learnable scalar; on ShapeNeRF-Text the learnable-fusion variant, TAP-Res (Learnt), achieves the top S-BERT and SimCSE scores for both short and detailed heading generation. A sympathetic reader would take the paper to be claiming that a cheap, complexity-driven variable-resolution grid and a lightweight adaptive pooling layer can make 3D-language systems faster and at least slightly more semantically accurate.

What carries the argument

The argument runs on two mechanisms. DR-MSV (Dynamic Resolution Multiscale Voxelization) is a complexity-driven voxel pyramid builder: it measures dot density, surface roughness, normal coherence, PCA linearity, PCA planarity, spatial entropy, and curvature in each initial voxel, classifies a voxel as complex when its metric crosses the 75th percentile of the scene's distribution, and iteratively merges 2x2x2 blocks whose eight children are all non-complex. TAP-LME (Token-level Adaptive Pooling for Lightweight Meta-Embedding) is a pooling layer that assigns each token an attention weight via a shared MLP and a learned vector, computes a weighted sum, and combines that sum with max-pooling through a trainable scalar lambda. The first mechanism carries the efficiency-and-accuracy claim for point clouds; the second carries the semantic-embedding claim for NeRF weights.

What would settle it

On the same ShapeNet setup, give the fixed-resolution baseline the same total training time as DR-MSV and compare Chamfer distance and voxel IoU; if the baseline then matches or exceeds DR-MSV, the claimed advantage is an artifact of unequal compute. Separately, re-run DR-MSV with a different percentile threshold or after removing each of the seven metrics and watch reconstruction error; a large swing would show the merging decision, not the pyramid idea, is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that feature extraction from point clouds and from NeRF weights can both be improved by replacing fixed operations with adaptive ones. For point clouds, the paper argues that a fixed-resolution voxel grid wastes computation on simple regions; its DR-MSV method classifies each voxel by whether it exceeds the 75th percentile on seven metrics, then merges entire 2x2x2 blocks only when all eight children are non-complex, producing a pyramid of voxel sizes that is coarse where the geometry is simple and fine where it is not. This is claimed to be what produces the reported speed-up and the better Chamfer distance, F1 scores, and voxel IoU. For NeRF weights, the paper argues that max-pooling over tokens discards the relative importance of individual substructures; its TAP-LME method computes a softmax attention weight for each token through a shared MLP, forms an attention-weighted sum, and fuses it with max-pooling using a trainable scalar lambda. The learned-fusion version is claimed to outperform both pure max-pooling and fixed-ratio fusion on semantic similarity and generation metrics.

Load-bearing premise

The whole speed-up rests on the premise that seven hand-picked geometric metrics combined with one 75th-percentile cutoff correctly identify which voxel regions can be merged without losing information that reconstruction needs; if that classification is wrong, the method's gain is simply discarded detail.

Editorial extensions

If this is right

  • On ShapeNet, DR-MSV lowers total training time from 19.87 to 12.24 hours while improving Chamfer distance from 0.195 to 0.150 and voxel IoU from 0.45 to 0.56, making real-time point-cloud processing more feasible at comparable reconstruction budget.
  • Ablations that remove multiscale fusion (DR-SV-MC) or reduce the task to binary classification (DR-MSV-BC) run faster but lose accuracy, indicating that both the multiscale pyramid and multiclass supervision contribute to the reported gain.
  • On ShapeNeRF-Text, replacing max-pooling with TAP-LME improves S-BERT and SimCSE scores for both short and detailed heading generation, with the learnable-fusion variant performing best.
  • A learnable fusion coefficient removes manual tuning of the balance between attention pooling and max-pooling, letting the model allocate between the two during training.
  • The TAP-Weight-only variant, which sets attention weights to the mean, underperforms the true attention-weighted version, so the learned attention mechanism itself is claimed to be responsible for the improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper compares DR-MSV only against fixed-resolution voxelization; testing it against other adaptive voxelization schemes would place the efficiency claim in a broader context that the paper does not supply.
  • Because the S-BERT and SimCSE gains are under one point, a natural next test is human evaluation of the generated headings; the paper reports only automated metrics.
  • The 75th-percentile rule is data-driven and may transfer to new datasets without retuning, but the paper does not demonstrate that transfer, leaving it an open extension.
  • The dynamic grid is built from purely geometric cues; a learned merging policy optimized end-to-end against the reconstruction loss could reveal how much of the hand-crafted metric set is actually necessary.
  • The conclusion names the framework 'LLaNA-DRVNet' while the title and abstract call it 'NeuroVoxel-LM', so the exact identity of the proposed system is left ambiguous.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes NeuroVoxel-LM, a framework that combines dynamic-resolution multiscale voxelization (DR-MSV) with a token-level adaptive pooling mechanism (TAP-LME) for language-aligned 3D perception. DR-MSV computes seven hand-crafted complexity metrics per voxel, thresholds them at the 75th percentile, and iteratively merges 2x2x2 blocks of non-complex voxels into coarser voxels. TAP-LME replaces pure max-pooling over NeRF-derived token embeddings with an attention-weighted aggregation fused with max-pooling via a learnable scalar. Experiments on ShapeNet report that DR-MSV reduces total training time from 19.87 to 12.24 hours while improving Chamfer Distance from 0.195 to 0.150 and voxel IoU from 0.45 to 0.56 relative to a fixed-resolution baseline. Experiments on ShapeNeRF-Text report that TAP-LME variants slightly improve S-BERT, SimCSE, BLEU, ROUGE-L, and METEOR over the LLaNA-7b baseline. The central claims are that DR-MSV improves feature-extraction efficiency and accuracy and that TAP-LME captures finer-grained semantics than conventional max-pooling.

Significance. The ideas are relevant and the proposed modules are simple enough to be reproducible: the complexity-metric formulation is concrete, the merging rule is explicit, and the pooling mechanism is lightweight. If the reported gains are robust, DR-MSV would be a useful efficiency improvement for voxel-based point-cloud processing, and TAP-LME a modest but generally applicable enhancement to NeRF-weight embedding. However, the significance is currently limited by evaluation gaps: the voxel-level accuracy metrics may be scored against labels derived from the method's own complexity thresholds, the non-circular reconstruction metrics lack error bars and external baselines, and the TAP-LME gains are small and unreplicated. The paper also does not compare against existing adaptive or octree-based voxelization methods, so the claimed state-of-the-art positioning is not established.

major comments (4)
  1. [Section 4.2.1, Tables 1-2] The paper never defines the ground-truth labels used to compute Acc, Prec, Rec, F1(Vox), and VoxIoU. Section 4.2.1 states that DR-MSV 'predicts multiclass complexity labels,' and those labels appear to be constructed from the same hand-crafted indicators and 75th-percentile thresholds (Eqs. 8-9) that Section 3.1 uses to decide which 2x2x2 blocks to merge. If the voxel metrics are scored against these metric-derived labels, the reported gains partly measure agreement with the method's own complexity prior rather than recovered geometry. Please specify the label-generation protocol (e.g., occupancy derived from the CAD mesh) and report CD, F1(PC), and GeoIoU with error bars over multiple seeds and train/test splits.
  2. [Section 3.1, Eqs. (8)-(9)] The set-builder notation 'S_complex = {∀m ∈ {d, σs, cos θ, λlinear, λplanar, Hs, κ}, m ≥ τm}' is ambiguous: read literally, it requires every metric to exceed its threshold, whereas the prose says 'if its metric exceeds' the threshold, and the caption of Figure 2 suggests a more permissive criterion. The logical combination of the seven indicators changes which voxels are classified as non-complex, hence which 2x2x2 blocks are merged and the reported speed/accuracy trade-off in Tables 1-2. Please state the exact aggregation rule (e.g., AND, OR, or majority voting) and correct the text, which says 'six indicators' while listing seven metrics.
  3. [Section 4.1.2, Tables 3-4] The TAP-LME gains over the LLaNA-7b baseline are small (S-BERT +0.47 on short headings and +0.90 on detailed headings, with differences of a similar size between TAP-Res variants), and the paper itself concedes that 'the improvement in indicators is minimal.' No error bars, confidence intervals, or statistical significance tests are reported, and no information is given about the number of repeated runs or the variance across test samples. Please add such analyses; without them, the claim that TAP-LME 'outperforms' max-pooling is not supported beyond a single run.
  4. [Section 4.1.1, Tables 1-2] The efficiency and accuracy comparison is conducted only against a fixed-resolution baseline (FRV/FPV) trained with Focal Loss. Because the central claim is that dynamic adaptive voxelization improves accuracy, the comparison should include an existing adaptive or octree-based voxelization baseline, or at least a fixed-resolution baseline matched for compute and model capacity. Without such a baseline, the reported improvements in CD, F1(PC), and GeoIoU may reflect the weakness of the specific fixed-resolution SIREN baseline rather than a general advantage of the proposed method.
minor comments (5)
  1. [Section 5] The conclusion refers to the framework as 'LLaNA-DRVNet,' whereas the title and abstract use 'NeuroVoxel-LM'; please unify the name throughout.
  2. [Section 3.1] The phrase 'the following six indicators' should read 'seven indicators,' since Eqs. (1)-(7) list dot density, surface roughness, normal coherence, linearity, planarity, spatial entropy, and curvature.
  3. [References] Reference [58] appears twice in the citation list [58, 1, 20, 58, 21] in Section 2; please remove the duplicate citation.
  4. [Section 4.1.1 and Table 1] The baseline is called 'FRV' in the running text and 'FPV' in Table 1; please use a single acronym consistently.
  5. [Section 4.2.2 and Eqs. (10)-(12)] The fixed fusion ratio is written 'α = 0.5' in the text but the fusion coefficient in Eq. (12) is λ; please use one symbol consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical benchmark comparisons, not derived quantities that reduce to their inputs.

full rationale

The central claims are empirical: DR-MSV is compared with a fixed-resolution baseline (FRV/FPV) on ShapeNet using training time, Chamfer distance, point-cloud F1, geometric IoU, and voxel-level metrics, while TAP-LME is compared with max-pooling baselines on ShapeNeRF-Text using S-BERT, SimCSE, BLEU, ROUGE-L, and METEOR. No equation in the paper defines a predicted success metric in terms of the same threshold or fitted parameter that is then reported as an independent improvement. The complexity indicators and the 75th-percentile threshold in Eqs. (1)-(9) are internal design choices of the proposed method, not free parameters fitted to a subset of data and then relabeled as predictions. The ablation variants (DR-SV-MC, DR-MSV-BC, TAP-only, TAP-Res, TAP-Weight-only) are compared against baselines and against one another, so module contributions are not forced by construction. No load-bearing argument relies on a self-citation: the main baseline LLaNA is by different authors, and no uniqueness theorem or prior-work authority is used to rule out alternatives. The underspecified voxel-label ground truth and the absence of error bars are reproducibility and evidence-quality concerns, not circularity. Accordingly, no circular step can be exhibited from the manuscript text, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on learned thresholds and hand-designed complexity metrics, plus inherited assumptions about NeRF weight semantics and baseline validity. No new physical entities are introduced.

free parameters (2)
  • 75th percentile complexity threshold = 75th percentile of each metric across the training scene
    Eqs. (8)-(9) define complex vs non-complex voxels by comparing each of seven metrics to a threshold; the paper sets this adaptively using the 75th percentile of the training data. The choice of percentile is a hyperparameter that directly controls how many voxels get merged, so it affects both speed and accuracy claims.
  • Initial voxel grid resolution = 16x16x16 (example; exact value used in experiments not stated clearly)
    Section 3.1 says the point cloud is divided into a fixed-size voxel grid (e.g., 16x16x16). The actual value used in experiments is not reported, and this coarse/fine granularity affects the accuracy/speed tradeoff.
assumptions (4)
  • domain assumption NeRF MLP weights contain sufficient semantic information for language-aligned captioning and QA after encoding through an nf2vec-style meta-encoder.
    The whole TAP-LME component operates on token features extracted from NeRF weights; if these features lack semantic information, the pooling mechanism cannot recover it. This assumption is inherited from LLaNA and is not tested independently.
  • ad hoc to paper The seven hand-crafted complexity metrics (density, roughness, normal coherence, linearity, planarity, entropy, curvature) are sufficient to identify regions where voxel merging is lossless.
    Section 3.1 defines complex/non-complex regions using these metrics and a 75th percentile threshold. There is no analysis showing that these metrics capture all structure relevant to reconstruction; this is the load-bearing design choice for DR-MSV.
  • domain assumption The ShapeNeRF-Text benchmark and its Sentence-BERT/SimCSE metrics measure meaningful caption quality.
    Tables 3 and 4 evaluate on this benchmark; the paper does not discuss whether the small S-BERT/SimCSE gains are perceptible to human readers or statistically reliable.
  • domain assumption The FPV baseline is a faithful and representative fixed-resolution voxelization method.
    The accuracy and efficiency gains of DR-MSV are relative to this baseline; if the baseline is weak or mis-implemented, the gains are overstated. No code or detailed baseline configuration is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding." pith.science (2026). https://pith.science/paper/J4Z633WN

@misc{pith2026250720110,
  author       = {Pith},
  title        = {Pith review of: NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J4Z633WN}},
  note         = {Machine review of arXiv:2507.20110}
}
read the original abstract

Recent breakthroughs in Visual Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have significantly advanced 3D scene perception towards language-driven cognition. However, existing 3D language models struggle with sparse, large-scale point clouds due to slow feature extraction and limited representation accuracy. To address these challenges, we propose NeuroVoxel-LM, a novel framework that integrates Neural Radiance Fields (NeRF) with dynamic resolution voxelization and lightweight meta-embedding. Specifically, we introduce a Dynamic Resolution Multiscale Voxelization (DR-MSV) technique that adaptively adjusts voxel granularity based on geometric and structural complexity, reducing computational cost while preserving reconstruction fidelity. In addition, we propose the Token-level Adaptive Pooling for Lightweight Meta-Embedding (TAP-LME) mechanism, which enhances semantic representation through attention-based weighting and residual fusion. Experimental results demonstrate that DR-MSV significantly improves point cloud feature extraction efficiency and accuracy, while TAP-LME outperforms conventional max-pooling in capturing fine-grained semantics from NeRF weights.

Figures

Figures reproduced from arXiv: 2507.20110 by the authors.

Figure 1
Figure 1. Overall architecture of NeuroVoxel-LM. This section discusses the proposed Neural Voxel-based Multimodal Language Model (NeuroVoxel￾LM) (shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. DR-MSV methods. After voxel processing of the input 3D point cloud data, the complexity of voxel units is determined based on key metrics, followed by iterative merging of non-complex regions. During the initialisation stage, the input point cloud is divided into a fixed-size voxel grid (e.g., 16 × 16 × 16) to create a fine-grained geometric representation. Then, each voxel cell in the initialised voxel network is d… view at source ↗
Figure 3
Figure 3. ), is proposed. By assigning learnable attentional weights to each token, TAP-LME improves the understanding of fine-grained meta-embedding by keeping it lightweight. Assignment of token weights (Token-by-token operations) … … Max Pooling Attention-based Aggregation Token Sequences ( ) dot product of vectors Residual Fusion Results Shared MLP [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

    cs.CV 2026-07 reject novelty 4.0 of 10

    A multi-scale gated fusion plus question-conditioned token router is specified to cut VLM visual tokens ~80% while improving accuracy, but all reported gains are simulated placeholders.

Reference graph

Works this paper leans on

73 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. In 9 Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 422–440. Springer, 2020

    work page 2020

  3. [3]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems , 35:23716–23736, 2022

    2022

  4. [4]

    ShapeNeRF–Text: A Dataset of Neural Object Renderings with Rich Language Descriptions

    Andrea Amaduzzi, Pierluigi Zama Ramirez, Giuseppe Lisanti, Samuele Salti, and Luigi Di Stefano. ShapeNeRF–Text: A Dataset of Neural Object Renderings with Rich Language Descriptions. https://service.tib.eu/ldmservice/dataset/shapenerf-text, 2024. Accessed: 2025-05-16

    work page 2024

  5. [5]

    Llana: Large language and nerf assistant

    Andrea Amaduzzi, Pierluigi Zama Ramirez, Giuseppe Lisanti, Samuele Salti, and Luigi Di Ste- fano. Llana: Large language and nerf assistant. Advances in Neural Information Processing Systems, 37:1162–1195, 2024

    work page 2024

  6. [6]

    Scanqa: 3d question answering for spatial scene understanding

    Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19129–19139, 2022

    work page 2022

  7. [7]

    Componerf: Text-guided multi-object compositional nerf with editable 3d scene layout

    Haotian Bai, Yuanhuiyi Lyu, Lutao Jiang, Sijia Li, Haonan Lu, Xiaodong Lin, and Lin Wang. Componerf: Text-guided multi-object compositional nerf with editable 3d scene layout. arXiv preprint arXiv:2303.13843, 2023

    arXiv 2023

  8. [8]

    Connecting nerfs images and text

    Francesco Ballerini, Pierluigi Zama Ramirez, Roberto Mirabella, Samuele Salti, and Luigi Di Stefano. Connecting nerfs images and text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 866–876, 2024

    work page 2024

Show all 73 references
  1. [9]

    Neural processing of tri-plane hybrid neural fields

    Adriano Cardace, Pierluigi Zama Ramirez, Francesco Ballerini, Allan Zhou, Samuele Salti, and Luigi Di Stefano. Neural processing of tri-plane hybrid neural fields. arXiv preprint arXiv:2310.01140, 2023

    2023 arXiv

  2. [10]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015

    2015 arXiv

  3. [11]

    Tensorf: Tensorial radiance fields

    Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European conference on computer vision, pages 333–350. Springer, 2022

    2022

  4. [12]

    Scanrefer: 3d object localization in rgb-d scans using natural language

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, pages 202–221. Springer, 2020

    2020

  5. [13]

    Videollm: Modeling video sequence with large language models

    Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, et al. Videollm: Modeling video sequence with large language models. arXiv preprint arXiv:2305.13292, 2023

    2023 arXiv

  6. [14]

    Minigpt-v2: large language model as a unified interface for vision-language multi-task learning

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310...

    arXiv

  7. [15]

    Language conditioned spatial relation reasoning for 3d object grounding

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Language conditioned spatial relation reasoning for 3d object grounding. Advances in neural information processing systems, 35:20522–20535, 2022

    2022

  8. [16]

    Mvt: Multi-view vision transformer for 3d object recognition

    Shuo Chen, Tan Yu, and Ping Li. Mvt: Multi-view vision transformer for 3d object recognition. arXiv preprint arXiv:2110.13083, 2021

    2021 arXiv

  9. [17]

    End-to-end 3d dense captioning with vote2cap-detr

    Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Gang Yu, and Tao Chen. End-to-end 3d dense captioning with vote2cap-detr. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11124–11133, 2023. 10

    2023

  10. [18]

    V ote2cap-detr++: Decoupling localization and describing for end-to-end 3d dense captioning

    Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, and Tao Chen. V ote2cap-detr++: Decoupling localization and describing for end-to-end 3d dense captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(11):7331– ...

    2024

  11. [19]

    Scan2cap: Context-aware dense captioning in rgb-d scans

    Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang. Scan2cap: Context-aware dense captioning in rgb-d scans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3193–3203, 2021

    2021

  12. [20]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023...

    2023

  13. [21]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1– 113, 2023

    2023

  14. [22]

    4d spatio-temporal convnets: Minkowski convolutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3075–3084, 2019

    2019

  15. [23]

    Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    2023

  16. [24]

    Deep learning on implicit neural representations of shapes

    Luca De Luigi, Adriano Cardace, Riccardo Spezialetti, Pierluigi Zama Ramirez, Samuele Salti, and Luigi Di Stefano. Deep learning on implicit neural representations of shapes. arXiv preprint arXiv:2302.05438, 2023

    2023 arXiv

  17. [25]

    Plenoxels: Radiance fields without neural networks

    Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5501–5510, 2022

    2022

  18. [26]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15180–15190, 2023

    2023

  19. [27]

    Submanifold sparse convolutional networks

    Benjamin Graham and Laurens Van der Maaten. Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307, 2017

    2017 arXiv

  20. [28]

    Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following. arXiv preprint arX...

    2023 arXiv

  21. [29]

    Imagebind-llm: Multi-modality instruction tuning

    Jiaming Han, Renrui Zhang, Wenqi Shao, Peng Gao, Peng Xu, Han Xiao, Kaipeng Zhang, Chris Liu, Song Wen, Ziyu Guo, et al. Imagebind-llm: Multi-modality instruction tuning. arXiv preprint arXiv:2309.03905, 2023

    2023 arXiv

  22. [30]

    3d-llm: Injecting the 3d world into large language models

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482–20494, 2023

    2023

  23. [31]

    Unlocking textual and visual wisdom: Open-vocabulary 3d object detection enhanced by comprehensive guidance from text and image

    Pengkun Jiao, Na Zhao, Jingjing Chen, and Yu-Gang Jiang. Unlocking textual and visual wisdom: Open-vocabulary 3d object detection enhanced by comprehensive guidance from text and image. In European Conference on Computer Vision, pages 376–392. Springer, 2024

    2024

  24. [32]

    Cg-nerf: Conditional generative neural radiance fields for 3d-aware image synthesis

    Kyungmin Jo, Gyumin Shim, Sanghun Jung, Soyoung Yang, and Jaegul Choo. Cg-nerf: Conditional generative neural radiance fields for 3d-aware image synthesis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 724–733, 2023. 11

    2023

  25. [33]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19729–19739, 2023

    2023

  26. [34]

    Parameter prediction for unseen deep architectures

    Boris Knyazev, Michal Drozdzal, Graham W Taylor, and Adriana Romero Soriano. Parameter prediction for unseen deep architectures. Advances in Neural Information Processing Systems , 34:29433–29448, 2021

    2021

  27. [35]

    Decomposing nerf for editing via feature field distillation

    Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitzmann. Decomposing nerf for editing via feature field distillation. Advances in neural information processing systems , 35:23311–23330, 2022

    2022

  28. [36]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  29. [37]

    Visual instruction tuning.Advances in neural information processing systems , 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems , 36:34892–34916, 2023

    2023

  30. [38]

    Sqa3d: Situated question answering in 3d scenes

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint arXiv:2210.07474, 2022

    2022 arXiv

  31. [39]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12663–12673, 2023

    2023

  32. [40]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021

    2021

  33. [41]

    Reference-guided controllable inpainting of neural radiance fields

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G Derpanis, and Igor Gilitschenski. Reference-guided controllable inpainting of neural radiance fields. In Proceedings of the IEEE/CVF international conference on com...

    2023

  34. [42]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1– 15, 2022

    2022

  35. [43]

    Equivariant architectures for learning in deep weight spaces

    Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, and Haggai Maron. Equivariant architectures for learning in deep weight spaces. In International Conference on Machine Learning, pages 25790–25816. PMLR, 2023

    2023

  36. [44]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:277...

    2022

  37. [45]

    Masked autoencoders for point cloud self-supervised learning

    Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud self-supervised learning. In European conference on computer vision, pages 604–621. Springer, 2022

    2022

  38. [46]

    Clip-guided vision-language pre-training for question answering in 3d scenes

    Maria Parelli, Alexandros Delitzas, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, and Thomas Hofmann. Clip-guided vision-language pre-training for question answering in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2023

  39. [47]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017. 12

    2017

  40. [48]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017

    2017

  41. [49]

    Gpt4point: A unified framework for point-language understanding and generation

    Zhangyang Qi, Ye Fang, Zeyi Sun, Xiaoyang Wu, Tong Wu, Jiaqi Wang, Dahua Lin, and Hengshuang Zhao. Gpt4point: A unified framework for point-language understanding and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26417–...

    2024

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  43. [51]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  44. [52]

    Deep learning on 3d neural fields

    Pierluigi Zama Ramirez, Luca De Luigi, Daniele Sirocchi, Adriano Cardace, Riccardo Spezialetti, Francesco Ballerini, Samuele Salti, and Luigi Di Stefano. Deep learning on 3d neural fields. CoRR, 2023

    2023

  45. [53]

    Self-supervised representation learning on neural network weights for model characteristic prediction

    Konstantin Schürholt, Dimche Kostadinov, and Damian Borth. Self-supervised representation learning on neural network weights for model characteristic prediction. Advances in Neural Information Processing Systems, 34:16481–16493, 2021

    2021

  46. [54]

    Hyp-nerf: Learning improved nerf priors using a hypernetwork

    Bipasha Sen, Gaurav Singh, Aditya Agarwal, Rohith Agaram, Madhava Krishna, and Srinath Sridhar. Hyp-nerf: Learning improved nerf priors using a hypernetwork. Advances in Neural Information Processing Systems, 36:51050–51064, 2023

    2023

  47. [55]

    Ditto-nerf: Diffusion-based iterative text to omni-directional 3d model

    Hoigi Seo, Hayeon Kim, Gwanghyun Kim, and Se Young Chun. Ditto-nerf: Diffusion-based iterative text to omni-directional 3d model. arXiv preprint arXiv:2304.02827, 2023

    2023 arXiv

  48. [56]

    Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction

    Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5459–5469, 2022

    2022

  49. [57]

    Nerfeditor: Differentiable style decomposition for 3d scene editing

    Chunyi Sun, Yanbin Liu, Junlin Han, and Stephen Gould. Nerfeditor: Differentiable style decomposition for 3d scene editing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 7306–7315, 2024

    2024

  50. [58]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  51. [59]

    Predicting neural network accuracy from weights

    Thomas Unterthiner, Daniel Keysers, Sylvain Gelly, Olivier Bousquet, and Ilya Tolstikhin. Predicting neural network accuracy from weights. arXiv preprint arXiv:2002.11448, 2020

    2002 arXiv

  52. [60]

    Nerf-art: Text-driven neural radiance fields stylization

    Can Wang, Ruixiang Jiang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Nerf-art: Text-driven neural radiance fields stylization. IEEE Transactions on Visualization and Computer Graphics, 2023

    2023

  53. [61]

    Sparsenerf: Distilling depth ranking for few-shot novel view synthesis

    Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9065–9076, 2023

    2023

  54. [62]

    Is attention all that nerf needs? arXiv preprint arXiv:2207.13298, 2022

    Peihao Wang, Xuxi Chen, Tianlong Chen, Subhashini Venugopalan, Zhangyang Wang, et al. Is attention all that nerf needs? arXiv preprint arXiv:2207.13298, 2022

    2022 arXiv

  55. [63]

    Dynamic graph cnn for learning on point clouds

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1–12, 2019. 13

    2019

  56. [64]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  57. [65]

    Pointllm: Empowering large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In European Conference on Computer Vision, pages 131–147. Springer, 2024

    2024

  58. [66]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference on c...

    2023

  59. [67]

    Ulip-2: Towards scalable multimodal pre-training for 3d understanding

    Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scalable multimodal pre-training for 3d understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision...

    2024

  60. [68]

    3d question answering

    Shuquan Ye, Dongdong Chen, Songfang Han, and Jing Liao. 3d question answering. IEEE Transactions on Visualization and Computer Graphics, 30(3):1772–1786, 2022

    2022

  61. [69]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19313–19322, 2022

    2022

  62. [70]

    X-trans2cap: Cross-modal knowledge transfer using transformer for 3d dense captioning

    Zhihao Yuan, Xu Yan, Yinghong Liao, Yao Guo, Guanbin Li, Shuguang Cui, and Zhen Li. X-trans2cap: Cross-modal knowledge transfer using transformer for 3d dense captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8563–8573, 2022

    2022

  63. [71]

    Llama-adapter: Efficient fine-tuning of language models with zero-init attention

    Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023

    2023 arXiv

  64. [72]

    Multi3drefer: Grounding text description to multiple 3d objects

    Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drefer: Grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15225–15236, 2023

    2023

  65. [73]

    Toward explainable 3d grounded visual question answering: A new bench- mark and strong baseline

    Lichen Zhao, Daigang Cai, Jing Zhang, Lu Sheng, Dong Xu, Rui Zheng, Yinjie Zhao, Lipeng Wang, and Xibo Fan. Toward explainable 3d grounded visual question answering: A new bench- mark and strong baseline. IEEE Transactions on Circuits and Systems for Video Technology, 33(6):29...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.