Pith. sign in

REVIEW 4 major objections 8 minor 2 cited by

Laser: Efficient Language-Guided Segmentation in Neural Radiance Fields

T0 review · 4 major / 8 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Laser distills dense pixel-level CLIP features into a NeRF to segment 3D scenes from text, cutting training from roughly 158 to about 11 minutes per scene while matching or beating prior accuracy.

desk verdict Solid efficiency win for language-guided 3D segmentation, but the performance claim is thinner than the abstract implies and needs error bars. read the letter →

arxiv 2501.19084 v1 pith:VVKS25NT submitted 2025-01-31 cs.CV

Xingyu Miao , Haoran Duan , Yang Bai , Tejal Shah , Jun Song , Yang Long , Rajiv Ranjan , Ling Shao This is my paper · ORCID
classification cs.CV
keywords 3Dsemanticsegmentationlanguage-guidedneuralradiancefieldsCLIPfeaturedistillationdensepixel-levelfeaturesopen-vocabularyefficienttrainingNeRF
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that language-guided 3D segmentation does not require the multi-scale CLIP patch features and extra DINO features used by prior NeRF-based methods. It proposes Laser, which extracts dense per-pixel CLIP features from a frozen, lightly modified CLIP encoder and distills them directly into a NeRF segmentation branch, backed by four lightweight mechanisms that clean noise, sharpen edges, stabilize segmentation across views, and reduce text-image ambiguity. On the 3D-OVS benchmark, Laser reports an average mIoU of 88.1 and accuracy of 97.3, compared with 87.1 and 96.2 for 3D-OVS, while reducing per-scene training time from 158 minutes to about 11 minutes and keeping memory under 10 GB. If correct, the result makes open-vocabulary 3D segmentation practical on a single consumer GPU without large precomputed feature banks.

What carries the argument

The load-bearing device is the dense pixel-level CLIP feature stream from a frozen, lightly modified CLIP encoder, combined with a small adapter f_theta and a self-cross-training loss. This replaces the multi-scale patch cropping pipeline of earlier methods, which stored about 35 GB of features per scene, with a single dense feature map computed on the fly. Around this stream, the low-rank transient query attention factorizes the attention matrix through a small learnable query T, cutting complexity from O($S^{2}$D) to O(SsD), and the label volume converts segmentation into classification by rendering label vectors and optimizing a smoothed pseudo-label cross-entropy. The simplified text augmentation repeats each text prompt once and normalizes each class's relevance map to [0,1]. The paper's claim is that this combination is sufficient to make noisy dense CLIP features competitive with multi-scale features.

What would settle it

A decisive experiment would take a scene with known object masks and train Laser twice, once with dense features from a frozen CLIP encoder modified only at the final pooling layer and once with features from a CLIP encoder fine-tuned for dense prediction. If the paper's central claim is right, the frozen-dense version should remain competitive; if the bottleneck is feature quality, the fine-tuned version should substantially outperform it even when all downstream modules are identical.

Watch

Extended reading notes

Core claim

The central discovery is that dense pixel-level CLIP features, obtained by modifying the final pooling or attention layers of a frozen CLIP encoder, can serve as direct supervision for a NeRF segmentation field, provided the noise they contain is explicitly handled. The paper shows that an adapter with a residual connection plus a self-cross-training loss, in which rendered and reconstructed features supervise each other, reduces that noise; a low-rank transient query attention sharpens object edges while cutting attention complexity from O($S^{2}$D) to O(SsD); a label volume converts segmentation into a per-ray classification task to enforce cross-view consistency; and a simplified text augmentation technique, repeating each text prompt and normalizing each class relevance map, reduces ambiguity in CLIP text-image matching. Together these components allow end-to-end training in roughly 11 minutes per scene with under 10 GB memory, while achieving mIoU and accuracy scores above the multi-scale feature baselines on the evaluated benchmarks.

Load-bearing premise

The whole pipeline rests on the assumption that dense pixel-level CLIP features, extracted from a frozen CLIP encoder with its final pooling and attention layers altered, carry enough per-pixel semantic alignment with text to supervise accurate 3D segmentation; the paper itself notes these features are often noisy and less refined than multi-scale features.

Editorial extensions

If this is right

  • Per-scene training time for language-guided NeRF segmentation drops by roughly a factor of 14 relative to 3D-OVS, from 158 minutes to about 11 minutes, with memory under 10 GB.
  • Large precomputed multi-scale CLIP feature banks and DINO features are not necessary for accurate open-vocabulary 3D segmentation on the evaluated benchmarks.
  • Cross-view segmentation consistency can be improved by reformulating segmentation as a classification task with a label volume rather than by direct feature regression.
  • A low-rank transient query attention recovers most of vanilla self-attention's benefit on 3D point features at a fraction of the FLOPs and training time, about 1.48G versus 26.91G FLOPs at hidden dimension 32.
  • The ablation trajectory from 59.4 to 88.1 mIoU indicates that the adapter, self-cross-training, transient-query attention, label volume, and text augmentation each contribute independently to the final result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same dense-feature distillation recipe could transfer to 3D Gaussian splatting backbones; the paper notes its TensoRF base limits rendering speed, and the efficiency gains would likely be larger in a real-time renderer.
  • Because the method's accuracy is capped by the quality of dense pixel-level CLIP features, a testable improvement path is to fine-tune or adapt the CLIP encoder itself on segmentation data, something the paper explicitly says it cannot currently do.
  • The simplified text augmentation, repeating prompts and normalizing each class's relevance map, is a parameter-free technique that could reduce CLIP text-image ambiguity in other dense prediction tasks, such as 2D open-vocabulary segmentation, without requiring any architectural change.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes Laser, a language-guided 3D segmentation method for NeRF scenes. Instead of extracting multi-scale CLIP patch features as in LERF/3D-OVS, Laser uses dense pixel-level CLIP features from a modified frozen CLIP encoder and introduces four components: an adapter with self-cross-training to denoise dense features, a low-rank transient query attention for edge refinement, a label volume that recasts segmentation as classification, and a simplified text augmentation/normalization scheme. On the 3D-OVS benchmark the method reports about 11 minutes of training per scene versus 158 minutes for 3D-OVS, with average mIoU 88.1 vs. 87.1 and accuracy 97.3 vs. 96.2; it also reports higher mIoU on all evaluated LERF and Replica scenes. The code is open-sourced.

Significance. The efficiency result is genuinely valuable if it holds: replacing multi-scale CLIP feature extraction and DINO supervision with online dense CLIP features reduces per-scene training time by roughly an order of magnitude and cuts memory use. The component designs are clearly motivated, and the ablation study in Section 4.6 shows that each component contributes to the reported score. However, the performance superiority claim is currently under-supported: the average gain over 3D-OVS on the main benchmark is about one mIoU point, it is uneven across scenes, no variance or seed information is reported, and several hyperparameters are tuned on the same benchmark that is later used for the headline comparison. The paper's own Section 4.7 concedes that dense CLIP features remain noisy and less refined than multi-scale features. The central speed claim is credible; the performance claim needs substantially stronger evidence before the abstract's 'surpasses current state-of-the-art' can be accepted.

major comments (4)
  1. [Table 1 / Section 4.5.2] The claimed performance superiority over 3D-OVS is not robust to the reported numbers. The overall mIoU advantage is +1.0 (88.1 vs. 87.1), but Laser loses on room (85.9 vs. 92.8), bench (88.3 vs. 89.3), and table (88.5 vs. 88.8), and the aggregate gain is almost entirely due to the sofa scene (+12.0 mIoU). No standard deviation or number of seeds is reported anywhere in Section 4, so a one-point average difference from a single run is not convincing evidence of systematic improvement, especially given the acknowledged weakness on small and color-similar objects in Section 4.7. Please provide per-seed results, per-category breakdowns, and a discussion of the scene-dependent gains.
  2. [Section 3.3, Eqs. (18)-(20)] The pseudo-label LpGT in Eq. (18) is computed from L(r), the rendered output of the very label volume being optimized. The cross-entropy in Eq. (20) then uses LpGT as the target for zI(r) and z(r), and because LpGT depends on L(r), gradients flow back into the label volume. This is a closed loop with no external anchor: if the label volume drifts, the pseudo-labels drift with it. The paper should demonstrate, for example by comparing label-volume-derived labels against image/text-derived labels at intermediate checkpoints, that this self-supervision cannot converge to a degenerate or mutually reinforcing solution.
  3. [Section 3.1, Eq. (9)] The self-cross-training loss maximizes four cosine similarities between fθa(F(r)), F(r), fθa(F_hat(r)), and F_hat(r). Because F_hat(r) is a trainable rendered feature and no stop-gradient is applied, this is a mutual-supervision loop between the adapter and the NeRF feature branch. The paper does not analyze whether this loop can collapse to a trivial fixed point where all four terms become constant or where the features lose semantic structure. An experiment with stop-gradient on F_hat(r), or a feature-scatter plot before and after training, would address this concern.
  4. [Section 4.6, Tables 5-8] Hyperparameters α (Table 5), β (Table 6), γ (Table 8), and the attention hidden dimension (Table 7) are all tuned on the same 3D-OVS benchmark that is later used for the headline comparison with 3D-OVS. Since the final performance gain over 3D-OVS is about one mIoU point, selection on the test set is a material concern that can inflate the reported advantage. The authors should either use a validation split for hyperparameter selection or report sensitivity on held-out scenes and on the LERF/Replica datasets separately.
minor comments (8)
  1. [Section 4.5.2, Table 1] The text says Laser is better in 3 of 6 scenes and comparable in 1, but the table shows losses on room, bench, and table; please correct this inconsistency.
  2. [Section 3.3, Eq. (17)] The symbol D is used for both the CLIP feature dimension and the label-vector dimension; use N for the number of classes, and fix the type 'Lx ∈ R3' which should presumably be R^N.
  3. [Section 4.6.2] The text says the low-rank transient query attention has '0.01MB more parameters' and that mIoU improves by '0.02%'; Table 7 shows a difference of 0.01M parameters and 0.12 mIoU. Please make the numbers consistent.
  4. [Section 4.6.2, Table 7] The column header 'FLOPs ↓' is ambiguous; please state whether this is total FLOPs per iteration or another quantity, and give the unit explicitly.
  5. [Section 4.1] The phrase 'self-crossing loss' should be 'self-cross-training loss' for consistency with the rest of the paper.
  6. [Section 4.6.3 / Eq. (20)] The sentence 'By minimizing LsGT' is a typo; the loss being minimized is LCE.
  7. [Section 2.2] The text says 3D-OVS is the first work on implicit neural field open-vocabulary segmentation but then says it is 'essentially trained and tested on a closed vocabulary set'; please clarify this apparent contradiction.
  8. [References] Reference [13] has a malformed author list ('Z. T. Zheng Ding, Jieke Wang'); please correct it to the official citation for MaskCLIP.

Circularity Check

3 steps flagged · score 3.0 of 10

Self-referential training objectives (label-volume pseudo-labels, self-cross-training, and self-normalization) are present, but the central benchmark claim is externally validated and not circular.

  1. self definitional [Section 3.3, Eq. 17-19]
    "L(r) = Softmax(Σ wi Li) ∈ [0,1]^D. ... We can calculate pseudo labels LpGT using classes' text features Ft: LpGT = cos ⟨L(r), Ft⟩ ∈R^N. (18) We can then use cross-entropy to optimize zI (r) and z(r): LCE = −Σ LpGT_i · (log(zI (r)i) + log(z(r)i)) , (19)"

    The pseudo-label LpGT is defined as the cosine similarity between the volume-rendered label vector L(r) and the text features Ft. Since L(r) is itself rendered from the label volume L being optimized, the cross-entropy target in Eq. 19 is generated by the very model that the loss is meant to train. This is a self-training loop: the label volume provides its own supervision, so the claimed view-consistency gain is a self-consistency constraint rather than a prediction from independent labels. The external mIoU benchmark makes the final claim non-circular, but the internal 'pseudo-label' derivation is self-referential by construction.

  2. other [Section 3.1, Eq. 9]
    "we propose a self-cross-training strategy that aims to reduce noise and improve rendered CLIP features by optimizing adapters... Lr = −β( fθa(F (r))·F (r)/|fθa(F (r))| |F (r)| + fθa(ˆF (r))·ˆF (r)/|fθa(ˆF (r))| |ˆF (r)| ) −(1−β)( fθa(ˆF (r))·F (r)/|fθa(ˆF (r))| |F (r)| + fθa(F (r))·ˆF (r)/|fθa(F (r))| |ˆF (r)| )"

    The self-cross-training loss optimizes the adapter fθa to maximize agreement among fθa(F), F, fθa(Fhat), and Fhat. However, the rendered feature Fhat is itself trained in Eq. 3 to match the same dense CLIP feature F. Thus the mutual supervision loop contains no independent denoising signal: both terms ultimately derive from the same noisy F. Whether this bootstrapping reduces noise is an empirical property demonstrated by ablations against ground-truth mIoU, not a logical consequence of the objective. The circularity is limited to the internal training mechanism and does not invalidate the external evaluation.

1 more flagged steps
  1. other [Section 3.4, Eq. 22-23]
    "¯zI = zI − min(zI ) max(zI ) − min(zI ) ∈ [0, 1]N ×H×W . (22) ... Laug = cos ⟨zI (r), ¯zI (r)⟩ . (23)"

    The augmented target ¯zI is computed by min-max normalizing zI itself, so Eq. 23 optimizes zI to agree with a deterministic rescaling of its own logits. There is no new external information in the target; the loss enforces self-consistency with a transformed version of the same prediction. The reported disambiguation benefit is therefore a regularizer effect rather than a prediction derived from independent text or image data. The final segmentation quality is still measured against external ground-truth masks, so the central empirical claim is not circular, but this particular objective is self-referential by construction.

full rationale

The paper's central claim is that Laser surpasses prior methods in training speed and segmentation performance. That claim is supported by external benchmarks (3D-OVS, LERF, Replica) using ground-truth masks and frozen CLIP text embeddings, so the empirical evaluation is not circular: the final mIoU/accuracy numbers do not reduce to any fitted parameter of the paper. Three internal objectives are self-referential in the sense that the supervision target is a function of the model's own output: the label-volume pseudo-labels (Eq. 18) come from the same label volume being optimized; the self-cross-training loss (Eq. 9) lets rendered and raw features supervise each other even though both are derived from the same noisy dense CLIP features; and the text-augmentation loss (Eq. 23) uses a normalized version of the prediction as its own target. These are training regularizers, not predictions, and the ablation study validates them against ground truth, so they do not make the central claim circular. The limitation in Section 4.7 explicitly acknowledges that performance depends on noisy dense CLIP features and is weaker for small or color-similar objects; that is a robustness concern, not a circularity concern. No load-bearing self-citation chain or imported uniqueness theorem is present. Overall, the paper contains self-referential training loops but the headline empirical claim remains externally grounded; circularity score 3 reflects those internal self-referential designs rather than a collapse of the main result.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The method leans on CLIP as an external semantic anchor; the modules themselves add free parameters that are tuned on the evaluation benchmark. The label volume creates a self-referential pseudo-label loop, but final scoring uses external ground-truth masks. No new physical entities are proposed beyond the learnable label-volume representation.

free parameters (5)
  • Residual ratio alpha (Eq. 8) = 0.2
    Ablated in Section 4.6.1, Table 5. alpha=0.2 gives best mIoU on the 3D-OVS table scene; the final model uses it. Selection is on the same benchmark later used for headline numbers.
  • Self-cross-training weight beta (Eq. 9) = 0.3
    Ablated in Table 6; beta=0.3 best. No separate validation split is reported.
  • Ensemble weight gamma (Eq. 20) = 0.5
    Ablated in Table 8; gamma=0.5 best. Tuned on the benchmark used for final results.
  • Low-rank transient query attention hidden dimension = 32
    Ablated in Table 7; D=32 best, D=16 collapses performance (mIoU 37.18), showing sensitivity.
  • Rank s of transient query matrix T = not reported
    Eq. 12 requires s << min(S,D) and complexity O(SsD), but the exact value of s is not given; it is a hand-chosen design parameter.
assumptions (5)
  • domain assumption Frozen CLIP ViT-B/16 embeddings are semantically aligned between text and image for the long-tail categories used.
    Entire method builds on CLIP text and image features (Section 3, Eq. 4-5, Eq. 21). No fine-tuning of CLIP is performed.
  • domain assumption Dense pixel-level CLIP features from MaskCLIP/SCLIP retain enough semantic alignment to act as training targets for 3D segmentation.
    Section 3.1 replaces multi-scale patch features with modified CLIP encoder outputs. The limitation section (4.7) says performance is strongly influenced by the quality of these noisy features.
  • standard math NeRF volume rendering weights apply unchanged to CLIP and label-volume feature fields.
    Equations (1)-(2) reuse the RGB transmittance weights wi for all rendered feature channels; this assumes features are composable along rays.
  • ad hoc to paper Rendered CLIP features F_hat(r) are cleaner than dense CLIP features F(r), making mutual supervision useful.
    Section 3.1 and Section 4.6.1 state this as an observation supporting the self-cross-training loss (Eq. 9); no proof is provided.
  • domain assumption Cosine similarity between label-volume features and CLIP text features yields valid pseudo-labels for classification.
    Section 3.3, Eq. 18-20 uses LpGT as the target for cross-entropy, while L(r) is simultaneously optimized; this is a self-training loop anchored to CLIP text.
invented entities (1)
  • Label volume L
    purpose: Learnable per-point feature volume that renders ray-wise label vectors, converts segmentation into classification, and generates pseudo-labels via cosine similarity to CLIP text features (Section 3.3).
    Introduced by this paper; validity is supported only by in-paper ablations (Section 4.6.3, Figure 10). There is no independent external prediction tied to the label volume as such.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Laser: Efficient Language-Guided Segmentation in Neural Radiance Fields." pith.science (2026). https://pith.science/paper/VVKS25NT

@misc{pith2026250119084,
  author       = {Pith},
  title        = {Pith review of: Laser: Efficient Language-Guided Segmentation in Neural Radiance Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVKS25NT}},
  note         = {Machine review of arXiv:2501.19084}
}
read the original abstract

In this work, we propose a method that leverages CLIP feature distillation, achieving efficient 3D segmentation through language guidance. Unlike previous methods that rely on multi-scale CLIP features and are limited by processing speed and storage requirements, our approach aims to streamline the workflow by directly and effectively distilling dense CLIP features, thereby achieving precise segmentation of 3D scenes using text. To achieve this, we introduce an adapter module and mitigate the noise issue in the dense CLIP feature distillation process through a self-cross-training strategy. Moreover, to enhance the accuracy of segmentation edges, this work presents a low-rank transient query attention mechanism. To ensure the consistency of segmentation for similar colors under different viewpoints, we convert the segmentation task into a classification task through label volume, which significantly improves the consistency of segmentation in color-similar areas. We also propose a simplified text augmentation strategy to alleviate the issue of ambiguity in the correspondence between CLIP features and text. Extensive experimental results show that our method surpasses current state-of-the-art technologies in both training speed and performance. Our code is available on: https://github.com/xingy038/Laser.git.

Figures

Figures reproduced from arXiv: 2501.19084 by the authors.

Figure 1
Figure 1. Workflows of existing methods and Ours: (a) The core of the LERF/3D-OVS process initially adopts a cutting strategy that subdivides the training images into patches of different sizes. These patches are then fed into the CLIP encoder to extract multi-scale CLIP features, which are subsequently saved. At the same time, the original image is also input into DINO to extract DINO features. Afterwards, these multi-scale … view at source ↗
Figure 2
Figure 2. Interaction comparison of different modality. M=modality, N=new modality. (a) Fusion of features from different modalities and then interactive processing. When new modalities are included, retraining is required. (b) Directly interact with features of different modalities. When new modalities are added, they also need to be retrained. (c) Directly interact with the features of different modalities. After introducin… view at source ↗
Figure 3
Figure 3. Modality Graphs. T=text feature, I=image feature, S=segmentation feature of NeRF, L=label volume feature of NeRF. (a) Previous methods only distilled the image modality capabilities of CLIP into the segmentation branch of NeRF. (b) and (c) demonstrate our attempt to align the segmentation modality of NeRF with the image modality of CLIP, as discussed in Section 3.1, where we introduced an adapter and a self-cross tr… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Employing label volume to generate cluster centroids. Our method progressively aggregates points lying on the same ray into a shared cluster centroid during training. This process effectively groups 3D points, which are spatially represented on similar-looking features…
Figure 5
Figure 5. Figure 5: Mitigating the ambiguity in CLIP features. We employ a simplified text augmentation strategy to standardize relevance maps. Observing the original relevance maps Za and Zb in (a), we note that the relevance of class a within the red-highlighted area is higher than in o…
Figure 6
Figure 6. Figure 6: Qualitative comparison. Visualization of segmentation results in 3D-OVS dataset 3 scenes [6]. Note that we generate object masks in this dataset requiring the complete category list. cannot handle the ambiguity between original CLIP features and textual correlations, a…
Figure 8
Figure 8. Figure 8: Qualitative comparison. Visualization of segmentation results in Replica [58] dataset. TABLE 2 Comparison of inference time and FPS for our and 3D-OVS. Method Inference Time (s/Frame) FPS 3D-OVS 6.156 0.16 Our 9.125 0.11 open-vocabulary 3D semantic segmentation and fea…
Figure 9
Figure 9. Figure 9: Ablation studies visualization results on the table scene of 3D-OVS [6]. We present the visualization results of the adapter layer, the label volume, the self-cross-training (SCT) strategy, the low-rank transient query (TQ) attention, and the full module. advantages of…
Figure 10
Figure 10. Figure 10: Ablation on the label volume. 3D points with similar feature expressions are gathered into the same category in the feature space, thereby enhancing the classification effect based on light consistency. However, increasing the dimension also leads to a decline in perf…
Figure 11
Figure 11. Figure 11: Ablation on simplified text augmentation. The x-axis and y￾axis represent the pixel position of the image, while the z-axis indicates the degree of similarity between the text and the image. w/o simplified text augmentation, there is notable ambiguity within the CLIP …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hi-LSplat: Hierarchical 3D Language Gaussian Splatting

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Hi-LSplat trains language-augmented 3D Gaussians with a three-level semantic tree and instance/part contrastive losses, improving open-vocabulary 3D segmentation and localization on eight datasets.

  2. Decoding Visual Neural Representations by Multimodal with Dynamic Balancing

    cs.CV 2025-09 conditional novelty 4.0 of 10

    A multimodal EEG-image-text contrastive framework with dynamic gradient balancing and stochastic noise improves zero-shot object recognition from EEG on ThingsEEG, raising top-1 accuracy from 13.8% to 15.8%.

Reference graph

Works this paper leans on

59 extracted references · 44 canonical work pages · cited by 2 Pith papers

  1. [1]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P . P . Srinivasan, M. Tancik, J. T. Barron, R. Ra- mamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021

    2021

  2. [2]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” ACM Transactions on Graphics, vol. 42, no. 4, July 2023. [Online]. Avail- able: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

    work page 2023

  3. [3]

    Panoptic lifting for 3d scene understanding with neural fields,

    Y. Siddiqui, L. Porzi, S. R. Bul `o, N. M ¨uller, M. Nießner, A. Dai, and P . Kontschieder, “Panoptic lifting for 3d scene understanding with neural fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9043–9052

    work page 2023

  4. [4]

    Contrastive lift: 3d object instance segmentation by slow-fast contrastive fusion,

    Y. Bhalgat, I. Laina, J. F. Henriques, A. Zisserman, and A. Vedaldi, “Contrastive lift: 3d object instance segmentation by slow-fast contrastive fusion,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=bbbbbov4Xu

    work page 2023

  5. [5]

    Lerf: Language embedded radiance fields,

    J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, “Lerf: Language embedded radiance fields,” in International Con- ference on Computer Vision (ICCV), 2023

    work page 2023

  6. [6]

    Weakly supervised 3d open-vocabulary segmentation,

    K. Liu, F. Zhan, J. Zhang, M. Xu, Y. Yu, A. El Saddik, C. Theobalt, E. Xing, and S. Lu, “Weakly supervised 3d open-vocabulary segmentation,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023

    work page 2023

  7. [7]

    Decomposing nerf for editing via feature field distillation,

    S. Kobayashi, E. Matsumoto, and V . Sitzmann, “Decomposing nerf for editing via feature field distillation,” Advances in Neural Information Processing Systems, vol. 35, pp. 23 311–23 330, 2022

    work page 2022

  8. [8]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P . Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

Show all 59 references
  1. [9]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” arXiv preprint arXiv:2301.12597, 2023

    2023 arXiv

  2. [10]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,” in International Conference on Machine Learning . PMLR, 2022, pp. 12 888–12 900

    2022

  3. [11]

    Bert: Pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language under- standing,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  4. [12]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P . Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the International Conference on Com- puter Vision (ICCV), 2021

    2021

  5. [13]

    Open-vocabulary universal image segmentation with maskclip,

    Z. T. Zheng Ding, Jieke Wang, “Open-vocabulary universal image segmentation with maskclip,” in International Conference on Ma- chine Learning, 2023

    2023

  6. [14]

    Cat-seg: Cost aggregation for open-vocabulary semantic segmentation,

    S. Cho, H. Shin, S. Hong, S. An, S. Lee, A. Arnab, P . H. Seo, and S. Kim, “Cat-seg: Cost aggregation for open-vocabulary semantic segmentation,” 2023

    2023

  7. [15]

    Sclip: Rethinking self-attention for dense vision-language inference,

    F. Wang, J. Mei, and A. Yuille, “Sclip: Rethinking self-attention for dense vision-language inference,” arXiv preprint arXiv:2312.01597, 2023

    2023 arXiv

  8. [16]

    Plenoptic sam- pling,

    J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sam- pling,” in Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 2000, pp. 307–318

    2000

  9. [17]

    Learning- based view synthesis for light field cameras,

    N. K. Kalantari, T.-C. Wang, and R. Ramamoorthi, “Learning- based view synthesis for light field cameras,” ACM Transactions on Graphics (TOG), vol. 35, no. 6, pp. 1–10, 2016

    2016

  10. [18]

    The lumigraph,

    S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996, pp. 43–54

    1996

  11. [19]

    Light field rendering,

    M. Levoy and P . Hanrahan, “Light field rendering,” in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996, pp. 31–42

    1996

  12. [20]

    Light field image processing: An overview,

    G. Wu, B. Masia, A. Jarabo, Y. Zhang, L. Wang, Q. Dai, T. Chai, and Y. Liu, “Light field image processing: An overview,” IEEE Journal of Selected Topics in Signal Processing , vol. 11, no. 7, pp. 926–954, 2017

    2017

  13. [21]

    Unstructured lumigraph rendering,

    C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, “Unstructured lumigraph rendering,” in Proceedings of the 28th annual conference on Computer graphics and interactive techniques , 2001, pp. 425–432

    2001

  14. [22]

    Modeling and rendering architecture from photographs: A hybrid geometry-and image- based approach,

    P . E. Debevec, C. J. Taylor, and J. Malik, “Modeling and rendering architecture from photographs: A hybrid geometry-and image- based approach,” in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996, pp. 11–20

    1996

  15. [23]

    Deepstereo: Learning to predict new views from the world’s imagery,

    J. Flynn, I. Neulander, J. Philbin, and N. Snavely, “Deepstereo: Learning to predict new views from the world’s imagery,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 5515–5524

    2016

  16. [24]

    Deep blending for free-viewpoint image-based ren- dering,

    P . Hedman, J. Philip, T. Price, J.-M. Frahm, G. Drettakis, and G. Brostow, “Deep blending for free-viewpoint image-based ren- dering,” ACM Transactions on Graphics (TOG) , vol. 37, no. 6, pp. 1–15, 2018

    2018

  17. [25]

    Scalable inside-out image-based rendering,

    P . Hedman, T. Ritschel, G. Drettakis, and G. Brostow, “Scalable inside-out image-based rendering,” ACM Transactions on Graphics (TOG), vol. 35, no. 6, pp. 1–11, 2016

    2016

  18. [26]

    First-person hyper-lapse videos,

    J. Kopf, M. F. Cohen, and R. Szeliski, “First-person hyper-lapse videos,” ACM Transactions on Graphics (TOG) , vol. 33, no. 4, pp. 1–10, 2014

    2014

  19. [27]

    Soft 3d reconstruction for view synthe- sis,

    E. Penner and L. Zhang, “Soft 3d reconstruction for view synthe- sis,” ACM Transactions on Graphics (TOG) , vol. 36, no. 6, pp. 1–11, 2017

    2017

  20. [28]

    Free view synthesis,

    G. Riegler and V . Koltun, “Free view synthesis,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIX 16. Springer, 2020, pp. 623–640

    2020

  21. [29]

    Stable view synthesis,

    ——, “Stable view synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 12 216–12 225

    2021

  22. [30]

    Neural sparse voxel fields,

    L. Liu, J. Gu, K. Zaw Lin, T.-S. Chua, and C. Theobalt, “Neural sparse voxel fields,” Advances in Neural Information Processing Systems, vol. 33, pp. 15 651–15 663, 2020

    2020

  23. [31]

    Nerf in the dark: High dynamic range view synthesis from noisy raw images,

    B. Mildenhall, P . Hedman, R. Martin-Brualla, P . P . Srinivasan, and J. T. Barron, “Nerf in the dark: High dynamic range view synthesis from noisy raw images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 190–16 199

    2022

  24. [32]

    Space-time neural irradiance fields for free-viewpoint video,

    W. Xian, J.-B. Huang, J. Kopf, and C. Kim, “Space-time neural irradiance fields for free-viewpoint video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 9421–9431

    2021

  25. [33]

    Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering,

    Y. Xiangli, L. Xu, X. Pan, N. Zhao, A. Rao, C. Theobalt, B. Dai, and D. Lin, “Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering,” in Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Par...

    2022

  26. [34]

    Point-nerf: Point-based neural radiance fields,

    Q. Xu, Z. Xu, J. Philip, S. Bi, Z. Shu, K. Sunkavalli, and U. Neu- mann, “Point-nerf: Point-based neural radiance fields,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5438–5448

    2022

  27. [35]

    pixelnerf: Neural radiance fields from one or few images,

    A. Yu, V . Ye, M. Tancik, and A. Kanazawa, “pixelnerf: Neural radiance fields from one or few images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 4578–4587

    2021

  28. [36]

    Tensorf: Tensorial ra- diance fields,

    A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su, “Tensorf: Tensorial ra- diance fields,” in European Conference on Computer Vision. Springer, 2022, pp. 333–350

    2022

  29. [37]

    Instant neural graphics primitives with a multiresolution hash encoding,

    T. M ¨uller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM Transactions on Graphics (ToG), vol. 41, no. 4, pp. 1–15, 2022

    2022

  30. [38]

    Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction,

    C. Sun, M. Sun, and H.-T. Chen, “Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction,” in Pro- IEEE TRANSACTIONS ON PATTERN ANAL YSIS AND MACHINE INTELLIGENCE 13 ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2022

  31. [39]

    Plenoc- trees for real-time rendering of neural radiance fields,

    A. Yu, R. Li, M. Tancik, H. Li, R. Ng, and A. Kanazawa, “Plenoc- trees for real-time rendering of neural radiance fields,” in Proceed- ings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 5752–5761

    2021

  32. [40]

    Plenoxels: Radiance fields without neural net- works,

    S. Fridovich-Keil, A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa, “Plenoxels: Radiance fields without neural net- works,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5501–5510

    2022

  33. [41]

    Zero-shot semantic segmentation,

    M. Bucher, T.-H. Vu, M. Cord, and P . P ´erez, “Zero-shot semantic segmentation,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  34. [42]

    Semantic projection network for zero-and few-label semantic segmenta- tion,

    Y. Xian, S. Choudhury, Y. He, B. Schiele, and Z. Akata, “Semantic projection network for zero-and few-label semantic segmenta- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8256–8265

    2019

  35. [43]

    Efficient esti- mation of word representations in vector space,

    T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient esti- mation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  36. [44]

    Groupvit: Semantic segmentation emerges from text supervision,

    J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang, “Groupvit: Semantic segmentation emerges from text supervision,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 134–18 144

    2022

  37. [45]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,” 2022

    2022

  38. [46]

    Scaling open-vocabulary image segmentation with image-level labels,

    G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, “Scaling open-vocabulary image segmentation with image-level labels,” in European Confer- ence on Computer Vision. Springer, 2022, pp. 540–557

    2022

  39. [47]

    Panoptic nerf: 3d-to-2d label transfer for panoptic urban scene segmentation,

    X. Fu, S. Zhang, T. Chen, Y. Lu, L. Zhu, X. Zhou, A. Geiger, and Y. Liao, “Panoptic nerf: 3d-to-2d label transfer for panoptic urban scene segmentation,” in 2022 International Conference on 3D Vision (3DV). IEEE, 2022, pp. 1–11

    2022

  40. [48]

    Anything-3d: Towards single-view anything reconstruction in the wild,

    Q. Shen, X. Yang, and X. Wang, “Anything-3d: Towards single-view anything reconstruction in the wild,” arXiv preprint arXiv:2304.10261, 2023

    2023 arXiv

  41. [49]

    Open-vocabulary panoptic segmentation with text-to-image dif- fusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. De Mello, “Open-vocabulary panoptic segmentation with text-to-image dif- fusion models,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2023, pp. 2955–2966

    2023

  42. [50]

    In-place scene labelling and understanding with implicit scene represen- tation,

    S. Zhi, T. Laidlow, S. Leutenegger, and A. J. Davison, “In-place scene labelling and understanding with implicit scene represen- tation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 838–15 847

    2021

  43. [51]

    Neural feature fusion fields: 3d distillation of self-supervised 2d image represen- tations,

    V . Tschernezki, I. Laina, D. Larlus, and A. Vedaldi, “Neural feature fusion fields: 3d distillation of self-supervised 2d image represen- tations,” in 2022 International Conference on 3D Vision (3DV). IEEE, 2022, pp. 443–453

    2022

  44. [52]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al. , “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

  45. [53]

    Langsplat: 3d language gaussian splatting,

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister, “Langsplat: 3d language gaussian splatting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 20 051–20 060

    2024

  46. [54]

    Language em- bedded 3d gaussians for open-vocabulary scene understanding,

    J.-C. Shi, M. Wang, H.-B. Duan, and S.-H. Guan, “Language em- bedded 3d gaussians for open-vocabulary scene understanding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 5333–5343

    2024

  47. [55]

    Feature 3dgs: Supercharging 3d gaus- sian splatting to enable distilled feature fields,

    S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P . Chari, S. You, Z. Wang, and A. Kadambi, “Feature 3dgs: Supercharging 3d gaus- sian splatting to enable distilled feature fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024,...

    2024

  48. [56]

    Low-rank modeling and its applications in image analysis,

    X. Zhou, C. Yang, H. Zhao, and W. Yu, “Low-rank modeling and its applications in image analysis,” ACM Computing Surveys (CSUR), vol. 47, no. 2, pp. 1–33, 2014

    2014

  49. [57]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y. Zhao, H. Zhang, P . Zhang, P . Va- jda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7061–7070

    2023

  50. [58]

    The replica dataset: A digital replica of indoor spaces,

    J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma et al., “The replica dataset: A digital replica of indoor spaces,” arXiv preprint arXiv:1906.05797, 2019

    1906 arXiv

  51. [59]

    Self-supervised hypergraphs for learning multiple world interpretations,

    A. Marcu, M. Pirvu, D. Costea, E. Haller, E. Slusanschi, A. N. Belbachir, R. Sukthankar, and M. Leordeanu, “Self-supervised hypergraphs for learning multiple world interpretations,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 983–992

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.