Pith. sign in

REVIEW 4 major objections 4 minor 67 references

A sparse learned point cloud can carry an object's identity, letting one network detect, segment, and locate objects it has never seen.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:39 UTC pith:4AR4PWJZ

load-bearing objection NeMO is a genuinely useful representation for few-shot detection and segmentation, but the absolute 6DoF pose numbers rest on a ground-truth alignment that the abstract does not disclose. the 4 major comments →

arxiv 2602.04343 v1 pith:4AR4PWJZ submitted 2026-02-04 cs.CV

Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception

Sebastian Jung , Leonard Kl\"upfel , Rudolph Triebel , Maximilian Durner This is my paper
classification cs.CV
keywords few-shot perceptionobject-centric representation6DoF pose estimationunseen object detectionamodal segmentationpoint cloudmodel-free perceptionneural memory object
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper aims to show that a single network can detect, segment, and estimate the 6DoF pose of objects it has never seen, given only a handful of RGB photos of the object. The key move is to store the object's visual and geometric identity in a representation called NeMO — a sparse point cloud with learned per-point features — rather than in the network's weights. Because the object information lives in the representation, the same decoder works for any object, and the query-time cost does not grow with the number of template views. If this is right, adding a new object to a perception system reduces to recording a few views and running the encoder offline, with no retraining and no CAD model.

Core claim

The paper's central claim is that a single encoder-decoder network, never fine-tuned on the target object, can detect, segment, reconstruct, and locate objects it has never seen, provided it is handed a representation built from a few ordinary RGB photos. The encoder fuses unordered template views by cross-attending learned 3D point queries to image features, and uses a learned unsigned distance field to pull those points onto the object's estimated surface; the resulting sparse point cloud — the NeMO — carries per-point semantic and geometric features. The decoder then attends a query image to these points and produces modal and amodal masks, a dense 2D-3D correspondence map, and a confiden

What carries the argument

The load-bearing object is the NeMO itself: a sparse, object-centric point cloud whose points carry learned feature vectors. It is constructed by a multi-view encoder with cross-attention between randomly sampled 3D points and image features; a learned unsigned distance field (UDF) regresses each point to the nearest surface point, so the cloud becomes a light geometric and semantic memory of the object. The decoder treats this cloud as the object, using cross-attention between query image features and NeMO points to regress dense predictions. Because the object lives in the cloud rather than in the weights, the cloud can be rotated, scaled, translated, or extended with more points, and the

Load-bearing premise

The absolute pose numbers rest on two conditions the paper sets aside: at test time a PnP solver needs camera intrinsics, and the NeMO coordinate frame must be aligned to ground-truth template poses for evaluation; without both, the method yields pose only up to an unknown similarity transform.

What would settle it

Run the pose pipeline on a benchmark object while withholding ground-truth template poses and camera intrinsics at test time, and check whether the recovered poses can still be expressed in one shared metric frame and compared to ground truth. If they cannot, the 'no camera-specific parameters' claim is not literal and the method is not truly absolute-6DoF on its own.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Onboarding a novel object for detection, segmentation, and pose becomes a matter of recording a handful of RGB views and running an offline encoder — no CAD model, no per-object training, no template matching at query time.
  • Because the NeMO is precomputed before inference, adding more template views improves accuracy without increasing per-image runtime or memory at the decoder.
  • The same network weights serve every object and every task, so a deployed system could scale to many objects at constant marginal cost per new object.
  • The representation is manipulable: rotating, translating, scaling, or merging point clouds adjusts the prediction, which suggests parts of an object could be transformed online.
  • When a CAD model is available, sampling input points from its surface improves reconstruction and pose, so the method covers both model-free and model-based settings with one architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the load-bearing claim is right, the technique could extend to articulated objects by composing several NeMOs — one per rigid part — and letting the decoder reason about their relative placement; the paper names this as future work, and the point-cloud structure makes it a natural next step.
  • The detection and segmentation results do not depend on the evaluation-time alignment to ground-truth template poses; only absolute pose does. A deployment without ground-truth poses could still use the method for amodal detection and relative pose.
  • A direct test: measure pose accuracy on objects with known but withheld template poses and no intrinsics at test time. If performance holds, the 'no camera parameters' claim is literal; if it collapses, the pose claim is contingent on calibration and alignment.
  • The failure mode on symmetric, textureless objects suggests the bottleneck is the training distribution's coverage of such shapes, not the representation itself — a testable claim by retraining on a dataset enriched with symmetric objects.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Neural Memory Object (NeMO), a sparse object-centric point cloud with per-point features generated by an encoder from a small set of unordered RGB template views, and a shared decoder that outputs modal/amodal masks, a dense 2D-3D correspondence map, and a confidence map. The authors claim that this representation enables few-shot detection, segmentation, surface reconstruction, and 6DoF pose estimation of objects unseen during training, without camera-specific parameters or fine-tuning. The method is evaluated on BOP benchmark datasets: model-based (T-LESS, TUD-L, YCB-V) and model-free (HOPEv2, HANDAL), with ablations on the number of templates, number of NeMO points, coordinate-system transformations, and NeMO extension. The paper also introduces a large synthetic training dataset.

Significance. If the central pose claim holds, the paper makes a substantial contribution: it would decouple object knowledge from network weights, enabling one network to perform multiple perception tasks on arbitrary objects with constant query-time cost and without retraining. The detection and segmentation results are valuable and appear independent of the pose-alignment issue. The ablations—particularly the template-count and point-count studies—are thoughtful, and the release of a synthetic dataset is a community contribution. However, the absolute 6DoF pose results are produced only after a ground-truth similarity transform alignment, so the pose claim as stated is not yet established.

major comments (4)
  1. [Abstract, Sec. 3.1 vs. Supp. 7.5] The abstract and Sec. 3.1 state that the method works 'without requiring any camera-specific parameters' and 'without the need for intrinsic or extrinsic camera parameters.' This is contradicted by the pose-estimation pipeline in Supp. 7.5, which uses OpenCV's solvePnPRansac; PnP requires camera intrinsics. The pose claim is therefore not parameter-free. Please revise the claim or explicitly state that intrinsics are needed for pose estimation.
  2. [Supp. 7.4, Algorithm 1] All absolute pose numbers in Tables 2, 5, 8, and 9 are produced after fitting a similarity transform (scale, rotation, translation) to the ground-truth poses of the k=5 best template views. The NeMO coordinate frame is defined by the anchor image, and the scale is set by the anchor crop, so without this GT-based alignment the method outputs poses only up to an unknown similarity transform. Thus the reported pose accuracy does not demonstrate absolute model-free 6DoF pose estimation in the deployment scenario described in the abstract—where no CAD model and no ground-truth template poses are available. Please either qualify the pose claims as 'relative' or 'up to similarity,' or provide an evaluation in which the alignment is obtained from metric scale, known object dimensions, or another deployment-available cue rather than from GT template poses.
  3. [Tables 1-5, 8-9] No error bars, confidence intervals, or significance tests are reported for any benchmark number. Many headline comparisons are within 1-3 pp (e.g., Table 1 HOPEv2: 0.411 vs. 0.384; Table 2 HANDAL: 0.235 vs. 0.213 with NeMO detections). Without variance estimates, it is impossible to assess whether the claimed state-of-the-art results are robust. Please report standard deviations across seeds or object subsets for at least the main comparisons.
  4. [Tables 3-4 and Abstract] On T-LESS, detection AP is 0.183 and segmentation AP is 0.169, far below the next published method (CNOS 0.395, NOCTIS 0.479). The authors attribute this to textureless, symmetric objects and cluttered scenes. This is plausible, but the abstract's claim of 'competitive and state-of-the-art results on various datasets' is too broad. The T-LESS failure should be quantitatively analyzed (e.g., per-object breakdown) or the claim should be restricted to the datasets where the method is competitive.
minor comments (4)
  1. [Sec. 5] The limitation section lists symmetric objects, textureless objects, and merged bounding boxes, but does not mention the two concerns identified above: (a) pose estimation requires camera intrinsics, and (b) absolute pose evaluation relies on GT template poses for similarity alignment. Please add these to the limitations.
  2. [Sec. 2] Typo: 'presentation' should be 'representation' in 'Instead of learning a presentation to render template images.' Also 'matchs' should be 'matches' in the description of ZeroPose.
  3. [Fig. 2 caption] Minor formatting issue: 'T emplate' is split in the figure header; should be 'Template.'
  4. [Tables 2 and 5] It would be helpful to state explicitly which rows use ICP refinement, especially in Table 2 where HOPEv2 uses ICP with depth and HANDAL does not. This affects comparability with OPFormer.

Circularity Check

1 steps flagged

Pose results depend on a ground-truth similarity fit; detection/segmentation are independent.

specific steps
  1. fitted input called prediction [Supplementary Sec. 7.4 (Object Alignment) and Algorithm 1; also Sec. 4 Experimental Setup]
    "As the BOP challenge compares the predicted object pose against the ground truth pose, we need to align the coordinate system of our NeMO with the ground truth object pose to be able to evaluate our approach. ... As we are given the object-to-camera transformation ... in the model-free setting, where we are given the ground truth pose, we can align our coordinate system by optimizing the scale, rotation and center-offset."

    Every reported BOP pose score (Tables 2, 5, 8, 9) is computed after applying the similarity transform (scale s, rotation R, translation t) learned in Algorithm 1 from ground-truth template poses of the same object. The final pose is thus T_est * T_corr, where T_corr is fitted to GT template poses, not predicted from RGB templates. This means the absolute 6DoF pose claim is not free-standing: the NeMO frame (defined by the anchor image) is stitched to the metric object frame using GT annotations. Detection and segmentation metrics do not use this alignment and remain independent.

full rationale

The representation learning and the detection/segmentation tasks are self-contained: the encoder is trained on a large synthetic dataset with object surface regression, and the detection and segmentation numbers are evaluated against external BOP benchmarks without any GT-based postprocessing. No load-bearing self-citation circularity was found; the cited prior work (e.g., Leap) is not by the present authors and is used only as architectural inspiration. However, the central 6DoF pose claim is partially circular in its evaluation. The paper explicitly aligns the NeMO coordinate system to the ground-truth object frame using a similarity transform fitted to ground-truth template poses (Supp. 7.4, Algorithm 1), and all absolute pose results are reported after that correction. In a deployment without CAD models or ground-truth template poses, the method yields poses only up to an unknown similarity transform tied to the anchor image, so the reported absolute pose accuracy cannot be taken as evidence for the abstract's unqualified claim of estimating 6DoF pose without camera-specific parameters or target-object supervision. This is a partial circularity affecting the pose claim, not the overall method; detection, segmentation, and relative geometry results retain independent content.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central claims rest on a domain assumption that synthetic-to-real transfer works for unseen objects, plus two instrumentation assumptions that the benchmark numbers depend on: calibrated intrinsics for PnP and ground-truth template poses for the NeMO-to-object-frame alignment. These are not proven false, but they are load-bearing and not stated in the abstract.

free parameters (5)
  • Loss weights α, β = α=1.0, β=0.2
    Hand-chosen weights in Supp. Sec 7.2 for Ltotal = α(Lχ+L2D3D) + β(Lcertain+Luncertain+Lmodal+Lamodal); no sweep reported.
  • Pointmap confidence threshold = 0.1
    Supp. Sec 7.5: pixels with confidence below 0.1 are discarded before solvePnPRansac.
  • Alignment similarity transform params = k=5, Huber δ, Adam lr, max iterations N
    Algorithm 1 (Supp. Sec 7.4) fits scale/rotation/translation to ground-truth template poses; affects all reported absolute pose numbers.
  • Number of NeMO points M = 1500 for training; 500–1000 at test
    Tab 6 shows AP plateaus around 500–1000 points; memory/precision trade-off.
  • Number of template views K = 32 for benchmarks (5 in training)
    Benchmark protocol choice; Fig 6 shows AP increases with K but saturates.
axioms (5)
  • domain assumption Synthetic-to-real transfer: a network trained on BlenderProc renders of Objaverse/GSO/OmniObject3D generalizes to real BOP test images of never-seen objects without fine-tuning.
    Core empirical premise of the method (Sec 3.2, 4); T-LESS results suggest the premise fails for textureless/symmetric objects.
  • domain assumption Object-centric frame consistency: the anchor-image camera frame provides a valid coordinate system and the learned UDF yields consistent surface points across different template sets.
    Sec 3.1 builds NeMO in the anchor frame; the decoder is trained with random transforms to be frame-agnostic, but the UDF regression (Eq 2-3) has no pose supervision.
  • domain assumption Ground-truth template poses are available for evaluation alignment.
    Algorithm 1 (Supp. 7.4) uses GT object-to-camera transforms of template views to align NeMO to the BOP object frame; without this the output is a similarity-up-to pose.
  • domain assumption Calibrated camera intrinsics are available at inference for PnP.
    Supp. Sec 7.5 invokes OpenCV solvePnPRansac with a camera matrix; contradicts the abstract's 'without camera-specific parameters'.
  • domain assumption Reported leaderboard values from external entries are accurate and directly comparable.
    Tables 1-5 compare against values from BOP leaderboards and [41]; differences of 1-3pp are treated as improvements without statistical testing.
invented entities (2)
  • NeMO (Neural Memory Object) independent evidence
    purpose: A sparse point cloud of estimated surface points with per-point semantic/geometric features, learned from RGB templates without camera poses; decouples object knowledge from network weights.
    Benchmark results on HOPEv2, HANDAL, TUD-L, YCB-V and PCA visualization support it as an informative representation; however the absolute pose numbers depend on GT alignment.
  • Object-centric synthetic dataset (Objaverse/GSO/OmniObject3D renders) independent evidence
    purpose: Training data with 11,077 objects rendered in cluttered scenes to teach the encoder/decoder without pose annotations at inference.
    Used to train the network; claimed to be released, but no download link or metadata is provided in the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 18542 in / 20709 out tokens · 214178 ms · 2026-08-03T04:39:45.679143+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception." pith.science (2026). https://pith.science/paper/4AR4PWJZ

@misc{pith2026260204343,
  author       = {Pith},
  title        = {Pith review of: Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4AR4PWJZ}},
  note         = {Machine review of arXiv:2602.04343}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present Neural Memory Object (NeMO), a novel object-centric representation that can be used to detect, segment and estimate the 6DoF pose of objects unseen during training using RGB images. Our method consists of an encoder that requires only a few RGB template views depicting an object to generate a sparse object-like point cloud using a learned UDF containing semantic and geometric information. Next, a decoder takes the object encoding together with a query image to generate a variety of dense predictions. Through extensive experiments, we show that our method can be used for few-shot object perception without requiring any camera-specific parameters or retraining on target data. Our proposed concept of outsourcing object information in a NeMO and using a single network for multiple perception tasks enhances interaction with novel objects, improving scalability and efficiency by enabling quick object onboarding without retraining or extensive pre-processing. We report competitive and state-of-the-art results on various datasets and perception tasks of the BOP benchmark, demonstrating the versatility of our approach. https://github.com/DLR-RM/nemo

Figures

Figures reproduced from arXiv: 2602.04343 by Leonard Kl\"upfel, Maximilian Durner, Rudolph Triebel, Sebastian Jung.

Figure 1
Figure 1. Figure 1: Overview. Our method uses a multi-view encoder to generate an object-centric geometric encoding called Neural Memory Object (NeMO) with its own coordinate system from a set of RGB images depicting an object unseen during training. A decoder uses the NeMO to retrieve dense predictions allowing us to detect, segment, estimate the objects surface and determine the camera-to-object position on an RGB query ima… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the NeMO approach. RGB template views are first processed by a ViT [13] and a multi-view encoder [27], producing updated image features. To incorporate spatial information, a randomly-sampled point cloud is processed through a MLP and attended to the image features in our proposed Geometric Mapping block, yielding feature-enhanced 3D points that form the NeMO. A decoder attends a query image wi… view at source ↗
Figure 3
Figure 3. Figure 3: Geometric Mapping Block. We fuse the updated im￾age features (key-value pairs) with the pre-processed geometric features (queries) in multiple transformer decoder blocks. The features are then forwarded to a UDF that estimates the unsigned distance of the initial point cloud to the estimated object surface. After further processing these points via a MLP, we combine them with the updated geometric features… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Example of Model-Free Few-Shot Detec￾tion and Pose Estimation on HOPEv2. Left shows the scene without annotations, right shows NeMO detections in green and pose estimations with refinement as rendered overlays. Even in the underexposed scene the model predicts reasonable results. DAL datasets compared to all other publicly listed results. Our method achieves state-of-the art performance on both… view at source ↗
Figure 5
Figure 5. Figure 5: Object Surface Reconstruction and Camera Pose Estimation on Unseen Objects. We show object surface points and camera poses as predicted by the decoder based on four images of randomly chosen objects in different scenarios: (Left) A static coffee machine standing on a table, captured by a dynamic camera. (Middle) A label machine in different environments, including occlusions. (Right) An espresso mug manipu… view at source ↗
Figure 6
Figure 6. Figure 6: Memory consumption and AP on 6DoF Pose Estima￾tion vs. Number of template images. While the offline NeMO generation requires more memory as the number of template im￾ages increases, the inference memory remains constant. Addition￾ally, more template images increase the AP on 6DoF Pose Estima￾tion on YCB-V with ground truth detections. different point cloud sizes, allowing for a dynamic adapta￾tion of the m… view at source ↗
Figure 7
Figure 7. Figure 7: MLP Block with Skip-Connections. 7.1. Synthetic Training Data Generation For the generation we proceed as following: We pick a ran￾dom number of objects, sample them in a scene and render 20 PBR images from random camera positions focusing on a random object in the scene for each camera position. The scenes vary in objects placed (i) randomly on the ground, (ii) flying in space, or (iii) in an upright stan… view at source ↗
Figure 8
Figure 8. Figure 8: Training Images. From left to right, respectively: RGB input, ground truth 2D3D mapping, ground truth amodal segmentation, estimated 2D-3D mapping and estimated confidence, estimated modal segmentation and estimated amodal segmentation. Shown are three different training samples, each sample consists of 10 images. The first 5 images are used to generate a NeMO, all 10 images are used for dense predictions.… view at source ↗
Figure 9
Figure 9. Figure 9: NeMOs from benchmark objects. We show the NeMOs (left, with PCA feature visualization), dense predictions (middle) and object surface reconstruction (right) from objects selected from different datasets. From top to bottom, the datasets are: HANDAL, HOPEv2, YCB-V and T-LESS. We use 32 template images to generate the NeMOs and use the same 32 images to show the dense prediction results. We show real templat… view at source ↗
Figure 10
Figure 10. Figure 10: Pointmap predictions on the YCB-V dataset. We only show estimations with confidences > 0.1. 0 50 100 150 200 250 300 350 z-Rot in [deg] of NeMO 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 Chamfer Distance in [-] dense_rot2gt_obj_surface_rot_handal dense_fix2gt_obj_surface_rot_handal dense_rot2gt_obj_surface_rot_hopev2 dense_fix2gt_obj_surface_rot_hopev2 dense_rot2gt_obj_surface_rot_YCBV dense_fix2gt_obj_… view at source ↗
Figure 11
Figure 11. Figure 11: Chamfer distance between dense predicted and ro￾tated ground truth point cloud averaged across all objects of respective dataset. In case objects and NeMOs are rotated analo￾gously, we observe small errors i.e. the point clouds are very simi￾lar. M-shaped error trajectory, with minimum around 180 degrees of rotation around objects’ z-axis, in case of fixed 2D-3D dense representation but rotated ground tru… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 5 linked inside Pith

  1. [1]

    ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers

    Philipp Ausserlechner, David Haberger, Stefan Thalhammer, Jean-Baptiste Weibel, and Markus Vincze. ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers. In 2024 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 463–469, 2024. 3, 6

    2024

  2. [2]

    Model-based unseen object 2d detection leaderboard bop-classic.https://bop.felk.cvut

    BOP Authors. Model-based unseen object 2d detection leaderboard bop-classic.https://bop.felk.cvut. cz/leaderboards/detection- unseen- bop23/ bop-classic-core/, . Accessed: 2025-07-30. 6

    2025

  3. [3]

    Model-based unseen object 6d detection leaderboard bop-classic.https://bop.felk.cvut

    BOP Authors. Model-based unseen object 6d detection leaderboard bop-classic.https://bop.felk.cvut. cz / leaderboards / pose - detection - unseen - bop24/bop- classic- core/, . Accessed: 2025-07-

    2025

  4. [4]

    Model-based unseen object 2d segmentation leaderboard bop-classic.https://bop.felk.cvut

    BOP Authors. Model-based unseen object 2d segmentation leaderboard bop-classic.https://bop.felk.cvut. cz/leaderboards/detection- unseen- bop23/ bop-classic-core/, . Accessed: 2025-07-30. 6

    2025

  5. [5]

    Model-free unseen object 2d detection leaderboard bop-h3.https : / / bop

    BOP Authors. Model-free unseen object 2d detection leaderboard bop-h3.https : / / bop . felk . cvut . cz / leaderboards / modelfree - detection - unseen-bop24/bop-h3/, . Accessed: 2025-07-30. 5

    2025

  6. [6]

    Model-free unseen object 6d detection leaderboard bop-h3.https://bop.felk.cvut.cz/ leaderboards / modelfree - pose - detection - unseen-bop24/bop-h3/,

    BOP Authors. Model-free unseen object 6d detection leaderboard bop-h3.https://bop.felk.cvut.cz/ leaderboards / modelfree - pose - detection - unseen-bop24/bop-h3/, . Accessed: 2025-07-30. 5

    2025

  7. [7]

    Besl and Neil D

    P.J. Besl and Neil D. McKay. A method for registration of 3-d shapes.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 14(2):239–256, 1992. 6

    1992

  8. [8]

    Freeze: Training-free zero-shot 6d pose estimation with geometric and vision foundation models

    Andrea Caraffa, Davide Boscaini, Amir Hamza, and Fabio Poiesi. Freeze: Training-free zero-shot 6d pose estimation with geometric and vision foundation models. InEuropean Conference on Computer Vision (ECCV), 2024. 14

    2024

  9. [9]

    Zeropose: Cad- prompted zero-shot object 6d pose estimation in cluttered scenes.IEEE Transactions on Circuits and Systems for Video Technology, 35(2):1251–1264, 2025

    Jianqiu Chen, Zikun Zhou, Mingshan Sun, Rui Zhao, Li- wei Wu, Tianpeng Bao, and Zhenyu He. Zeropose: Cad- prompted zero-shot object 6d pose estimation in cluttered scenes.IEEE Transactions on Circuits and Systems for Video Technology, 35(2):1251–1264, 2025. 3

    2025

  10. [10]

    Obja- verse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Obja- verse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022. 5

    Pith/arXiv arXiv 2022

  11. [11]

    Strobl, Matthias Humt, and Rudolph Triebel

    Maximilian Denninger, Dominik Winkelbauer, Martin Sun- dermeyer, Wout Boerdijk, Markus Knauer, Klaus H. Strobl, Matthias Humt, and Rudolph Triebel. Blenderproc2: A procedural pipeline for photorealistic rendering.Journal of Open Source Software, 8(82):4901, 2023. 1, 5

    2023

  12. [12]

    SuperPoint: Self-Supervised Interest Point De- tection and Description

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. SuperPoint: Self-Supervised Interest Point De- tection and Description. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 337–33712, Salt Lake City, UT, USA,

  13. [13]

    Dosovitskiy

    A. Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 3

    Pith/arXiv arXiv 2010

  14. [14]

    McHugh, and Vincent Vanhoucke

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B. McHugh, and Vincent Vanhoucke. Google scanned objects: A high- quality dataset of 3d scanned household items. In2022 In- ternational Conference on Robotics and Automation (ICRA), pages 2553–2560, 2022. 5

    2022

  15. [15]

    A point set generation network for 3d object reconstruction from a single image

    Haoqiang Fan, Hao Su, and Leonidas Guibas. A point set generation network for 3d object reconstruction from a single image. In2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2463–2471, 2017. 8

    2017

  16. [16]

    Fischler and Robert C

    Martin A. Fischler and Robert C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Commun. ACM, 24(6):381–395, 1981. 4

    1981

  17. [17]

    Noctis: Novel object cyclic threshold based instance seg- mentation, 2025

    Max Gandyra, Alessandro Santonicola, and Michael Beetz. Noctis: Novel object cyclic threshold based instance seg- mentation, 2025. 2, 6

    2025

  18. [18]

    Dounseen: Tuning-free class- adaptive object detection of unseen objects for robotic grasp- ing, 2023

    Anas Gouda and Moritz Roidl. Dounseen: Tuning-free class- adaptive object detection of unseen objects for robotic grasp- ing, 2023. 5

    2023

  19. [19]

    HAN- DAL: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions

    Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. HAN- DAL: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions. In IROS, 2023. 5

    2023

  20. [20]

    Cambridge University Press, 2 edition, 2004

    Richard Hartley and Andrew Zisserman.Multiple View Ge- ometry in Computer Vision. Cambridge University Press, 2 edition, 2004. 4

    2004

  21. [21]

    Onepose++: Keypoint-free one- shot object pose estimation without CAD models

    Xingyi He, Jiaming Sun, Yuang Wang, Di Huang, Hujun Bao, and Xiaowei Zhou. Onepose++: Keypoint-free one- shot object pose estimation without CAD models. InAd- vances in Neural Information Processing Systems, 2022. 2

    2022

  22. [22]

    FS6D: Few-Shot 6D Pose Estimation of Novel Ob- jects

    Yisheng He, Yao Wang, Haoqiang Fan, Jian Sun, and Qifeng Chen. FS6D: Few-Shot 6D Pose Estimation of Novel Ob- jects. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6804–6814, New Orleans, LA, USA, 2022. IEEE. 2

    2022

  23. [23]

    Bop: Benchmark for 6d object pose estima- tion

    Tomas Hodan, Frank Michel, Eric Brachmann, Wadim Kehl, Anders GlentBuch, Dirk Kraft, Bertram Drost, Joel Vidal, Stephan Ihrke, Xenophon Zabulis, Caner Sahin, Fabian Man- hardt, Federico Tombari, Tae-Kyun Kim, Jiri Matas, and Carsten Rother. Bop: Benchmark for 6d object pose estima- tion. InProceedings of the European Conference on Com- puter Vision (ECCV)...

    2018

  24. [24]

    T-LESS: An RGB-D dataset for 6D pose estimation of texture-less ob- jects.IEEE Winter Conference on Applications of Computer Vision (WACV), 2017

    Tom ´aˇs Hoda ˇn, Pavel Haluza, ˇStˇep´an Obdrˇz´alek, Jiˇr´ı Matas, Manolis Lourakis, and Xenophon Zabulis. T-LESS: An RGB-D dataset for 6D pose estimation of texture-less ob- jects.IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. 5

    2017

  25. [25]

    BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects.Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024

    Tom ´aˇs Hoda ˇn, Martin Sundermeyer, Yann Labb ´e, Van Nguyen Nguyen, Gu Wang, Eric Brachmann, Bertram Drost, Vincent Lepetit, Carsten Rother, and Ji ˇr´ı Matas. BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects.Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024. 1, 5, 7 9

    2023

  26. [26]

    A survey of loss functions for semantic seg- mentation

    Shruti Jadon. A survey of loss functions for semantic seg- mentation. In2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pages 1–7, 2020. 5

    2020

  27. [27]

    Leap: Liberate sparse-view 3d modeling from camera poses

    Hanwen Jiang, Zhenyu Jiang, Yue Zhao, and Qixing Huang. Leap: Liberate sparse-view 3d modeling from camera poses. ArXiv, 2310.01410, 2023. 3, 4

    Pith/arXiv arXiv 2023

  28. [28]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4015–4026, 2023. 2, 6

    2023

  29. [29]

    MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare

    Yann Labb ´e, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay, Justin Carpentier, Mathieu Aubry, Dieter Fox, and Josef Sivic. MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare. InProceedings of The 6th Conference on Robot Learning, pages 715–725. PMLR, 2023. 2, 3, 6

    2023

  30. [30]

    Epnp: An accurate o(n) solution to the pnp problem, 2009

    Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. Epnp: An accurate o(n) solution to the pnp problem, 2009. 4

    2009

  31. [31]

    Ground- ing Image Matching in 3D with MASt3R

    Vincent Leroy, Yohann Cabon, and Jerome Revaud. Ground- ing Image Matching in 3D with MASt3R. InComputer Vi- sion – ECCV 2024, pages 71–91. Springer Nature Switzer- land, Cham, 2025. 2, 4

    2024

  32. [32]

    Zhang, Deva Ramanan, and Shubham Tulsiani

    Amy Lin, Jason Y . Zhang, Deva Ramanan, and Shubham Tulsiani. RelPose++: Recovering 6D Poses from Sparse- view Observations. In2024 International Conference on 3D Vision (3DV), pages 106–115, 2024. ISSN: 2475-7888. 2

    2024

  33. [33]

    Sam-6d: Segment anything model meets zero-shot 6d object pose es- timation

    Jiehong Lin, Lihua Liu, Dekun Lu, and Kui Jia. Sam-6d: Segment anything model meets zero-shot 6d object pose es- timation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 27906–27916, 2024. 3, 6

    2024

  34. [34]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEuro- pean conference on computer vision, pages 38–55. Springer,

  35. [35]

    Gfreedet: Ex- ploiting gaussian splatting and foundation models for model- free unseen object detection in the bop challenge 2024

    Xingyu Liu, Gu Wang, Chengxi Li, Yingyue Li, Chenyang- guang Zhang, Ziqin Huang, and Xiangyang Ji. Gfreedet: Ex- ploiting gaussian splatting and foundation models for model- free unseen object detection in the bop challenge 2024. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), CV4MR Workshop,

    2024

  36. [36]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. InInternational Conference on Learning Representations, 2019. 5

    2019

  37. [37]

    Adapting pre-trained vision models for novel instance detection and segmentation

    Yangxiao Lu, Jishnu Jaykumar P, Yunhui Guo, Nicholas Ruozzi, and Yu Xiang. Adapting pre-trained vision models for novel instance detection and segmentation. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024. 2, 6

    2024

  38. [38]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis, 2020. 2

    2020

  39. [39]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In2016 Fourth International Conference on 3D Vision (3DV), pages 565–571, 2016. 5

    2016

  40. [40]

    Genflow: Generalizable recurrent flow for 6d pose refinement of novel objects

    Sungphill Moon, Hyeontae Son, Dongcheol Hur, and Sang- wook Kim. Genflow: Generalizable recurrent flow for 6d pose refinement of novel objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10039–10049, 2024. 6

    2024

  41. [41]

    Co-op: Correspondence-based novel object pose estimation

    Sungphill Moon, Hyeontae Son, Dongcheol Hur, and Sang- wook Kim. Co-op: Correspondence-based novel object pose estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11622–11632, 2025. 6, 7, 14

    2025

  42. [42]

    PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Track- ing

    Van Nguyen Nguyen, Yuming Du, Yang Xiao, Micha ¨el Ramamonjisoa, and Vincent Lepetit. PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Track- ing. In2022 International Conference on 3D Vision (3DV), pages 515–525, 2022. 2

    2022

  43. [43]

    CNOS: A Strong Base- line for CAD-based Novel Object Segmentation

    Van Nguyen Nguyen, Thibault Groueix, Georgy Ponimatkin, Vincent Lepetit, and Tomas Hodan. CNOS: A Strong Base- line for CAD-based Novel Object Segmentation. In2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 2126–2132, Paris, France, 2023. IEEE. 2, 5, 6, 7

    2023

  44. [44]

    GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence

    Van Nguyen Nguyen, Thibault Groueix, Mathieu Salzmann, and Vincent Lepetit. GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9903–9913, Seattle, W A, USA,

  45. [45]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El- Nouby, et al. Dinov2: Learning robust visual features with- out supervision.arXiv preprint arXiv:2304.07193, 2023. 2, 5

    Pith/arXiv arXiv 2023

  46. [46]

    Found- pose: Unseen object pose estimation with foundation fea- tures

    Evin Pınar ¨Ornek, Yann Labb ´e, Bugra Tekin, Lingni Ma, Cem Keskin, Christian Forster, and Tomas Hodan. Found- pose: Unseen object pose estimation with foundation fea- tures. InEuropean Conference on Computer Vision, pages 163–182. Springer, 2024. 6

    2024

  47. [47]

    LatentFusion: End-to-End Differentiable Reconstruc- tion and Rendering for Unseen Object Pose Estimation

    Keunhong Park, Arsalan Mousavian, Yu Xiang, and Dieter Fox. LatentFusion: End-to-End Differentiable Reconstruc- tion and Rendering for Unseen Object Pose Estimation. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 10707–10716, Seattle, W A, USA, 2020. IEEE. 2

    2020

  48. [48]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12179–12188, 2021. 4, 5

    2021

  49. [49]

    Iqbal H. Sarker. Machine learning: Algorithms, real-world applications and research directions.SN Computer Science, 2(3):160, 2021. Epub 2021 Mar 22. PMID: 33778771. PMC7983091. Accessed 2025-07-29. 3 10

    2021

  50. [50]

    SuperGlue: Learning Feature Matching With Graph Neural Networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning Feature Matching With Graph Neural Networks. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4937–4946, Seattle, W A, USA, 2020. IEEE. 2

    2020

  51. [51]

    Crisp: Object pose and shape estimation with test- time adaptation.arXiv preprint arXiv:2412.01052, 2024

    Jingnan Shi, Rajat Talak, Harry Zhang, David Jin, and Luca Carlone. Crisp: Object pose and shape estimation with test- time adaptation.arXiv preprint arXiv:2412.01052, 2024. 2

    Pith/arXiv arXiv 2024

  52. [52]

    OSOP: A Multi-Stage One Shot Object Pose Estimation Framework

    Ivan Shugurov, Fu Li, Benjamin Busam, and Slobodan Ilic. OSOP: A Multi-Stage One Shot Object Pose Estimation Framework. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6825–6834, New Orleans, LA, USA, 2022. IEEE. 3

    2022

  53. [53]

    LoFTR: Detector-Free Local Feature Match- ing with Transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-Free Local Feature Match- ing with Transformers. In2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8918–8927, Nashville, TN, USA, 2021. IEEE. 2

    2021

  54. [54]

    OnePose: One-Shot Object Pose Estimation without CAD Models

    Jiaming Sun, Zihao Wang, Siyu Zhang, Xingyi He, Hongcheng Zhao, Guofeng Zhang, and Xiaowei Zhou. OnePose: One-Shot Object Pose Estimation without CAD Models. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6815–6824, New Orleans, LA, USA, 2022. IEEE. 2

    2022

  55. [55]

    Implicit 3d orientation learning for 6d object detection from rgb images

    Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, and Rudolph Triebel. Implicit 3d orientation learning for 6d object detection from rgb images. InProceedings of the European Conference on Computer Vi- sion (ECCV), 2018. 2

    2018

  56. [56]

    6-dof pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark

    Stephen Tyree, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Jeffrey Smith, and Stan Birchfield. 6-dof pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark. InInternational Con- ference on Intelligent Robots and Systems (IROS), 2022. 5

    2022

  57. [57]

    Conditional latent diffusion models for zero-shot instance segmentation, 2025

    Maximilian Ulmer, Wout Boerdijk, Rudolph Triebel, and Maximilian Durner. Conditional latent diffusion models for zero-shot instance segmentation, 2025. 2

    2025

  58. [58]

    Method: Frtpose.v1 (sam6d-fastsam).https: //bop.felk.cvut.cz/method_info/784/, 2024

    Unknown. Method: Frtpose.v1 (sam6d-fastsam).https: //bop.felk.cvut.cz/method_info/784/, 2024. Accessed: 2025-07-30. 1

    2024

  59. [59]

    Vggt: Vi- sual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi- sual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025. 4

    2025

  60. [60]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20697–20709, 2024. 4, 12

    2024

  61. [61]

    BundleSDF: Neural 6-DoF Track- ing and 3D Reconstruction of Unknown Objects

    Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas M ¨uller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. BundleSDF: Neural 6-DoF Track- ing and 3D Reconstruction of Unknown Objects. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 606–617, Vancouver, BC, Canada, 2023. IEEE. 2

    2023

  62. [62]

    FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects. In2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 17868– 17879, Seattle, W A, USA, 2024. IEEE. 2

    2024

  63. [63]

    Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation

    Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Liang Pan Jiawei Ren, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 5

    2023

  64. [64]

    Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes

    Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. 2018. 5

    2018

  65. [65]

    Zhang, Deva Ramanan, and Shubham Tulsiani

    Jason Y . Zhang, Deva Ramanan, and Shubham Tulsiani. Rel- Pose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild. InComputer Vision – ECCV 2022, pages 592–611. Springer Nature Switzerland, Cham, 2022. 2

    2022

  66. [66]

    Fast segment any- thing, 2023

    Xu Zhao, Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, and Jinqiao Wang. Fast segment any- thing, 2023. 2, 6 11

    2023

  67. [67]

    Supplementary Material Input Layer Linear Linear Linear Output Layer Figure 7.MLP Block with Skip-Connections. 7.1. Synthetic Training Data Generation For the generation we proceed as following: We pick a ran- dom number of objects, sample them in a scene and render 20 PBR images from random camera positions focusing on a random object in the scene for ea...

    arXiv