Location Field Descriptors: Single Image 3D Model Retrieval in the Wild

Alexander Grabner; Peter M. Roth; Vincent Lepetit

REVIEW 4 major objections 5 minor 73 references

Location Field Descriptors: Single Image 3D Model Retrieval in the Wild

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that location-field descriptors—per-pixel 3D coordinates predicted from a single RGB image and embedded into a learned pose-invariant space—match the correct 3D model up to 20 percentage points more often than…

desk verdict A solid empirical retrieval paper with a genuinely useful intermediate representation, but the 'in the wild' scope is narrower than advertised and the baseline set is thin. read the letter →

arxiv 1908.02853 v1 pith:HE7LVLA3 submitted 2019-08-07 cs.CV

Alexander Grabner , Peter M. Roth , Vincent Lepetit This is my paper · ORCID

classification cs.CV

keywords 3Dmodelretrievallocationfieldssingle-imagepose-invariantdescriptorembeddinglearningcenterlosstriplet-centersyntheticdata

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that single-image 3D model retrieval works better if a photo and a 3D model are first brought into a shared low-level picture—a location field, which records a 3D surface coordinate for each pixel of an object—rather than mapped directly into an abstract embedding. From those location fields the method computes pose-invariant descriptors, so the retrieval step compares shapes, not appearances. The claim is that this intermediate representation regularizes training, lets most of the system learn from virtually unlimited synthetic renderings, and makes failures visually interpretable. On three real-world datasets covering beds, chairs, sofas, tables, and cars, the paper reports matching the correct 3D model in the top position up to 20 percentage points more often than prior methods, and it still retrieves useful models when the database contains only 3D models never seen in training.

What carries the argument

The load-bearing object is the location field: an image-like tensor whose three channels store the X/Y/Z coordinates, in the object's canonical frame, of the surface point visible at each pixel. It explicitly encodes both shape and pose while discarding texture, lighting, and material. Around it the method builds two mechanisms: a Location Field CNN branch that predicts a 56x56 location field per detected object, with the mask branch thresholding out background, and a Descriptor CNN with a combined softmax, Center, and Triplet-Center loss that learns one center descriptor per 3D model. Because each center descriptor is the average of embeddings over 100 rendered location fields, descriptors become pose-invariant, and new 3D databases can be added at test time by rendering and averaging, with no retraining.

What would settle it

Render two 3D models that differ only in a structure thinner than the 56x56 location-field grid (for example, two chairs identical except for leg thickness, or two cars identical except for mirror shape), compute their center descriptors from the rendered location fields, and measure the descriptor distance. If the distance is near zero while a human viewer can see the difference, the location-field bottleneck cannot support the fine-grained discriminations the method claims.

Watch

Extended reading notes

Core claim

The central discovery is that location fields—image-like maps in which each object pixel carries the X/Y/Z coordinate of the corresponding point on the object's canonical surface—are a sufficient and advantageous common representation for retrieval. Rendered location fields are generated from 3D meshes by interpolating per-vertex coordinates; predicted location fields are produced from RGB images by a region-based CNN at 56x56 resolution and thresholded with predicted masks. A Descriptor CNN embeds each location field into a 270-dimensional pose-invariant descriptor, and each 3D model is represented by a learned center descriptor obtained by averaging the embeddings of 100 rendered views. Retrieval is nearest-neighbor matching in this space. The paper's experiments show that this pipeline beats direct-embedding baselines on three real-world furniture and car datasets, including on unseen 3D model collections, and that the same predicted location fields can be fed to a PnP solver to recover the object's 3D pose.

Load-bearing premise

The load-bearing premise is that a 56x56 location field predicted from a single photo keeps enough 3D shape detail to tell similar models apart, even though thin structures such as chair legs and car side mirrors are routinely missed.

Editorial extensions

If this is right

The same predicted location field that drives retrieval can also feed a PnP solver, so 3D model retrieval and 3D pose estimation come out of a single representation.
The descriptor network can be trained mostly on synthetic data: rendered location fields are cheap to generate, and the real-image supervision is concentrated in the location-field prediction branch.
Adding a new 3D model database at test time requires no retraining—render each model about 100 times, average its embeddings, and match.
When retrieval fails, the predicted location field shows whether the error was a bad per-pixel 3D estimate or a genuinely ambiguous shape, turning a black box into an inspectable pipeline.
Since location fields eliminate appearance, retrieval accuracy should be insensitive to lighting, texture, and material changes, which the paper identifies as a core advantage over RGB-rendering baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the resolution of the location-field bottleneck is the most likely scaling lever: the ablation shows half-resolution fields hurt accuracy, so a higher-resolution or multi-scale location field could push retrieval further on thin structures like chair legs and mirrors.
A natural transfer test is fine-grained category-level pose: because location fields encode dense 2D-3D correspondences, the same network could replace current keypoint-based pose pipelines for objects with few stable keypoints.
Retrieval databases built from raw scanned point clouds, not watertight meshes, could be enrolled by training the Descriptor CNN on point-cloud-derived fields, which would widen the approach to non-CAD data.
The paper leaves open whether the bottleneck also helps under occlusion; datasets are filtered to non-occluded, non-truncated objects, so testing on partially visible objects would show whether the mask-thresholded location field still preserves enough shape.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a credible, well-executed paper. The core idea—predict location fields from RGB, render them from 3D models, and compute pose-invariant descriptors from those fields—is genuinely new for retrieval, and it pays off. The ablation is honest: synthetic data and resolution matter most, while the loss design and feature mapping give smaller gains. The seen/unseen database evaluation is a nice touch, and the qualitative results show the failure modes clearly.

The main contributions hold up. The paper is the first to bring location fields into 3D retrieval, and it demonstrates clear gains over the two baselines it compares against, with up to 20% absolute Top-1 improvement on Pix3D. The synthetic-data transfer story is convincing because the bottleneck is meaningful and interpretable.

Now the soft spots, in order of importance. First, the 'in the wild' claim is overstated. The evaluation on Pix3D restricts to non-occluded, non-truncated samples—exactly the hard cases they admit to failing on. That is a legitimate experimental choice for a first paper, but the title and abstract overpromise. Second, the baseline set is thin: one baseline from the same authors, and no error bars or significance tests. The margins are large enough that they probably survive, but I would want to see variance across runs or resamples. Third, no code or models are released, so reproducibility is harder to verify. The feature mapping and data augmentation are described well enough that a determined team could reproduce, but it is real work.

One small note on the stress-test: the concern that baselines may not be evaluated on the identical subset does not fully stick. The paper says they feed the same detections to both baselines, and the experimental setup implies the same filtered test split. So the comparison is probably fair. The bigger issue is the scope of the claim, not the protocol.

Who is this for? Anyone working on single-image 3D retrieval, AR/robotics, or synthetic-to-real transfer. It is a solid contribution that deserves serious referee time. I would send it to peer review with a request for code release and a discussion of the occlusion/truncation limitation in the main text.

Referee Report

4 major / 5 minor

Summary. The paper proposes Location Field Descriptors for single-image 3D model retrieval. Images and 3D models are first mapped to a common low-level representation: location fields that encode, for each object pixel, the corresponding 3D surface coordinate in a canonical object coordinate system. A Descriptor CNN then maps these location fields to pose-invariant descriptors in an embedding space learned with a combination of softmax, Center, and Triplet-Center losses, with center descriptors for each 3D model computed offline. The Location Field CNN predicts location fields from RGB images, and the Descriptor CNN is trained jointly on predicted and rendered location fields, allowing large amounts of synthetic data to be exploited. The method is evaluated on Pix3D, Comp, and Stanford for retrieval from both seen and unseen 3D model databases, with an ablation study on Pix3D. The central claim is that the approach significantly outperforms the state of the art by up to 20% absolute in multiple retrieval metrics.

Significance. If the claims hold, this is a useful and interesting contribution. Using location fields as an intermediate representation is well motivated: the representation is interpretable, explicitly encodes 3D shape and pose, and enables training on abundant synthetic renderings while reducing the appearance domain gap. The evaluation is broader than in many prior works, covering three datasets, seen and unseen database splits, several retrieval metrics, and an ablation study. The paper also honestly identifies failure modes and limitations. However, the strength of the empirical claim is weakened by protocol choices and missing statistical support, and the training objective is stated inconsistently between equations.

major comments (4)

[Sec. 6 (supplementary) and Table 1] The evaluation is restricted to non-occluded and non-truncated Pix3D samples, as stated in Sec. 6, yet the title, abstract, and Sec. 1 claim retrieval "in the wild," and Sec. 8 lists complex occlusion and extreme pose as failure modes. The headline improvements in Table 1 are therefore computed on exactly the cases the method handles best, and the paper does not explicitly state whether the baselines [3] and [14] were re-evaluated on the identical filtered split with identical detections. Please clarify the exact protocol and either report results on the full Pix3D set or qualify the "in the wild" claim.
[Table 1 and Sec. 4.1] The claim that the method "significantly outperform[s] the state-of-the-art" is not supported by error bars, confidence intervals, or significance tests. In categories with very few database models (Pix3D bed: 19 models; Pix3D sofa: 20 models; see Table 3), an absolute improvement of roughly 20% in AccTop-1 may correspond to only a handful of test samples. Please report variance across train/test splits or at least bootstrap confidence intervals for the main comparisons.
[Eq. (3) and Eq. (5)] Equation (3) defines the Descriptor loss as L_D = L_softmax + alpha L_C + beta L_TC, but Eq. (5) gives the system loss as L = L_D + L_softmax + alpha L_C + beta L_TC + gamma L_LF + delta L_FM, which double-counts the softmax, Center, and Triplet-Center terms. This ambiguity affects reproducibility of the training objective and should be resolved.
[Sec. 8, Fig. 5, and Table 1] The method's ability to rank visually similar 3D models depends on the predicted location fields preserving fine shape details, but Sec. 8 and Fig. 5 acknowledge that thin structures and side mirrors are missed. The car categories, where fine details are most discriminative, show markedly lower AccTop-1 (20.5% on Comp, 29.5% on Stanford in Table 1), which is consistent with this limitation. Please provide an analysis of whether incorrect exact-model retrievals coincide with missing fine details in the predicted location fields, rather than relying only on aggregate dHAU/dIOU.

minor comments (5)

[Sec. 3.1 vs. Sec. 7] Section 3.1 states that location fields are predicted at a resolution of 56x56x3, while Sec. 7 states 58x58x3. Please harmonize the reported resolution.
[Sec. 6] The Pix3D train/test split is described as randomly choosing 50% of images per 3D model, but no random seed or exact split is given. Please release the exact split for reproducibility.
[Sec. 4.1 and Table 1] The text says the method "correctly detect[s] 95% of all objects in the images on average," but Table 1 reports per-dataset AccD0.5 values of 94.6% (Pix3D mean), 99.9% (Comp), and 99.6% (Stanford). Please clarify which average is meant.
[Sec. 4.1] The comparison set contains only two baselines, one of which ([14]) is the authors' own prior work. Adding an independently published third-party baseline would strengthen the empirical claim.
[Eq. (4)] The symbol D is used both for the Euclidean distance between vertices in Eq. (4) and for the Huber distance between descriptors in Eqs. (1)-(2). Please use distinct notation for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the retrieval pipeline is a supervised embedding learned from rendered and predicted location fields and evaluated on held-out data.

full rationale

We walked the derivation chain: (i) location fields are rendered from 3D meshes and predicted from RGB images by a CNN trained with Huber loss against rendered location fields; (ii) a Descriptor CNN maps both rendered and predicted location fields into an embedding, trained with Center loss (Eq. 1), Triplet-Center loss (Eq. 2), softmax, and a system loss (Eq. 5); (iii) retrieval is nearest-neighbor matching against center descriptors, which are either learned weights or averages of rendered-location-field embeddings for unseen databases. None of these steps defines the output in terms of the claimed result: the ground-truth 3D model labels are used only as supervision, not as retrieval inputs, and the unseen-database protocol uses a database disjoint from training. The comparison to [14] (a prior paper by two of the authors) is a baseline comparison, not a load-bearing premise; Feature Mapping [44] is shown by the ablation (Table 2: 60.0% vs. 60.7% AccTop-1) to be non-essential. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from self-citation, and no ansatz smuggled in via citation. The only notable caveat is scope: Supplementary Sec. 6 explicitly restricts Pix3D training/evaluation to non-occluded and non-truncated samples, which limits the 'in the wild' generalization claim but does not make the derivation circular. Verdict: no circularity, score 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. 'Location Field Descriptors' is a new learned representation, but it is a method, not an entity. The location field itself predates this paper.

free parameters (7)

alpha (Center loss weight) = 0.01
Set experimentally; balances intra-class variance minimization in the Descriptor loss (Supplement Sec 7).
beta (Triplet-Center loss weight) = 0.1
Set experimentally; balances inter-class variance maximization (Supplement Sec 7).
gamma (Location Field loss weight) = 10
Set experimentally; weights the location field regression loss in the system loss (Supplement Sec 7).
delta (Feature Mapping loss weight) = 0.01
Set experimentally; controls domain-gap mapping loss (Supplement Sec 7).
margin m (Triplet-Center) = 1
Margin in the triplet-center loss (Eq. 2).
Huber threshold = 1
Threshold for Huber distance in multiple losses (Supplement Sec 7).
predicted:rendered location field training ratio = 1:3
Ratio of predicted to rendered location fields used to train the Descriptor CNN (Supplement Sec 7).

assumptions (5)

domain assumption Ground truth 3D models and image-to-model annotations in Pix3D, Comp, and Stanford are correct.
The evaluation metrics and retrieval accuracy rely on the correctness of these annotations.
domain assumption The evaluation metrics (modified Hausdorff distance, voxel IOU) reflect meaningful 3D shape similarity.
Used to assess retrieval quality, especially for unseen databases without exact matches.
domain assumption Location fields predicted from a single RGB image (at 56x56 resolution, masked) retain enough 3D shape detail for fine-grained discrimination.
The whole retrieval pipeline depends on this; the paper's own failure cases show thin structures and mirrors are missed.
domain assumption Rendered location fields from the training set 3D models are representative enough to train a descriptor that generalizes to unseen ShapeNet models.
The unseen database experiments assume this generalization; no retraining is done for ShapeNet.
ad hoc to paper Excluding occluded and truncated Pix3D samples is acceptable for benchmarking 'in the wild' retrieval.
Supplementary Sec 6: the paper restricts to non-occluded, non-truncated samples and categories with more than 300 samples, which narrows the claim of 'in the wild'.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Location Field Descriptors: Single Image 3D Model Retrieval in the Wild." pith.science (2026). https://pith.science/paper/HE7LVLA3

@misc{pith2026190802853,
  author       = {Pith},
  title        = {Pith review of: Location Field Descriptors: Single Image 3D Model Retrieval in the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HE7LVLA3}},
  note         = {Machine review of arXiv:1908.02853}
}

read the original abstract

We present Location Field Descriptors, a novel approach for single image 3D model retrieval in the wild. In contrast to previous methods that directly map 3D models and RGB images to an embedding space, we establish a common low-level representation in the form of location fields from which we compute pose invariant 3D shape descriptors. Location fields encode correspondences between 2D pixels and 3D surface coordinates and, thus, explicitly capture 3D shape and 3D pose information without appearance variations which are irrelevant for the task. This early fusion of 3D models and RGB images results in three main advantages: First, the bottleneck location field prediction acts as a regularizer during training. Second, major parts of the system benefit from training on a virtually infinite amount of synthetic data. Finally, the predicted location fields are visually interpretable and unblackbox the system. We evaluate our proposed approach on three challenging real-world datasets (Pix3D, Comp, and Stanford) with different object categories and significantly outperform the state-of-the-art by up to 20% absolute in multiple 3D retrieval metrics.

Figures

Figures reproduced from arXiv: 1908.02853 by the authors.

**Figure 2.** An image of an object and its location field. Loca [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overview of our approach. Given a single RGB image and a 3D model database, we use CNNs to predict a location [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Qualitative results for 3D model retrieval from ShapeNet. From left to right, we show the input image, the ground [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Qualitative examples of our predicted location [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Qualitative results for 3D pose estimation and 3D [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Additional qualitative results for 3D model retrieval from ShapeNet. From left to right, we show the input image, [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Failure cases of our approach. For each example image, the top row shows the ground truth and the bottom row [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Additional qualitative examples of our predicted location fields. For each example image, the top row shows the [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

**Figure 10.** Figure 10: Additional qualitative results for 3D pose estimation and 3D model retrieval from both seen and unseen databases. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

**Figure 11.** Figure 11: Additional qualitative results for 3D pose estimation and 3D model retrieval from both seen and unseen databases. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 55 canonical work pages

[3]

Aubry and B

M. Aubry and B. Russell. Understanding Deep Features with Computer-Generated Imagery. In Conference on Computer Vision and Pattern Recognition, pages 2875–2883, 2015

work page 2015
[14]

Grabner, P

A. Grabner, P. M. Roth, and V . Lepetit. 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. In Con- ference on Computer Vision and Pattern Recognition, pages 3022–3031, 2018

2018
[1]

Aspert, D

N. Aspert, D. Santa-Cruz, and T. Ebrahimi. Mesh: Measur- ing Errors between Surfaces using the Hausdorff Distance. In International Conference on Multimedia and Expo, pages 705–708, 2002

work page 2002
[2]

Aubry, D

M. Aubry, D. Maturana, A. Efros, B. Russell, and J. Sivic. Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models. In Conference on Computer Vision and Pattern Recognition , pages 3762– 3769, 2014

work page 2014
[4]

S. Bai, X. Bai, W. Liu, and F. Roli. Neural Shape Codes for 3D Model Retrieval. Pattern Recognition Letters, 65(1):15– 21, 2015

work page 2015
[5]

S. Bai, X. Bai, Z. Zhou, Z. Zhang, and L. Jan Latecki. Gift: A Real-Time and Scalable 3D Shape Search Engine. InCon- ference on Computer Vision and Pattern Recognition, pages 5023–5032, 2016

work page 2016
[6]

Besl and N

P. Besl and N. McKay. Method for Registration of 3-D Shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, pages 586–607. International Society for Optics and Photonics, 1992

work page 1992
[7]

Brachmann, A

E. Brachmann, A. Krull, F. Michel, S. Gumhold, J. Shotton, and C. Rother. Learning 6D Object Pose Estimation using 3D Object Coordinates. In European Conference on Com- puter Vision, pages 536–551, 2014

2014

Show all 73 references

[8]

Brachmann, F

E. Brachmann, F. Michel, A. Krull, M. Ying Yang, S. Gumhold, and C. Rother. Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image. In Conference on Computer Vision and Pattern Recognition, pages 3364–3372, 2016

2016
[9]

Chang, T

A. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. Shapenet: An Information-Rich 3D Model Repository. Technical report, Stanford University — Prince- ton University — Toyota Technological Institut...

2015
[10]

Chen, X.-P

D.-Y . Chen, X.-P. Tian, Y .-T. Shen, and M. Ouhyoung. On Visual Similarity Based 3D Model Retrieval. In Computer Graphics Forum, pages 223–232, 2003

2003
[11]

Esteves, C

C. Esteves, C. Allen-Blanchette, A. Makadia, and K. Dani- ilidis. Learning SO(3) Equivariant Representations with Spherical CNNs. In European Conference on Computer Vi- sion, pages 52–68, 2018

2018
[12]

H. Fan, H. Su, and L. Guibas. A Point Set Generation Net- work for 3D Object Reconstruction from a Single Image. In Conference on Computer Vision and Pattern Recognition, pages 605–613, 2017

2017
[13]

Girdhar, D

R. Girdhar, D. Fouhey, M. Rodriguez, and A. Gupta. Learn- ing a Predictable and Generative Vector Representation for Objects. In European Conference on Computer Vision, pages 484–499, 2016

2016
[15]

Grabner, P

A. Grabner, P. M. Roth, and V . Lepetit. GP 2C: Geometric Projection Parameter Consensus for Joint 3D Pose and Focal Length Estimation in the Wild. In International Conference on Computer Vision, 2019

2019
[16]

Hartley and A

R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003

2003
[17]

K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick. Mask R- CNN. In International Conference on Computer Vision , pages 2980–2988, 2017

2017
[18]

K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016

2016
[19]

K. He, X. Zhang, S. Ren, and J. Sun. Identity Mappings in Deep Residual Networks. In European Conference on Com- puter Vision, pages 630–645, 2016

2016
[20]

X. He, Y . Zhou, Z. Zhou, S. Bai, and X. Bai. Triplet-Center Loss for Multi-View 3D Object Retrieval. In Conference on Computer Vision and Pattern Recognition , pages 1945– 1954, 2018

1945
[21]

Huang, Z

G. Huang, Z. Liu, L. van der Maaten, and K. Weinberger. Densely Connected Convolutional Networks. In Conference on Computer Vision and Pattern Recognition , pages 2261– 2269, 2017

2017
[22]

Huang, S

S. Huang, S. Qi, Y . Zhu, Y . Xiao, Y . Xu, and S.-C. Zhu. Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image. In European Conference on Computer Vision, pages 187–203, 2018

2018
[23]

P. Huber. Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35(1):73–101, 1964

1964
[24]

Hueting, P

M. Hueting, P. Reddy, E. Yumer, V . Kim, N. Carr, and N. Mi- tra. SeeThrough: Finding Objects in Heavily Occluded In- door Scene Images. In International Conference on 3D Vi- sion, pages 267–276, 2018

2018
[25]

Izadinia, Q

H. Izadinia, Q. Shan, and S. Seitz. IM2CAD. In Conference on Computer Vision and Pattern Recognition, 2017

2017
[26]

O. H. Jafari, S. K. Mustikovela, K. Pertsch, E. Brachmann, and C. Rother. iPose: Instance-Aware 6D Pose Estimation of Partly Occluded Objects. In Asian Conference on Computer Vision, 2018

2018
[27]

Kanezaki, Y

A. Kanezaki, Y . Matsushita, and Y . Nishida. Rotation- Net: Joint Object Categorization and Pose Estimation using Multiviews from Unsupervised Viewpoints. In Conference on Computer Vision and Pattern Recognition , pages 5010– 5019, 2018

2018
[28]

Keller, Z

M. Keller, Z. Chen, F. Maffra, P. Schmuck, and M. Chli. Learning Deep Descriptors with Scale-Aware Triplet Net- works. In Conference on Computer Vision and Pattern Recognition, pages 2762–2770, 2018

2018
[29]

Klokov and V

R. Klokov and V . Lempitsky. Escape from Cells: Deep KD- Networks for the Recognition of 3D Point Cloud Models. In International Conference on Computer Vision , pages 863– 872, 2017

2017
[30]

B. Kulis. Metric Learning: A Survey. Foundations and Trends in Machine Learning, 5(4):287–364, 2013

2013
[31]

Lee, Y .-L

T. Lee, Y .-L. Lin, H. Y . Chiang, M.-W. Chiu, W. Hsu, and P. Huang. Cross-Domain Image-Based 3D Shape Retrieval by View Sequence Learning. In International Conference on 3D Vision, pages 258–266, 2018

2018
[32]

Y . Li, H. Su, C. R. Qi, N. Fish, D. Cohen-Or, and L. Guibas. Joint Embeddings of Shapes and Images via CNN Image Pu- 9 riﬁcation. ACM Transactions on Graphics, 34(6):234, 2015

2015
[33]

T.-Y . Lin, P. Doll ´ar, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie. Feature Pyramid Networks for Object Detection. In Conference on Computer Vision and Pattern Recognition, pages 2117–2125, 2017

2017
[34]

T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Doll´ar, and C. L. Zitnick. Microsoft COCO: Com- mon Objects in Context. In European Conference on Com- puter Vision, pages 740–755, 2014

2014
[35]

Mandikal, K

P. Mandikal, K. L. Navaneet, M. Agarwal, and R. V . Babu. 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Im- age. In British Machine Vision Conference , pages 55.1– 55.12, 2018

2018
[36]

Massa, B

F. Massa, B. Russell, and M. Aubry. Deep Exemplar 2D- 3D Detection by Adapting from Real to Rendered Views. In Conference on Computer Vision and Pattern Recognition, pages 6024–6033, 2016

2016
[37]

Mottaghi, Y

R. Mottaghi, Y . Xiang, and S. Savarese. A Coarse-To-Fine Model for 3D Pose Estimation and Sub-Category Recogni- tion. In Conference on Computer Vision and Pattern Recog- nition, pages 418–426, 2015

2015
[38]

K. L. Navaneet, P. Mandikal, M. Agarwal, and V . Babu. CAPNet: Continuous Approximation Projection for 3D Point Cloud Reconstruction using 2D Supervision. In AAAI Conference on Artiﬁcial Intelligence, 2019

2019
[39]

Nooruddin and G

F. Nooruddin and G. Turk. Simpliﬁcation and Repair of Polygonal Models Using V olumetric Techniques. IEEE Transactions on Visualization and Computer Graphics , 9(2):191–205, 2003

2003
[40]

Oh Song, Y

H. Oh Song, Y . Xiang, S. Jegelka, and S. Savarese. Deep Metric Learning via Lifted Structured Feature Embedding. In Conference on Computer Vision and Pattern Recognition, pages 4004–4012, 2016

2016
[41]

C. Qi, H. Su, M. Nießner, A. Dai, M. Yan, and L. Guibas. V olumetric and Multi-View CNNs for Object Classiﬁcation on 3D Data. In Conference on Computer Vision and Pattern Recognition, pages 5648–5656, 2016

2016
[42]

C. R. Qi, H. Su, K. Mo, and L. Guibas. Pointnet: Deep Learning on Point Sets for 3D Classiﬁcation and Segmenta- tion. In Conference on Computer Vision and Pattern Recog- nition, pages 652–660, 2017

2017
[43]

C. R. Qi, L. Yi, H. Su, and L. Guibas. Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In Advances in Neural Information Processing Sys- tems, pages 5099–5108, 2017

2017
[44]

M. Rad, M. Oberweger, and V . Lepetit. Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Syn- thetic Images. In Conference on Computer Vision and Pat- tern Recognition, pages 4663–4672, 2018

2018
[45]

S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems, pages 91–99, 2015

2015
[46]

Russakovsky, J

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet Large Scale Visual Recognition Challenge. International Journal of Computer Vision , 115(3):211–252, 2015

2015
[47]

Santa Cruz and J

C. Santa Cruz and J. Dorronsoro. A Nonlinear Discrimi- nant Algorithm for Feature Extraction and Data Classiﬁca- tion. IEEE Transactions on Neural Networks , 9(6):1370– 1376, 1998

1998
[48]

Savva, F

M. Savva, F. Yu, H. Su, et al. SHREC’16 Track Large-Scale 3D Shape Retrieval from ShapeNet Core55. InEurographics Workshop on 3D Object Retrieval, 2016

2016
[49]

Savva, F

M. Savva, F. Yu, H. Su, et al. SHREC’17 Track Large-Scale 3D Shape Retrieval from ShapeNet Core55. InEurographics Workshop on 3D Object Retrieval, 2017

2017
[50]

Schroff, D

F. Schroff, D. Kalenichenko, and J. Philbin. FaceNet: A Uniﬁed Embedding for Face Recognition and Clustering. In Conference on Computer Vision and Pattern Recognition , pages 815–823, 2015

2015
[51]

Stockman and L

G. Stockman and L. Shapiro. Computer Vision. Prentice Hall PTR, 2001

2001
[52]

H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller. Multi- View Convolutional Neural Networks for 3D Shape Recog- nition. In Conference on Computer Vision and Pattern Recognition, pages 945–953, 2015

2015
[53]

X. Sun, J. Wu, X. Zhang, Z. Zhang, C. Zhang, T. Xue, J. Tenenbaum, and W. Freeman. Pix3D: Dataset and Meth- ods for Single-Image 3D Shape Modeling. In Conference on Computer Vision and Pattern Recognition , pages 2974– 2983, 2018

2018
[54]

Szeliski

R. Szeliski. Computer Vision: Algorithms and Applications. Springer Science & Business Media, 2010

2010
[55]

F. P. Tasse and N. Dodgson. Shape2Vec: Semantic-Based Descriptors for 3D Shapes, Sketches and Images. ACM Transactions on Graphics, 35(6):208, 2016

2016
[56]

Tatarchenko, S

M. Tatarchenko, S. Richter, R. Ranftl, Z. Li, V . Koltun, and T. Brox. What Do Single-View 3D Reconstruction Networks Learn? In Conference on Computer Vision and Pattern Recognition, 2019

2019
[57]

Taylor, J

J. Taylor, J. Shotton, T. Sharp, and A. Fitzgibbon. The Vit- ruvian Manifold: Inferring Dense Correspondences for One- Shot Human Pose Estimation. In Conference on Computer Vision and Pattern Recognition, pages 103–110, 2012

2012
[58]

Tulsiani, S

S. Tulsiani, S. Gupta, D. Fouhey, A. Efros, and J. Malik. Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene. In Conference on Computer Vision and Pattern Recognition, pages 302–310, 2018

2018
[59]

H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. Guibas. Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation. In Conference on Computer Vision and Pattern Recognition , pages 2642–2651, 2019

2019
[60]

S. Wang, J. Wu, X. Sun, W. Yuan, W. Freeman, J. Tenen- baum, and E. Adelson. 3D Shape Perception from Monoc- ular Vision, Touch, and Shape Priors. In International Con- ference on Intelligent Robots and Systems, pages 1606–1613, 2018

2018
[61]

Y . Wang, X. Tan, Y . Yang, X. Liu, E. Ding, F. Zhou, and L. S. Davis. 3D Pose Estimation for Fine-Grained Object Categories. In European Conference on Computer Vision Workshops, 2018

2018
[62]

Weinberger and L

K. Weinberger and L. Saul. Distance Metric Learning for Large Margin Nearest Neighbor Classiﬁcation. Journal of Machine Learning Research, 10:207–244, 2009. 10

2009
[63]

Y . Wen, K. Zhang, Z. Li, and Y . Qiao. A Discriminative Feature Learning Approach for Deep Face Recognition. In European Conference on Computer Vision, pages 499–515, 2016

2016
[64]

Wohlhart and V

P. Wohlhart and V . Lepetit. Learning Descriptors for Ob- ject Recognition and 3D Pose Estimation. In Conference on Computer Vision and Pattern Recognition , pages 3109– 3118, 2015

2015
[65]

J. Wu, C. Zhang, X. Zhang, Z. Zhang, W. Freeman, and J. Tenenbaum. 3D ShapeNets: A Deep Representation for V olumetric Shapes. In Conference on Computer Vision and Pattern Recognition, pages 1912–1920, 2015

1912
[66]

Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. Learning Shape Priors for Single-View 3D Comple- tion and Reconstruction. In European Conference on Com- puter Vision, pages 646–662, 2018

2018
[67]

Xiang, W

Y . Xiang, W. Kim, W. Chen, J. Ji, C. Choy, H. Su, R. Mot- taghi, L. Guibas, and S. Savarese. ObjectNet3D: A Large Scale Database for 3D Object Recognition. In European Conference on Computer Vision, pages 160–176, 2016

2016
[68]

Xiang, R

Y . Xiang, R. Mottaghi, and S. Savarese. Beyond Pascal: A Benchmark for 3D Object Detection in the Wild. In IEEE Winter Conference on Applications of Computer Vi- sion, pages 75–82, 2014

2014
[69]

Zhang, Z

X. Zhang, Z. Zhang, C. Zhang, J. Tenenbaum, W. Freeman, and J. Wu. Learning to Reconstruct Shapes from Unseen Classes. In Advances in Neural Information Processing Sys- tems, pages 2257–2268, 2018. 11 Location Field Descriptors: Single Image 3D Model Retrieval in the Wild Supp...

2018
[70]

These datasets have only been released recently and, to the best of our knowledge, we are the ﬁrst to report results for 3D model retrieval on all of them

Datasets and Evaluation Setup We evaluate our proposed approach for 3D model re- trieval in the wild on three challenging real-world datasets with different object categories: Pix3D [53] ( bed, chair, sofa, table), Comp [61] ( car), and Stanford [61] ( car). These datasets hav...
[71]

For our Descriptor CNN, we use a DenseNet-50 architec- ture [21] with 3 dense blocks and a growth rate of 24

Implementation and Training Details For our Location Field CNN, we use a Feature Pyramid Network [33] on top of a ResNet-101 backbone [18, 19]. For our Descriptor CNN, we use a DenseNet-50 architec- ture [21] with 3 dense blocks and a growth rate of 24. For our implementation,...
[72]

8 shows failure cases of our approach

Failure Cases Fig. 8 shows failure cases of our approach. Most failure cases relate to incorrect location ﬁeld predictions. For ex- ample, if the 3D pose of the object in the image is far from the 3D poses seen during training, or if multiple objects are detected as a single o...
[73]

Additional Qualitative Results Finally, we present additional qualitative results which complement those presented in the main paper. Fig. 9 presents further qualitative examples of our pre- dicted location ﬁelds. We upscale and pad the predicted location ﬁelds to match the in...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [3]

Aubry and B

M. Aubry and B. Russell. Understanding Deep Features with Computer-Generated Imagery. In Conference on Computer Vision and Pattern Recognition, pages 2875–2883, 2015

work page 2015

[2] [14]

Grabner, P

A. Grabner, P. M. Roth, and V . Lepetit. 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. In Con- ference on Computer Vision and Pattern Recognition, pages 3022–3031, 2018

2018

[3] [1]

Aspert, D

N. Aspert, D. Santa-Cruz, and T. Ebrahimi. Mesh: Measur- ing Errors between Surfaces using the Hausdorff Distance. In International Conference on Multimedia and Expo, pages 705–708, 2002

work page 2002

[4] [2]

Aubry, D

M. Aubry, D. Maturana, A. Efros, B. Russell, and J. Sivic. Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models. In Conference on Computer Vision and Pattern Recognition , pages 3762– 3769, 2014

work page 2014

[5] [4]

S. Bai, X. Bai, W. Liu, and F. Roli. Neural Shape Codes for 3D Model Retrieval. Pattern Recognition Letters, 65(1):15– 21, 2015

work page 2015

[6] [5]

S. Bai, X. Bai, Z. Zhou, Z. Zhang, and L. Jan Latecki. Gift: A Real-Time and Scalable 3D Shape Search Engine. InCon- ference on Computer Vision and Pattern Recognition, pages 5023–5032, 2016

work page 2016

[7] [6]

Besl and N

P. Besl and N. McKay. Method for Registration of 3-D Shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, pages 586–607. International Society for Optics and Photonics, 1992

work page 1992

[8] [7]

Brachmann, A

E. Brachmann, A. Krull, F. Michel, S. Gumhold, J. Shotton, and C. Rother. Learning 6D Object Pose Estimation using 3D Object Coordinates. In European Conference on Com- puter Vision, pages 536–551, 2014

2014