{"id":"7794705c-f251-40f6-89bd-4192f20df597","arxiv_id":"2412.18605","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Orient Anything predicts an object's front-facing 3D orientation in a single image using a model trained on 2M rendered views, with zero-shot transfer to real images.","lead":"A new model estimates the 3D orientation of objects in a single image by learning from 2 million rendered views of 3D models. It claims state-of-the-art accuracy on both synthetic and real images, and could give vision-language systems a missing spatial reasoning skill.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3's zero-shot real-image SOTA claim rests on an unspecified conversion from each benchmark's native pose annotation to the paper's spherical front-axis convention; without that conversion the five-dataset comparison is not verifiable.","rationale":"The reader's weakest_assumption points to VLM annotation bias in the training labels (Table 4, 86-92% accuracy). That is a real concern, but it affects training noise and the COCO benchmark indirectly. The more immediately load-bearing issue is the benchmark ground-truth conversion: even if Gemini's front-face labels were perfect, Table 3 can be invalid if the five real datasets' native pose annotations are not mapped into the same spherical front-axis convention used for training and for the COCO evaluation. Without this conversion, the paper's strongest quantitative evidence for zero-shot real-image SOTA is not reproducible or interpretable. The reader did list 'a precise description of how ground-truth orientations from the five real benchmarks are converted into the paper's spherical convention' as one of three conditions, but did not elevate it to the weakest assumption. I therefore partially agree. The verdict should remain CONDITIONAL: the paper's method and rendered-image results are credible, but the real-image SOTA claim needs either the conversion code or an independent reconstruction. This does not change the reader's conditional assessment.","tokens_in":19561,"tokens_out":2905,"duration_ms":30331,"concrete_test":"Independently reconstruct the ground truth for Table 3: obtain raw annotations for 100 randomly selected objects per benchmark (SUN RGB-D, KITTI, nuScenes, Objectron, ARKitScenes), implement the paper's spherical convention from Sec. 4 Step 3, and recompute azimuth/polar/rotation errors. Check two invariants: (1) converted ground-truth angles are invariant to camera/viewing direction and align with the object front axis as defined by the training labels; (2) applying the same conversion to Cube R-CNN's raw outputs reproduces the reported Table 3 numbers. If either fails, or if another reasonable conversion changes the errors by more than the gap to the next best method, the zero-shot SOTA claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim for real images (Table 3) depends on converting each benchmark's native annotation into the paper's spherical convention: polar θ, azimuth φ, and rotation δ relative to a VLM-defined front axis (Sec. 5.1, Eq. 1-2). The paper never specifies this conversion. SUN RGB-D, KITTI, and nuScenes provide 3D bounding boxes or yaw in camera/world coordinates, Objectron provides 9-DoF poses, and ARKitScenes provides 3D boxes; each requires a non-trivial choice of object-front axis and camera-up alignment before azimuth/polar/rotation errors can be computed. Cube R-CNN's raw outputs must be transformed into the same convention for a fair comparison. If those choices are inconsistent with the training definition, the low errors in Table 3 are not measuring the capability claimed in the abstract, and the comparison against Cube R-CNN and VLMs is not well-defined. This is load-bearing because Table 3 is the main quantitative evidence for 'state-of-the-art ... in real images.' The rendered-image results (Table 2) and the COCO 8-direction benchmark are less affected, but the five-dataset zero-shot claim is unverifiable without the conversion. The paper's Appendix E only describes a template for converting predicted azimuth into horizontal 8-direction text, not the reverse mapping of benchmark ground truth into spherical angles.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Orient Anything, a single-image object orientation estimation model trained on 2M rendered images from Objaverse. The training labels are produced by a pipeline that first filters 3D assets in canonical poses, then uses Gemini-1.5-Pro to select the object's front face from four orthographic views (with symmetry analysis as a filter), and finally renders 40 random viewpoints per object with spherical angle annotations (polar theta, azimuth phi, camera rotation delta). The model uses a DINOv2 visual encoder and multiple prediction heads that fit discretized Gaussian distributions over the three angles, plus a confidence head for whether a meaningful front face exists. The authors evaluate on a manually annotated rendered test set, a COCO-based 8-direction real-image benchmark, five real-world datasets (SUN RGB-D, KITTI, nuScenes, Objectron, ARKitScenes), and a new VQA benchmark (Ori-Bench), reporting state-of-the-art results against Cube R-CNN, GPT-4o, and Gemini-1.5-Pro. They also demonstrate applications in spatial VQA, generation scoring, and 3D model orientation voting.","tokens_in":19829,"tokens_out":6518,"duration_ms":63880,"significance":"If the results are validated, this is a potentially significant contribution: it is the first large-scale, category-agnostic single-image orientation estimator that works on free-view images, and the probability-distribution-fitting objective is a simple and effective alternative to direct regression. The paper's strengths include a clear data-generation pipeline, extensive ablations (learning objective, number of views, initialization, augmentation), a new benchmark (Ori-Bench), and reproducible methodology in principle. However, the central 'state-of-the-art in real images' claim depends on an unspecified annotation-conversion protocol for the five real datasets, and the definition of 'meaningful front face' is inherited from a VLM rather than from an independent human-consensus protocol. These issues are load-bearing for the paper's main claims and need to be addressed before the results can be fully trusted.","major_comments":[{"comment":"The zero-shot real-image evaluation on SUN RGB-D, KITTI, nuScenes, Objectron, and ARKitScenes requires converting each dataset's native orientation annotations (3D bounding boxes, yaw angles, 9-DoF poses) into the paper's spherical convention (polar theta, azimuth phi, rotation delta relative to a front axis). The paper does not specify this conversion for any dataset, nor how Cube R-CNN's raw outputs are mapped to the same convention. Without this protocol, the numeric errors in Table 3 are not verifiable, and the comparison against Cube R-CNN is not well-defined. The appendix template (Appendix E) only describes converting predicted azimuth to 8-direction text, not the reverse mapping of ground truth into spherical angles. The authors should provide a detailed, per-dataset conversion description, including the definition of the object front axis for each object category and how camera-up alignment is handled.","section":"Sec. 6.3, Table 3"},{"comment":"The training labels are produced by Gemini-1.5-Pro's judgment of the front face from four orthographic views, and the paper's own ablation (Table 4) reports only 86–92% agreement between this VLM and manual annotations on 300 objects. Since Gemini is also a baseline in the experiments, any systematic bias in Gemini's notion of 'front' (e.g., for keyboards, remote controls, shoes) is baked into the trained model and into the comparison. The manual annotation protocol for the test sets (Sec. 6.2 and Sec. 6.3) is not described: no annotation instructions, inter-annotator agreement, or criteria for 'meaningful front face' are given. This is load-bearing because the paper defines the task as estimating the 'meaningful front face' and claims the model is the first to align with it. The authors need to specify the manual annotation protocol and show that it is independent of the VLM's notion, or explicitly discuss the subjectivity of the front-face definition and its impact on results.","section":"Sec. 4 (Step 2), Table 4, Sec. 6.2"},{"comment":"No error bars, confidence intervals, or statistical significance tests are reported for any of the main quantitative results. For example, the COCO 8-direction benchmark has 1,600 samples, and the difference between Orient Anything (72.44%) and GPT-4o (45.78%) is large, but the difference between ViT-B (70.19%) and ViT-L (72.44%) may not be meaningful without variance estimates. Similarly, the rendered test set has only 300 objects (4,800 images), and the manual annotation may introduce additional variance. The authors should report standard deviations over multiple runs or bootstrap confidence intervals for the key tables, and at least state the number of independent training runs.","section":"Sec. 6.2, Sec. 6.3, Tables 2 and 3"},{"comment":"Ori-Bench consists of only 200 base images (100 COCO and 100 DALL-E 3), with each mirrored pair counted only if both versions are answered correctly. This yields small effective sample sizes, especially for the Spatial Part Reasoning task (39+39 samples). The paper reports overall accuracy of 51.50% for Orient Anything+LLM versus 33.00% for Gemini, but does not report per-subset results (COCO vs. DALL-E 3) or any confidence interval. Additionally, the LLM used in the Orient Anything+LLM pipeline appears to be the same GPT-4o model used as a baseline, so the comparison partly measures the benefit of additional spatial-context prompting rather than a new reasoning capability. The authors should report per-task and per-source breakdowns, state the statistical significance of the differences, and clarify the role of the LLM in the pipeline.","section":"Sec. 3, Table 1 (Ori-Bench)"}],"minor_comments":[{"comment":"The text says \"sigma_phi is the variance for polar distribution\" but the context refers to the azimuth distribution; this is a typo that should be corrected.","section":"Sec. 5.1, after Eq. (2)"},{"comment":"The heading \"Sythetic-to-Real Transferring\" contains a typo; it should be \"Synthetic-to-Real Transferring\".","section":"Sec. 5.2 heading"},{"comment":"In the Training Initialization block, \"DIONv2\" is a typo for \"DINOv2\".","section":"Sec. 6.4, Table 5"},{"comment":"The rotation angle is denoted as both gamma (in the text before Eq. (2)) and delta (in Eq. (2) and elsewhere); please use a single symbol consistently.","section":"Sec. 5.1, Eq. (2) and surrounding text"},{"comment":"The symmetry threshold is described as \"similarity exceeds the threshold\" without specifying the threshold value or how SIFT, SSIM, and color similarity are combined; please provide the exact criterion for reproducibility.","section":"Sec. 4, Step 2"},{"comment":"For the five real-world datasets, the paper states that \"1,000 objects with 3D orientation annotations are randomly selected and cropped\" but does not specify whether ground-truth boxes or an external detector is used for cropping, and whether the same crops are used for all baselines; this should be clarified.","section":"Sec. 6.3"},{"comment":"The appendix says all Ori-Bench samples are in the attached file, but the main text does not state whether this file will be released publicly; please clarify availability of the benchmark and annotations.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the core idea is sound, but the evaluation protocol for the real-image SOTA claim (Table 3) is not yet verifiable due to the missing ground-truth conversion. The front-face definition also needs to be grounded in a transparent manual protocol to avoid circularity with the VLM annotation. These are fixable within the scope of a revision. I would be willing to review a revised version that addresses these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real step forward for a useful capability. The paper builds the first large-scale training set for front-face-aligned 3D orientation from single images, and the model demonstrably does something GPT-4o and Gemini struggle with. The COCO 8-direction results and the Ori-Bench improvement are credible evidence that the capability exists. I'd credit the data pipeline—canonical-pose filtering, VLM-assisted front-face annotation with symmetry checks, and random-view rendering—as a smart way to get scalable supervision. The distribution-fitting objective, with circular Gaussians for periodic angles, is sensible and adequately ablated. DINOv2 initialization and the masking/cropping augmentations are reasonable choices, and the paper reports the relevant ablations rather than hiding them.\n\nThe soft spot is real and load-bearing: the five-dataset zero-shot comparison in Table 3. The paper never says how SUN RGB-D, KITTI, nuScenes, Objectron, and ARKitScenes ground-truth annotations are converted into its spherical convention (polar θ, azimuth φ, rotation δ relative to a VLM-defined front axis). Each of those datasets uses its own native pose representation—3D boxes in camera or world coordinates, 9-DoF poses, yaw angles—and the front-axis choice is non-trivial. Without that protocol, the low absolute errors in Table 3 cannot be interpreted, and the comparison against Cube R-CNN is not well-defined. This is not a nitpick; it's the main quantitative evidence for the 'state-of-the-art in real images' claim. The rendered-image results and the COCO 8-direction benchmark are less affected, and the manual test labels there give me confidence the core method works.\n\nOther issues are proportionate. No error bars anywhere, and the manual rendered test set is small (300 objects). The VLM annotation noise (86–92% agreement) means the training set has systematic label bias for certain object categories; the category-level table honestly shows the model fails on remote and keyboard, which tempers the 'general' claim. Using Gemini for training labels and as a baseline is awkward but not circular, since the main test labels are human.\n\nBottom line: worth engaging with seriously. I'd send it to review, but the authors need to release code and data, add error bars, and spell out the GT conversion for Table 3. With that, it could become a reference point for single-image orientation.","headline":"A genuinely new single-image orientation estimator with a clever data pipeline, but the headline real-image SOTA claim is unverifiable until the paper specifies how each benchmark's native pose labels were converted to its spherical convention.","tokens_in":20375,"tokens_out":2954,"would_cite":false,"duration_ms":28585,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces a model trained on two million rendered views of 3D objects that estimates an object's front-facing 3D orientation from a single real image, and reports large accuracy gains over both expert methods and…","keywords":["object orientation estimation","single-image 3D pose","front-face annotation","probability distribution fitting","rendered image training","synthetic-to-real transfer","zero-shot evaluation","spatial reasoning"],"falsifier":"Annotate a random subset of the evaluation objects with several independent human labelers, asking them to pick the front face from the same orthogonal views, and measure inter-labeler agreement; if agreement on the annotator model's choices drops well below the 86 to 92 percent range the paper reports for its own annotation ablation, or if the model's accuracy falls sharply when evaluated against human-chosen fronts, the claim of robust, convention-independent orientation estimation is not established.","tokens_in":19357,"feed_emoji":"🧭","tokens_out":10972,"duration_ms":94337,"temperature":0.7,"pith_summary":"Object orientation—which way an object's meaningful front face points in 3D—has been a missing perceptual primitive in vision models, largely because orientation labels are scarce. This paper tries to establish that the missing primitive can be learned at scale by generating the labels rather than collecting them: it renders two million images from random viewpoints of 3D models whose front faces were chosen by a large vision-language model aided by symmetry checks, and trains a single-image estimator on those renders. The estimator predicts three angles (azimuth, polar, and in-plane rotation) as probability distributions over one-degree bins, a target the paper shows is far more trainable than direct regression. The authors report large accuracy gains over existing expert methods and vision-language baselines on both rendered and real images, and they show that handing the estimated angles to an LLM as text lifts spatial question-answering accuracy. If the central claim holds, orientation becomes an available input for object-centric reasoning, generation, and 3D asset alignment.","feed_headline":"2M rendered views teach a model to read object orientation in one photo","feed_subtitle":"Adding the model's angles to an LLM lifts spatial question scores from 33% to 51.5%.","key_machinery":"The load-bearing mechanism is orientation-as-distribution fitting. Ground-truth azimuth, polar, and rotation angles are converted into target probability masses over 1-degree bins: a Gaussian for the polar angle and a circular (wrapped) Gaussian for azimuth and rotation, with variances fixed at 2 degrees, 20 degrees, and 1 degree respectively. The network is trained with cross-entropy to reproduce those masses, and at inference the highest-probability bin is the prediction. This reformulation turns an unstable continuous regression into a smooth, well-behaved classification-adjacent objective that captures the near-equivalence of adjacent angles such as 29, 30, and 31 degrees. The second load-bearing component is the data pipeline: canonical-pose filtering, vision-language-model front-face annotation from orthogonal views with symmetry checks, and free-view rendering, which together remove the manual-labeling bottleneck and make training set scale a matter of rendering time.","core_discovery":"The paper's central claim is that object orientation can be estimated from a single free-view image by a model trained entirely on synthetic renders. Its model, Orient Anything, takes an image and outputs a confidence that the object has a meaningful front face, plus three angles specifying that front face relative to the camera: azimuth, polar, and in-plane rotation. The training labels are generated by a pipeline that filters 3D assets to canonical upright poses, asks a vision-language model to pick the front face from four orthogonal views with symmetry information, and then renders 40 random views per object to create two million annotated images. On top of this data, the paper argues, the decisive design is to predict each angle as a discretized probability distribution rather than a single number, which stabilizes training and yields transfers to real photos. Evaluated on five real-world benchmarks it has never seen, the model reports substantially lower angle errors than an expert 3D detection baseline, and when its outputs are converted to text for an LLM, spatial question accuracy on the paper's Ori-Bench rises from about 33 percent for the best baseline to 51.5 percent.","pith_inferences":["The definition of 'front' is a labeling convention, not a geometric invariant. A useful stress test would compare the model's accuracy when the same benchmark objects are re-annotated by several independent human labelers, or by a different vision-language model, to see how much the reported numbers depend on the annotator's notion of front.","Because the target distributions are fixed-width Gaussians, the loss assumes all angular errors of the same size are equally bad; predicting per-instance variance or learning the distribution shape could improve calibration on ambiguous, partly occluded, or symmetric objects.","The architecture is category-agnostic, so coupling it with an open-world detector and tracker should extend it to video, where temporal smoothing across frames could further reduce angle errors; the paper evaluates only independent still images."],"forward_implications":["Single-image orientation estimation becomes practical for arbitrary object categories without CAD models, reference views, or depth information as prerequisites.","Adding Orient Anything's angle estimates as text to an LLM lifts performance on orientation-based spatial questions, including part reasoning (left versus right paw or hand) and perspective-taking relations, far above what current vision-language models achieve directly.","The same estimates can score whether generated images obey a requested orientation or perspective, giving text-to-image systems a spatial reward signal they currently lack.","Because multi-view predictions are consistent, the model can vote across views to recover and re-align the orientation of 3D assets, which can then be used to expand the rendering pipeline.","The confidence head handles objects with no meaningful front face, so the method applies uniformly to symmetric or front-less objects instead of crashing or returning arbitrary angles."],"supporting_citations":[{"why":"Supplies the pool of 3D assets that the pipeline filters, annotates, and renders into training images.","marker":"[9]"},{"why":"The vision-language model that selects front faces from orthogonal views, thereby defining the annotation ground truth; also one of the comparison baselines.","marker":"[42]"},{"why":"The expert 3D detection baseline whose orientation accuracy the paper must beat.","marker":"[6]"},{"why":"A leading vision-language baseline for Ori-Bench and the LLM used in the Orient Anything+LLM spatial answering pipeline.","marker":"[18]"},{"why":"Provides the real-image pretrained visual encoder used for initialization, key to synthetic-to-real transfer.","marker":"[30]"},{"why":"Supplies the real images used for the Ori-Bench benchmark and the horizontal direction recognition evaluation.","marker":"[23]"},{"why":"Supplies real object-centric videos with pose annotations used in zero-shot evaluation.","marker":"[2]"},{"why":"Supplies real indoor AR scenes used in zero-shot evaluation.","marker":"[4]"},{"why":"Supplies real street-scene images used in zero-shot evaluation.","marker":"[11]"},{"why":"Supplies real indoor RGB-D scenes used in zero-shot evaluation.","marker":"[39]"}],"fun_headline_variants":["2M synthetic views teach single-image orientation","Orientation from one photo, trained on 2M renders","Single-image orientation via 2M synthetic views","From 2M renders: accurate orientation in a single photo","2M model renders unlock one-shot orientation estimation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the front-face labels used for training and evaluation are trustworthy: every label comes from one large vision-language model's choice among four orthogonal views, filtered by symmetry, so any systematic bias in that model's notion of 'front' is inherited by both the training signal and the accuracy numbers.","fun_headline_variants_meta":{"raw":{"variants":["2M synthetic views teach single-image orientation","Orientation from one photo, trained on 2M renders","Single-image orientation via 2M synthetic views","From 2M renders: accurate orientation in a single photo","2M model renders unlock one-shot orientation estimation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000446,"raw_usage":{"total_tokens":2264,"prompt_tokens":969,"completion_tokens":1295,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":1219}},"tokens_in":585,"tokens_out":1295,"duration_ms":9402,"temperature":1.0,"reasoning_tokens":1219,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:35:41.820089+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Annotate a random subset of the evaluation objects with several independent human labelers, asking them to pick the front face from the same orthogonal views, and measure inter-labeler agreement; if agreement on the annotator model's choices drops well below the 86 to 92 percent range the paper reports for its own annotation ablation, or if the model's accuracy falls sharply when evaluated against human-chosen fronts, the claim of robust, convention-independent orientation estimation is not established.","supporting_citations":[{"cited_title":"Objaverse: A universe of annotated 3d objects","cited_arxiv_id":null,"evidence_quote":"Supplies the pool of 3D assets that the pipeline filters, annotates, and renders into training images."},{"cited_title":"Omni3d: A large benchmark and model for 3d object detection in the wild","cited_arxiv_id":null,"evidence_quote":"The expert 3D detection baseline whose orientation accuracy the paper must beat."},{"cited_title":"Sun rgb-d: A rgb-d scene understanding benchmark suite","cited_arxiv_id":null,"evidence_quote":"Supplies real indoor RGB-D scenes used in zero-shot evaluation."}],"review_version":1}