{"id":"ad14d75e-6ffc-435e-ad4c-9312cf115be1","arxiv_id":"2411.15860","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Two-side generation and matching of intermediate views with a score distillation loss improves single-reference object pose estimation under large viewpoint changes.","lead":"This paper introduces a method to estimate an object's 3D pose from one reference photo and one query photo, using a diffusion model to generate intermediate views from both images and matching them to find the relative pose. The result is a large accuracy gain for large viewpoint changes, with no training and no 3D model required.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's pose search is driven by an unvalidated DreamFusion-style surrogate (Eq. 4) that is not shown to be minimized at the true pose; without a demonstrated equivalence to the two-sided matching objective in Eq. (3), the reported gains rest on a proxy.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing point: Eq. (4) is substituted for Eq. (3) 'without a proof that the loss is minimized at the correct pose.' I agree that this is the most serious concern about the central claim. The paper's empirical results are strong and consistent, and the two-side matching idea is plausible, but the entire pose search is driven by this surrogate. If the surrogate is biased, the reported gains could be an artifact of the specific hyperparameters and test sets rather than evidence that two-side matching works. A direct comparison between Eq. (4) and a concrete implementation of Eq. (3) on a subset would settle this concern. Because the concern is testable and the current evidence does not disprove the method, the conditional verdict remains appropriate; no change to the reader's verdict is needed.","tokens_in":16052,"tokens_out":6910,"duration_ms":71432,"concrete_test":"On a held-out subset of GSO/NAVI pairs (or new objects), compute the argmin of Eq. (5) on the coarse grid, and compare it to the argmin of a direct image-space matching score: generate both reference-side and query-side images at the same intermediate viewpoints with a fixed seed, and use a perceptual metric (e.g., LPIPS or DINO feature distance) to implement Eq. (3). If the two argmins disagree by more than about 15 degrees on a substantial fraction of queries, Eq. (4) is not a faithful proxy and the method's success is not explained by two-side matching. Also run the same search with M=16 and t in {0.2, 0.6}; if accuracy swings by more than the reported gaps, the loss landscape is too noisy to support the claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that minimizing Eq. (4) recovers the true relative pose because the DreamFusion-style score-distillation loss is a faithful surrogate for the two-sided matching objective in Eq. (3). This is load-bearing and unproven. Eq. (4) does not generate images from the query side at all; it uses only one set of reference-generated images Ir->i and treats the denoiser eps_Theta(· | Iq, Delta_theta_qi, Delta_phi_qi) as a pose-conditional similarity oracle. There is no derivation showing that the expected denoising residual is minimized at the true (theta_q, phi_q), nor that its global minimum is separated from wrong-pose minima on the search grid. Because the fixed time step t=0.4, sample count M=4, intermediate-view count N=64, and the coarse-to-fine schedule were all chosen empirically on the evaluation sets, a biased or flat surrogate could be masked by these hyperparameters. The reported 13-19 point gains therefore rest on an unvalidated proxy rather than on a demonstrated equivalence to Eq. (3).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a generalizable single-view object pose estimation method, Gen2SM, that uses a pre-trained diffusion model (Zero123) to estimate the relative elevation and azimuth between a reference image and a query image. Instead of directly generating the query view from the reference, the method generates images at N intermediate viewpoints from the reference side and matches them against the query image through a DreamFusion-style score distillation loss, followed by coarse-to-fine grid search and gradient refinement. Experiments on the GSO and NAVI datasets report large improvements over prior methods, especially for large viewpoint changes, along with ablations on the number of intermediate views, Monte-Carlo samples, time step, and refinement iterations.","tokens_in":16341,"tokens_out":3579,"duration_ms":32378,"significance":"If the reported gains are sound, the two-side matching idea is a meaningful step for training-free, category-agnostic pose estimation and could benefit downstream applications such as AR and 3D reconstruction. The paper includes extensive ablations, a runtime analysis, visualizations, and a promise of code release. However, the central loss in Eq. (5) is a heuristic surrogate whose equivalence to the stated two-sided matching objective is not established, and the key hyperparameters are selected on the test set, so the significance of the empirical claims is currently conditional.","major_comments":[{"comment":"The replacement of the two-sided matching objective in Eq. (3) by the DreamFusion-style score distillation loss in Eq. (4) is load-bearing but not justified. Eq. (5) is asymmetric: images are generated only from the reference side, and the query image is used as a conditioning signal for denoising reference-side generated views. There is no derivation showing that the expected denoising residual is minimized at the true (θ_q, ϕ_q), nor an empirical demonstration that the global minimum on the search grid is separated from wrong-pose minima. Please provide either a proof or a quantitative validation on held-out objects, e.g., a plot of f(θ_q, ϕ_q) against pose error or a comparison with directly optimizing Eq. (3) on a subset of the data.","section":"§3.3, Eqs. (4)–(5)"},{"comment":"All core hyperparameters (N=64, M=4, t=0.4, and 3 refinement iterations) are selected on the NAVI test set, and the paper reports no error bars, per-object variance, or significance tests. Since Table 5 shows a drop from 45.58% to 35.33% at 15° when t changes from 0.4 to 0.8, the reported margin over E2VG could partly reflect test-set selection. Please report results on a validation split or with confidence intervals across objects, and state the exact hyperparameter selection procedure.","section":"§4.1 and Tables 3–5"},{"comment":"The evaluation uses the benchmark constructed in the authors' prior work E2VG, including the filtering of NAVI to 27 objects and the random sampling of reference-query pairs. The paper does not state whether all baselines are evaluated on the exact same reference-query pairs or whether the baseline numbers are taken from E2VG's paper. To avoid a favorable selection, please release the exact test pairs and verify that all methods are run under identical conditions, or use an independently constructed benchmark.","section":"§4.1–4.2, Table 1"},{"comment":"The refinement ablation compares the proposed two-side matching refinement with a 'naive refinement' baseline, but the naive baseline is not specified in sufficient detail. It is unclear whether the naive approach uses the same initialization, step count, and optimization variables as the proposed method, differing only in the two-sided generation. Without this control, the conclusion that two-side matching improves refinement accuracy at 15° could be confounded by implementation differences.","section":"§4.3, Fig. 8"}],"minor_comments":[{"comment":"The abstract contains a typo: 're leased' should be 'released'. Also, the URL appears with a space before the comma.","section":"Abstract"},{"comment":"The caption has a typo: 'asd' should be 'and'.","section":"Table 1 caption"},{"comment":"The dataset name is written inconsistently as 'NA VI' and 'NAVI'; please use a single spelling.","section":"Throughout"},{"comment":"The term 'score function' for f in Eq. (5) is nonstandard, since f is a loss to be minimized rather than a score in the statistical sense; consider renaming it 'matching loss' or 'score distillation loss' for clarity.","section":"§3.3"},{"comment":"The sentence 'the comparison between our method and them serves as an ablation study of the two-sided matching' is grammatically unclear; consider rephrasing.","section":"§4.2"},{"comment":"The phrase 'restricts the application in some real-time scenarios' should be 'restricts its application in some real-time scenarios'.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's benchmark is inherited from the authors' own E2VG paper and the strongest baseline is their prior work, so the comparison is unusually favorable. I would encourage the editor to require the authors to release the exact test pairs and seeds, and ideally to report results on an additional benchmark or a held-out split. The proxy validation of Eq. (4)–(5) is essential before the claimed gains can be fully trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the two-side generating and matching scheme is a real, plausible extension of the diffusion-based pose estimation line (E2VG/IDPose), and the reported gains are consistent across synthetic and real data. The paper is worth a serious referee, but the evaluation protocol needs tightening before I'd trust the numbers.\n\nWhat's actually new: instead of generating the query view directly from the reference (E2VG) or inverting the diffusion process (IDPose), they generate intermediate views from both reference and query sides and match in that shared space. The use of the DreamFusion-style score distillation loss to approximate the image matching objective is a neat trick that avoids explicit image generation during search. The motivation section (Fig. 2) is clear, and the idea of using intermediate viewpoints to break large viewpoint changes into smaller steps is intuitive and directly addresses a known bottleneck.\n\nThe experiments show large and consistent improvements: roughly 13-19 points at 15 degrees over E2VG on both datasets, and even larger relative gains in the large-viewpoint-change subset. They also include a runtime analysis and a light version, which is useful for practitioners. The limitations section is honest about dependence on the diffusion model and the inability to handle scenes directly.\n\nThe soft spots are mostly in the experimental protocol. First, the key hyperparameters (N=64, M=4, t=0.4, 3 refinement iterations) are all selected on the NAVI test set. That inflates the reported numbers and makes the comparisons less clean. Second, there are no error bars or significance tests, so we don't know if the 13-19 point gaps are stable. Third, the benchmark is inherited from the authors' own E2VG paper, and the strongest baseline is their own prior work; an independent evaluation on a different benchmark (e.g., CO3D or a held-out split) would help. The stress-test note about the surrogate loss is a legitimate theoretical concern: Eq. (4) is not shown to be minimized at the true pose. But in practice this is a heuristic from DreamFusion, and the paper explicitly says they approximate. The bigger issue is that the heuristic's behavior is not analyzed at all. A simple experiment plotting the loss landscape vs. pose error would go a long way.\n\nWho is this for: anyone working on category-agnostic or single-reference pose estimation, and people interested in using diffusion models as priors for geometric vision. It reads like a solid workshop-to-conference paper with revisions.\n\nRecommendation: send to peer review. Require code release, validation-based hyperparameter selection (or at least a discussion of the test-set tuning), error bars, and ideally a loss-landscape analysis. Conditionally accept after that.","headline":"Two-side matching with score distillation is a plausible extension that deserves serious review, but test-set-tuned hyperparameters and an unvalidated surrogate loss mean the numbers should be treated with caution.","tokens_in":16842,"tokens_out":2195,"would_cite":true,"duration_ms":19643,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that estimating object pose by generating and matching intermediate views from both the reference and query images, rather than generating the query view directly, substantially improves single-image pose estimation…","keywords":["object pose estimation","single-view","diffusion model","two-side matching","novel view generation","score distillation","category-agnostic","large viewpoint change"],"falsifier":"A direct test: take a set of objects with known 3D models, render a reference view and a query view at a known relative pose, run the method, and record the score function values across the coarse search grid. If the minimum of the score function does not coincide with the true pose for a substantial fraction of test cases, or if the score is flat or multi-modal near the true pose, the score proxy is biased and the central claim would fail. This test can be run offline without any retraining.","tokens_in":15873,"feed_emoji":"🎯","tokens_out":6432,"duration_ms":49795,"temperature":0.7,"pith_summary":"This paper presents a method to estimate an object's pose from a single RGB image, using only one reference image of the same object. The method uses a pre-trained diffusion model to generate novel views, but instead of generating the query view directly from the reference, it generates intermediate views from both images and matches them on a fixed set of intermediate viewpoints. The two-sided matching reduces the generation burden when the viewpoint change is large, and the paper reports that it achieves the best rotation accuracy on both the synthetic GSO and real NAVI benchmarks, outperforming the strongest prior diffusion-based method by about 13 to 19 percentage points at the 15-degree threshold. No training on the object, no CAD model, and no multiple views are required.","feed_headline":"Two-side matching lifts pose accuracy by up to 19 points","feed_subtitle":"Generating intermediate views from both images handles large viewpoint changes that break prior methods.","key_machinery":"The key machinery is the two-side matching objective and its score-function approximation. The paper replaces the direct generation of the query view from the reference (which degrades at large viewpoint changes) with a set of intermediate viewpoints sampled evenly on the upper hemisphere via the Fibonacci sphere algorithm. For each intermediate viewpoint, it generates an image from the reference and, conceptually, from the query, and matches them. Because generating both sets of images is costly, the matching distance is approximated by the noise-prediction loss of the pre-trained diffusion model, as in DreamFusion: one adds noise to the intermediate image generated from the reference and asks the denoiser, conditioned on the query image and the relative viewpoint change, to predict that noise; the L2 residual is the matching score. A coarse-to-fine search over azimuth and elevation, followed by gradient descent refinement, finds the pose that minimizes the score.","core_discovery":"The central discovery is that a two-side generating-and-matching scheme makes diffusion-based, category-agnostic object pose estimation accurate even for large viewpoint changes. Given a reference image with known pose and a query image of the same object, the method first generates images of the object from N intermediate viewpoints using a pre-trained novel-view diffusion model, once conditioned on the reference and once on the query. It then searches over the query pose by minimizing a score function that approximates the matching distance between these two generated image sets; the score function is the DreamFusion-style score distillation loss between the denoiser's noise prediction and the added noise. The paper reports that this approach achieves the best rotation accuracy on both GSO and NAVI, and that under viewpoint changes of 120 degrees or more it roughly doubles the accuracy of the strongest prior method.","pith_inferences":["The approximation in Eq. (4) is not proven to have its minimum at the true pose; if the noise-prediction residual is systematically biased, the pose search could be wrong even when intermediate views align. This could be tested by evaluating the score function at ground-truth poses against nearby poses on a large set of known objects.","The intermediate viewpoints are fixed and object-independent; adapting them to an object's shape or refining them iteratively might improve accuracy or reduce the number of generated views needed.","The method assumes a single centered object in both images; extending to scenes or occluded objects would require detection or segmentation, which the paper only sketches.","The reported gains are on two datasets with specific object distributions; broader testing across more diverse objects, lighting, and backgrounds would clarify how far the two-side matching advantage extends."],"forward_implications":["A single reference image and no 3D model or multi-view capture suffice, so the method can be applied to arbitrary novel objects at inference time.","Large viewpoint changes, the failure case of prior diffusion pose estimators, become tractable because the matching is done on small, well-conditioned steps.","The pipeline is modular with respect to the generative prior: improving the novel-view diffusion model should directly raise pose accuracy.","The coarse-to-fine search and refinement are parallelizable, and the paper's light variant processes a query in about one second on a single GPU.","Accurate single-image pose estimates enable downstream applications such as augmented-reality object insertion, 3D reconstruction, and robot manipulation from a single photo."],"supporting_citations":[{"why":"Supplies the pre-trained Zero-1-to-3 diffusion model that generates novel-view images used in both the naive and two-side matching setups.","marker":"[34]"},{"why":"Supplies the DreamFusion score distillation loss used to approximate the two-side matching distance in Eq. (4).","marker":"[49]"},{"why":"The prior diffusion-based method E2VG whose naive generation-and-matching scheme is the main baseline and bottleneck the paper improves upon.","marker":"[58]"},{"why":"The other diffusion-based method IDPose that inverts the diffusion process; used as a baseline and comparison for the matching scheme.","marker":"[9]"},{"why":"Provides the elevation predictor used to initialize the coarse pose search.","marker":"[33]"},{"why":"The synthetic GSO dataset used for evaluation of rotation accuracy.","marker":"[13]"},{"why":"The real NAVI dataset used for evaluation of rotation accuracy.","marker":"[22]"}],"fun_headline_variants":["Two-sided view synthesis matches pose for unseen objects","One reference, no 3D: pose estimation generalizes","Dual generation handles extreme viewpoint changes","Generate both views, match, and get pose without 3D model","Single-view pose estimation that doubles accuracy at 120°"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the DreamFusion score distillation loss used in Eq. (4) is a valid proxy for the two-side image matching objective: the paper assumes that minimizing the noise-prediction residual is equivalent to finding the pose that makes the generated image sets match, without proving that the loss is minimized at the correct pose.","fun_headline_variants_meta":{"raw":{"variants":["Two-sided view synthesis matches pose for unseen objects","One reference, no 3D: pose estimation generalizes","Dual generation handles extreme viewpoint changes","Generate both views, match, and get pose without 3D model","Single-view pose estimation that doubles accuracy at 120°"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001679,"raw_usage":{"total_tokens":6606,"prompt_tokens":845,"completion_tokens":5761,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":5682}},"tokens_in":461,"tokens_out":5761,"duration_ms":31923,"temperature":1.0,"reasoning_tokens":5682,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:48:42.255064+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: take a set of objects with known 3D models, render a reference view and a query view at a known relative pose, run the method, and record the score function values across the coarse search grid. If the minimum of the score function does not coincide with the true pose for a substantial fraction of test cases, or if the score is flat or multi-modal near the true pose, the score proxy is biased and the central claim would fail. This test can be run offline without any retraining.","supporting_citations":[{"cited_title":"Zero-1-to-3: Zero-shot one image to 3d object","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained Zero-1-to-3 diffusion model that generates novel-view images used in both the naive and two-side matching setups."},{"cited_title":"Google scanned objects: A high- quality dataset of 3d scanned household items","cited_arxiv_id":null,"evidence_quote":"The synthetic GSO dataset used for evaluation of rotation accuracy."},{"cited_title":"NA VI: Category- agnostic image collections with high-quality 3d shape and pose annotations","cited_arxiv_id":null,"evidence_quote":"The real NAVI dataset used for evaluation of rotation accuracy."}],"review_version":1}