{"id":"214743ee-feae-4730-9634-edaa1b1f9802","arxiv_id":"2412.14464","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A two-stage pipeline, volume-triplane reconstruction plus latent diffusion with iterative view feedback, reports state-of-the-art single and few-view novel view synthesis on CO3D, GSO, and SRN-Car.","lead":"LiftRefine creates new views of an object from one or a few photos by first building a rough 3D model and then using an image-diffusion model to fill in hidden areas. It feeds newly generated views back into the 3D model in a loop, gradually sharpening the final images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The progressive-feedback novelty is not isolated: no ablation compares iterative feedback against one-shot diffusion refinement with equal sampling budget, so the central claim that feedback improves the 3D representation is unsupported.","rationale":"The reader's weakest assumption identifies the same structural risk: the progressive inference loop feeds diffusion-generated views back into a reconstructor trained only on real views, and this feedback is never ablated against an equal-cost one-shot alternative. I agree with that assessment and with the CONDITIONAL verdict, because the concern is substantial but addressable; it does not by itself invalidate the deterministic reconstruction results or the diffusion model's ability to generate sharp views. The proposed controlled comparison would settle whether the progressive mechanism is genuinely responsible for the reported FID/LPIPS gains, and whether error compounding from inconsistent pseudo-views is a real problem. I keep the verdict unchanged rather than moving it, since the paper already requires additional evidence before its central claims can be fully accepted.","tokens_in":14738,"tokens_out":3685,"duration_ms":32423,"concrete_test":"Run a controlled ablation on CO3D-Hydrant (or GSO) with a fixed budget of K diffusion calls and the same K intermediate camera poses. Condition A is the proposed progressive loop: generate each intermediate view conditioned on the current buffer and append it. Condition B is one-shot refinement: generate the K intermediate views in parallel, conditioned only on the input view and its first-stage feature map, then append all K at once and run the final reconstructor. Use identical DDIM steps, guidance, seeds, and final rendering. If B matches or beats A on FID and LPIPS while A does not show better multi-view consistency, the progressive-feedback mechanism is not supported. Also append ground-truth intermediate views to the buffer as an upper-bound check on whether the reconstructor can exploit extra views at all.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 defines the central contribution: generated intermediate views are appended to an image buffer and the reconstructor is re-run on the growing buffer. The reconstructor (Section 3.1 and supplement Section 7) was trained with 1-3 ground-truth input views only. At inference, after one iteration the buffer contains a diffusion sample that the supplement (Section 9, GSO) itself describes as having 'minor inconsistencies across views.' The paper never tests whether the reconstructor can assimilate such out-of-distribution pseudo-views without corrupting the triplane, nor does it isolate the feedback mechanism from the extra diffusion calls. Table 5 varies the iteration count (0, 1, 2, 4, 6, 8), but every nonzero run uses feedback, so the FID improvement could come from more diffusion sampling or from conditioning on more views, not from progressive feedback. Without an equal-budget comparison to one-shot refinement (same number of diffusion calls, same intermediate poses), the paper's second contribution and the claimed 'consistently surpasses' results in the diffusion setting are not established. This is the load-bearing concern because if feedback does not help or actively hurts, the progressive-refinement novelty collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LiftRefine, a two-stage method for single- and few-view novel view synthesis. Stage 1 trains a reconstruction model that lifts input images into a coarse volume and then a fine tri-plane representation, which can be rendered to images and feature maps. Stage 2 trains a latent diffusion model conditioned on the rendered feature map and a CLIP embedding of the input to hallucinate details in occluded regions. The paper's third contribution is a progressive inference procedure in which diffusion-generated intermediate views are appended to an image buffer and the reconstructor is re-run on the growing buffer, eventually rendering the final view. Experiments on CO3D, GSO, and ShapeNet-SRN Cars report improvements over several baselines, with ablations on the 3D representation, backbone, and number of progressive iterations.","tokens_in":14990,"tokens_out":5708,"duration_ms":43601,"significance":"The volume-plus-triplane reconstruction design is a sensible engineering contribution that appears to give strong deterministic reconstruction results, and the FID improvements from diffusion refinement are consistent with the literature on generative view synthesis. If the progressive-inference claim were properly isolated, the method would be a useful addition to the single/few-view reconstruction literature. The paper provides detailed architecture and training details in the supplement, and the qualitative results show clear improvements in occluded regions. However, the key novelty--iterative feedback of generated views into the reconstructor--is not supported by an experiment that separates the feedback mechanism from the extra diffusion compute, and the abstract's Objaverse claim is unsupported. These gaps prevent acceptance as is.","major_comments":[{"comment":"The central claim that progressive inference improves reconstruction quality is not isolated. In Table 5, iteration 0 is the deterministic baseline, and every nonzero iteration includes both a diffusion sampling step and a feedback of the generated intermediate view into the reconstructor buffer. The improvements in FID and LPIPS between iterations 1 and 8 could therefore be caused by the additional diffusion calls, by the increased number of conditioning views, or by the progressive feedback itself. Please add an equal-budget comparison: hold the number of diffusion calls fixed (e.g., 4) and compare (a) one-shot diffusion applied to the final target view feature map, (b) diffusion applied to the final target view after feeding all generated intermediate views into the buffer in a single batch, and (c) the proposed progressive loop. Only (c) versus (b) isolates the iterative nature, and only (c) versus (a) measures the cost-effectiveness of feedback.","section":"Section 3.3, Table 5"},{"comment":"The abstract claims empirical superiority on the 'large-scale Objaverse dataset,' but no quantitative evaluation on Objaverse appears anywhere in the main paper or supplement. Objaverse appears only as a training set. Either add Objaverse evaluation results (even a table in the supplement) or revise the abstract to remove this claim.","section":"Abstract and Section 4"},{"comment":"The progressive feedback loop assumes the reconstruction model generalizes to diffusion-generated pseudo-views, but the reconstructor was trained only on 1-3 ground-truth views (Supplementary Section 7), and Supplementary Section 9 states that the diffusion-generated GSO samples exhibit 'minor inconsistencies across views.' No experiment measures whether feeding such out-of-distribution views into the reconstructor degrades the triplane representation compared with feeding a ground-truth view at the same pose. Since this premise is structural to the method, it should be tested directly, for example by replacing the generated intermediate view with the ground-truth image at the same camera pose and measuring the change in final reconstruction metrics.","section":"Supplementary Section 9 / Section 3.3"}],"minor_comments":[{"comment":"The text contains the typo 'SpareFusion' in the sentence discussing FID; it should be 'SparseFusion.'","section":"Section 4.1"},{"comment":"The sentence 'one ought to to utilize' has a duplicated 'to'; remove the repetition.","section":"Related Work"},{"comment":"The word 'empowerd' should be 'empowered' in the final paragraph.","section":"Conclusion"},{"comment":"The supplement lists both ResNet34 and Dino-v2 as possible feature extractors, but the main paper never states which one is used in the reported experiments; please specify and note whether results differ between the two.","section":"Supplementary Section 6.1"},{"comment":"Table 4 lists OpenLRM as an image-to-triplane backbone, but OpenLRM is a full reconstruction method; clarify what part is used as the backbone and how it is trained in this ablation.","section":"Table 4"},{"comment":"Table 5 reports single runs with no variance or significance information; the FID differences between adjacent settings are small (e.g., 34.7 vs 34.0), so error bars or multiple seeds are needed to support the ranking.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a competent systems paper with promising deterministic reconstruction results and plausible FID gains. The missing isolation experiment for progressive inference is a real gap, but it is fixable within a revision: the authors can add an equal-budget one-shot refinement comparison and a pseudo-view corruption analysis. The unsupported Objaverse claim should also be corrected. I recommend major revision rather than rejection because the core approach is sound and the main defect is an experimental design issue rather than a fundamental error."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read LiftRefine. The engineering is solid and the two-stage volume-triplane reconstructor plus latent diffusion yields real gains on CO3D and SRN-Car. But the paper's marquee contribution—progressive inference—is not isolated as the cause of the improvement, and the abstract promise of Objaverse superiority never materializes in the experiments.\n\nWhat's new: the volume-to-triplane lifting is a sensible integration of previously separate ideas (ViewsetDiffusion's volume, LRM's triplane, GeNVS-style conditioning). The ablations for the reconstructor are useful: they show the coarse volume matters and the triplane upsampler is not just interpolation. The 4x-A100 training footprint compared to LRM's 128 GPUs is a genuine practical win.\n\nSoft spots: the stress-test note is right. Table 5 varies the number of progressive iterations, but every nonzero row uses feedback; there is no baseline that spends the same number of diffusion calls but adds all intermediate views at once, or that refines the final view without feeding them back into the reconstructor. So you cannot attribute the FID drop to the feedback loop rather than to simply sampling more views from the diffusion model. The reconstructor was trained on 1–3 real views; adding diffusion outputs that the supplement itself calls 'minorly inconsistent' (Sec. 9) is a distribution shift that needs explicit testing. Also, the abstract's 'large-scale Objaverse dataset' claim is unsupported—no Objaverse evaluation table exists in the main text or the supplement. GSO baseline setup is under-specified, and there are no error bars anywhere.\n\nNone of this is fatal: the core reconstruction result is credible, and the deterministic-setting numbers stand on their own. But the progressive-refinement novelty stays under-supported until the authors run the equal-budget ablation. If I were handling this, I'd send it to review and ask for that experiment, plus error bars and a substantiated or removed Objaverse claim. Then it would be a solid contribution.","headline":"Solid two-stage view synthesis with real benchmark gains, but the progressive-refinement novelty is not isolated and the Objaverse claim is unsupported.","tokens_in":15513,"tokens_out":2790,"would_cite":true,"duration_ms":23547,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LiftRefine's progressive 3D lifting surpasses prior view synthesis.","keywords":["novel view synthesis","neural radiance field","tri-plane representation","volumetric rendering","latent diffusion","progressive inference","single-view reconstruction","few-view reconstruction"],"falsifier":"Run a controlled comparison on CO3D-Hydrant that keeps the total number of diffusion denoising steps fixed and compares (a) one diffusion pass straight to the target view against (b) the progressive loop with four interpolated views; if (b) does not beat (a) on FID or LPIPS, the progressive-refinement claim is not supported. Additionally, measure the multi-view consistency of the generated buffer (e.g., by re-rendering from the final tri-plane and checking reprojection error against the intermediate views) and check whether consistency degrades as the number of iterations grows.","tokens_in":14564,"feed_emoji":"🖼️","tokens_out":8378,"duration_ms":59076,"temperature":0.7,"pith_summary":"This paper proposes a two-stage method for synthesizing novel views of an object from one or a few input images. The first stage lifts the input into a coarse 3D volume and then into a fine tri-plane representation, which is rendered to both an image and a feature map at the target camera pose. The second stage uses a latent diffusion model, conditioned on that feature map and a CLIP embedding of the input, to hallucinate details in occluded regions. The added contribution is progressive inference: the output of the diffusion model at an intermediate camera pose is appended to the input buffer, and the two stages are repeated so that the reconstructor gradually sees more of the object. The paper reports consistent gains over SparseFusion, ViewsetDiffusion, OpenLRM, Splatter Image, and LaRa on CO3D, GSO, and ShapeNet-SRN Cars, with the diffusion-based setting improving LPIPS and FID while pixel-wise PSNR eases slightly.","feed_headline":"LiftRefine's progressive 3D lifting surpasses prior view synthesis","feed_subtitle":"A volume-triplane reconstructor and a diffusion refiner iterate, sharpening occluded regions while keeping views consistent.","key_machinery":"The load-bearing mechanism is the progressive inference loop, supported by a two-stage model. Stage 1, the Lift reconstructor, projects image features into a low-resolution volume, then upsamples that volume through a tri-plane decoder into a high-resolution tri-plane that can be rendered to images and feature maps. Stage 2, the Refine diffusion model, is a latent diffusion U-Net initialized from Zero123 that denoises a target latent conditioned on the rendered feature map and on the CLIP embedding of the input. In progressive inference, the reconstructor renders a feature map at an interpolated camera pose, the diffusion model generates an intermediate view, that view is appended to the image buffer, and the process repeats until the target view is rendered. This loop is what the paper claims couples 3D consistency with 2D realism.","core_discovery":"The central claim is that a 3D reconstructor and a 2D diffusion refiner, when combined through a progressive loop, give both the multi-view consistency of a neural field and the sharpness of a generative model. The reconstructor alone produces blurry but view-consistent results in unseen regions; the diffusion model alone produces sharp but somewhat inconsistent views. Progressive inference lets the two correct each other: each newly generated view is treated as an additional input to the reconstructor, and the final tri-plane is used to render the target. On the reported benchmarks the method beats the deterministic and probabilistic baselines, and the paper emphasizes that the volume-plus-triplane design needs far less compute than transformer-based LRM (4 A100 GPUs for 7 days versus 128 for 3 days).","pith_inferences":["The feedback loop is a form of self-conditioning at test time; a direct extension would weight generated views by the diffusion model's estimated confidence so that uncertain pseudo-views do not degrade the reconstructor (the paper does not test such weighting).","A same-budget ablation—same total diffusion denoising steps spent on one target view versus spread over n interpolated views—would isolate whether the gain comes from the loop itself or simply from more sampling; the paper does not report this ablation.","Since the diffusion model is initialized from a pretrained image-to-3D model, the method likely inherits that model's biases on categories or viewpoints it was not trained on; testing on rare CO3D categories beyond Hydrant, Teddybear, Vase, and Plant would map those failure modes.","The supplement notes 'minor inconsistencies' in generated views; a quantitative multi-view consistency metric on the final render could tell whether progressive inference converges or drifts as iterations grow."],"forward_implications":["If the claims hold, single-image synthesis closes much of the gap to few-view synthesis in occluded regions, since the CO3D comparison shows one-view diffusion output approaching three-view deterministic quality.","The progressive loop gives a direct quality knob: more interpolation steps trade pixel-wise metrics for distribution-level realism, with 4 iterations offering the best balance on CO3D-Hydrant.","The two-stage design appears more data- and compute-efficient than transformer-based tri-plane reconstruction, which matters for scaling to large 3D datasets.","The diffusion setting's lower FID and LPIPS suggest the rendered views are perceptually closer to real novel-view distributions, not just pixel-wise closer to ground truth.","Because the reconstructor is a generalizable neural field, the method supports arbitrary output poses, unlike fixed-set multi-view generators such as SyncDreamer."],"supporting_citations":[{"why":"Supplies the volume-lifting encoder architecture and serves as a key baseline for comparison.","marker":"Szymanowicz, Rupprecht, and Vedaldi 2023b"},{"why":"Provides the latent diffusion formulation and loss used in the Refine stage.","marker":"Rombach et al. 2021"},{"why":"Supplies the conditional diffusion U-Net architecture and pretrained initialization.","marker":"Liu et al. 2023a"},{"why":"The transformer-based tri-plane baseline whose compute and small-data performance the paper contrasts with its volume-plus-triplane design.","marker":"Hong et al. 2024"},{"why":"The SparseFusion baseline compared on CO3D.","marker":"Zhou and Tulsiani 2023"},{"why":"The LaRa baseline compared in the few-view GSO experiments.","marker":"Chen et al. 2024"},{"why":"The Splatter Image baseline compared in the single-view GSO experiments.","marker":"Szymanowicz, Rupprecht, and Vedaldi 2023a"}],"fun_headline_variants":["LiftRefine's iterative 3D lifting and diffusion refine views","Progressive reconstruction plus diffusion sharpens occluded views","Volume-triplane and diffusion loop beat prior view synthesis","LiftRefine: coarse-to-fine 3D lifting with diffusion correction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The progressive loop assumes that diffusion-generated intermediate views—which the supplement concedes are slightly inconsistent across angles—can be safely added to the input buffer and will strengthen rather than corrupt the volume-triplane reconstruction, even though the reconstructor was trained on real views only.","fun_headline_variants_meta":{"raw":{"variants":["LiftRefine's iterative 3D lifting and diffusion refine views","Progressive reconstruction plus diffusion sharpens occluded views","Volume-triplane and diffusion loop beat prior view synthesis","LiftRefine: coarse-to-fine 3D lifting with diffusion correction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000505,"raw_usage":{"total_tokens":2443,"prompt_tokens":901,"completion_tokens":1542,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":1481}},"tokens_in":517,"tokens_out":1542,"duration_ms":10420,"temperature":1.0,"reasoning_tokens":1481,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:12:52.651985+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled comparison on CO3D-Hydrant that keeps the total number of diffusion denoising steps fixed and compares (a) one diffusion pass straight to the target view against (b) the progressive loop with four interpolated views; if (b) does not beat (a) on FID or LPIPS, the progressive-refinement claim is not supported. Additionally, measure the multi-view consistency of the generated buffer (e.g., by re-rendering from the final tri-plane and checking reprojection error against the intermediate views) and check whether consistency degrades as the number of iterations grows.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The transformer-based tri-plane baseline whose compute and small-data performance the paper contrasts with its volume-plus-triplane design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The SparseFusion baseline compared on CO3D."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The LaRa baseline compared in the few-view GSO experiments."}],"review_version":1}