{"id":"5dbe0d07-1365-4b68-b0f2-75dca94e7084","arxiv_id":"2504.20026","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A feed-forward transformer reconstructs shape, PBR materials, and view-dependent radiance from 3 to 6 posed images in under a second, rivaling slower optimization-based inverse rendering.","lead":"The paper presents LIRM, a transformer that reconstructs a 3D object's shape, materials, and view-dependent appearance from only a few photos in under a second. It is built to make relightable 3D content creation practical, with results that rival optimization-based methods that take hours.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-data comparison is confounded by ground-truth-mask cropping and by using 18 images instead of the advertised 3-6, so the headline edge over optimization baselines is not yet established.","rationale":"The reader's weakest_assumption identified the mask-based cropping on Stanford-ORB; my analysis agrees and sharpens it into two distinct confounds: the method requires ground-truth masks as an input, and the strongest reported results use 18 images rather than the advertised 3-6. Both conditions bear directly on the central comparison with optimization-based methods, because the mask crop changes the effective resolution and the image count changes the task difficulty. The paper provides no no-mask ablation and no matched preprocessing for baselines, so the Chamfer-distance advantage in Table 5 could stem from preprocessing rather than model quality. I do not see an internal inconsistency in the method itself: the architecture, loss functions, and ablations are coherent, and the synthetic experiments support the component-wise claims. The baseline-provenance issue for MeshLRM (reimplemented rather than official) is real but secondary, since the headline claim is about optimization-based methods from the leaderboard. The concern does not warrant rejection; it warrants making the stated scope precise, which is already reflected in the reader's CONDITIONAL verdict. Hence no verdict change.","tokens_in":23272,"tokens_out":8558,"duration_ms":87268,"concrete_test":"Re-run the Stanford-ORB evaluation under three matched conditions: (1) LIRM with full-image inputs resized to the model's input resolution, no mask; (2) LIRM with masks produced by an off-the-shelf segmenter instead of ground-truth masks; and (3) Neural-PBIR/InvRender with the same object-centric mask crop and resize used for LIRM. Recompute the Table 5 relighting, view-synthesis, and CD metrics. If LIRM's advantage persists under (1)/(2) and baselines do not improve under (3), the concern is resolved; otherwise the headline claim should be restricted to 'given foreground masks' and to 18-input-image evaluation. Reporting the 6-image (1st-set) results for all methods would also clarify the advertised 3-6 image regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LIRM reconstructs shape, materials, and view-dependent radiance from 3-6 posed images with quality on par or better than dense-view optimization methods (Sec 4). The real-data evidence for this claim is Table 5 on Stanford-ORB. Two load-bearing conditions are left untested. First, the inputs are not just posed images: Sec 4 states the model receives masked images plus background images, and 'only crop and resize the region of interest based on the object's foreground mask.' A ground-truth foreground mask is therefore required to form the inputs and to crop; the paper reports no evaluation without masks. Second, the best numbers use 18 images (3 sets of 6), not 3-6; the 1st-set results (6 images) are notably weaker (LIRM-NDE 1st relighting PSNR-H 24.25 vs Neural-PBIR 26.01). Because Stanford-ORB objects occupy a small central region, mask-based cropping and resizing gives LIRM substantially more effective pixels on the object than optimization baselines that process full images; this alone could explain the Chamfer-distance win (0.31 vs 0.43). Without ablating the mask input or normalizing pre-processing across methods, the comparison is confounded and the practical scope of the claim is narrower than the abstract states.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"LIRM is a transformer-based feed-forward inverse rendering model that reconstructs shape (as a neural SDF), BRDF materials, and view-dependent radiance from sparse posed images in under a second. The paper makes three main technical contributions: a progressive update module that refines reconstruction as additional input view sets are provided without growing the transformer token count; a hexa-plane neural SDF representation that improves texture detail relative to tri-planes; and a neural directional encoding (NDE) mechanism with multiple predicted panoramas for view-dependent appearance. The model is trained on a large synthetic dataset (600k objects, 38M rendered images) with PBR materials and HDR environment lighting. Experiments on GSO, ABO, and DTC show strong view synthesis and inverse rendering results, and on Stanford-ORB the paper claims reconstruction quality on par with or better than dense-view optimization-based inverse rendering methods at a fraction of the runtime.","tokens_in":23455,"tokens_out":6124,"duration_ms":62081,"significance":"If the claims hold, LIRM would be a notable advance toward practical sparse-view relightable 3D reconstruction, combining the speed of large reconstruction models with material decomposition and view-dependent radiance. The paper has considerable strengths: each of the three architectural components is ablated (update stages, hexa-plane versus tri-plane, NDE versus diffuse-only, camera trajectories); the training dataset is large and designed to mimic real capture conditions; the training pipeline avoids the stability issues of differentiable marching cubes; and the authors include an honest limitations section. The main risk is that the central real-data comparison rests on an evaluation protocol that is narrower than the abstract's promise, as detailed in the major comments.","major_comments":[{"comment":"The headline claim that LIRM 'achieves reconstruction quality on par and even better than state-of-the-art optimization-based methods' from '3 to 6 posed images' is not supported by the real-data experiment as reported. The best LIRM rows in Table 5 use 18 images (three sets of six), not 3-6; the one-set rows are consistently below Neural-PBIR on relighting (e.g., LIRM-NDE 1st PSNR-H 24.25 vs Neural-PBIR 26.01). In addition, the inputs are masked images plus background images, and Sec. 4 states that the authors 'only crop and resize the region of interest based on the object's foreground mask,' so a ground-truth foreground mask is required and LIRM receives an object-centric crop at higher effective resolution than the optimization baselines, which process the full images. This preprocessing asymmetry alone could explain part of the Chamfer-distance improvement (0.31 vs 0.43). The paper reports no experiment without masks or with matched preprocessing, so the practical scope of the central claim is narrower than the abstract states. Please add such experiments or explicitly restrict and rephrase the claim.","section":"Sec. 4, Table 5, Abstract"},{"comment":"The rows labeled 'MeshLRM [85]' in Tables 1 and 2 are not the official released model (which has not been open-sourced); Supplementary Sec. 8 discloses that the authors trained their own 'LRM-V olSDF' model with the same architecture on the authors' Shutterstock-based dataset. The main text does not disclose this provenance. Because MeshLRM is the primary LRM baseline and the comparison is the basis for the claim that LIRM outperforms the prior state of the art, the main text should state that these numbers come from a re-implementation trained on a different dataset, and the comparison should be framed accordingly. If official numbers become available, they should be used or the discrepancy discussed.","section":"Sec. 4, Tables 1-2; Supplementary Sec. 8"}],"minor_comments":[{"comment":"The row header 'LIRM-NDE 3nd' should read 'LIRM-NDE 3rd'.","section":"Table 5"},{"comment":"The main-text paragraph beginning 'Supplementary We will include more implementation details...' contains placeholder language ('We will include', 'We will add comparisons'), even though the arXiv supplementary already contains these items; this text should be replaced with a description of the actual supplementary contents.","section":"Sec. 5, Supplementary paragraph"},{"comment":"The overview states that 'Our supplementary material consists of three parts' but then lists four bullet points; the enumeration should be corrected.","section":"Supplementary Sec. 6"},{"comment":"In the paragraph on testing datasets, the sentence beginning 'esting datasets captured under uniform lighting...' is missing a leading 'T'; fix the typo.","section":"Supplementary Sec. 7"},{"comment":"The naming of the re-implemented baseline is inconsistent: the main text uses 'MeshLRM [85]' while the supplementary calls it 'LRM-V olSDF'; use one name and clearly identify it as a re-implementation.","section":"Tables 1-2 and Supplementary Sec. 8"},{"comment":"The 'CD' entry for MetaLRM is listed as '-'; clarify whether geometry was not evaluated for this method or the value is unavailable.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an arXiv v1 with some leftover placeholder text ('We will include...') that should be cleaned up before resubmission. Also, several baselines are from the authors' own group (Neural-PBIR, MetaLRM) and the MeshLRM comparison is a re-implementation; this is not disqualifying, but the editorial process should ensure that the re-implemented baseline is clearly labeled and, where possible, independently verified. The main concern is the Stanford-ORB protocol: mask-based cropping and the use of 18 images should be disclosed in the abstract and the claims tempered accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a serious system paper, and most of what it claims at the architectural level holds up. The update module — recycling the predicted 3D tokens through self-attention with new image tokens — is the real contribution. It lets you add input views without growing GPU memory, and the ablations show consistent gains from the 1st to the 3rd set plus robustness to input order, which the supplement tests explicitly. The hexa-plane SDF and the multi-panorama NDE are both well-motivated and ablated; the overfitting test in Fig. 5 shows the NDE design does real work on specular highlights, and the paper is upfront that mirror reflections still defeat it. Evaluation is extensive: GSO, ABO, and Stanford-ORB, plus BRDF metrics against optimization methods, training-stage ablations, and camera-trajectory tests. No circularity — the central claims are empirical on external benchmarks.\n\nThe soft spot is the one the stress test flags, and I think it lands. Table 5's real-data numbers carry the headline claim, but LIRM's inputs are masked and cropped to the object using ground-truth foreground masks, and the best numbers use 18 images (3 sets of 6), not the 3–6 promised in the abstract. The optimization baselines process inputs under their own protocols, so LIRM gets substantially more effective pixels on the object. Without an ablation that strips the mask/crop advantage, the Chamfer-distance win (0.31 vs 0.43) cannot be cleanly attributed to the model. The one-set (6-image) results trail Neural-PBIR on relighting and view synthesis, so “on par and even better than optimization” is plausible but not yet established on real data.\n\nTwo smaller issues. Tables 1–2 label the baseline “MeshLRM [85],” but the supplement reveals it is a reimplementation trained on the authors' own dataset — disclosed, but the main text should say so. And there is no code or weights; with a proprietary 600k-object training set, independent verification is difficult.\n\nWho this is for: anyone working on feed-forward inverse rendering or LRM-style reconstruction; the update-module idea alone is worth citing. My recommendation: send it to peer review, and require the mask-crop ablation and a corrected baseline label before acceptance.","headline":"Serious feed-forward inverse-rendering system with a genuinely useful progressive-update module; the real-data claim of beating optimization methods is plausible but rests on masked crops and 18 images rather than the advertised 3–6.","tokens_in":24183,"tokens_out":5530,"would_cite":true,"duration_ms":49651,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new transformer turns 3–6 photos into 3D objects in under a second","keywords":["inverse rendering","large reconstruction model","progressive reconstruction","hexa-plane neural SDF","neural directional encoding","view-dependent radiance fields","relighting","sparse-view 3D reconstruction"],"falsifier":"Run LIRM on Stanford-ORB full frames without foreground-mask cropping: if relighting and view-synthesis metrics degrade markedly relative to the masked-crop numbers in Table 5, the posed-image claim depends on segmentation, not just pose.","tokens_in":22959,"feed_emoji":"🧊","tokens_out":7425,"duration_ms":65240,"temperature":0.7,"pith_summary":"LIRM is a feed-forward transformer that takes three to six posed images of an object and, in under a second, outputs geometry, PBR material maps (albedo, metallic, roughness), and a view-dependent radiance field suitable for relighting in standard graphics pipelines. The paper's central claim is that sparse-view inverse rendering no longer needs dense captures or long optimization: with three progressive sets of six images, LIRM reaches reconstruction quality on par with, and on geometry metrics better than, optimization-based methods that run for hours on dense views. A sympathetic reader would care because this is the difference between an asset-creation workflow that takes minutes or hours and one that feels interactive, while still producing relightable content rather than view-baked radiance. The authors report that additional views can be added progressively without increasing GPU memory, and identify two remaining limits: mirror reflections and very fine texture detail.","feed_headline":"A new transformer turns 3–6 photos into 3D objects in under a second","feed_subtitle":"LIRM matches hour-long optimization-based inverse rendering on relighting and geometry.","key_machinery":"The load-bearing representation is the hexa-plane neural SDF: six feature planes divide the bounding volume into eight octants, each with its own tri-plane, so front and back textures do not share a single plane and leak into each other. From these features, small MLPs predict signed distance, albedo, metallic, roughness, normal, and view-dependent color, with a signed-distance bias $\\|x\\|-0.1R$ to stabilize training. View dependence is handled by neural directional encoding (NDE): the transformer outputs multiple 128x128 feature panoramas, and at each surface point a small network selects which panorama to query with the reflection direction, modeling near-field and occluded reflections without a second-bounce computation. The update module feeds the previous set of hexa-plane and NDE tokens back through self-attention together with new image tokens, so extra views refine the reconstruction at constant GPU memory.","core_discovery":"The paper claims that a single transformer, trained on 38 million rendered images from 600k objects with ground-truth PBR materials, jointly predicts a hexa-plane neural SDF and four NDE panoramas that encode view-dependent appearance. At inference, volume rendering from the SDF yields a mesh via marching cubes, and BRDF maps are extracted on surface points; relighting is then direct. On Stanford-ORB, LIRM with three sets of input images achieves the lowest chamfer distance among compared methods (0.31 vs 0.43 for the best optimization baseline) and competitive relighting and view-synthesis PSNR, while running in about 0.3 seconds per update step on an A100. The authors state this is 'on par and even better than state-of-the-art optimization-based methods, which take dense views as inputs and several hours to run,' and they explicitly acknowledge that mirror-like reflections and the finest texture details remain unsolved.","pith_inferences":["If the real-world protocol is switched to full images without foreground-mask cropping, the gap between LIRM and optimization-based methods may widen; a direct masked-versus-unmasked comparison would settle whether the 'posed images' promise includes busy backgrounds.","The multi-panorama NDE design suggests a general recipe for feed-forward models: replace spherical-harmonic bottlenecks with learned directional feature volumes, which could improve other large reconstruction models whenever shiny materials matter.","The update module's robustness to camera-trajectory order (shown in supplementary sequential-sampling experiments) hints that the same network could serve as an online object refiner, updating a 3D asset as a person walks around it.","A stress test the authors did not run: evaluate on a dataset with explicit mirror-like BRDFs to quantify how much the NDE panoramas can be pushed before requiring a second-bounce term."],"forward_implications":["A capture loop can be interactive: after an initial reconstruction from 3–6 images, users add views of unseen regions and the same transformer refines the asset without retraining or memory growth.","Relightable assets become cheap enough for mass production: mesh plus PBR maps come out of a feed-forward pass, so standard game, film, and VFX pipelines can consume them directly.","Sparse-view inverse rendering closes much of the quality gap to dense-view optimization: on Stanford-ORB, geometry (chamfer distance) is the best among compared methods, suggesting hours-long optimization may be avoidable for many objects.","Because background images are fed alongside masked object images, the model can separate lighting color from material color, improving relighting fidelity under novel environments.","The same progressive-update mechanism may extend to scenes too large for one capture, as the paper demonstrates on a changing scene and on egocentric captures without fine-tuning."],"supporting_citations":[{"why":"Supplies the base LRM transformer design (Plücker-ray tokenization, tri-plane output, rendering losses) that LIRM enlarges and modifies.","marker":"MeshLRM [85]"},{"why":"Establishes the sparse-view large reconstruction model paradigm that LIRM builds on.","marker":"LRM [28]"},{"why":"Provides the directional feature-volume idea LIRM adapts into multiple predicted panoramas for view-dependent radiance.","marker":"NDE [86]"},{"why":"Gives the SDF-based volume ray marching that turns the hexa-plane into rendered images and BRDF maps.","marker":"[93]"},{"why":"Is the real-object benchmark used for the headline comparison against dense-view optimization-based methods.","marker":"Stanford-ORB [35]"},{"why":"Supplies the 600k objects with ground-truth PBR materials used to render the 38M-image training set.","marker":"Shutterstock [1]"},{"why":"Serves as a synthetic evaluation dataset with high-quality materials for inverse rendering and relighting.","marker":"DTC [17]"},{"why":"Serves as a synthetic evaluation dataset for view synthesis and inverse rendering comparisons.","marker":"ABO [11]"},{"why":"Is the strongest optimization-based baseline; LIRM reports lower chamfer distance while running in seconds.","marker":"Neural-PBIR [73]"}],"fun_headline_variants":["Under a second: 3D shape, materials, and radiance from a few images","From a few views to relightable 3D in under a second","Progressive multi-view 3D reconstruction with relightable materials","Fast inverse rendering: matches hours of optimization in seconds","One transformer for shape, materials, and view-dependent radiance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"On real data, LIRM is evaluated with inputs cropped and resized to the object's foreground mask, so the practical claim of reconstruction from arbitrary posed images assumes foreground segmentation is available or accurate at deployment.","fun_headline_variants_meta":{"raw":{"variants":["Under a second: 3D shape, materials, and radiance from a few images","From a few views to relightable 3D in under a second","Progressive multi-view 3D reconstruction with relightable materials","Fast inverse rendering: matches hours of optimization in seconds","One transformer for shape, materials, and view-dependent radiance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000817,"raw_usage":{"total_tokens":3585,"prompt_tokens":957,"completion_tokens":2628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":2534}},"tokens_in":573,"tokens_out":2628,"duration_ms":17479,"temperature":1.0,"reasoning_tokens":2534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:37:43.770713+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LIRM on Stanford-ORB full frames without foreground-mask cropping: if relighting and view-synthesis metrics degrade markedly relative to the masked-crop numbers in Table 5, the posed-image claim depends on segmentation, not just pose.","supporting_citations":[],"review_version":1}