{"id":"07c2d805-66c3-4672-b00e-4332f4e06832","arxiv_id":"2412.16604","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A training-free feed-forward framework that uses Yin-Yang grid decomposition to make pretrained perspective-image 3D Gaussian splatting models work on omnidirectional images, achieving state-of-the-art feed-forward novel-view PSNR without fine-tuning.","lead":"OmniSplat adapts a perspective-image feed-forward 3D Gaussian splatting model to omnidirectional (360 degree) images without retraining, by splitting each 360 image into two quasi-uniform Yin-Yang grids before reconstruction and rendering. It reports higher novel-view PSNR than existing feed-forward baselines at the fastest inference speed, and adds attention-based 3D segmentation for editing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Manual reference-view selection makes the claimed feed-forward advantage untested under automatic capture; fixed or random reference-pair evaluation is needed.","rationale":"The reader's conditional verdict is appropriate. The paper describes a plausible training-free adaptation of MVSplat to omnidirectional input, with an ablation (Table 2) suggesting that both the Yin-Yang cross-view attention and the Yin-Yang rasterizer contribute. However, the evidence for the central reconstruction claim relies on manually selected reference views chosen to be favorable. This is acknowledged in the paper and visible in the per-scene tables in the supplementary material, and it is the most load-bearing weakness because the claimed practical advantage over perspective-trained feed-forward networks is exactly what would be used by practitioners. The segmentation evaluation is also circular (DEVA pseudo-ground truth is used to evaluate against DEVA), but it is secondary to the reconstruction claim. No code is released, though the method description is concrete enough for a reimplementation. The concern does not invalidate the method, but it means the central claim is not yet robustly established under typical or automatic reference-view selection. The verdict therefore remains CONDITIONAL, unchanged from the reader's assessment.","tokens_in":25831,"tokens_out":7363,"duration_ms":68946,"concrete_test":"Rerun the Table 1 evaluation on all six datasets with automatic reference-selection protocols: (i) adjacent-frame pairs; (ii) all pairs at a fixed separation (e.g., 5, 10, and 20 frames, plus the widest available pair); and (iii) randomly sampled pairs across several seeds. Use the same pretrained MVSplat weights, d_near=1, d_far=100, and the same test-view splitting. Report mean±std PSNR, SSIM, and LPIPS for OmniSplat, PixelSplat(P), MVSplat(P), and MVSplat(Y). If OmniSplat's PSNR advantage over the perspective baselines is not consistently positive across all protocols, the central claim should be restricted to the hand-picked wide-baseline setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract, Section 1) is that OmniSplat, a training-free feed-forward 3DGS framework, demonstrates higher reconstruction accuracy than existing perspective-trained feed-forward networks. The only evidence for this claim is Table 1, and the evaluation protocol in Section 4 (Experimental details) states that the two reference views per scene are manually selected 'to minimize the portion of occluded regions.' The supplementary tables show specific hand-picked indices, e.g., OmniBlender frames 14 and 86, Ricoh360 frames 35 and 69, and OmniPhotos frames 0 and 35. This protocol does not reflect how a user would obtain two omnidirectional images in practice, where the reference views are typically adjacent, automatically selected, or otherwise not chosen to minimize occlusion. If the reported 1-3 dB gains over PixelSplat(P) and MVSplat(P) are contingent on this favorable wide-baseline, low-occlusion pair selection, the headline claim is overstated. The paper provides no robustness analysis over reference-pair choice, no error bars, and no statistical significance test. Moreover, even under the favorable protocol, OmniSplat trails PixelSplat(P) on OmniPhotos by 1.35 dB (18.59 vs 19.94), so the advantage is not uniform across datasets. The manual reference-selection protocol is therefore the most load-bearing weakness in the evaluation of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"OmniSplat proposes a training-free feed-forward 3D Gaussian splatting pipeline for omnidirectional novel-view synthesis. It decomposes each equirectangular input into two Yin-Yang grid images, feeds the four decomposed images (two views times two grids) into an unmodified MVSplat network with a modified cross-view attention that mixes features across grids under a spherical sweep, and renders novel views with a Yin-Yang rasterizer. The method is evaluated on six omnidirectional datasets against perspective-adapted and omnidirectional-adapted feed-forward baselines as well as the optimization-based ODGS, and a secondary attention-based segmentation/editing capability is presented.","tokens_in":1638,"tokens_out":1614,"duration_ms":66722,"significance":"If the central claim holds, this is a useful and practical contribution: it shows that a perspective-trained feed-forward 3DGS model can be applied to omnidirectional images without fine-tuning, using a principled spherical representation (Yin-Yang grid) and a matching rasterizer. The paper is commendably transparent in several ways: it provides per-scene supplementary tables, an ablation separating the encoder and rasterizer contributions (Table 2), and the method itself is not circular in the novel-view-synthesis setting because it reuses pretrained MVSplat weights and a published omnidirectional rasterizer without fitting to the target views. The main weaknesses are in the evaluation protocol: the reference views are manually selected to favor low occlusion, no error bars or significance tests are given, and the segmentation evaluation is circular. These issues do not invalidate the technical core, but they currently prevent the strongest claims from being fully supported.","major_comments":[{"comment":"The central claim that OmniSplat 'demonstrates higher reconstruction accuracy than existing feed-forward networks trained on perspective images' is evaluated only under manually selected reference pairs. The text states that 'we select two indices as reference views for each scene where the two images are taken from a distance to minimize the portion of occluded regions,' and the supplementary tables confirm hand-picked indices such as OmniBlender frames 14 and 86, Ricoh360 frames 35 and 69, and OmniPhotos frames 0 and 35. This protocol does not reflect automatic or typical capture, where reference views may be adjacent or otherwise not chosen to minimize occlusion. I request an evaluation with fixed, random, or automatically selected reference pairs, reporting the distribution of results (mean, standard deviation, or paired per-scene differences) and including adjacent-frame pairs. Without this, the reported gains over PixelSplat(P) and MVSplat(P) may be contingent on the favorable wide-baseline, low-occlusion choice.","section":"Section 4, Experimental details; Tables B-G"},{"comment":"No error bars or statistical significance tests are reported. The per-scene tables show that the aggregate margins are small and inconsistent: on OmniPhotos, OmniSplat is 1.35 dB below PixelSplat(P) (18.59 vs 19.94), and on OmniBlender, PixelSplat(P) beats OmniSplat in several scenes (e.g., bistro square 20.11 vs 18.43, restroom 24.95 vs 23.02). A paired test or a per-scene visualization of differences is needed to support the claim that OmniSplat is broadly 'higher reconstruction accuracy' rather than better on a selected subset.","section":"Table 1 and Tables B-G"},{"comment":"The segmentation evaluation is circular. The paper states: 'Due to the absence of ground truth segment maps in the datasets, we applied DEVA tracking over full image sequences and treated the result as pseudo ground truth for evaluation.' DEVA is then used as the baseline in Table 3, so the comparison measures how well OmniSplat matches DEVA-derived pseudo labels against DEVA itself. This cannot support the claim of 'superior segmentation correspondence.' Please evaluate on human-annotated segment correspondences, or use an independent matching oracle, or clearly reframe the result as agreement with DEVA rather than superiority.","section":"Section 4.2, Table 3"},{"comment":"The text overstates the comparison with ODGS and with the feed-forward baselines. Section 1 says OmniSplat 'shows the fastest synthesis speed while reaching the highest PSNR than any other models,' and Section 4.1 says that even after 30,000 iterations ODGS results 'do not outperform than the feed-forward networks.' Table 1 contradicts this: ODGS has higher PSNR than OmniSplat on OmniBlender (22.23 vs 21.22), OmniPhotos (20.25 vs 18.59), 360Roam (18.72 vs 17.53), and 360VO (22.66 vs 20.63); and PixelSplat(P) has higher PSNR than OmniSplat on OmniPhotos. Please revise these statements to be scoped to feed-forward comparisons or to the PSNR-runtime trade-off, and correct the ODGS discussion.","section":"Section 4.1, Table 1"}],"minor_comments":[{"comment":"The definition of the Yin grid uses both an elevation interval and a 270-degree azimuth interval; it would help to state explicitly which spherical coordinates are used (elevation vs latitude, azimuth convention) and to include a small schematic of the Yin-Yang overlap, since the geometric setup is central to the method.","section":"Section 3.1, Eq. (1)"},{"comment":"The attention-based segment matching is described only qualitatively ('matches the most focused pixel'). Please provide the exact algorithm: how attention scores are aggregated across depth candidates and Yin/Yang grids, how the query points are propagated, and how labels are assigned to Gaussians.","section":"Section 4.2"},{"comment":"The PanoGRF baseline is not tagged with an input-domain label, and it is unclear whether it is used as released or adapted for this task. Please specify the exact protocol for this baseline.","section":"Section 4.1, Table 1"},{"comment":"The reported runtimes should state the hardware and the exact pipeline endpoints (e.g., whether pose estimation via OpenMVG and image I/O are excluded). The runtime comparison is a major selling point, so this information is needed for reproducibility.","section":"Section 4, Experimental details"},{"comment":"The limitation about interpolation artifacts when transforming the Yang image to the omnidirectional domain is not quantified. A short experiment or even a per-dataset LPIPS breakdown for the seam region would help readers understand how much this interpolation affects the reported numbers.","section":"Section 5, Limitations"}],"recommendation":"major_revision","confidential_remarks":"The technical core is sound and the paper is likely to be acceptable after revision, but the headline claim currently rests on a favorable reference-view selection protocol and the segmentation claim rests on a circular evaluation. The authors should be asked to provide an automatic/random reference-pair evaluation with error bars, correct the ODGS discussion, and replace or reframe the segmentation evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the central trick is real — using a Yin-Yang grid to make equirectangular images look enough like perspective images that a frozen MVSplat works on them, with a matching rasterizer to avoid polar artifacts. That is new, it is clean (no learned parameters, no fine-tuning), and the paper shows it on six datasets. The main caveat is the evaluation protocol: reference views are manually chosen to minimize occlusion, and there are no error bars, so the reported gains over perspective baselines are not yet demonstrated for automatic or adjacent capture. The segmentation section is circular, but clearly secondary.\n\nWhat the paper does well: the decomposition plus cross-view attention is a sensible way to exploit a perspective-trained encoder; the ablations in Table 2 show both the Yin-Yang encoder and the Yin-Yang rasterizer matter; runtime is genuinely fast; and the 'OmniSplat+opt' variant is a sensible use of a small number of optimization steps. Credit is due for reusing external weights and a known rasterizer rather than training a new model — that makes the claim checkable. The citation pattern looks appropriate: the Yin-Yang grid is credited to Kageyama and Sato, and the baseline comparisons cover the relevant feed-forward and optimization-based methods.\n\nSoft spots, in order. First, the manual reference selection. The supplementary lists specific indices, e.g., OmniBlender 14/86 and Ricoh360 35/69, picked 'to minimize the portion of occluded regions.' That is not how people capture two 360 frames in practice. Without a fixed or random reference-pair evaluation, the headline claim of 'higher reconstruction accuracy than existing feed-forward networks' is probably true for well-chosen pairs but unproven for typical ones. Second, no error bars or significance tests; some per-scene tables show high variance, so a few lucky scenes could be driving averages. Third, the advantage is not uniform: on OmniPhotos, OmniSplat trails PixelSplat(P) by about 1.35 dB. Fourth, the segmentation evaluation uses DEVA as pseudo ground truth to claim superiority over DEVA, which is circular; the editing results are suggestive but not a clean evaluation.\n\nThe paper is re-implementable from the text, but no code is released. I'd want a reference-pair robustness analysis and error bars before fully trusting the numbers — that is a revision, not a rejection.\n\nThis is for anyone working on generalizable 3DGS or omnidirectional reconstruction. It deserves a serious referee, and I would bring it to reading group.","headline":"A genuinely new training-free adaptation of perspective feed-forward 3DGS to 360 images via Yin-Yang decomposition; the result is plausible but the manual reference-view selection and circular segmentation evaluation keep it from being fully convincing.","tokens_in":26660,"tokens_out":2774,"would_cite":true,"duration_ms":22344,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OmniSplat claims that reshaping omnidirectional images onto a Yin-Yang grid lets an off-the-shelf perspective-trained feed-forward 3D Gaussian splatting network reconstruct a full 360-degree scene from two photos, with no fine-tuning.","keywords":["omnidirectional images","3D Gaussian splatting","feed-forward reconstruction","Yin-Yang grid","novel view synthesis","equirectangular projection","sparse-view 3D reconstruction","attention-based segmentation"],"falsifier":"Evaluate on the same six benchmarks using automatically selected or randomly chosen reference pairs instead of the manually selected low-occlusion pairs, and compare PSNR against the cubemap and equirectangular baselines; if the margin shrinks to insignificance or reverses, the training-free advantage would be refuted.","tokens_in":25661,"feed_emoji":"🌐","tokens_out":15119,"duration_ms":110226,"temperature":0.7,"pith_summary":"OmniSplat claims that an unmodified perspective-trained feed-forward 3D Gaussian splatting network can reconstruct a complete 360-degree scene from just two omnidirectional photos, as long as the images are first reshaped onto a Yin-Yang grid. The grid's quasi-uniform square tiles look enough like perspective images that the pretrained model's priors transfer, eliminating the need for omnidirectional training data or fine-tuning. The paper also introduces a Yin-Yang rasterizer that renders the two tiles separately and stitches them in pixel space, removing near-pole stripe artifacts. If the claim holds, 360-degree cameras become drop-in inputs for instant sparse-view 3D reconstruction, and the attention maps used for depth estimation double as a free multiview-consistent segmentation signal for 3D editing.","feed_headline":"No retraining: two 360° photos make a 3D scene in half a second","feed_subtitle":"Reshaping 360° images onto a quasi-uniform grid lets a perspective-trained model beat optimized baselines without tuning.","key_machinery":"The load-bearing object is the Yin-Yang grid, an overset spherical coordinate system made of two overlapping square lattices: Yin, defined by $\\theta\\in[-\\pi/4,\\pi/4]$ and $\\phi\\in[-3\\pi/4,3\\pi/4]$, and Yang, obtained by rotating Yin with the matrix $M$. Its two properties do the work: quasi-uniformness makes each tile nearly perspective-like, so a CNN or Transformer trained on perspective photos can be reused unchanged, while the structured square lattice keeps the grid CNN-compatible, unlike icosahedral or triangular tilings. The companion mechanism is the Yin-Yang rasterizer, which renders the two tiles separately from the predicted 3D Gaussians, normalizes each by its $\\alpha$ map, and warps and combines them into one equirectangular image, suppressing the stripe artifacts caused by non-uniform sampling near the poles.","core_discovery":"The paper's central claim is that feed-forward 3D Gaussian splatting from omnidirectional images can be achieved without any training or fine-tuning by changing the image grid rather than the network. Each equirectangular input is cut into two quasi-uniform square tiles, the Yin grid and the Yang grid (the latter is the former rotated by the matrix $M$), so the four tiles from two reference views look enough like perspective photographs that a pretrained perspective-image estimator extracts correct features without adaptation. A mask-weighted cross-view attention warps each grid feature to the other view along depth candidates and builds cost volumes for all four grid combinations, preventing information loss at grid boundaries. The estimated Gaussians are then rendered not onto an equirectangular raster but separately onto Yin and Yang images, which are divided by their alpha maps and fused in pixel space into the final omnidirectional view. On six datasets the paper reports that this pipeline attains the best PSNR-runtime trade-off among feed-forward methods, and that a short 100-step color/opacity optimization makes the result competitive with or better than a 30,000-step optimization baseline; the same attention correspondences also yield multiview-consistent segmentation used for clean 3D editing.","pith_inferences":["If the wrapper is as generic as it looks, any perspective-trained 3D estimator could be ported to 360-degree input by the same Yin-Yang front-end, not just the specific feed-forward 3DGS network tested here.","The hand-picked reference pairs in the evaluation leave open whether the margin survives automatic or arbitrary view selection; testing random and occluded pairs would settle how much of the gain comes from the grid versus the favorable baseline setup.","The acknowledged horizontal interpolation when converting a Yang tile back to equirectangular space is the most likely quality bottleneck, so seam-aware blending or an invertible grid parameterization is a natural next step.","Segmentation from attention scores could grow into a full zero-shot 3D editing pipeline, where selecting an object in one view edits or removes it consistently across all views without per-scene optimization; the paper shows removal but not yet an end-to-end editing interface."],"forward_implications":["Two reference omnidirectional images synthesize novel 360-degree views in roughly 0.5 seconds, fast enough for interactive capture and preview.","The method beats cubemap- and equirectangular-adapted perspective feed-forward baselines on PSNR in most of the six tested datasets, while running substantially faster than the perspective baselines that must process many cubemap faces.","A short 100-step optimization that updates only color and opacity (about 12 seconds) raises quality further without the overfitting that plagues optimization-only omnidirectional 3DGS.","Attention-score correspondence gives multiview-consistent segmentation that supports clean removal and editing of Gaussians with no extra computation.","The same decomposition-and-rasterization recipe transfers to large-motion capture settings (360Roam, OmniScenes, 360VO) as well as dense small-baseline video datasets."],"supporting_citations":[{"why":"It defines the Yin-Yang overset spherical grid whose quasi-uniform decomposition OmniSplat uses to make omnidirectional images perspective-like.","marker":"[16]"},{"why":"It supplies the pretrained perspective-image feed-forward 3DGS model and cross-view attention machinery that OmniSplat reuses without fine-tuning.","marker":"[4]"},{"why":"It introduces 3D Gaussian splatting, the representation that OmniSplat estimates and rasterizes.","marker":"[17]"},{"why":"It provides the optimization-based omnidirectional 3DGS baseline and omnidirectional rasterizer whose polar artifacts motivate the Yin-Yang rasterizer.","marker":"[20]"},{"why":"It is a perspective feed-forward 3DGS baseline compared through cubemap decomposition and equirectangular adaptation.","marker":"[3]"},{"why":"It is a perspective feed-forward baseline compared on all six datasets.","marker":"[32]"},{"why":"It is an omnidirectional-adapted feed-forward baseline used in the quantitative comparisons.","marker":"[34]"},{"why":"It supplies the OmniPhotos real-world outdoor 360-degree dataset used in the main evaluation.","marker":"[2]"},{"why":"It supplies the OmniBlender and Ricoh360 omnidirectional datasets used in the main evaluation.","marker":"[7]"},{"why":"It provides the video-tracking segmentation baseline that OmniSplat's attention-based matching is compared against for editing.","marker":"[6]"}],"fun_headline_variants":["Two 360° photos to instant 3D scene, no training","Yin-Yang grid enables feed-forward 3D from omni images","OmniSplat: zero-training 3D reconstruction from 360° images","No retraining: 360° images become 3D via clever grid"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hand-picked reference pairs, chosen to minimize occluded regions, represent how the method would actually be used, and that the reported gains survive ordinary or automatically chosen view pairs.","fun_headline_variants_meta":{"raw":{"variants":["Two 360° photos to instant 3D scene, no training","Yin-Yang grid enables feed-forward 3D from omni images","OmniSplat: zero-training 3D reconstruction from 360° images","No retraining: 360° images become 3D via clever grid"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1544,"prompt_tokens":1029,"completion_tokens":515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":432}},"tokens_in":645,"tokens_out":515,"duration_ms":35610,"temperature":1.0,"reasoning_tokens":432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:24:56.157571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate on the same six benchmarks using automatically selected or randomly chosen reference pairs instead of the manually selected low-occlusion pairs, and compare PSNR against the cubemap and equirectangular baselines; if the margin shrinks to insignificance or reverses, the training-free advantage would be refuted.","supporting_citations":[{"cited_title":"yin-yang grid","cited_arxiv_id":null,"evidence_quote":"It defines the Yin-Yang overset spherical grid whose quasi-uniform decomposition OmniSplat uses to make omnidirectional images perspective-like."},{"cited_title":"Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images","cited_arxiv_id":null,"evidence_quote":"It supplies the pretrained perspective-image feed-forward 3DGS model and cross-view attention machinery that OmniSplat reuses without fine-tuning."},{"cited_title":"Odgs: 3d scene reconstruction from omnidirectional images with 3d gaussian splattings","cited_arxiv_id":null,"evidence_quote":"It provides the optimization-based omnidirectional 3DGS baseline and omnidirectional rasterizer whose polar artifacts motivate the Yin-Yang rasterizer."},{"cited_title":"Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction","cited_arxiv_id":null,"evidence_quote":"It is a perspective feed-forward 3DGS baseline compared through cubemap decomposition and equirectangular adaptation."},{"cited_title":"latentsplat: Autoencoding variational gaus- sians for fast generalizable 3d reconstruction","cited_arxiv_id":null,"evidence_quote":"It is a perspective feed-forward baseline compared on all six datasets."},{"cited_title":"Omniphotos: casual 360 vr photography","cited_arxiv_id":null,"evidence_quote":"It supplies the OmniPhotos real-world outdoor 360-degree dataset used in the main evaluation."},{"cited_title":"Bal- anced spherical grid for egocentric view synthesis","cited_arxiv_id":null,"evidence_quote":"It supplies the OmniBlender and Ricoh360 omnidirectional datasets used in the main evaluation."},{"cited_title":"Deva: Tracking anything with decoupled video segmentation","cited_arxiv_id":null,"evidence_quote":"It provides the video-tracking segmentation baseline that OmniSplat's attention-based matching is compared against for editing."}],"review_version":1}