{"id":"982d842e-b36e-4d73-9518-a9c8fd99207c","arxiv_id":"2411.16172","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"U2NeRF jointly performs novel view synthesis and unsupervised underwater image restoration by disentangling each rendered patch into scene radiance, transmission maps, and background light.","lead":"This paper introduces a neural network that renders new views of underwater scenes while cleaning up their color and haze without needing clean training images. It combines a transformer-based radiance field with a physics-style underwater image model and a new 12-scene dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The restoration claim rests on an unidentifiable decomposition: Eq. 5 admits infinite (J, TD, TB, A) solutions, and the no-reference metrics used to validate J are aligned with the regularizers, so the reported gains do not establish physical restoration.","rationale":"I agree with the reader's weakest-assumption analysis: the decomposition in Eq. 5 is the point where the restoration claim is least secure. The mathematical underdetermination is not merely theoretical; the paper's own synthetic results provide a partial falsification, because on the easy split the method does not beat the UIESS baseline on the only reference-based restoration metric reported. The no-reference metrics used on real scenes cannot resolve the ambiguity because they are correlated with the loss terms. I do not see a reason to change the conditional verdict: the UVS dataset and the generalization experiments are potentially useful, and the rendering comparisons on real scenes are suggestive, but the restoration claim needs the proposed ground-truth component check (or an equivalent controlled experiment) before it can be accepted. The concrete test above would settle whether the concern lands: if U2NeRF recovers known J/T/A accurately across seeds, the identifiability objection is mitigated; if not, the restoration claim should be downgraded to 'enhancement consistent with heuristics' rather than physical restoration. The reader's requested revisions, including error bars and code/data release, remain appropriate, so the verdict stays conditional.","tokens_in":12796,"tokens_out":6698,"duration_ms":77177,"concrete_test":"Construct a synthetic underwater scene (or use the easy split) with known clean radiance J*, known transmission maps T*D/T*B, and known background light A* generated from Eq. 5. Train U2NeRF from scratch on only degraded views with the published losses and hyperparameters. Measure per-pixel RMSE/SSIM between the predicted J, TD, TB, A and the known maps, and repeat with three random seeds. If the predicted J is not substantially closer to J* than the input degraded image, or if the three seeds yield very different J with nearly equal training loss, the disentanglement is not identifiable and the restoration claim is not supported. A complementary check is to ablate Lcon/Lcol/Lglob; if predicted J changes drastically, the restored output is driven by those heuristics rather than by the physical model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that U2NeRF restores scene radiance requires that the self-supervised objective in Eq. 6 selects the physically meaningful factorization in Eq. 5, I(i)=J(i)TD(i)+(1-TB(i))A, rather than an arbitrary factorization that fits the training images. This is not established. For fixed observed I and a chosen global A, Eq. 5 is one equation in three unknown per-pixel fields (J, TD, TB); without constraints tying transmission to scene depth or requiring TD=TB, there is a large solution manifold. The regularizers in Eq. 6 and Appendix A are heuristics: Lcon pushes V(J)-S(J) to zero, Lcol pushes channel means to 0.5, Ltrans fixes ratios of log TB, and Lglob smooths A. They can be satisfied by degenerate outputs, e.g., a desaturated, color-balanced J with TD chosen to reconstruct I, and nothing in the paper shows that the optimizer lands on the physical J. The real-scene validation uses UIQM and UCIQE, no-reference metrics that reward contrast and colorfulness, which are exactly the qualities Lcon and Lcol enforce; improved scores may therefore reflect the training losses, not fidelity to true scene radiance. On the synthetic easy split, where ground truth exists, U2NeRF is not the best restoration method (LPIPS 0.230 vs UIESS 0.174), corroborating the concern. Until the predicted components are checked against ground-truth radiance and transmission in a controlled setting, the restoration half of the central claim remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes U2NeRF, an extension of the generalizable NeRF transformer (GNT) that simultaneously renders novel views and restores underwater images. Instead of predicting single pixel colors, U2NeRF predicts image patches and decomposes each patch into scene radiance J, direct transmission TD, backscatter transmission TB, and global background light A, combining them via the image formation model I(i) = J(i)TD(i) + (1 − TB(i))A (Eq. 5). Training is fully self-supervised using a photometric reconstruction loss plus five regularizers (contrast, color constancy, KL, transmission consistency, global smoothness) largely borrowed from the UPIFM baseline [4]. The authors introduce the UVS dataset with 12 scenes split into easy (synthetic), medium (real-world high quality), and hard (real-world low quality) splits, and report single-scene and cross-scene experiments against rendering-only and restoration-only baselines. The central claims are simultaneous rendering and restoration with improvements in LPIPS, UIQM, and UCIQE.","tokens_in":13162,"tokens_out":6283,"duration_ms":57000,"significance":"If the decomposition in Eq. 5 were identifiable and the evaluations sound, U2NeRF would be a meaningful step toward combining multi-view geometry with unsupervised underwater restoration, and the UVS dataset would be a useful benchmark. The paper has potentially valuable components: a novel dataset with SfM-calibrated real underwater scenes, an architecture that predicts patches rather than pixels to enable restoration, and an attempt at a physics-informed self-supervised NeRF. However, as presented, the central restoration claim is not well supported by the reported tables, and the identifiability of the decomposition from the self-supervised losses is not established. The significance of the contribution therefore depends on additional controlled experiments that are not currently in the paper.","major_comments":[{"comment":"The claims of superior restoration quality are also not supported on the only split with ground truth, which is a load-bearing issue for the paper's central thesis.","section":"§4.4, Tables 1a–1c, Tables 5–6"},{"comment":"This is a load-bearing issue because the paper's stated contribution is simultaneous rendering and restoration; if the factorization is arbitrary, the 'restored' output J is just a color-adjusted reconstruction, not the true scene radiance.","section":"§3.2, Eq. (5), Appendix A, §4.7"},{"comment":"This concern is specific to the real-scene evaluation, which is the only setting where the paper claims both rendering and restoration improvements.","section":"§4.1 (Metrics), §4.4, Appendix A (Eqs. 8–9)"}],"minor_comments":[{"comment":"The total-loss weights are inconsistent: Section 4.1 sets λ6 = 0.1, while Appendix A sets λ6 = 1. Please harmonize these values and state which were used for the reported results.","section":"§4.1 vs Appendix A"},{"comment":"The text refers to 'Fig. 8 and Fig. ??' in Appendix B; the unresolved reference 'Fig. ??' should be fixed.","section":"Appendix B"},{"comment":"The phrase '12 scenes, equally split into easy (synthetic), medium (real-world high quality), hard (real-world low quality)' is awkward because the splits are 4+4+4, not 'equal' in a ternary sense; consider rewording to 'split into three groups of four scenes each'.","section":"§4.2"},{"comment":"The reconstruction loss is denoted Lrec in Eq. (6) but LRec in Eq. (7); please use a single notation consistently throughout.","section":"Eq. (6) and Eq. (7)"},{"comment":"The description of LPIPS (gray) is unclear: 'LPIPS scores between the gray scale image of rendered and restored target views using [7] (to remove color differences)' — please specify whether the images are converted to grayscale before computing LPIPS or processed by UIESS first, and what exactly [7] is used for.","section":"§4.1, Table 1b"},{"comment":"The denoising experiment (Section 4.6, Fig. 6) is qualitative only; reporting PSNR or other metrics on noisy inputs would strengthen the claim.","section":"§4.6, Fig. 6"},{"comment":"The explanation of the VAE for estimating A is brief; it is not clear how the global background light estimated from the nearest source image is made consistent across different target views, especially since Lglob only enforces spatial smoothness within a single prediction.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's abstract and Section 4.4 substantially overstate the empirical results relative to Tables 1a–1c and Tables 5–6, particularly on the hard split and the ground-truth easy split. The heavy reliance on UPIFM [4] both as the source of the regularizers and as a comparison baseline creates a circularity that needs to be addressed head-on: the identifiability of the decomposition is the key scientific question, and the current evidence does not resolve it. The UVS dataset and the generalizable patch-based NeRF formulation are potentially useful contributions, but they are currently secondary to unsupported restoration claims. I would recommend a major revision requiring controlled synthetic experiments (J vs. ground truth, transmission vs. depth) and a more honest reporting of metric comparisons. This is not a reject because the core idea is defensible and the gaps are addressable within the manuscript's scope, but the authors must provide the missing evidence and correct the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a reasonable engineering contribution, not a scientific breakthrough. The authors take GNT's generalizable transformer NeRF, render image patches instead of pixels, and add the UPIFM physics-based unsupervised restoration losses. They also introduce a 12-scene UVS dataset (easy/medium/hard splits). That combination is new and the dataset could be useful to the underwater vision community. The patch-based rendering idea is sensible and the ablations on patch size and source views are honest work.\n\nWhere it gets shaky is the restoration claim. The decomposition in Eq. 5, I = J*TD + (1-TB)*A, is one equation in three unknown per-pixel fields plus a global A. Nothing in the paper proves the self-supervised losses select the physically meaningful factorization over a degenerate one. The regularizers are heuristics from UPIFM—contrast, color constancy, transmission ratio, global smoothness—and can be satisfied by a desaturated, color-balanced J that has nothing to do with true scene radiance. The real-scene evaluation uses UIQM and UCIQE, which explicitly reward contrast and colorfulness, exactly what Lcon and Lcol enforce. So the reported gains may just be the training losses coming back. The synthetic easy split, where ground truth exists, shows U2NeRF underperforming UIESS on PSNR, SSIM, and LPIPS (0.230 vs 0.174), which is consistent with the concern.\n\nThere are also more mundane problems. No error bars or significance tests anywhere. Code and data are not actually released. The abstract's \"up to 11% LPIPS, 5% UIQM, 4% UCIQE\" average claim is not supported by the tables—on the hard split U2NeRF actually loses to NeRF+Clean on UCIQE in one scene and to Clean+NeRF on LPIPS in another. There is also a discrepancy in the loss weights: main text says lambda6=0.1, Appendix A says lambda6=1. Given that the whole method borrows heavily from GNT and UPIFM, the omission of prior underwater NeRF work (e.g., SeaThru-NeRF) is a gap that should be fixed.\n\nWho is this for? Someone working on underwater imaging or generalizable NeRF who wants a baseline dataset and a proof-of-concept that transformers can do simultaneous rendering and restoration. It deserves a serious referee because the problem is real and the dataset has potential, but it needs major revision: show the decomposition actually recovers ground-truth J and transmission on synthetic scenes, add error bars, compare against underwater-specific NeRF baselines, fix the claims and the lambda inconsistency, and release the code and data.\n\nFor peer review: I would send it out, but with a clear expectation of substantial revision before acceptance.","headline":"A plausible engineering combo—transformer NeRF plus UPIFM-style physics losses—with a useful new dataset, but the restoration claim rests on an unverified decomposition and metrics that mirror the training losses.","tokens_in":13694,"tokens_out":2015,"would_cite":false,"duration_ms":21865,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A transformer-based radiance field, trained without clean reference images, renders and restores underwater views by decomposing each predicted patch into scene radiance, transmission maps, and background light.","keywords":["underwater image restoration","neural radiance fields","novel view synthesis","self-supervised learning","image formation model","multi-view geometry","transformer"],"falsifier":"On a scene with available clean reference views, compare the network's predicted scene radiance patch against the true clean image of the same view; if the predicted radiance is not substantially closer to the clean image than the degraded input is, while still satisfying the self-supervised losses, the restoration claim is not supported.","tokens_in":12569,"feed_emoji":"🌊","tokens_out":6352,"duration_ms":52934,"temperature":0.7,"pith_summary":"This paper asks whether a neural radiance field can perform underwater image restoration at the same time as novel view synthesis, using no clean ground-truth images at all. The authors adapt a generalizable transformer-based NeRF to predict image patches rather than single pixel colors, and decompose each patch through a physics-based underwater image formation model into scene radiance, two transmission maps, and global background light; the decomposition is trained only by reconstructing the original degraded views plus heuristic regularizers. On their new 12-scene Underwater View Synthesis benchmark, the resulting U2NeRF outperforms both rendering-plus-restoration baselines and restoration-only methods on perceptual and color-quality metrics, by up to 11% in LPIPS and 5% in UIQM on average. The broader claim is that multi-view geometry and spatial context can substitute for supervision, baking physical decomposition into the network implicitly.","feed_headline":"U2NeRF renders and restores underwater views with no clean labels","feed_subtitle":"Transformer-based NeRF disentangles scene radiance from scattering, using only multi-view images of the scene itself.","key_machinery":"The load-bearing object is the disentangled image formation model of Eq. 5, in which the observed underwater color $I$ at a pixel is written as $J \\cdot T_D + (1 - T_B) \\cdot A$, with $J$ scene radiance, $T_D$ direct transmission, $T_B$ backscatter transmission, and $A$ global background light. Around this, the method builds on a generalizable NeRF transformer architecture from the paper's reference [30]: a view transformer aggregates multi-view image features along epipolar lines, a ray transformer composes point features along a ray, and the final ray feature is upsampled by convolutions to emit a $p \\times p$ patch for each component through separate output heads, while a small VAE estimates $A$. The reconstruction loss on recombining the components is the only direct supervision; the other five losses act as regularizers that guide the factorization toward physically meaningful values.","core_discovery":"The central discovery is that unsupervised restoration and rendering can be achieved in one network if the radiance field predicts a local image patch instead of a point color, and if that patch is forced through the underwater image formation model $I(i) = J(i)T_D(i) + (1 - T_B(i))A$. With the patch providing spatial context and the formation model providing structure, the self-supervised reconstruction loss plus lightweight regularizers (contrast, gray-world color constancy, backscatter consistency, smooth global light) are sufficient to separate scene radiance $J$ from attenuation and backscatter. Visualizations show that the estimated transmission maps track scene depth and the background light converges to the brightest region, indicating physical grounding. The method generalizes across scenes after pretraining and, with a short finetune, matches or beats a per-scene vanilla NeRF.","pith_inferences":["A testable extension is to verify on synthetic scenes with known clean references whether the predicted scene radiance $J$ itself matches the true clean image, not just the final recombined output, which would confirm that the factorization is physically faithful.","If the decomposition is truly physical, the same patch-based self-supervision recipe should transfer to other scattering media such as haze, fog, or turbid water by swapping the formation model, and to video by enforcing temporal consistency.","The released benchmark with easy, medium, and hard splits gives the community a way to compare joint rendering-restoration methods, and the easy split's known clean references allow future work to test physical correctness directly."],"forward_implications":["Joint rendering and restoration requires no clean data: a single underwater scene's posed views are enough to train U2NeRF end-to-end.","Multi-view geometry helps restoration: U2NeRF beats restoration-only baselines that assume direct access to the target view, because the geometry gives consistent scene information across views.","Spatial context is what makes unsupervised restoration work in a radiance field: pixel-level NeRF output cannot restore, while predicting patches of size 4 strikes the best balance.","The same pretrained network generalizes to unseen underwater scenes, and fine-tuning for only 50k steps suffices to match or exceed a per-scene vanilla NeRF.","Even without explicit denoising training, patch prediction gives some robustness to Gaussian noise in the input views."],"supporting_citations":[{"why":"Supplies the physics-informed underwater image formation model, the disentanglement into scene radiance, transmission maps, and background light, and the regularizer losses that the method adapts.","marker":"[4]"},{"why":"Supplies the generalizable NeRF transformer architecture (view and ray transformers) that U2NeRF modifies to predict image patches.","marker":"[30]"},{"why":"Supplies the multi-view source-target view sampling strategy used to construct training pairs for the radiance field.","marker":"[33]"},{"why":"Provides a restoration baseline and the grayscale LPIPS evaluation protocol used for real underwater scenes.","marker":"[7]"},{"why":"Supplies the light-field scenes that are synthetically degraded to create the easy split of the UVS dataset.","marker":"[18]"}],"fun_headline_variants":["U2NeRF: Unsupervised underwater restoration and rendering in one network","No clean labels needed: U2NeRF restores and renders underwater views","Underwater image restoration and novel views via self-supervised NeRF","U2NeRF: single network for unsupervised underwater restoration and rendering","Self-supervised NeRF restores underwater images and renders new views"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the reconstruction loss plus the heuristic regularizers push the network to a physically correct factorization of the degraded image into scene radiance, transmission, and background light, even though infinitely many decompositions satisfy the same image-formation equation.","fun_headline_variants_meta":{"raw":{"variants":["U2NeRF: Unsupervised underwater restoration and rendering in one network","No clean labels needed: U2NeRF restores and renders underwater views","Underwater image restoration and novel views via self-supervised NeRF","U2NeRF: single network for unsupervised underwater restoration and rendering","Self-supervised NeRF restores underwater images and renders new views"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000749,"raw_usage":{"total_tokens":3320,"prompt_tokens":915,"completion_tokens":2405,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":2310}},"tokens_in":531,"tokens_out":2405,"duration_ms":15244,"temperature":1.0,"reasoning_tokens":2310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:27:01.474074+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a scene with available clean reference views, compare the network's predicted scene radiance patch against the true clean image of the same view; if the predicted radiance is not substantially closer to the clean image than the degraded input is, while still satisfying the self-supervised losses, the restoration claim is not supported.","supporting_citations":[{"cited_title":"Unsupervised and untrained underwater im- age restoration based on physical image formation model","cited_arxiv_id":null,"evidence_quote":"Supplies the physics-informed underwater image formation model, the disentanglement into scene radiance, transmission maps, and background light, and the regularizer losses that the method adapts."},{"cited_title":"Ibr- net: Learning multi-view image-based rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-view source-target view sampling strategy used to construct training pairs for the radiance field."},{"cited_title":"Domain adaptation for underwater image enhancement via content and style sepa- ration","cited_arxiv_id":null,"evidence_quote":"Provides a restoration baseline and the grayscale LPIPS evaluation protocol used for real underwater scenes."},{"cited_title":"Local light field fusion: Practical view syn- thesis with prescriptive sampling guidelines","cited_arxiv_id":null,"evidence_quote":"Supplies the light-field scenes that are synthetically degraded to create the easy split of the UVS dataset."}],"review_version":1}