{"id":"c453090f-48a1-43af-ab6c-15adaf2c9aff","arxiv_id":"2506.21680","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PhotonSplat adapts 3D Gaussian Splatting to learn 3D scenes directly from binary SPAD frames, using a photon-counting loss, spatial smoothing, and single-image colorization.","lead":"This paper presents PhotonSplat, a 3D reconstruction system that takes noisy binary images from single-photon SPAD sensors and turns them into clean 3D scenes with color. A smart generalist might care because it points toward high-speed, low-light 3D capture for drones, robotics, and AR/VR.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed advantage over prior work is not established because QRF, the closest SPAD-specific baseline, is absent from Table 1, and the only quantitative geometry results come from a single simulated benchmark without error bars.","rationale":"The reader's weakest_assumption focuses on the alpha-blended compositing in Eq. 6 as a non-derived approximation of expected photon flux. I find that concern less decisive: Eq. 6 is exactly the transmittance-weighted sum used throughout volumetric rendering, and if cgray_k represents per-Gaussian photon emission, the sum is a natural estimate of the mean photon count at the pixel. The subsequent Bernoulli nonlinearity is then the correct likelihood. The paper's ablations (Fig. 9) also show that introducing the photon loss and the spatial smoothing improves results, which is consistent with the forward model being useful. A genuine weakness in the argument lies instead in the evidence for the central comparative claim. The method is introduced as a successor to QRF, yet QRF is never included in Table 1. The two baselines that are included are not SPAD-specific, so the comparison may be unintentionally favorable. Further, all numbers in Table 1 are from one simulated benchmark with no error bars, and the real-capture results are qualitative. The paper itself acknowledges limitations in low light and SfM dependence. I therefore agree with the CONDITIONAL verdict but for a different primary reason than the reader's stated weakest assumption: the reported advantage over prior work is not yet convincingly supported by the experiments. The concrete test I propose—adding QRF and repeated simulation seeds—would settle whether the margin in Table 1 is real. If it survives, the central claim stands; if not, the paper would need either stronger comparisons or a more modest claim.","tokens_in":15227,"tokens_out":5246,"duration_ms":65009,"concrete_test":"Re-run the Table 1 benchmark on the same simulated SPAD frames, same COLMAP poses, same train/test split, and same iteration budget, adding (1) QRF as a baseline (adapted to consume the provided binary frames), and (2) 5 independent simulation seeds per scene to get error bars. If QRF's mean PSNR/SSIM/LPIPS falls within PhotonSplat's standard error, or if PhotonSplat's margin over Deblur-GS shrinks to <0.5 dB, the claimed advantage is not supported and the verdict should remain CONDITIONAL pending stronger evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 (Table 1) reports PhotonSplat's PSNR/SSIM/LPIPS only against NANeRF and Deblur-GS on simulated captures. The closest prior SPAD reconstruction method, QRF (Jungerman and Gupta, 'Radiance fields from photons'), is not benchmarked even though the introduction frames it as the direct predecessor. Because NANeRF (burst-denoising NeRF) and Deblur-GS (deblurring 3DGS) are not designed for binary single-photon input, their lower scores may reflect modality mismatch rather than PhotonSplat's superiority. Moreover, the binary input simulation (Sec 4.2) follows [15] but the exact scaling/thresholding is not specified; without error bars or multiple simulation runs, the margins in Table 1 (e.g., PSNR 15.61 vs 14.50) cannot be assessed as statistically significant. The central comparative claim therefore rests on a single, non-adapted baseline set. This is more load-bearing than the Eq. 6 compositing question: alpha-blended sums are the standard transmittance-weighted estimate of mean photon counts, so that approximation is a reasonable design choice absent evidence of bias.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PhotonSplat extends 3D Gaussian Splatting to multi-view binary SPAD images. The key idea is to replace the RGB radiance attribute with a grayscale photon-flux attribute cgray and to supervise rendered binary predictions via a Bernoulli nonlinearity and binary cross-entropy loss (Sec. 3.1, Eqs. 6-7). A spatial smoothing regularizer perturbs camera translations and enforces L1 consistency (Eq. 8). Color is then baked into the splats either from a single motion-blurred reference image by jointly optimizing deformations (Eq. 9) or from a 2D generative prior with denoising. The authors contribute PhotonScenes, a real 9-scene SPAD dataset, and report experiments on simulated and real captures.","tokens_in":15567,"tokens_out":6778,"duration_ms":71693,"significance":"If the quantitative claims survive revision, this is a useful combination of a physics-based sensor model with a fast explicit scene representation, and the dataset is a tangible community contribution. The method is falsifiable: the forward model is explicit, and the code and data are promised. The main unresolved issue is the thinness of the comparative evaluation, not the internal logic of the loss.","major_comments":[{"comment":"The central comparative claim is not yet established because QRF [6], the closest prior SPAD-based reconstruction method, is absent from the benchmark. The introduction and Sec. 2.3 explicitly motivate PhotonSplat by QRF's limitations (averaged frames, slow NeRF optimization), so a reader would expect QRF to be the primary baseline; instead Table 1 compares only NANeRF and Deblur-GS, both designed for ordinary RGB input. Their lower scores may reflect input modality mismatch rather than PhotonSplat's superiority. Please add QRF on the same simulated captures and on PhotonScenes, and specify its frame-averaging setting.","section":"Sec. 4.3, Table 1"},{"comment":"The quantitative results are single point estimates with no error bars, no number of seeds, and no statistical significance test. Since the margins are modest (e.g., PSNR 15.61 vs 14.50), the reported advantage cannot be assessed. Report mean and standard deviation over multiple simulation runs and random initializations, with the same evaluation protocol for all methods.","section":"Sec. 4.3, Table 1"},{"comment":"Quantitative geometry evaluation is restricted to simulated captures that use the same Poisson-Bernoulli forward model as the training loss, while real-capture results (Fig. 4) are qualitative only. This creates a risk that the simulated gains are partly self-confirming. The situation is further confused by an internal inconsistency: Sec. 4.1 states that sharp ground-truth images were recorded, whereas Sec. 4.3 says 'We do not have access to ground truth RGB images.' Please either report quantitative metrics on PhotonScenes using the recorded ground truth, or explain why that is impossible and justify why the simulated numbers carry the central claim.","section":"Secs. 4.2 and 4.3"},{"comment":"The paper identifies the alpha-blended Gaussian sum Cgray with the expected photon count λ before the Bernoulli nonlinearity, but this compositing rule is inherited from the 3DGS radiance model and is not derived from the Poisson model in Eqs. (4)-(5). Because all geometry gradients flow through the BCE loss, a biased compositing of λ would directly bias geometry. Please add a derivation or, at minimum, a synthetic test comparing Cgray against a volume-rendered photon-flux estimate with known ground-truth flux to show that the approximation does not bias geometry. If Eq. 6 is intended as a standard transmittance-weighted estimate, the manuscript should say so explicitly.","section":"Sec. 3.1, Eqs. (6)-(7)"}],"minor_comments":[{"comment":"The sentence 'Table. 1 qualitatively presents these results' should read 'quantitatively presents'.","section":"Sec. 4.3"},{"comment":"Reference [41] is cited for COLMAP, but the listed reference is 'Colmap-free 3D Gaussian Splatting,' which is a different method; please cite the actual COLMAP paper.","section":"Sec. 4.2"},{"comment":"Equation (8) uses p both as the number of perturbed viewpoints and as a variable ranging over perturbed poses; please use distinct symbols.","section":"Eq. (8)"},{"comment":"The pointer 'see Fig. 10' for the low-light SfM failure appears to reference the wrong figure; the failure case appears to be shown in Fig. 11.","section":"Sec. 4.1"},{"comment":"The no-reference colorization results are explicitly declared to have no quantitative metrics; this should be framed in the text as a qualitative demonstration rather than as 'outperforming' the baselines.","section":"Sec. 4.4"},{"comment":"The dynamic-scene results are qualitative only and do not include a quantitative metric or a dynamic baseline; the claim of 4D reconstruction should be phrased accordingly.","section":"Sec. 4.6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope and has a promising core. The missing QRF baseline, the absence of error bars, and the internal inconsistency about ground-truth image availability should be resolved before the paper can be accepted. The incorrect COLMAP citation should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take a look at this one. The central idea is sound: instead of averaging binary SPAD frames and feeding grayscale-ish images to a splatting pipeline, PhotonSplat optimizes Gaussians directly against the binary frames using a BCE loss derived from the Poisson-to-Bernoulli photon model. That is a real difference from QRF's splatting variant, which averages frames, and it is the right kind of move for high-speed capture where averaging either leaves noise or introduces blur. The colorization module, using a single blurry reference and jointly estimating camera motion, is also practically motivated, and the real PhotonScenes dataset is a genuine contribution.\n\nThe physics is on solid ground. Equation 6 treats the alpha-blended Gaussian sum as the expected photon count before the 1-exp(-lambda) nonlinearity. That is an approximation, but it is the standard transmittance-weighted compositing, and the stress-test note is right that this is a reasonable design choice rather than a load-bearing flaw.\n\nThe soft spots are in the evaluation. Table 1 compares against NANeRF and Deblur-GS, neither of which is designed for binary single-photon input, and omits QRF, the direct predecessor. That makes the headline numbers hard to interpret. The results are also single point estimates with no error bars, and the simulated benchmark is generated with the same Poisson model the method is trained against, which biases the comparison. Real-capture and dynamic-scene results are mostly qualitative. None of this kills the paper, but it does mean the claimed advantage over prior work is not yet established.\n\nWhat would make me confident: add QRF to the benchmark table, report variance across multiple simulation runs or seeds, and make the code and data public. The project page promises code and data; the paper would be stronger if they were linked in the preprint.\n\nBottom line: this deserves a serious referee. A reviewer should push on the baselines and the statistical support, but the core approach is sensible, the dataset is useful, and the writing is honest about failure cases. I would bring it to our reading group to discuss the evaluation gap.","headline":"Directly training 3DGS on binary SPAD frames is a real step forward, but the paper undersells its own case by skipping QRF in the benchmark and reporting no error bars.","tokens_in":16058,"tokens_out":2624,"would_cite":true,"duration_ms":28258,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PhotonSplat reconstructs 3D scenes directly from multi-view binary SPAD images, without frame averaging.","keywords":["SPAD sensors","Gaussian splatting","binary images","photon noise","novel view synthesis","3D reconstruction","colorization","dynamic scenes"],"falsifier":"Render a calibrated static scene from SPAD captures while disabling the smoothness loss and compare the predicted $\\hat{\\lambda}$ against the empirical mean of thousands of averaged binary frames at the same viewpoint: a systematic mismatch would falsify the claim that $\\alpha$-blended Gaussian sums equal expected photon counts. A second check would use a scene with known ground-truth depth and test whether the BCE-optimized geometry converges to the true surface rather than fitting photon noise.","tokens_in":15046,"feed_emoji":"📸","tokens_out":7062,"duration_ms":70212,"temperature":0.7,"pith_summary":"This paper is trying to establish that 3D scene geometry can be recovered directly from multi-view single-bit SPAD frames, without averaging frames to fight photon noise. It matters because SPAD arrays image at rates around 100,000 frames per second, which avoids motion blur in fast camera or object motion, but each frame records only whether zero or at least one photon arrived. The proposed method folds this detection model into Gaussian splatting, optimizes with a binary cross-entropy photon loss plus a pose-perturbation smoothness term, and then adds view-consistent color from a single blurry reference or a generative prior. On simulated captures it reports better reconstruction quality than the two baselines, and it contributes a real-world multi-view SPAD dataset for further work.","feed_headline":"Single-photon frames rebuild 3D scenes without averaging","feed_subtitle":"A new pipeline treats every pixel as a photon count, recovering blur-free geometry and color from high-speed SPAD captures.","key_machinery":"The central mechanism is the photon-aware Gaussian splat: each splat carries a nonnegative flux attribute that plays the role of $\\lambda$ in the Poisson-to-Bernoulli model, the rasterizer $\\alpha$-composites these fluxes into $C_{\\mathrm{gray}}(x')$, and the binary cross-entropy loss against the measured frame, $1-e^{-C_{\\mathrm{gray}}}$ versus $B_{\\mathrm{target}}$, back-propagates into positions, opacities, and fluxes. Around it, the spatial smoothing loss, an $L_1$ distance between renders of a pose and perturbed nearby viewpoints, removes noise, and the colorization loss jointly estimates camera-motion deformations so a single blurred RGB reference can color the splats.","core_discovery":"On its own terms, the paper's discovery is that raw binary SPAD frames carry enough information to drive 3D Gaussian-splat optimization. The scene is represented by splats whose per-point attribute is an expected photon count; $\\alpha$-blending these counts yields a grayscale render that is mapped to a detection probability by $1-e^{-\\lambda}$ and supervised with binary cross-entropy against the observed single-bit frames. A pose-perturbation smoothness term suppresses photon noise, and a second stage attaches color either from one motion-blurred RGB reference, modeling the blur as averaged spline deformations, or from a generative colorization prior. The authors report that this recovers geometry and appearance from multi-view SPAD captures in both static and dynamic scenes, and contributes a real-world multi-view SPAD dataset.","pith_inferences":["A natural next experiment the paper does not run is to compare the predicted photon flux $\\hat{\\lambda}$ from the optimized splats against long-exposure averaged SPAD frames in a calibrated static scene; that would directly test whether linear Gaussian compositing is the right photon-count model.","The same Bernoulli-supervision idea could transfer to other single-bit or quantized sensors, such as event cameras or quanta image sensors, where the measurement is also a thresholded photon count.","The pose bottleneck may be the real limiter at extreme speed: if camera poses could be estimated directly from binary frames instead of from averaged SfM inputs, the system would be closer to real-time operation.","The paper's view-consistency colorization uses one reference; multiple references or a learned prior over color could reduce ambiguity in textureless regions, though the paper does not test this."],"forward_implications":["Fast, motion-blur-free 3D capture becomes possible: a SPAD array taking single-bit frames at about 100,000 frames per second can feed the pipeline directly, with no scene-specific choice of how many frames to average.","The same photon-aware splat representation should support high-dynamic-range and low-light scenes, since the photon model naturally handles sparse detections and avoids saturation at high flux.","A single motion-blurred RGB reference is enough to attach view-consistent color to the reconstructed geometry, opening the renders to segmentation, detection, and instruction-guided editing.","Because the photon loss and colorization act at the representation level, the approach extends to dynamic scenes by swapping in a time-deformed Gaussian splatting backbone.","Color-filtered SPAD hardware can be used directly by running the photon model per channel on single-bit color frames."],"supporting_citations":[{"why":"supplies the Gaussian splatting representation and its fast rasterizer, which the photon-aware model modifies.","marker":"[2]"},{"why":"supplies the Poisson/Bernoulli photon-arrival model and the prior radiance-field-from-photons baseline this work replaces with explicit splats.","marker":"[6]"},{"why":"provides the photon-starved simulation recipe used to generate synthetic binary SPAD captures for evaluation.","marker":"[15]"},{"why":"is one of the spline-based deblurring works whose motion-deformation idea motivates the colorization module's blur modeling.","marker":"[31]"},{"why":"is the deblurring Gaussian-splat baseline compared in the experiments and the other source of the spline-deformation idea.","marker":"[32]"},{"why":"is the noise-aware NeRF baseline for joint denoising and novel view synthesis that PhotonSplat is compared against.","marker":"[43]"},{"why":"is the dynamic Gaussian-splatting framework extended to 4D scenes in the dynamic experiments.","marker":"[27]"},{"why":"is the pre-trained colorization model used when no reference color image is available.","marker":"[7]"}],"fun_headline_variants":["PhotonSplat: 3D scenes directly from SPAD binary frames","Single-photon SPAD captures rebuild 3D scenes without averaging","PhotonSplat: 3D reconstruction from binary SPAD images","From SPAD photon bits to 3D scenes: PhotonSplat","PhotonSplat: 3D scenes without averaging from SPAD binary data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the $\\alpha$-blended sum of Gaussian photon attributes equals the expected photon count at a pixel, so photon statistics only enter through the final $1-e^{-\\lambda}$ step; if that compositing is biased, the binary cross-entropy gradients will push the geometry in the wrong direction.","fun_headline_variants_meta":{"raw":{"variants":["PhotonSplat: 3D scenes directly from SPAD binary frames","Single-photon SPAD captures rebuild 3D scenes without averaging","PhotonSplat: 3D reconstruction from binary SPAD images","From SPAD photon bits to 3D scenes: PhotonSplat","PhotonSplat: 3D scenes without averaging from SPAD binary data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000619,"raw_usage":{"total_tokens":2862,"prompt_tokens":928,"completion_tokens":1934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":1835}},"tokens_in":544,"tokens_out":1934,"duration_ms":16644,"temperature":1.0,"reasoning_tokens":1835,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:22:07.263665+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a calibrated static scene from SPAD captures while disabling the smoothness loss and compare the predicted $\\hat{\\lambda}$ against the empirical mean of thousands of averaged binary frames at the same viewpoint: a systematic mismatch would falsify the claim that $\\alpha$-blended Gaussian sums equal expected photon counts. A second check would use a scene with known ground-truth depth and test whether the BCE-optimized geometry converges to the true surface rather than fitting photon noise.","supporting_citations":[{"cited_title":"Radiance fields from photons,","cited_arxiv_id":null,"evidence_quote":"supplies the Poisson/Bernoulli photon-arrival model and the prior radiance-field-from-photons baseline this work replaces with explicit splats."},{"cited_title":"Photon-starved scene inference using single photon cameras,","cited_arxiv_id":null,"evidence_quote":"provides the photon-starved simulation recipe used to generate synthetic binary SPAD captures for evaluation."},{"cited_title":"Deblur-gs: 3d gaussian splatting from camera motion blurred images,","cited_arxiv_id":null,"evidence_quote":"is one of the spline-based deblurring works whose motion-deformation idea motivates the colorization module's blur modeling."},{"cited_title":"Nan: Noise-aware nerfs for burst- denoising,","cited_arxiv_id":null,"evidence_quote":"is the noise-aware NeRF baseline for joint denoising and novel view synthesis that PhotonSplat is compared against."},{"cited_title":"Ddcolor: Towards photo-realistic image colorization via dual decoders,","cited_arxiv_id":null,"evidence_quote":"is the pre-trained colorization model used when no reference color image is available."}],"review_version":1}