{"id":"b95caf8c-feb6-498a-a2fa-614a4acf2be9","arxiv_id":"2510.25824","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new ray-tracing MCMC sampler keeps ray speed constant, making it far more robust to stochastic gradients and able to sample billion-parameter neural networks on one GPU.","lead":"The paper derives a Markov Chain Monte Carlo sampler that follows light-ray paths through a medium whose refractive index is the likelihood. It claims orders-of-magnitude better noise tolerance than Hamiltonian Monte Carlo, enabling Bayesian sampling of billion-parameter neural networks on one GPU.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ergodicity proof for hole/island crossing fails where L=0: the advertised 'cross any barrier, including holes' claim is unsupported, although the core sampler may remain valid for positive-likelihood targets.","rationale":"I read the paper's central theoretical claim in good faith. The derivation of stationarity via conservation of basic radiance/étendue is internally consistent for continuous, positive-likelihood media: with n=L^{1/(D-1)}, the deterministic ray flow preserves the phase-space measure n^{D-1} dx du, and the acceptance criterion corrects for imperfect integrators. The Gaussian and standard-distribution tests support this. The reader's weakest assumption focuses on the barrier-crossing proof, and I agree that Section 2.5 is the least secure part, but I do not think the reader's specific bending objection is decisive: for a smooth boundary with L>0, there is an open cone of escaping directions, so a rigorous proof could be constructed. The genuinely load-bearing gap is the claim to cross holes and islands where L=0. There n=0 makes the ray equation singular, the escaping cone has measure zero, and the blackbody-radiator extension in Appendix A is not shown to preserve the target distribution. Since this hole/island-crossing claim is advertised in the abstract and conclusions, it is a real overclaim. It does not invalidate the core sampler for ordinary continuous posteriors, but it does make the paper's 'any barrier' phrasing too strong. The reader's conditional verdict remains appropriate; I simply sharpen the reason.","tokens_in":41759,"tokens_out":34828,"duration_ms":384883,"concrete_test":"In 2D, take L(x) = exp(-(x1^2-1)^2/0.01 - x2^2/2) and set L=0 for |x1|<0.5, creating a zero-likelihood barrier between two positive modes. Run Algorithm 1 exactly with isotropic direction refresh from a point in one mode and record whether the other mode is ever reached over, say, 10^8 refresh events. Then run the Appendix A blackbody modification and compare the stationary mass in each mode with the normalized integral of L over that mode. If the base sampler has zero crossings, or if the modified sampler's mode masses are biased away from ∫mode L / ∫all L, the Section 2.5 'holes' claim is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.5 claims the sampler is ergodic and 'can even reach parameter space islands', with the abstract claiming the ability to cross arbitrary likelihood barriers, including holes. The proof assumes the ray flow is a one-to-one reversible map on a bounded reachable set B and that from a point arbitrarily close to a boundary, a step of length ε+δ reaches outside. This argument only works if the refractive index remains positive and the ray equation is nondegenerate. When the 'hole' is a region with L=0, n=0, the Eikonal/ray equation degenerates: no ray can propagate into the zero-index region, and the set of directions that could cross has measure zero. The optional blackbody-radiator boundary treatment in Appendix A is not part of Algorithm 1, and no stationarity proof is given for it. Thus the advertised hole/island-crossing property is unsupported. For smooth positive L, the proof can likely be repaired via an open cone of escaping directions, so this is an overclaim rather than a refutation of the core sampler; but the 'any barrier including holes' claim is a real soft spot that contradicts the paper's own strongest claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new MCMC family based on ray propagation in a medium with refractive index chosen as a function of the desired likelihood. The main theoretical claim is that the conserved quantity of geometric optics — basic radiance — can be used to construct a Markov chain with stationary distribution proportional to L(x). Concretely, the authors set n(x)=L(x)^{1/(D-1)} and propose a deterministic, reversible integrator with a Metropolis-style acceptance step (Eq. 17) intended to correct discretization error. The paper further claims ergodicity and the ability to cross arbitrary likelihood barriers, including parameter-space holes, and demonstrates the sampler on a 10,000-dimensional Gaussian, a 1,433-parameter MLP, a 22-million-parameter ResNet, and a 1.5-billion-parameter GPT-2 model on a single consumer GPU. It also presents a generalized ray-tracing framework in which HMC, microcanonical HMC, Gibbs, Metropolis, and Monte Carlo integration appear as special cases.","tokens_in":42103,"tokens_out":10734,"duration_ms":106777,"significance":"If the central stationarity claim is fully established, this is a novel and potentially important contribution: the geometric-optics derivation is elegant, the idea of using radiance conservation rather than energy conservation as a basis for exact MCMC is original, and the stochastic-gradient robustness results (§3.1.2) are striking. The public implementation in C, JAX, and PyTorch is a concrete strength, as is the honest Appendix D showing that ray tracing does not dramatically outperform HMC on standard distributions with perfect gradients. The GPT-2 experiment, although approximate, is a useful scalability demonstration. However, the paper currently asserts rather than proves a key ingredient of the discrete algorithm — the Jacobian of its integrator — and several advertised properties, especially hole-crossing ergodicity, are not supported by the arguments given.","major_comments":[{"comment":"The paper claims that the acceptance probability P(x→y)=min(1, L(y)L(x)/(L(x)L(y))) guarantees exact sampling even with imperfect integrators. The radiance-conservation derivation applies to continuous ray bundles crossing interfaces; Algorithm 1, however, uses a discrete drift-kick-drift map in which the direction is updated at a midpoint while position advances along straight segments. For a reversible non-volume-preserving map, a Metropolis-Hastings correction must include the Jacobian of the proposal transformation. The manuscript does not compute det ∂(y,v′)/∂(x,v) for the actual integrator, nor does it prove that this determinant equals the sine-ratio product in Eq. (16). Without this step, the central claim of exact sampling for imperfect integrators is unsupported. I recommend adding a direct derivation or a numerical verification of the Jacobian for one integration step, followe","section":"§2.4, Eqs. (16)–(17); §2.8, Algorithm 1"},{"comment":"The ergodicity and hole-crossing claims are not established. The proof in §2.5 assumes that from a point arbitrarily close to the boundary a step of length ε+δ exits the reachable set. This requires the ray equation to remain nondegenerate. When the 'hole' is a region with L=0, the refractive index n=L^{1/(D-1)} vanishes and the ray equation degenerates; there is no continuous ray path into the zero-index region, so the argument fails. The blackbody-radiator treatment in Appendix A is not part of Algorithm 1, and no stationarity proof is given for it. Thus the abstract's claim of crossing 'including holes in parameter space' is an overclaim. The core sampler may be valid for smoothly varying positive likelihoods, but the advertised island-crossing property needs either a rigorous proof under explicit assumptions or a weakening of the claim.","section":"§2.5; Appendix A; abstract"},{"comment":"The neural-network experiments sample with respect to an ad hoc likelihood L(x)=exp(-f_loss(x) D_eff/(2 Δf_loss)), where D_eff and Δf_loss are user-chosen. This is not derived from a Bayesian model and is acknowledged as heuristic in the text. The abstract and conclusions nevertheless state that the method 'sample[s] the posterior distributions of neural network outputs' and conclude that it 'provides fair sampling.' These statements go beyond what Eq. (32) defines. Either the manuscript should explicitly state that the neural-network experiments target a user-specified loss-tolerance distribution and consistently use 'approximate' language, or it should provide a principled calibration for L(x) that connects D_eff and Δf_loss to a well-defined posterior.","section":"§2.11.1, Eq. (32); §3.2–§3.4"}],"minor_comments":[{"comment":"The symbol L is used for both the likelihood and the radiance; the notation becomes confusing, especially in Eq. (17) where the two quantities appear together. Please use a distinct symbol (e.g., script L or R) for radiance.","section":"Eq. (17)"},{"comment":"The pseudocode reassigns Δs to √D Δs inside the procedure. Since Δs is also the function argument, this is a source of confusion; use a separate variable for the scaled step size or state explicitly that the scaling is applied once at initialization.","section":"Algorithm 1"},{"comment":"The blackbody-radiator boundary treatment is described in the text and appendix but is not included in Algorithm 1. If it is intended to be part of the proposed sampling method, a separate pseudocode block would help. If it is only an extension, the abstract's 'including holes' claim should be qualified.","section":"§2.5 / Appendix A"},{"comment":"The claimed equivalence between Goodman–Weare stretch moves and a limiting ray-tracing cone is sketched informally. Since this is one of the paper's stated special cases, a short derivation of the acceptance rule from the ray-tracing formalism would significantly strengthen the unification claim.","section":"§2.9.2"},{"comment":"The abstract mentions 'extending past work on isokinetic sampling,' but the text does not explain or cite this connection. Please add a reference or remove the phrase.","section":"Abstract / §2.9.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to generate substantial interest because of the GPT-2 demonstration and the simple geometric optics framing. My concern is primarily theoretical: the discrete integrator's Jacobian is not computed, and the acceptance criterion's correctness for non-volume-preserving maps is asserted rather than proved. This is fixable — a focused derivation or numerical Jacobian test plus a detailed balance proof would address it. The hole-crossing overclaim should also be toned down or rigorously treated. The empirical sections are interesting but are approximate demonstrations under a user-defined likelihood; the framing should be aligned accordingly. I would not reject the paper, but I would require these corrections before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The ray tracing sampler is a real idea, not a repackaging. The specific refractive-index choice n = L^{1/(D-1)} with constant-speed propagation and the radiance-conservation correction in Eq. 17 is new, and the unification of HMC, microcanonical HMC, Gibbs, Metropolis, and Monte Carlo as special cases of a generalized ray-tracing framework is a useful synthesis. The derivation of basic radiance conservation in D dimensions is standard optics but cleanly applied. Credit where due: the paper ships public code, gives explicit algorithm pseudocode, and reports honest comparisons — including Appendix D where ray tracing and HMC perform roughly equally with exact gradients, which makes the stochastic-gradient advantage more credible.\n\nThe strongest evidence is the stochastic-gradient resilience result (Sections 3.1.2): the 250x larger noise tolerance and the O(1/B) vs. O(1/sqrt(B)) compute scaling are striking and well-illustrated. The neural network demos, including the 1.5B-parameter GPT-2 run, are impressive engineering and suggest the method can indeed explore high-dimensional weight spaces at scale without the trajectory heating that plagues stochastic HMC.\n\nThe soft spots are real but not fatal to the core sampler. First, the ergodicity proof in Section 2.5 — specifically the claim that the sampler can cross arbitrary likelihood barriers, including parameter-space islands with L=0 — does not hold as written. When the refractive index n = L^{1/(D-1)} goes to zero, the ray equation degenerates; rays cannot propagate into zero-index regions, and the boundary-crossing argument based on a single step of length epsilon+delta fails. The blackbody-radiator treatment in Appendix A is not part of Algorithm 1 and has no stationarity proof. This is an overclaim, not a refutation: for smooth positive likelihoods the argument can likely be repaired, and the paper's core sampling mechanism does not rely on crossing zero-likelihood holes.\n\nSecond, the neural network posteriors are not proper Bayesian posteriors. The likelihood is constructed ad hoc via Eq. 32 with hand-tuned D_eff and loss tolerance, and the GPT-2 demo uses post-hoc masking of 4% of samples. That makes the posterior interpretation heuristic. The authors acknowledge this, but the abstract and conclusions present it more strongly.\n\nThird, the paper does not prove that the discrete integrator's Jacobian exactly equals the sine-ratio factor used in the acceptance criterion; it is consistent with a Metropolis-Hastings correction, but a rigorous proof for the actual leapfrog-like update would strengthen the stationarity claim.\n\nOverall: a promising method, honestly reported, with one clearly overclaimed property. It deserves a serious referee, not a desk reject. I would want the referee to focus on the ergodicity claim and on clarifying what posterior is actually being sampled for neural networks.","headline":"A genuinely new sampler with an attractive noise-resilience story and an honest, well-documented empirical section; the advertised ability to cross zero-likelihood holes is not proven, but the core method deserves a serious referee.","tokens_in":42506,"tokens_out":1141,"would_cite":false,"duration_ms":12844,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Markov chain sampler that follows light-like rays through a likelihood-dependent refractive index can draw fair posterior samples from billion-parameter neural networks on a single consumer GPU.","keywords":["ray tracing sampler","Markov chain Monte Carlo","stochastic gradient noise","Bayesian neural networks","Hamiltonian Monte Carlo","Snell's law","high-dimensional sampling"],"falsifier":"Run the ray-tracing sampler on a two-mode target distribution separated by a broad, deep valley, using a small step size and no momentum refresh. If the chain essentially never leaves one mode in the long-time limit, while the local acceptance test continues to pass, the barrier-crossing claim is falsified.","tokens_in":41643,"feed_emoji":"🧭","tokens_out":4639,"duration_ms":45561,"temperature":0.7,"pith_summary":"This paper introduces an MCMC method in which a walker follows ray paths through parameter space whose refractive index is a power of the likelihood. By choosing the refractive index as the likelihood raised to the power 1/(D-1) in D dimensions, the geometric density of ray bundles becomes proportional to the likelihood, giving fair sampling; a Metropolis-like acceptance test restores exactness when the integrator errs. The central claim is that this makes Bayesian sampling of neural-network posteriors practical at billion-parameter scale on ordinary hardware, because the method tolerates far more stochastic-gradient noise than Hamiltonian Monte Carlo and can cross arbitrary likelihood barriers. A sympathetic reader would care because it offers a route to uncertainty quantification for models that are currently only trained, not sampled.","feed_headline":"Ray-tracing sampler handles 1.5B-parameter networks on one GPU","feed_subtitle":"Ray paths in a likelihood-dependent medium tolerate far more gradient noise than HMC, making Bayesian neural nets practical.","key_machinery":"The load-bearing identity is the conservation of basic radiance R = n^(1-D) L and its companion étendue invariant G = n^(D-1) dA cosθ dΩ along ray paths. Choosing n(x)=L(x)^(1/(D-1)) makes the radiance L proportional to the target likelihood L(x), so the ray density is the target density. The paper uses Snell's law in differential form, dθ/ds = -sinθ |∇ ln n(x)|, to update propagation direction, and the acceptance test based on the basic-radiance ratio to correct for integrator error.","core_discovery":"The paper's central discovery is that ray tracing with refractive index n(x)=L(x)^(1/(D-1)) produces a Markov chain whose stationary distribution is proportional to the likelihood L(x). The proof rests on conservation of basic radiance, R = n^(1-D) L, and étendue along ray bundles: Snell's law compresses the ray-bundle phase space in exactly the right way to cancel the likelihood difference between starting and ending points. For imperfect integrators, exactness is restored by the acceptance criterion P(x→y) = min(1, L(y)L(x)/(L(x)L(y))), where L is the ray-bundle radiance. The paper further claims that this sampler is far more resistant to heating from stochastic gradients than Hamiltonian","pith_inferences":["If the stochastic-gradient resilience holds at scale, practitioners can shrink mini-batch sizes by large factors without paying the compute penalty HMC incurs, so sampling and training can share a single compute budget.","The paper's own demonstrations on large models are approximate: convergence is verified in function space, not weight space, and the GPT-2 run shows quantization spikes and an effective parameter count far below the nominal count. A fair reader should expect the 'everyone' promise to depend on function-space convergence tests.","The information-flow measure the paper introduces for comparing network architectures could be reused as an objective for automated architecture search, though the paper presents it only as a diagnostic.","Because standard HMC is a special case of generalized ray tracing, the framework suggests an alternative route to stochastic HMC: instead of adding a cooling term, keep the ray speed fixed and reweight samples post hoc, which could sidestep the difficulty of tuning friction parameters."],"forward_implications":["Bayesian posterior sampling becomes feasible for neural networks with over a billion parameters on a single consumer GPU, not just on large compute clusters.","Stochastic mini-batch gradients cause orders of magnitude less chain heating than in Hamiltonian Monte Carlo, so small batch sizes can be used without runaway divergence.","The sampler's acceptance test makes the resulting distribution fair even when the numerical integrator is imperfect.","The method can cross arbitrary likelihood barriers and even holes in parameter space, which Hamiltonian-style methods generally cannot.","Prior algorithms, including Hamiltonian Monte Carlo, microcanonical HMC, Metropolis, Gibbs, and Monte Carlo integration, reappear as special cases of the generalized ray tracing framework.","The generalized weighting scheme allows sampling according to an arbitrary weighting function and supports likelihood-dependent tempering along the path."],"fun_headline_variants":["Ray tracing sampler: Bayesian NNs, 1.5B params, one GPU","Rays not Hamiltonians: sampler immune to gradient heating","Sampler uses ray paths to cross any likelihood barrier","Unified Bayesian sampler: HMC, Gibbs, Metropolis as special cases","Bayesian neural nets via ray tracing, even GPT-2 on one GPU"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that the sampler can cross any likelihood barrier rests on the assumption that a single step of length ε+δ from a point arbitrarily close to a boundary exits the reachable region; because ray directions bend continuously, the ray can be deflected away from the boundary before that distance is traveled, potentially leaving the sampler trapped in one mode.","fun_headline_variants_meta":{"raw":{"variants":["Ray tracing sampler: Bayesian NNs, 1.5B params, one GPU","Rays not Hamiltonians: sampler immune to gradient heating","Sampler uses ray paths to cross any likelihood barrier","Unified Bayesian sampler: HMC, Gibbs, Metropolis as special cases","Bayesian neural nets via ray tracing, even GPT-2 on one GPU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1331,"prompt_tokens":780,"completion_tokens":551,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":457}},"tokens_in":524,"tokens_out":551,"duration_ms":5561,"temperature":1.0,"reasoning_tokens":457,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T07:25:27.631309+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ray-tracing sampler on a two-mode target distribution separated by a broad, deep valley, using a small step size and no momentum refresh. If the chain essentially never leaves one mode in the long-time limit, while the local acceptance test continues to pass, the barrier-crossing claim is falsified.","supporting_citations":[],"review_version":1}