{"id":"6ce9d48e-85f7-4cd4-ba95-08ff712ed71a","arxiv_id":"2412.03489","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new importance-sampling scheme estimates Hessians and Hessian-vector products of smoothed renderings, enabling Newton-type optimizers in inverse rendering with O(1) rendering evaluations per step.","lead":"Researchers derive Monte Carlo estimators for the Hessians and Hessian-vector products of the rendering operator, based on importance-sampling a smoothed convolution kernel. This gives inverse rendering optimizers second-order information, with reported 2.71x average speedups over first-order baselines in several tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"HVP estimator is a biased finite-difference surrogate, not the exact directional second derivative; its sign in Eq. 23 is reversed relative to Eq. 8, and no bias or epsilon analysis is provided.","rationale":"The reader identified the HVP finite-difference bias and the CNN Gauss-Newton truncation as weaknesses, and I agree those are real. My stress-test adds a sharper point: as written, Eq. 23 has the opposite sample order from the standard central-difference HVP in Eq. 8, so it estimates -Hv, not Hv. This makes the HVP construction not merely approximate but sign-inconsistent in the published text. Since the paper provides no code, the discrepancy cannot be resolved by inspection of an implementation, and it directly affects the estimator used for most of the reported speedups. The diagonal-Hessian CDF contradiction in Eq. 21 versus Eq. 52 is also unresolved, but Eq. 21 appears internally consistent and may be a typo in the supplement; the HVP issue is more consequential because it affects the scalability claims. I still believe the full-Hessian estimator and the convolution-based derivation are credible enough to justify conditional acceptance, provided the authors resolve the HVP sign and bias, document the epsilon schedule, and release code. Therefore I do not move the reader's verdict; the conditionality is already appropriate.","tokens_in":16962,"tokens_out":10348,"duration_ms":109671,"concrete_test":"Implement the exact directional second-derivative kernel H_N(tau) v for the Gaussian smoothing kernel and compare its expectation with Eq. 23 on the QUAD and BOX23 tasks for epsilon in {0.01, 0.1, 1}, using the analytic smoothed loss. If the expectation differs by a nonzero O(epsilon^2) term or has the wrong sign, the estimator is not an unbiased Hessian-vector product. Then re-run the MUG and SHADOW optimizations with the exact kernel in place of Eq. 23; if convergence speedups disappear, the headline result is attributable to the biased finite-difference surrogate rather than to exact second-order information.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the Hessian-vector-product construction. The central claim requires unbiased MC estimates of second-order differentials, and the HVP is the estimator that makes the O(n) and O(1) aggregate speedups possible (Sec. 3.5, Tab. 1). But Eq. 23 defines D_HVP kappa(tau) = (DG kappa(tau - epsilon v) - DG kappa(tau + epsilon v)) / (2 epsilon). This is a finite-difference approximation to the directional derivative of the gradient kernel, not the exact kernel H_N(tau) v = grad(grad N dot v). For any fixed epsilon it estimates the convolution of a biased finite-difference stencil with the renderer, introducing an O(epsilon^2) error in the HVP. The paper gives no epsilon schedule, no bias bound, and no ablation showing the reported convergence is insensitive to epsilon. The sign is also reversed relative to the standard central-difference formula in Eq. 8, which uses g(theta + epsilon v) - g(theta - epsilon v); Eq. 23 has the opposite order, so as written it estimates -Hv. Because every tabulated speedup from OursHVP/OursHVPA rests on this estimator, the 2.71x headline aggregates a quantity whose unbiasedness is unsupported and whose published formula is sign-inconsistent. The full-Hessian sampler may still be correct, but the paper's broad claim extends to HVPs and CNN grey-box Hessians (Suppl. Eq. E), which are also approximations. This is the load-bearing soft spot.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Monte Carlo framework for estimating first- and second-order derivatives of a black-box rendering operator by importance-sampling kernels that combine differentiation with Gaussian smoothing. Starting from the convolution identity in Eq. 15, the authors derive gradient, Hessian, and Hessian-vector-product estimators, introduce an aggregate sampling strategy that reuses one renderer evaluation across all differential dimensions (Table 1), and plug the estimators into Newton-CG and related optimizers. Experiments on analytic, rasterization, path-traced, and CNN-based inverse-rendering tasks report that the aggregate HVP variant converges on average 2.71x faster than previous methods. The supplement contains derivations, an analytic-Hessian validation, and robustness studies with respect to initialization, sample count, MC noise, and JPEG compression.","tokens_in":17299,"tokens_out":10548,"duration_ms":96240,"significance":"If the estimators were unbiased and the implementation faithful, the paper would be a useful practical contribution: it extends differentiable rendering to second-order information at O(1) per-iteration renderer evaluations, and the aggregate sampling idea is a plausible variance/render-cost tradeoff. The core derivation of the Gaussian gradient and Hessian kernels (Secs. 3.2-3.3, Suppl. A-D) is standard and appears correct, and the paper includes a validation against analytic Hessians in Suppl. G and explicit robustness experiments. However, the HVP construction and the inconsistent diagonal-CDF formulas are central to the headline convergence claims, so the significance can only be assessed after those issues are resolved.","major_comments":[{"comment":"The HVP estimator is not an unbiased estimator of the second-order differential. Eq. (23) defines D_HVP kappa(tau) = [D_G kappa(tau - epsilon v) - D_G kappa(tau + epsilon v)] / (2 epsilon), which is a fixed-epsilon central difference of gradient kernels rather than the exact directional derivative of the Gaussian-gradient kernel. For any fixed epsilon this introduces an O(epsilon^2) bias into the estimated HVP, and the sign is opposite to the central-difference convention used in Eq. (8), since Eq. (8) uses g(theta + epsilon v) - g(theta - epsilon v). As written, the estimator targets -Hv rather than Hv. No bias bound, epsilon schedule, or sensitivity analysis is given, and epsilon does not appear in the hyperparameter table (Table 3), although OursHVP and OursHVPA account for the paper's main convergence speedups. The full-Hessian sampler may be correct, but the HVP-based central claim is unsupported as written.","section":"Sec. 3.4, Eq. (23)"},{"comment":"The main text and the supplement give contradictory formulas for the diagonal-Hessian inverse-CDF sampling. Eq. (21) evaluates to 1/4 at u = -sigma, to 3/4 at u = +sigma, and is monotone in the middle interval, which is consistent with the described tail masses. Eq. (52), however, uses P = 1/2 + beta nabla N(tau) on [-sigma, sigma], which gives 3/4 at tau = -sigma and 1/4 at tau = +sigma and is therefore decreasing; the correct middle branch is 1/2 - beta nabla N(tau), and the correct upper branch is 1 + beta nabla N(tau). Since Eq. (21) and Eq. (52) are presented as the same construction and the implementation is described as a tabulation of 'this' CDF, the reader cannot determine which distribution the Hessian sampler actually uses. This directly affects the correctness of OursH and all experiments that use full Hessian information.","section":"Sec. 3.3, Eq. (21) vs Suppl. Sec. D, Eq. (52)"},{"comment":"The grey-box second-order chain rule is truncated to the Gauss-Newton term: nabla^2_z f(g(x)) is approximated by (nabla_x g)^T nabla^2_y f(g(x)) nabla_x g, omitting the term that contains nabla^2_x g (the second derivative of the inner function). The supplement marks this with 'approximately', but the CNN experiments in Sec. 4.1 use this approximation when reporting higher-order convergence gains. The magnitude of the dropped term is not analyzed for any of the CNN tasks, so the CNN rows in Table 2 do not demonstrate exact second-order information. This is acceptable only if the claim is explicitly limited to a Gauss-Newton approximation; as written, the paper's narrative that higher-order differentials are computed for grey-box compositions is stronger than what Eq. E supports.","section":"Suppl. Sec. E and Sec. 4.1 (CNN tasks)"}],"minor_comments":[{"comment":"Numerical footnote markers are embedded inside data cells (e.g., '4', '6', '8', '10' appear within numeric entries), which makes the table difficult to read; these markers should be moved outside the cells or replaced by a legend.","section":"Table 2"},{"comment":"The statement that all hyperparameters, including those of competitors, 'have been optimally chosen' is not backed by a tuning protocol or a sensitivity analysis; given the large variation across tasks, a description of the selection criterion would improve reproducibility.","section":"Table 3"},{"comment":"The phrase 'This is a transcendental equation whose inverse CDF cannot be expressed in closed form via elementary functions' is imprecise: the CDF in Eq. (21) is elementary; it is the inverse that requires numerical tabulation.","section":"Sec. 3.3, paragraph after Eq. (21)"}],"recommendation":"major_revision","confidential_remarks":"The inconsistency between Eq. (21) and Eq. (52), the sign in Eq. (23), and the missing bias analysis for the HVP estimator must be resolved before the convergence claims can be trusted. I do not see grounds for rejection: the full-Hessian derivation is standard, and the issues are localized to the HVP estimator, the diagonal-CDF formulas, and the grey-box extension. I would encourage the authors to release the sampling code or a numerical check of the inverse-CDF to pin down the actual distribution used in the experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my read. The genuinely new thing is a Monte Carlo estimator for Hessians and Hessian-vector products of a black-box rendering operator, built by importance-sampling the differentiated Gaussian smoothing kernel, plus an aggregate sampling scheme that cuts per-step rendering calls to O(1). That is a real extension of Fischer-Ritschel, and the full-Hessian sampler in Sec. 3.3 looks correct to me. The variance analysis in the supplement is also honest: aggregate sampling trades variance for function evaluations, and the paper says so.\n\nThe soft spots are real but mostly fixable. First, the diagonal-Hessian CDF has opposite signs in Eq. 21 and Suppl. Eq. 52; Eq. 21 behaves like a proper CDF, so the supplement has a sign error. Second, the HVP estimator in Eq. 23 is not an exact directional second-derivative kernel; it is a central finite difference of the gradient kernel, and the sign is reversed relative to Eq. 8. As written, the convolution gives -Hv rather than Hv. If the code flips it back, fine, but the paper does not say, and there is no epsilon sweep or bias bound. That matters because the headline speedups for OursHVP/OursHVPA rest on this estimator. Third, the grey-box CNN Hessian in Suppl. E drops the inner function's second derivative; the paper calls it an approximation, but the error is not analyzed. These issues do not kill the core idea, but they stop the paper from being self-consistent as written.\n\nThe experiments are broad, and the robustness checks (MC noise, JPEG compression, non-PSD Hessians) are a plus. The 2.71x speedup is softer than the abstract suggests: hyperparameters are per-task tuned, the average mixes heterogeneous ratios, and on BUNNY the new method is not fastest. No code or data is provided, so the numerical claims are not independently checkable. Citation pattern is fine; prior work gets proper credit.\n\nWho is this for? People working on differentiable rendering or inverse rendering optimization. It gives them a credible way to get second-order information with small render-call budgets. I would send it to review rather than desk-reject. The main revision needs are: fix the sign inconsistencies, add a bias/epsilon analysis for the HVP, and release code or at least precise sampling pseudocode.","headline":"A credible second-order extension of smoothed gradient sampling with a clever O(1) aggregate trick, but sign errors and an unquantified HVP bias need fixing before the numerical claims are trustworthy.","tokens_in":17830,"tokens_out":8068,"would_cite":true,"duration_ms":78059,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By importance sampling a differentiated Gaussian smoothing kernel, this paper derives Monte Carlo estimators for Hessians and Hessian-vector products of the rendering operator, and shows that Newton-conjugate-gradient optimization using…","keywords":["differentiable rendering","Hessian estimation","Hessian-vector product","importance sampling","second-order optimization","inverse rendering","Gaussian smoothing"],"falsifier":"On a one-dimensional Gaussian with known second derivative, draw many samples from the tabulated diagonal-Hessian inverse CDF and compare the empirical CDF against the analytic $P^{H}_{ii}(\\tau_i)$ in Eq. 21; a persistent mismatch would show the sampler is not faithful to the derived distribution and would break the unbiasedness claim.","tokens_in":16707,"feed_emoji":"🎨","tokens_out":9802,"duration_ms":85927,"temperature":0.7,"pith_summary":"The paper is trying to establish that second-order derivatives of a rendered image with respect to scene parameters can be estimated without any analytic differentiation of the renderer, purely by Monte Carlo sampling of a smoothed convolution. The trick is to blur the rendering integrand with a Gaussian kernel, differentiate the kernel instead of the scene, and importance-sample the resulting signed distribution; the same recipe yields gradients, Hessians, and Hessian-vector products. Because the renderer is only point-sampled, the method applies to both rasterization and path tracing, and ignores the internal structure of the rendering operator. If correct, this gives inverse rendering practical access to Newton-style optimization, and the paper reports an average 2.71x wall-clock speedup over prior gradient-based methods across its tasks.","feed_headline":"Sampled Gaussian kernels give renderers unbiased Hessians","feed_subtitle":"Curvature estimates from one renderer call per step let Newton-style optimizers cut inverse rendering time 2.71x on average.","key_machinery":"The load-bearing object is the combined smoothing-and-differentiation kernel $D\\kappa(\\tau)$: a Gaussian convolution kernel with the desired differential operator applied to it, used as the importance-sampling proposal for the convolution integral. Concretely, the gradient kernel is $-\\tau_i/\\sigma^2 N(\\tau,\\sigma)$, the Hessian-diagonal kernel is $(-1/\\sigma^2+\\tau_i^2/\\sigma^4)N(\\tau,\\sigma)$, the Hessian-off-diagonal kernel is $\\tau_i\\tau_j/\\sigma^4 N(\\tau,\\sigma)$, and the HVP kernel is the central difference of the gradient kernel along the direction $v$. Because the Gaussian factors, the dimensions not being differentiated are sampled independently from a Gaussian. The diagonal-Hessian CDF is transcendental, so its inverse is precomputed in a 1D table for $O(1)$ search; off-diagonals are sampled as products of two independent gradient samples; aggregate sampling substitutes the average over all kernels as a single proposal, justified as a mixture-style combination of unbiased estimators.","core_discovery":"The paper's central claim is that the differential of the Gaussian-smoothed rendering equation can be written exactly as the product-space integral $$D\\bar{L}(x,\\omega_o;\\$\\theta$)=\\int_\\$\\Omega$\\int_\\Theta D\\kappa(\\tau)R(\\omega_i;\\$\\theta$-\\tau)\\,d\\tau\\,d\\omega_i,$$ where $D$ is the differential operator of interest and $\\kappa$ is a Gaussian smoothing kernel. Since smoothing makes the integrand differentiable, the derivative passes through the integrals, and the resulting integral is estimated by importance-sampling $D\\kappa(\\tau)$. For the gradient this reproduces the known smooth-gradient estimator; for the Hessian diagonal the kernel is $(-1/\\sigma^2+\\tau_i^2/\\sigma^4)N(\\tau,\\sigma)$ and for off-diagonals $\\tau_i\\tau_j/\\sigma^4N(\\tau,\\sigma)$, each with its own sampling scheme; for Hessian-vector products the estimator is the central difference of the gradient kernel along the direction $v$. The paper further claims that 'aggregate' importance sampling, which draws one sample point from the average of all kernels and reuses it for every matrix element, keeps the estimator unbiased while reducing the cost to a single renderer call. On this basis it reports that Newton-conjugate-gradient optimizers converge faster than gradient descent, by an average factor of 2.71 in wall-clock time.","pith_inferences":["A natural extension, not tested in the paper, is to apply the same convolution-sampling recipe to other integral operators whose integrands are discontinuous in the parameters, such as differentiable physics simulators.","The aggregate sampling trick is a general variance-cost tradeoff: it cuts function evaluations to one but raises variance by sampling from the average kernel; where the crossover makes it unfavorable at high dimension is a testable quantification the paper only begins in its texture task.","The grey-box Hessian used in the CNN experiments drops the inner function's second derivative; including that term would test how much of the reported speedup on network tasks comes from true second-order information versus the smoother gradient landscape.","Choosing the HVP finite-difference step $\\epsilon$ adaptively could reduce the bias of that estimator on problems with heterogeneous curvature."],"forward_implications":["The aggregate Hessian and Hessian-vector product samplers reduce per-iteration renderer calls to a single evaluation, making second-order optimization of rendering problems practical at the same black-box cost as one gradient step.","The same black-box point-sampling interface applies to both rasterization and path tracing, so the method does not need renderer-specific differentiation code.","Plateau-type inverse rendering problems, where standard differentiable renderers produce zero gradients, become optimizable because the Gaussian smoothing gives the loss landscape a nonzero local slope and curvature.","In the paper's evaluations, Newton and conjugate-gradient optimizers using the estimators converge faster in wall-clock time than gradient-descent baselines, with the aggregate HVP variant the fastest overall by an average factor of 2.71."],"supporting_citations":[{"why":"Supplies the gradient-smoothing estimator and the Smirnov inverse-CDF sampling that this paper extends to Hessian and HVP differentials.","marker":"[7]"},{"why":"Introduces smoothing of the integrand as a way to make differentiation of non-smooth operators well-defined; the convolution in Eq. 13 builds on it.","marker":"[4]"},{"why":"Defines the rendering equation that is the operator being differentiated, so it supplies the target integral $L(x,\\omega_o;\\theta)$.","marker":"[12]"},{"why":"Provides the importance-sampling correctness foundation used to turn the differentiated convolution kernel into a PDF.","marker":"[23]"},{"why":"Supplies the central-differences recipe for Hessian-vector products that the HVP estimator uses.","marker":"[24]"},{"why":"Older reference for the same Hessian-vector product trick; cited alongside Pearlmutter for the finite-difference HVP.","marker":"[35]"},{"why":"Balance heuristic / MIS for combining sampling proposals; invoked to justify the aggregate mixture-style sampling.","marker":"[33]"},{"why":"Mitsuba 2 renderer used as black-box rendering engine and as the gradient-descent baseline in the inverse rendering experiments.","marker":"[20]"}],"fun_headline_variants":["One render pass yields unbiased Hessians for faster Newton optimization","Curvature from a single renderer call speeds Newton-style inverse rendering","Unbiased Hessians with one render call cut inverse rendering time 2.71x","Aggregate sampling gives Hessians in a single renderer call for faster inverse rendering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the implemented samplers, especially the tabulated inverse CDF for the diagonal Hessian, actually draw from the derived importance distributions, so the Monte Carlo estimates are unbiased in practice and not only in the derivation.","fun_headline_variants_meta":{"raw":{"variants":["One render pass yields unbiased Hessians for faster Newton optimization","Curvature from a single renderer call speeds Newton-style inverse rendering","Unbiased Hessians with one render call cut inverse rendering time 2.71x","Aggregate sampling gives Hessians in a single renderer call for faster inverse rendering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001018,"raw_usage":{"total_tokens":4279,"prompt_tokens":909,"completion_tokens":3370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":3289}},"tokens_in":525,"tokens_out":3370,"duration_ms":23043,"temperature":1.0,"reasoning_tokens":3289,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:21:40.540825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a one-dimensional Gaussian with known second derivative, draw many samples from the tabulated diagonal-Hessian inverse CDF and compare the empirical CDF against the analytic $P^{H}_{ii}(\\tau_i)$ in Eq. 21; a persistent mismatch would show the sampler is not faithful to the derived distribution and would break the unbiasedness claim.","supporting_citations":[{"cited_title":"Plateau-reduced differ- entiable path tracing","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-smoothing estimator and the Smirnov inverse-CDF sampling that this paper extends to Hessian and HVP differentials."},{"cited_title":"Smooth in- terpretation","cited_arxiv_id":null,"evidence_quote":"Introduces smoothing of the integrand as a way to make differentiation of non-smooth operators well-defined; the convolution in Eq. 13 builds on it."},{"cited_title":"The rendering equation","cited_arxiv_id":null,"evidence_quote":"Defines the rendering equation that is the operator being differentiated, so it supplies the target integral $L(x,\\omega_o;\\theta)$."},{"cited_title":"Safe and effective importance sam- pling","cited_arxiv_id":null,"evidence_quote":"Provides the importance-sampling correctness foundation used to turn the differentiated convolution kernel into a PDF."},{"cited_title":"Fast exact multiplication by the hes- sian","cited_arxiv_id":null,"evidence_quote":"Supplies the central-differences recipe for Hessian-vector products that the HVP estimator uses."},{"cited_title":"Backpropagation: Past and future","cited_arxiv_id":null,"evidence_quote":"Older reference for the same Hessian-vector product trick; cited alongside Pearlmutter for the finite-difference HVP."},{"cited_title":"Optimally combining sampling techniques for Monte Carlo rendering","cited_arxiv_id":null,"evidence_quote":"Balance heuristic / MIS for combining sampling proposals; invoked to justify the aggregate mixture-style sampling."},{"cited_title":"Mitsuba 2: A retargetable forward and inverse renderer","cited_arxiv_id":null,"evidence_quote":"Mitsuba 2 renderer used as black-box rendering engine and as the gradient-descent baseline in the inverse rendering experiments."}],"review_version":1}