{"id":"a070599e-62e9-40c6-a225-54ac3eaf465c","arxiv_id":"2607.17411","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A feature-conditioned diffusion model teamed with CMA-ES solves black-box inverse rendering tasks without gradients, beating scalar-loss and gradient-based baselines.","lead":"The paper introduces FIDE, an optimizer that treats renderers as black boxes and uses visual features from a frozen Vision Transformer to guide a diffusion model that proposes new scene parameters, refined by CMA-ES. It reports reliable convergence on five inverse rendering tasks where gradient-based and scalar-loss methods get stuck.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'no specific initialization' claim rests on a single fixed center init; without varying init or sigma0, FIDE's robustness claim is not established.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing gap: the fixed center initialization and the lack of any variation of it. My independent reading of the full text and supplement confirms that all experiments use mean 0.5 and sigma0=1/6, and that the paper's 'no specific initialization' language is not supported by any experiment that changes the initialization. The concern is not an internal inconsistency; it is an empirical-support gap. The paper has real strengths: the method is evaluated on five diverse tasks, with multiple loss functions, ablations on feature extractors and training epochs, and comparisons to strong baselines including CMA-ES with the same population size and budget. These support a conditional acceptance, not a rejection. The missing initialization sweep is addressable and should be required before the strong claims are accepted. Therefore I recommend keeping the reader's CONDITIONAL verdict; my analysis does not move it in either direction.","tokens_in":19541,"tokens_out":4201,"duration_ms":47227,"concrete_test":"Run FIDE on the 40-parameter Voronoi task (and, if feasible, the 8-parameter San Miguel task) with (a) initial mean at 0.1 and 0.9 in all coordinates, and (b) sigma0=0.01 instead of 1/6, keeping k=500, N=50, 20 epochs, and the same seeds. Also generate a target whose known optimum is at a corner (e.g., one Voronoi cell center at (0.05,0.05)) and initialize at 0.5. If FIDE's median final LPIPS remains comparable to the center-init runs, the initialization-robustness claim survives; if performance degrades or variance increases sharply, the paper must weaken the claim to 'works with a broad centered initialization.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Sec. 1) is that FIDE 'requires no gradients or specific initialization' and 'is able to converge to a global minimum, even in the presence of many local minima.' Supplement S1 fixes the CMA-ES mean at 0.5 and sigma0=1/6 on the normalized [0,1]^d domain, arguing the entire parameter space is within 3σ of the initialization. This is a strong domain-knowledge assumption: the initial Gaussian is centered at the midpoint and wide enough to place substantial mass on every coordinate's full range. All five benchmarks and all 10 seeds use exactly this initialization; no experiment varies the starting mean or initial covariance. If a task's optimum lies in a narrow basin away from the center, or if CMA-ES requires the initial distribution to cover the optimum to succeed, then the 'no specific initialization' and 'robust to far-from-optimum initialization' claims (stated in Sec. 1 and Sec. 4.3) are unsupported. The Voronoi task's 8! symmetry means many optima exist, but they are still at specific center/color configurations; the initial Gaussian centered at 0.5 may cover them. The Spline task's target is a font glyph that cannot be exactly reproduced by the B-spline, so 'global minimum' is not even verified; the reported metric is image loss, not parameter error. The concern is not that the method fails on the tested tasks, but that the evidence as presented cannot distinguish 'works without initialization' from 'works with this particular broad centered initialization.' This is load-bearing because a central contribution over derivative-free baselines is robustness to initialization; if FIDE only works when the initial Gaussian covers the optimum, its claimed advantage is much weaker.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Feature-Informed Diffusion Evolution (FIDE), a black-box optimization method for inverse rendering. FIDE alternates between (i) a diffusion-model candidate proposer conditioned on dense visual features of rendered images (extracted by a frozen ViT) and (ii) a CMA-ES evolutionary update that refines the search distribution. The method treats the renderer as an opaque image-producing function and does not compute gradients. The authors claim that FIDE 'requires no gradients or specific initialization' and 'is able to converge to a global minimum, even in the presence of many local minima' (Sec. 1). The method is evaluated on five tasks (spline, robot arm, camera/light in San Miguel, Voronoi shader, and glass beads in a Cornell box) against CMA-ES, dual annealing, DiBO, and ZeroGrads, using MSE, LPIPS, and LOI losses, with 10 seeds per setup and median/quantile plots. Ablations cover feature extractors, training epochs, dimensionality, and a dual-annealing variant. The paper includes a supplementary with implementation details and additional results.","tokens_in":19932,"tokens_out":5357,"duration_ms":58653,"significance":"If the empirical results are accepted as stated, FIDE is a meaningful contribution to derivative-free inverse rendering: it is, to my knowledge, the first method to condition a generative diffusion proposer on dense image features extracted from a frozen pretrained network, and the idea of using CMA-ES's covariance to define the diffusion noise model is neat. The evaluation is broad and mostly careful: it includes several tasks, multiple loss functions, multiple baselines, 10 seeds, quantile plots, and ablations of feature extractors, epochs, and dimensionality. The paper does not ship machine-checked proofs or code, but the algorithmic description is sufficiently detailed to be replicable. The main weaknesses are that the strong claims of 'no specific initialization' and 'global minimum convergence' are not supported by the evidence as presented: the initialization is fixed to a centered broad Gaussian for all tasks, and no task with a known unique optimum is included. These issues are local and fixable, so the appropriate recommendation is major revision.","major_comments":[{"comment":"The claim that FIDE 'requires no specific initialization' and 'is robust to far-from-optimum initialization' is not supported by the experiments. Supplement S1 states that all tasks use the same initialization: mean 0.5 and sigma0 = 1/6, with the justification that the entire parameter space [0,1] is within 3 sigma. This is a strong, problem-specific prior centered at the middle of the domain. Moreover, in d dimensions a Gaussian with per-coordinate sigma = 1/6 concentrates most of its mass in a shell of radius roughly sqrt(d)/6; for d=40 the mass near the corners or boundaries of [0,1]^d is vanishingly small. If the optimum lies near a boundary or in a narrow basin outside this initial distribution, the method's claim of not requiring a specific initialization would break. The manuscript never varies the initial mean or initial covariance. I ask for an ablation that varies mu_0 (e.g., 0","section":"Sec. 1 / Supplement S1"},{"comment":"The statement that FIDE 'is able to converge to a global minimum, even in the presence of many local minima' is an empirical overreach. For the Spline task, the target is an ampersand font glyph that cannot be exactly reproduced by the optimized B-spline; the notion of 'global minimum' is therefore not defined (or at least not verifiable) for that task. For the other tasks, the paper reports only image losses (MSE/LPIPS/LOI), not parameter recovery error relative to a known ground truth. The evidence supports 'achieves lower image loss than baselines' or 'consistently finds better solutions', but not 'converges to a global minimum'. Please either remove the global-minimum claim or add an experiment with a known unique optimum and report parameter error.","section":"Sec. 1 and Fig. 5/6"},{"comment":"The comparison to differentiable rendering in Fig. 7 is used to support the broad conclusion that 'gradient-based methods require informed initialization' (Sec. 4.3). The figure shows four runs of Mitsuba 3's prb_projective integrator on the Beads task, all of which diverge, but the number of runs and the exact initialization are not specified in the text. This is acceptable as a motivating example, but it is not a systematic comparison; for a fair comparison, the paper should report the variance over seeds and the initialization used for Mitsuba, or soften the conclusion to 'on this task, the tested gradient-based method diverges'. As written, the claim 'gradient-based methods require informed initialization' is too broad for the evidence.","section":"Sec. 4.2 / Fig. 7"}],"minor_comments":[{"comment":"The legend 'Our overall best result' in Fig. 5 is unclear: it is not defined in the caption or text whether this is an oracle best over seeds, a separate curve, or something else. Please clarify.","section":"Fig. 5 / general"},{"comment":"The evaluation reports medians and quantiles over 10 seeds, but no significance tests or confidence intervals. Given that the performance differences are visually substantial on some tasks, this is not a blocking issue, but adding a simple paired test (e.g., Wilcoxon) over the 10 runs would strengthen the claim.","section":"Sec. 4.3 / statistical analysis"},{"comment":"The supplement says source code will be released 'upon acceptance'. For a method whose main novelty is algorithmic, I would encourage the authors to release code with the revision or at least a reference implementation, since the paper would be much easier to build upon with code available.","section":"Supplement S1"},{"comment":"Line 8 says 'Noise is sampled during training using Sigma', but the exact relationship between the CMA covariance matrix and the diffusion timestep is only described in the supplement (S1). Please include a brief explanation in the main text so that the algorithm is self-contained.","section":"Sec. 3.2 / Algorithm 1"},{"comment":"The 'simple patch embedding' is a learnable, one-layer patch embedding. The text calls the degradation 'graceful', but the quantitative gap on San Miguel and Beads appears substantial. Please add a sentence quantifying the final LPIPS gap.","section":"Sec. 4.3 / Fig. 9"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a solid empirical contribution, but the two central claims—'no specific initialization' and 'global minimum convergence'—are not supported by the current evidence. Both are fixable within the scope of a revision: add an initialization robustness experiment and rephrase the global-minimum claim. I would not reject, but I would not accept as-is. The paper's fit for the journal is good; the topic is inverse rendering and non-differentiable optimization, which is core to the readership. One concern for the editor: the paper does not yet provide code, which is increasingly expected for empirically driven methods; the authors state an intention to release upon acceptance, which is acceptable but should be followed through."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick read. The paper does something real: it replaces scalar-loss conditioning in diffusion-based black-box optimizers with dense ViT features of the current render, and pairs an online-trained denoiser with CMA-ES. The empirical study is solid: five tasks spanning path tracing, splines, Voronoi, robot kinematics, 10 seeds, medians and quantiles, ablations on feature extractors, loss functions, training epochs, and even an alternative evolution strategy in the supplement. Under LPIPS and LOI it consistently beats scalar-loss baselines. That is a genuine contribution to inverse rendering and DFO.\n\nThe soft spots are mostly in the framing, not the method. The introduction claims FIDE 'is able to converge to a global minimum,' but no formal argument is given, and for the Spline task the target is not even representable by the spline, so 'global minimum' is undefined. That should be softened to 'strong local optimum' or 'best-found solution.' The bigger issue: the 'no specific initialization' claim is not tested. All experiments start with mean 0.5 and sigma0=1/6, a Gaussian that covers the entire normalized domain. That is a specific, fairly generous initialization. The paper never varies the starting mean or sigma0, so the robustness-to-initialization claim is unsupported. If FIDE only works with that broad centered prior, the advantage over baselines is less impressive. The comparison to baselines uses the same initialization, so the core empirical result stands, but the generalization claim needs an experiment with off-center or narrow initializations.\n\nAlso, no code is released yet, which makes replication harder. The supplement describes the setup well, but source code would help.\n\nVerdict: worth a serious referee. The method is interesting, the experiments are above average, but the paper overreaches in its abstract and intro. I'd ask for an initialization sweep, a reworded 'global minimum' claim, and ideally code release. As is, it's a conditional accept, not a reject.","headline":"Solid method paper that overclaims 'no specific initialization' and 'global minimum'; the core feature-guided diffusion idea is worth a serious look.","tokens_in":20444,"tokens_out":4685,"would_cite":true,"duration_ms":44796,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Feature-guided diffusion solves inverse rendering without gradients, and it escapes local minima that stall differentiable renderers.","keywords":["inverse rendering","black-box optimization","diffusion models","feature guiding","CMA-ES","vision transformer","derivative-free optimization","global optimization"],"falsifier":"Apply FIDE to a benchmark whose global optimum is a narrow valley near a corner or boundary of the normalized parameter space, using the paper's default initialization (mean 0.5, sigma 1/6). If FIDE consistently fails to find it while succeeding from a hand-placed initial mean near that corner, then the 'no specific initialization' claim is falsified. A simpler check: rerun the five reported tasks with the CMA-ES initial mean shifted to 0.9 in every coordinate and record any convergence collapse.","tokens_in":19456,"feed_emoji":"🎨","tokens_out":5187,"duration_ms":53300,"temperature":0.7,"pith_summary":"The paper proposes Feature-Informed Diffusion Evolution (FIDE), a fully black-box method for inverse rendering that never computes or estimates gradients and needs no task-specific initial guess. Instead of reducing each candidate rendering to a scalar loss value, FIDE extracts dense visual features with a frozen vision transformer and uses them to train a diffusion model that proposes new parameter candidates. Those candidates are refined in a closed loop by CMA-ES, which also supplies the anisotropic search distribution that guides both training and sampling. The authors argue that this synergy lets the method converge to global minima on five diverse tasks within 25,000 rendering calls, outperforming scalar-loss baselines and differentiable rendering.","feed_headline":"Feature-guided diffusion solves inverse rendering without gradients","feed_subtitle":"A black-box optimizer escapes local minima that trap differentiable renderers.","key_machinery":"The carrying mechanism is a closed loop between two components: a diffusion denoiser (a small transformer with cross-attention over image feature tokens) that is trained online on the rendered population to predict clean parameters from noisy ones, conditioned on ViT features (DINOv3) of the target image; and CMA-ES, which supplies the anisotropic Gaussian search distribution used both to sample candidates and to set per-parameter noise during diffusion training. Only half of each iteration's samples are denoised before rendering, preserving diversity and preventing mode collapse. The synergy is that CMA-ES shrinks the region the denoiser must model, while the denoiser seeds CMA-ES with bett","core_discovery":"The central claim is that feature guiding—conditioning an online-trained diffusion proposer on dense visual features of rendered images rather than on scalar loss values—turns a challenging black-box inverse problem into a tractable one. FIDE treats the renderer as an opaque function, learns a mapping between image features and the parameters that produced them, and combines this with CMA-ES's covariance adaptation to narrow the search. The paper argues this resolves both the information bottleneck of scalar losses and the mode-averaging problem of direct inverse predictors, and supports it with experiments showing convergence across spline, robot-arm, camera-plus-light, Voronoi, and glass-b","pith_inferences":["Because the denoiser is trained online, it effectively builds a local inverse model of the specific scene being optimized, a property that could enable warm-starting on subsequent targets in the same scene—an extension the paper does not test.","The feature-guiding principle is likely transferable to other black-box functions with structured, spatial output (audio spectrograms, meshes, point clouds), provided a suitable feature extractor exists.","The paper's fixed initialization means its 'no specific initialization' claim is really a claim about a reasonably broad default Gaussian; a task with a tiny global basin near the domain boundary would stress-test that claim more severely than the five benchmarks do.","A natural next experiment is to ablate the ViT feature extractor against task-specific or cheaper features on higher-dimensional problems, since the paper shows graceful degradation but not whether learned features close the gap to pretrained ones."],"forward_implications":["Gradient-free inverse rendering becomes practical for small-to-medium parameter counts, removing the need to derive or implement differentiable renderers.","Feature guiding, not just the loss function, drives convergence: the method succeeds under MSE, LPIPS, and LOI losses, including tasks where all baselines fail under MSE.","A single generic initialization suffices across diverse tasks, while a differentiable renderer diverges from the same starting point on the Beads task.","Convergence scales smoothly to about 60 parameters, with 80 parameters requiring more than double the render budget; runtime per iteration stays roughly constant."],"fun_headline_variants":["Feature-guided diffusion beats gradients without any gradients","Escaping local minima: feature-guided diffusion for inverse rendering","Black-box rendering solved: feature-guided diffusion needs no gradients","Feature cues, not scalars: diffusion cracks inverse rendering","Gradient-free inverse rendering: feature-guided diffusion wins"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that FIDE needs no specific initialization rests on the assumption that a fixed initial Gaussian centered at the midpoint of the normalized parameter domain, with standard deviation 1/6 so the whole domain lies within three sigmas, gives the search enough probability mass on the global optimum; the paper never varies this starting point across experiments.","fun_headline_variants_meta":{"raw":{"variants":["Feature-guided diffusion beats gradients without any gradients","Escaping local minima: feature-guided diffusion for inverse rendering","Black-box rendering solved: feature-guided diffusion needs no gradients","Feature cues, not scalars: diffusion cracks inverse rendering","Gradient-free inverse rendering: feature-guided diffusion wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000773,"raw_usage":{"total_tokens":3245,"prompt_tokens":718,"completion_tokens":2527,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":2450}},"tokens_in":462,"tokens_out":2527,"duration_ms":18886,"temperature":1.0,"reasoning_tokens":2450,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T18:01:38.740069+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply FIDE to a benchmark whose global optimum is a narrow valley near a corner or boundary of the normalized parameter space, using the paper's default initialization (mean 0.5, sigma 1/6). If FIDE consistently fails to find it while succeeding from a hand-placed initial mean near that corner, then the 'no specific initialization' claim is falsified. A simpler check: rerun the five reported tasks with the CMA-ES initial mean shifted to 0.9 in every coordinate and record any convergence collapse.","supporting_citations":[],"review_version":1}