{"id":"6e9b37df-57b9-4b54-a544-b4d949323424","arxiv_id":"2602.08267","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A Lie-group diffusion sampler that inverts unknown data transformations at test time, using only an energy function, and improves pretrained models on affine/homography images and PDE solving.","lead":"This paper builds a diffusion sampler on Lie groups that, given a distorted input and an energy for the original data, samples likely inverse transformations to undo the distortion. The authors use it as a training-free way to make pretrained networks more robust to image warps, homographies, and PDE symmetries, and report gains over optimization and sampling baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Finite-N MC score estimator (N=2–10) has only asymptotic justification; no direct check that TIED samples the transformation posterior.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the practical sampler uses a biased Monte Carlo score estimator with very few samples (N=2–10) and no finite-N guarantee. The central claim that TIED samples from the transformation posterior and restores inputs to the training distribution depends on the accuracy of this estimator. The paper's theory (Prop. B.12) only ensures convergence as N→∞, which is not satisfied in the experiments. The empirical evaluation does not directly measure sampler quality; accuracy and FID can be high even if the sampler is not faithful to the posterior. The synthetic SO(10) result uses N=100 and a smooth energy, so it does not cover the rugged neural-network regime. I agree with the reader that this is the weakest assumption. My proposed test would settle the concern by comparing the N=2 score estimate against a high-N estimate on the actual task. If the error is small, the concern is resolved and the verdict stands; if large, the paper's central claim would need to be softened or the configuration changed. Since the reader already issued a CONDITIONAL verdict, my analysis does not move the verdict.","tokens_in":44808,"tokens_out":14640,"duration_ms":153425,"concrete_test":"Take 100 MNIST test images transformed by random affine maps. For each image, run the reverse SDE of Algorithm 1 with the classifier energy and the Table 3 configuration (N=2, Δt=1/50). At every timestep, also compute the score estimate with N=1024 MC samples of k_t, using the same random seed for the reverse-SDE Brownian increments. Record the relative L2 error ||s_N=2 − s_N=1024||_2 / ||s_N=1024||_2 in the Lie algebra (Aff(2) parameters). If the median relative error over timesteps and images exceeds ~30%, the N=2 score is unreliable; additionally compare the final inverse-transformed images between the two runs — if the average pixel/group distance is large, the sampler output is dominated by MC noise rather than the posterior.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 estimates the trivialized score via a logsumexp of N Monte Carlo draws from the noise density k_t (Prop. 4.3, Eq. 23). Prop. B.12 proves only consistency as N→∞; no non-asymptotic bound is given. In the image experiments N=2–4 and in PDE experiments N=10 (Table 3), while energies are rugged neural-network logits or boundary-distance functions. If the finite-N bias/variance is large, the drift in the reverse SDE (Eq. 4) is inaccurate, so the samples are not drawn from the transformation-inversion posterior p(g|x̃). Consequently the in-distribution guarantee of Prop. 3.1 and the equivariance rationale of Prop. 3.2 do not apply to the actual algorithm. The paper's evaluations report downstream accuracy and FID, which can be good even if the sampler is not sampling the posterior (e.g., if it acts as a deterministic optimizer). The synthetic SO(10) experiment uses N=100 and a smooth quadratic energy, so it does not validate the practical regime. This is the weakest link in the central claim that TIED 'samples from the posterior' and 'restores transformed inputs to the training distribution.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies transformation inversion: an observed datum x̃ is generated by applying an unknown element g of a Lie group G to an in-distribution sample x. The authors model the posterior p(g|x̃) as a Boltzmann distribution from an energy E(g) = E_x(g^{-1}·x̃) − log|det J_{g^{-1}}(x̃)| and propose Transformation-Inverting Energy Diffusion (TIED), a diffusion sampler on the group. The sampler uses a new 'trivialized target-score identity' (Prop. 4.2) that expresses the score of the noised posterior as an average of clean energy gradients, and a Monte Carlo estimator (Prop. 4.3, Eq. 23) suitable for implementation. Algorithm 1 discretizes the reverse trivialized SDE using exponential-map updates. The authors prove that the posterior is equivariant (Prop. 3.2), which they use for test-time equivariance of pretrained networks. Experiments cover SO(10) sampling, MNIST with affine and perspective transformations, and PDE solving under Lie point symmetries, reporting improvements over optimization and sampling baselines.","tokens_in":45207,"tokens_out":6051,"duration_ms":70706,"significance":"If the practical sampler is sound, the paper makes a valuable contribution: a diffusion-based posterior sampler on general Lie groups that does not require compactness, a bi-invariant metric, or linear actions, and that enables training-free test-time equivariance for generic pretrained models. The theoretical development is largely self-contained, and the SO(10) experiment shows a substantial speedup over kinetic Langevin in a smooth, low-dimensional posterior regime. The paper includes code and compares across several energy choices and baselines. The main caveat is that the deployed algorithm uses a finite-N Monte Carlo score estimator whose bias is not controlled, so the theoretical guarantees do not automatically transfer to the experimental settings.","major_comments":[{"comment":"The practical estimator replaces the integral in Proposition 4.3 with a logsumexp over N Monte Carlo draws from k_t (Eq. 23). Proposition B.12 establishes only almost-sure consistency as N→∞; no non-asymptotic bound is given. Table 3 uses N=2–10 in all practical experiments, while the SO(10) demonstration uses N=100 with a smooth quadratic energy. Because the drift in the reverse SDE (Eq. 4) is exactly this estimated score, a finite-N bias means the sampler output is not distributed according to p(g|x̃). Consequently the in-distribution guarantee of Prop. 3.1 and the equivariance guarantee of Prop. 3.2 do not apply to the deployed algorithm. The paper needs either a non-asymptotic error bound or a direct empirical check that the sampler produces samples from the posterior in the practical N regime.","section":"§4.3, Eq. (23); Prop. B.12; Table 3"},{"comment":"The evaluation reports classification accuracy and FID, but not whether TIED actually samples the posterior. Accuracy can improve even if the sampler degenerates into a deterministic optimizer, and FID measures closeness to the training image distribution, not agreement with p(g|x̃). The synthetic SO(10) experiment (Fig. 3) does validate posterior sampling in a smooth-energy, N=100 regime, but the image and PDE experiments use N=2–10 with rugged neural-network or boundary-distance energies and do not include any direct sampling-quality metric. The central claim that TIED 'samples from the posterior' and that the theoretical guarantees therefore hold at test time is thus not directly supported by the experiments. I recommend adding a direct measure of sampler quality for the practical N regime — for example, comparing TIED output histograms to ground-truth posterior densities on a small g","section":"§5, Tables 1–2, Fig. 3"}],"minor_comments":[{"comment":"The notation ∇_g log p_0(g_0) in the trivialized target-score identity is potentially misleading. The proof clarifies that the argument is g_t b with b ≡ g_t^{-1} g_0, and that the derivative is taken with respect to g_t. I recommend writing this as ∇_g log p_0(g_t b)|_{b = g_t^{-1} g_0} or using an explicit coordinate form to avoid confusion.","section":"§4.2, Eq. (5)"},{"comment":"The TIED accuracy for the Aff(2,R) classifier-energy row is 85.64% in Table 1 but 82.64±0.11% in Table 4 (five repeated tests). Please clarify which number is final and, if Table 1 is a single run, present the mean and standard deviation consistently across the tables.","section":"Tables 1 and 4"},{"comment":"The free action assumption is stated but not discussed. For data with nontrivial stabilizers, the posterior density on the group is not defined in the same way. A brief remark on how the method behaves or fails when the action is not free would help.","section":"§3.2, Prop. 3.1"},{"comment":"The initialization uses the approximation k_1 ≈ p_1 without empirical validation. Since the noise schedule is finite in all experiments, a sentence justifying this approximation (or a sensitivity check on γ_max) would strengthen the paper.","section":"§B.9, Algorithm 1, line 2"},{"comment":"FID is reported for MNIST using an inception-based feature space (Fatir, 2018). For 40×40 grayscale digits this may not be a meaningful metric; a simple distributional metric on pixels or on the classifier's feature space would be more interpretable.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The finite-N MC estimator is the main gate for acceptance. If the authors can provide a non-asymptotic bias bound, or — more practically — an experiment that directly validates posterior sampling in the N=2–10 rugged-energy regime, the paper would be acceptable. The theoretical identity and the experimental results are otherwise strong, and the source code is a plus. The discrepancy between Table 1 and Table 4 should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth taking seriously. It generalizes target-score diffusion sampling to general Lie groups via trivialization, removing bi-invariant metric assumptions, and applies it to transformation inversion for pretrained networks. That is genuinely new relative to De Bortoli et al. and Akhound-Sadegh et al.\n\nWhat is good: the trivialized target-score identity (Prop. 4.2) and the Monte Carlo estimator (Prop. 4.3) are clean, and the reverse-SDE proof is careful. The experiments on noncompact groups (Aff(2,R), PGL(3,R), PDE symmetry groups) are nontrivial, and TIED consistently beats gradient sampling and optimization baselines, sometimes by large margins. The code is released. The paper is also honest about the Jensen bias in the score estimator and only claims consistency as N grows.\n\nThe biggest soft spot is the gap between the theory and the practical sampler. Algorithm 1 uses N=2–10 Monte Carlo samples from a discretized noise density, and the score estimate is biased for finite N. Prop. B.12 only provides asymptotic consistency. For rugged neural-network energies, the bias and variance could be large, and the paper does not directly check whether the sampler produces samples from p(g|x̃). Accuracy and FID can improve even if the sampler is chasing local modes; the synthetic SO(10) experiment uses N=100 and a smooth quadratic energy, so it does not cover the practical regime. The authors should add a direct comparison of the sampled transformation distribution against a reference (for example, MCMC ground truth on small groups) for N=2–10.\n\nA smaller issue: Table 1 reports 85.64% for TIED with classifier energy on affine MNIST, but Table 4 reports 82.64±0.11% for the same setting. The main table omits standard deviations; the supplementary includes them. That discrepancy should be reconciled.\n\nAlso, the notation in Prop. 4.2, ∇_g log p0(g0), is easy to misread as a full gradient on G rather than the trivialized (Ad-twisted) derivative. The authors should make that convention explicit in the main text.\n\nBottom line: the central idea and principal results look sound. The posterior-sampling claim is not yet fully established, but that is fixable with modest additional evidence. This paper deserves a serious referee; I would send it out, asking reviewers to focus on the finite-N behavior of the score estimator and the reporting consistency.","headline":"Solid theory and strong empirical results on test-time equivariance, but the practical sampler's finite-N score estimate needs direct validation before the posterior-sampling claim is fully earned.","tokens_in":45617,"tokens_out":2002,"would_cite":true,"duration_ms":24605,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion sampling on the Lie group, driven by a new trivialized target-score identity, can invert unknown data transformations and bring distorted inputs back to the training distribution.","keywords":["transformation inversion","Lie groups","diffusion sampling","energy-based models","test-time equivariance","canonicalization","Lie algebra","Lie point symmetries"],"falsifier":"Take images with known ground-truth affine or homography transformations, run TIED with the same energy used in the paper, and compare the sampled inverse h against the true transformation g. If, at N=2–10 Monte Carlo samples, the recovered transformations do not concentrate near the true modes of p(g|x̃), or corrected images do not have lower energy than the uncorrected ones, then the sampler is not approximating the reverse SDE and the in-distribution guarantee collapses.","tokens_in":44781,"feed_emoji":"🔄","tokens_out":5939,"duration_ms":70484,"temperature":0.7,"pith_summary":"This paper tries to solve a blind inverse problem: a datum is changed by an unknown element of a Lie group, and the goal is to recover the inverse transformation. The authors model the posterior over group elements as a Boltzmann distribution defined by an energy on data space, then sample from that posterior using a diffusion process that stays on the group and works in the Lie algebra. Their key theoretical move is a new trivialized target-score identity, which lets the noisy score be estimated directly from clean energy gradients without needing clean samples or a bi-invariant metric. If the approach is correct, any pretrained neural network can be made robust to geometric transformations at test time simply by undoing the unknown transform before prediction.","feed_headline":"Group diffusion restores warped inputs to the training set","feed_subtitle":"Recovers inverse homographies and PDE symmetries, keeping pretrained models in-distribution with no retraining.","key_machinery":"The central machinery is the reverse trivialized SDE (Prop. 4.1) together with the trivialized target-score identity (Prop. 4.2) and its Monte Carlo form (Prop. 4.3). Trivialization means pulling gradients back to the Lie algebra via d(L_{g⁻¹})_g, so gradients evaluated at different group elements live in one common vector space and can be averaged without parallel transport. The identity expresses the noisy score as ∇_g log ∫ k_t(w) exp(−E(g w⁻¹) − log λ(w)) dµ(w), with λ the modular function accounting for non-unimodularity; this is what makes the sampler trainable from an arbitrary energy function. The reverse SDE is discretized with exponential-map updates, and the whole pipeline needs o","core_discovery":"The paper claims that the transformation-inversion posterior on any connected Lie group, p(g|x̃) ∝ exp(−E(g⁻¹·x̃))|det J(g⁻¹)|, is G-equivariant, so sampling h from it and applying h⁻¹·x̃ maps the transformed input back to the training distribution. The central discovery is a 'trivialized target-score identity': for the forward SDE dg_t = d(L_{g_t})_e[γ(t) dw_t], the trivialized score ∇_g log p_t(g_t) equals an average of initial trivialized scores over the conditional path, and therefore can be written as a Monte Carlo expression involving only the energy, the noise density, and a modular-function correction. Because all gradients are expressed in the Lie algebra, this score estimate works","pith_inferences":["Because only on-orbit energy differences matter, the method may tolerate fairly crude energies; a testable extension is to use an energy from a model trained on a different dataset and see whether inversion still pushes inputs in-distribution.","The finite-N Monte Carlo score estimate is a standing risk; a natural diagnostic is to measure the gap between TIED samples and the true Boltzmann posterior modes on a rugged energy, then adaptively increase N where the gap is large.","The trivialized identity suggests a route to sampling on other manifolds that admit a trivializing chart, and to faster samplers via distilling the reverse SDE into fewer steps, directions the paper hints at but does not pursue.","If the energy is provided by a learned generative model, TIED effectively turns blind inverse problems such as image registration or sensor-motion estimation into a single on-group sampling procedure, though that application is not demonstrated."],"forward_implications":["If TIED is correct, test-time equivariance no longer requires an equivariant architecture or retraining: any pretrained model can be made robust to Lie-group transformations using an energy function, including the model's own confidence.","Corrected inputs are guaranteed to be drawn from the data prior (Prop. 3.1), so downstream inference operates on in-distribution samples rather than on arbitrary transformed inputs.","The method covers noncompact, non-Abelian groups and nonlinear actions, needing only the Lie algebra and exponential map; this includes projective homographies and PDE symmetry groups such as SL(2,R)⋉H(1,R).","On the reported benchmarks, TIED restores accuracy and lowers FID compared with optimization, Langevin, and canonicalization baselines, and on the tested settings matches or exceeds specialized equivariant networks."],"fun_headline_variants":["Lie-group diffusion undoes unknown input transforms","Diffusion sampling inverts transformations on Lie groups","Restore warped data with energy-guided group diffusion","TIED: invert transformations by sampling on Lie groups","Group diffusion rescues pretrained models from input warp"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that averaging a handful of random noise draws per diffusion step gives a good enough estimate of the true noisy score; the proof only shows consistency in the limit of infinitely many draws.","fun_headline_variants_meta":{"raw":{"variants":["Lie-group diffusion undoes unknown input transforms","Diffusion sampling inverts transformations on Lie groups","Restore warped data with energy-guided group diffusion","TIED: invert transformations by sampling on Lie groups","Group diffusion rescues pretrained models from input warp"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1395,"prompt_tokens":753,"completion_tokens":642,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":569}},"tokens_in":497,"tokens_out":642,"duration_ms":7106,"temperature":1.0,"reasoning_tokens":569,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T03:20:33.758118+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take images with known ground-truth affine or homography transformations, run TIED with the same energy used in the paper, and compare the sampled inverse h against the true transformation g. If, at N=2–10 Monte Carlo samples, the recovered transformations do not concentrate near the true modes of p(g|x̃), or corrected images do not have lower energy than the uncorrected ones, then the sampler is not approximating the reverse SDE and the in-distribution guarantee collapses.","supporting_citations":[],"review_version":1}