{"id":"5812a909-c789-44ea-9283-1d0e46f6b8e8","arxiv_id":"1908.08767","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"KLDivNet, a neural estimator of the Donsker-Varadhan lower bound on KL-divergence, embedded in a VoxelMorph-style network, improves unsupervised multi-modality deformable registration Dice and ASD over MI and LNCC baselines.","lead":"A Fudan and SenseTime team trained a neural critic to estimate a KL-divergence-based similarity between images from different medical scanners, then used it as an unsupervised loss to drive deformable image registration. On brain and liver datasets the method edges out mutual-information baselines on Dice and surface distance, but gains are small, error bars are absent, and code is not yet released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (9) is not shown to be a faithful MI surrogate: voxel-shuffling corrupts local patch statistics if the CNN has spatial context, and critic tightness under one-stage training is never verified.","rationale":"The reader's verdict is CONDITIONAL and I agree. The paper's theoretical bridge from MI to the trained loss has two unverified links: the shuffled-image marginal estimator and the tightness of the finite-capacity critic under one-stage training. Section 4.2 shows the trained metric has a good global optimum on translation tests, which is real evidence, but it does not establish that Eq. (9) is an unbiased DV bound; a biased surrogate can still behave plausibly on a narrow test. The empirical margins over RegNet+MI are small and no error bars are given, so Tables 2-4 cannot independently validate the surrogate. The correct disposition remains conditional: the method is plausible and partially supported, but the central statistical claim needs direct verification. The proposed retraining test would settle whether the shuffle bias affects the actual registration pipeline, and the direct estimator comparison would separate bias in the metric from bias in the downstream task.","tokens_in":9197,"tokens_out":11647,"duration_ms":131310,"concrete_test":"Retrain DivRegNet replacing the shuffled-fixed-image marginal term in Eq. (9) with an unbiased product-of-marginals estimate: for each moved patch sample a fixed patch from a different scan (or from a large pool of independent fixed images), average sufficiently many draws, and use this value in the loss. Compare test Dice/ASD/HD on the three datasets against Tables 2-4, and also directly measure, for a fixed trained critic, the gap between the shuffled estimate and the unbiased estimate. If the loss or the registration accuracy changes materially, the shuffle approximation is load-bearing; if they match, repeat with a critic using 3x3x3 convolutions to resolve whether the receptive-field assumption changes the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core claim is that maximizing S in Eq. (8)-(9) provides an effective multi-modality similarity for registration. This requires Eq. (9) to estimate the Donsker-Varadhan bound E_{P_{T(M),F}}[Phi] - log E_{P_{T(M)}P_F}[e^Phi]. The paper computes the marginal term by feeding a voxel-shuffled fixed image F_s. If D_theta has any receptive field larger than one voxel, which the 'patch pairs' wording and downsampling in Sec. 3.1 imply, then shuffled-image patches are not draws from the marginal patch distribution of F, so mean_i e^{D_theta(T(M), F_s)_i} is biased. If instead the 1x1x1 convolutions are literal, the critic is per-voxel and the shuffle is valid, but then the method is essentially a neural histogram MI and the 'CNN similarity' framing is misleading. The paper does not resolve this ambiguity, nor does it compare Eq. (9) to a direct product-of-marginals estimate. A related gap is that Eq. (7)'s supremum is not solved: with a finite critic and one-stage joint training of RegNet and KLDivNet (Sec. 3.2), the trained D_theta need not be close to Phi*, so maximizing the lower bound need not maximize true MI. The reported margins over RegNet+MI are small (e.g., Table 4 Dice 0.8351 vs 0.8348) and lack error bars, so they do not independently establish surrogate fidelity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KLDivNet, a CNN-based estimator of the Donsker-Varadhan lower bound on KL divergence, and embeds it as a similarity metric into DivRegNet, an unsupervised deformable registration network built on a VoxelMorph-style backbone. The method is evaluated on three datasets (AAL Brain, LiTS Liver, Hospital Liver) for both intra- and inter-modality registration, with comparisons to LNCC, histogram-based MI, and conventional FFD SEMI using Dice, ASD, and HD. The central claim is that maximizing the estimated Donsker-Varadhan bound provides an effective, differentiable surrogate for mutual information in multi-modality registration.","tokens_in":9561,"tokens_out":4344,"duration_ms":40108,"significance":"If the method works as claimed, it offers a principled and differentiable alternative to histogram-based MI, which is a genuine need for deep-learning-based multi-modality registration. The use of external anatomical metrics (Dice, ASD, HD) reduces circularity, and the Donsker-Varadhan formulation is a standard and attractive starting point. However, the theoretical derivation in Section 3.1 is garbled, the surrogate fidelity of Eq. (9) is not established, and the reported gains over a well-tuned MI baseline are small (Dice deltas of +0.002 to +0.006) with no statistical analysis. The idea is promising, but the current evidence does not support the claim of 'superior performance'.","major_comments":[{"comment":"The derivation of the Donsker-Varadhan bound is mathematically garbled. Eq. (4) introduces a function u(x) and a change of measure that is not justified; the manipulations do not follow from the preceding line. More importantly, Eq. (7) is labeled an 'upper bound form' and the text says 'This upper bound is tight,' but the Donsker-Varadhan representation is a lower bound on KL divergence. This is not a mere terminology slip: the method's foundation is the lower-bound inequality, and the mislabeling obscures the direction of the optimization. The derivation should be rewritten cleanly, and the terminology corrected.","section":"Section 3.1, Eq. (4)-(7)"},{"comment":"The marginal expectation E_{P_{T(M)}P_F}[e^Phi] is computed as mean_i e^{D_theta(T(M), F_s)_i}, where F_s is a voxel-shuffled fixed image. This is valid only if the critic D_theta has no spatial context beyond a single voxel. The manuscript describes 'patch pairs' and the architecture includes downsampling layers; if the receptive field exceeds one voxel, shuffled-image patches are not draws from the product of the marginal patch distributions, so Eq. (9) is a biased estimator. The paper neither states the effective receptive field nor compares Eq. (9) to a direct product-of-marginals estimate. This is load-bearing because the claim that S(T(M),F) estimates MI depends on this estimator.","section":"Section 3.1, Eq. (9)"},{"comment":"The inner supremum in Eq. (7) over Phi is not actually solved. In the one-stage training described in Section 3.2, the registration network and KLDivNet are optimized jointly by maximizing S, so the trained D_theta may be far from the optimal Phi*. Maximizing a lower bound with a suboptimal critic need not maximize true MI. The manuscript provides no analysis of the finite-capacity or optimization gap, and no experiment measures how close the trained critic is to the Donsker-Varadhan optimum. Without this, the method should be positioned as a heuristic, not as a faithful MI surrogate.","section":"Section 3.2"},{"comment":"The reported differences between DivRegNet and the best RegNet+MI baseline are very small: e.g., on AAL T1-T2, Dice 0.9546 vs 0.9524 and ASD 0.72 vs 0.75; on LiTS, Dice 0.9030 vs 0.9016; on Hospital Liver, Dice 0.8351 vs 0.8348. No error bars, standard deviations, repeated runs, or significance tests are given. The claim that DivRegNet delivers 'superior' or state-of-the-art performance is not statistically supported. Please provide uncertainty estimates or formal significance testing.","section":"Tables 2, 3, and 4"}],"minor_comments":[{"comment":"The phrase 'most underlined processes' is unclear; it likely should be 'most fundamental processes' or 'most important processes'.","section":"Abstract"},{"comment":"The notation in Eq. (1), 'logµ(x)/λ(x)µ(dx)', is ambiguous; it should be written as log(µ(x)/λ(x)) µ(dx) to avoid confusion.","section":"Section 3.1, Eq. (1)"},{"comment":"Reference [McAllester and Statos, 2018] should be spelled 'McAllester and Stratos'.","section":"References"},{"comment":"The caption does not fully identify which contour corresponds to which metric; labeling the subplots directly would improve readability.","section":"Figure 3"},{"comment":"The abbreviation 'FFD SEMI' is used without definition; please expand it on first use.","section":"Section 4.3"},{"comment":"The PWDE-based MI loss is referenced but not described; a brief description or pseudocode would aid reproducibility.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the authors have chosen a principled starting point, but the theoretical presentation is incorrect in places and the experimental evidence for the central claim is thin. The small margins over the MI baseline, combined with the lack of significance tests, make it hard to judge whether KLDivNet offers a real advantage. The authors should also clarify the architecture's receptive field and provide a bias analysis of the shuffle estimator. I would welcome a revised version addressing these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper applies the Donsker-Varadhan / MINE lower bound as a training loss for unsupervised multi-modality deformable registration, with a Y-shaped critic network feeding a VoxelMorph-style regressor. As far as I know, that specific application is new, and it's a sensible direction: it replaces the non-differentiable histogram MI with a neural estimator that can be trained end-to-end. The experiments cover three datasets and use external anatomical Dice/ASD/HD, so the evaluation is not circular. That part is genuinely useful.\n\nThe soft spots are real but not fatal. The derivation in Eq. (4) is garbled, and the bound is mislabeled as an upper bound when it's a lower bound; a reader shouldn't have to decode that. More substantively, Eq. (9) estimates the marginal expectation by feeding the critic a voxel-shuffled fixed image. That's only correct if the critic treats each voxel independently. But the critic includes downsample layers, so it almost certainly has a spatial receptive field larger than one voxel; in that case, patches from a shuffled image are not draws from the marginal patch distribution, and the estimate is biased. The paper doesn't address this, and it doesn't compare against a proper negative-sample estimator. I also note the reported gains over a tuned MI baseline are small — Dice improvements around 0.002 to 0.006 on LiTS and Hospital — and there are no error bars or significance tests, so the \"state-of-the-art\" claim is stretched. Code is promised but not released, which limits reproducibility.\n\nIs the central claim undermined? Not necessarily. The method could still work as an effective similarity metric; the bias might be small in practice. But the evidence as presented doesn't establish it, and the ambiguity about the critic's receptive field creates a hole in the theoretical justification.\n\nWho is this for? Researchers working on learning-based deformable registration, especially cross-modality. They'll want to read it, and they'll probably try the idea. It deserves a serious review: I'd accept it for peer review and ask for (a) a clean derivation, (b) an explicit statement of the critic's receptive field and an analysis or experiment on the shuffle bias, (c) error bars or statistical testing, and (d) released code. If those hold up, it could be a useful method paper.","headline":"A plausible and novel application of MINE to multi-modality deformable registration, but the reported gains are thin and the estimator's shuffling trick has an unanalyzed bias.","tokens_in":10101,"tokens_out":4118,"would_cite":true,"duration_ms":41188,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN maximising a Donsker-Varadhan KL bound gives unsupervised multi-modality registration that outperforms histogram mutual information and LNCC on brain and liver datasets.","keywords":["multi-modality image registration","deformable registration","KL divergence","Donsker-Varadhan representation","mutual information estimation","unsupervised learning","convolutional neural network","medical image analysis"],"falsifier":"Take a synthetic image pair with a known ground-truth deformation and a known intensity model, then compare the KLDivNet estimate of the second term, $\\operatorname{mean}_i e^{D_\\theta(T(M),F_s)}$, with a Monte Carlo estimate of $\\mathbb{E}_{P_{T(M)}P_F}[e^{D_\\theta}]$ obtained from genuinely independent samples with the same intensity distribution. A large gap shows the shuffled-voxel estimator is biased. Separately, sweep translations of a synthetic T1/T2 pair and check whether the maximiser of $S$ coincides with the true alignment; if the peak of $S$ is systematically off-center, the claim that it acts as a cross-modality similarity fails.","tokens_in":9016,"feed_emoji":"🧠","tokens_out":9583,"duration_ms":88006,"temperature":0.7,"pith_summary":"Cross-modality image registration usually relies on mutual information estimated from intensity histograms, which needs bin tuning and complicated gradients in deep learning. This paper proposes to replace it with a CNN that maximises the Donsker-Varadhan lower bound on the KL divergence between the joint distribution of a moved/fixed image pair and the product of their marginals. The resulting similarity module, KLDivNet, is trained unsupervisedly and then attached to a U-Net-like deformable registration network, forming DivRegNet. On AAL Brain, LiTS Liver, and Hospital Liver datasets the authors report that DivRegNet achieves the best Dice and average surface distance among the compared methods, in both same-modality and cross-modality tasks. The point of the work is that a differentiable, histogram-free estimate of mutual information can serve as a practical training loss for multi-modality deformable registration.","feed_headline":"KL-divergence loss lets one network register CT, MR and brain scans","feed_subtitle":"Beats histogram mutual information on Dice and surface distance across three datasets while training without labels.","key_machinery":"The load-bearing identity is the Donsker-Varadhan variational representation of KL divergence: for probability measures $\\mu$ and $\\lambda$, $D_{KL}(\\mu\\|\\lambda)=\\sup_\\Phi \\mathbb{E}_\\mu[\\Phi]-\\log \\mathbb{E}_\\lambda[e^\\Phi]$. Applied to image registration, $\\mu$ is the joint distribution of the moved and fixed images and $\\lambda$ is the product of their marginals, so the KL divergence is exactly their mutual information. KLDivNet is a Y-shaped CNN that parameterises $\\Phi$ by $D_\\theta(T(M),F)$, estimates the joint expectation as the mean over its output feature map, and estimates the marginal-product expectation by feeding the same network a voxel-shuffled fixed image $F_s$ and averaging $e^{D_\\theta}$. Maximising this objective with respect to both $\\theta$ and the registration transform is what the paper claims substitutes for computing histogram-based MI during training.","core_discovery":"The paper's central claim is that the right-hand side of the Donsker-Varadhan representation, $S(T(M),F)=\\sup_\\theta \\operatorname{mean}[D_\\theta(T(M),F)]-\\log\\operatorname{mean}[e^{D_\\theta(T(M),F_s)}]$, is a valid and effective similarity measure for deformable image registration across modalities. Here $D_\\theta$ is a convolutional network, the first mean is over the moved image $T(M)$ paired with the fixed image $F$, and $F_s$ is the fixed image with voxels shuffled so that it is independent of $T(M)$ while sharing the marginal intensity distribution. The paper argues that maximising this lower bound behaves like maximising mutual information, and that embedding it in a U-Net-like registration network yields unsupervised training because no deformation ground truth is needed. The reported experiments show DivRegNet outperforming histogram-based MI, LNCC, and a conventional FFD method on Dice and average surface distance across intra- and inter-modality tasks on brain and liver data.","pith_inferences":["Inference: the shuffled-voxel trick is likely biased for a convolutional critic, because shuffling destroys local spatial correlations inside the network's receptive fields; an honest product-of-marginals sample would require permuting patches or sampling an independent image with the same intensity histogram. This bias is not tested in the paper.","Inference: if that bias is large, part of DivRegNet's edge over histogram MI could come from the U-Net's smoothness or other regularisation rather than from the KL objective itself; comparing DivRegNet against the same architecture with a corrected marginal estimator would isolate the effect.","Inference: the same variational recipe could extend to other neural-dual distances, such as Wasserstein or Sinkhorn divergences, producing alternative unsupervised multi-modality registration losses with different invariance properties.","Inference: alternating more critic updates per registration update, or pretraining KLDivNet on a large corpus of unlabelled modality pairs, would likely narrow the gap between the estimated lower bound and true mutual information."],"forward_implications":["DivRegNet can be trained end-to-end on unlabelled pairs of CT and MR scans, so deformation ground truth is not needed for cross-modality tasks.","The KL-divergence similarity removes histogram-bin selection: unlike the Parzen-window MI baseline, there is no number-of-bins hyperparameter to tune.","The same trained RegNet can handle both intra-modality and inter-modality registration, so a single architecture covers T2-T2, T1-T2 and PD-T2 brain tasks.","Because KLDivNet is differentiable, backpropagation goes directly through the similarity loss without estimating a joint intensity histogram."],"supporting_citations":[{"why":"Supplies the variational representation of KL divergence that the proposed similarity bound is derived from.","marker":"[Donsker and Varadhan, 1976]"},{"why":"Companion paper extending the same asymptotic variational result, cited alongside the 1976 paper for the lower-bound identity.","marker":"[Donsker and Varadhan, 1983]"},{"why":"Introduces neural estimation of mutual information via the Donsker-Varadhan bound; KLDivNet adapts this estimator to image similarity.","marker":"[Belghazi et al., 2018]"},{"why":"Provides the unsupervised encoder-decoder registration architecture that DivRegNet's RegNet follows.","marker":"[Balakrishnan et al., 2018]"},{"why":"Journal version of the same registration framework; supplies the architecture details and a baseline comparison.","marker":"[Balakrishnan et al., 2019]"},{"why":"Provides the autograd-compatible histogram-based mutual information loss that RegNet+MI uses as the baseline.","marker":"[Sandkühler et al., 2018]"},{"why":"Supplies the conventional FFD iterative registration baseline and its public software for comparison.","marker":"[Zhuang et al., 2011]"},{"why":"Provides the LiTS liver dataset used for single-modality evaluation.","marker":"[Bilic et al., 2019]"},{"why":"Provides the AAL template used to generate the brain phantom dataset.","marker":"[Rolls et al., 2019]"},{"why":"Recursive cascaded network whose reported LiTS results serve as a comparison baseline.","marker":"[Zhao et al., 2019]"}],"fun_headline_variants":["No labels needed: KL divergence powers multi-modality registration","Unsupervised KL-divergence network registers CT, MR, brain scans","Multi-modality registration trained without voxel ground truth","KL divergence similarity beats histogram MI in registration","Unsupervised deformable registration for multi-modality images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result stands on the assumption that averaging the critic over a voxel-shuffled fixed image equals averaging over true independent samples from the fixed image's intensity distribution, and that the CNN critic is flexible enough to make the lower bound tight; neither gap is measured.","fun_headline_variants_meta":{"raw":{"variants":["No labels needed: KL divergence powers multi-modality registration","Unsupervised KL-divergence network registers CT, MR, brain scans","Multi-modality registration trained without voxel ground truth","KL divergence similarity beats histogram MI in registration","Unsupervised deformable registration for multi-modality images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000767,"raw_usage":{"total_tokens":3388,"prompt_tokens":923,"completion_tokens":2465,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":2384}},"tokens_in":539,"tokens_out":2465,"duration_ms":17731,"temperature":1.0,"reasoning_tokens":2384,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:29:19.484054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a synthetic image pair with a known ground-truth deformation and a known intensity model, then compare the KLDivNet estimate of the second term, $\\operatorname{mean}_i e^{D_\\theta(T(M),F_s)}$, with a Monte Carlo estimate of $\\mathbb{E}_{P_{T(M)}P_F}[e^{D_\\theta}]$ obtained from genuinely independent samples with the same intensity distribution. A large gap shows the shuffled-voxel estimator is biased. Separately, sweep translations of a synthetic T1/T2 pair and check whether the maximiser of $S$ coincides with the true alignment; if the peak of $S$ is systematically off-center, the claim that it acts as a cross-modality similarity fails.","supporting_citations":[{"cited_title":"Asymptotic evaluation of cer- tain markov process expectations for large time—iii","cited_arxiv_id":null,"evidence_quote":"Supplies the variational representation of KL divergence that the proposed similarity bound is derived from."},{"cited_title":"Asymptotic evaluation of certain markov process expectations for large time","cited_arxiv_id":null,"evidence_quote":"Companion paper extending the same asymptotic variational result, cited alongside the 1976 paper for the lower-bound identity."},{"cited_title":"Mutual information neu- ral estimation","cited_arxiv_id":null,"evidence_quote":"Introduces neural estimation of mutual information via the Donsker-Varadhan bound; KLDivNet adapts this estimator to image similarity."},{"cited_title":"An unsupervised learning model for deformable medical im- age registration","cited_arxiv_id":null,"evidence_quote":"Provides the unsupervised encoder-decoder registration architecture that DivRegNet's RegNet follows."},{"cited_title":"V oxel- morph: a learning framework for deformable medical im- age registration","cited_arxiv_id":null,"evidence_quote":"Journal version of the same registration framework; supplies the architecture details and a baseline comparison."},{"cited_title":"A nonrigid registration framework using spatially encoded mutual in- formation and free-form deformations","cited_arxiv_id":null,"evidence_quote":"Supplies the conventional FFD iterative registration baseline and its public software for comparison."},{"cited_title":"Automated anatomical abelling atlas","cited_arxiv_id":null,"evidence_quote":"Provides the AAL template used to generate the brain phantom dataset."},{"cited_title":"Recursive cascaded networks for unsuper- vised medical image registration","cited_arxiv_id":null,"evidence_quote":"Recursive cascaded network whose reported LiTS results serve as a comparison baseline."}],"review_version":1}