{"id":"8ba05fa1-4d59-45ec-9fd7-579038e478f8","arxiv_id":"2506.07883","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Diffusion-based causal image counterfactuals with semantic abduction improve identity preservation at a small cost in intervention effectiveness, demonstrated on Morpho-MNIST, CelebA-HQ, and mammogram artifact removal.","lead":"This paper builds a counterfactual image editing pipeline on diffusion models, adding a semantic latent code and per-step noise refinement to keep the person's identity stable while changing causal attributes. The result is a set of mechanisms that trade off how faithfully an edit follows the intervention against how well the original face is preserved.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Semantic abduction's identity gain may be driven by z leaking the intervened attribute; the paper's acknowledged non-identifiability leaves this unresolved.","rationale":"The reader's weakest_assumption is exactly the entanglement of z with causal parents, and my read confirms it as the most load-bearing threat. It is not an external critique; it targets the mechanism that produces the reported advantage, and the authors' own Limitation statement concedes the missing guarantee. The direction of the observed numbers is consistent with leakage: semantic mechanisms obtain lower LPIPS and lower effectiveness on the target attribute, exactly what one would expect if z carried part of the intervened attribute. I do not see a separate flaw severe enough to move the verdict: the Morpho-MNIST setup uses a known ground-truth SCM, the code is available, and the effectiveness numbers are openly reported. However, until the entanglement check is run, the key comparison in Table 3 cannot be interpreted as evidence for a principled semantic trade-off. This is precisely the conditional nature of the reader's verdict, so I recommend no change.","tokens_in":559,"tokens_out":3779,"duration_ms":134778,"concrete_test":"On Morpho-MNIST, where the true parent d is known, train the semantic mechanism with the exact Eq. (13) objective. On held-out data, train a logistic classifier on z=μ_φ(x) to predict the true digit d. If test accuracy is far above chance (e.g., >90%), z leaks the intervened attribute. Then re-run the Table 1/2 protocol with a second semantic mechanism that enforces pa⊥z|x, either by adding an orthogonality/adversarial penalty or by using p(z|pa), and compare the semantic-vs-spatial IDP gap at matched ω. If the IDP advantage disappears once entanglement is removed, the central semantic-identity-preservation claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that semantic abduction improves identity preservation with only a minor effectiveness cost. This holds only if the semantic latent z is exogenous noise independent of the causal parents pa, so that holding z fixed under an intervention edits the target attribute without carrying it over. Equations (13)-(14) train z with a standard normal prior and a beta-weighted KL, and the decoder conditions on both z and pa; the paper explicitly disclaims pa⊥z|x in the Limitations. Without identifiability or disentanglement, z can absorb pa. If z encodes eyeglasses or smiling, then under do(g) the abducted z already contains 'eyeglasses' information, so the generated counterfactual stays closer to the original, lowering LPIPS and inflating IDP precisely because the intervention is only partially obeyed. This is consistent with the lower F1(g) seen in Table 3 (96.86 for semantic vs 99.07 for spatial at ω=2, p∅=0.1). The reported trade-off may therefore be an artifact of attribute leakage rather than a 'principled trade-off between faithful causal control and identity preservation'.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a suite of diffusion-based structural causal mechanisms for counterfactual image generation, framed through Pearl's abduction-action-prediction procedure. It proposes (i) a spatial mechanism using conditional DDIM inversion for abduction, (ii) a semantic mechanism that adds a variational encoder producing a latent z treated as an exogenous noise term and held fixed under interventions, and (iii) dynamic semantic abduction with counterfactual trajectory alignment (CTA), which optimizes guidance tokens to align inverse and guided trajectories. The methods are evaluated on Morpho-MNIST, CelebA-HQ, and EMBED mammography data using axiomatic soundness metrics (composition, reversibility, effectiveness) and LPIPS-based identity preservation, with comparisons against VAE, HVAE, VCI, and DiffSCM baselines. The central claim is that semantic abduction improves identity preservation over purely spatial abduction at a modest cost in intervention effectiveness, and that dynamic ablation further preserves backgrounds and facial structure.","tokens_in":34662,"tokens_out":6753,"duration_ms":76906,"significance":"If the claims hold, the paper is a useful empirical contribution: it extends diffusion-based deep SCMs with a high-level semantic latent, introduces a practical test-time optimization for counterfactual trajectory alignment, and evaluates on three datasets with established external metrics and reported standard deviations in the main CelebA-HQ table. The writing is clear and the limitations are honestly stated. However, the central interpretation depends on an exogeneity assumption for z that the paper explicitly does not guarantee, and the main quantitative tables do not include the dynamic mechanism that is presented as a headline contribution. These issues are concrete and addressable within the paper's scope, so the appropriate disposition is major revision rather than acceptance in the current form.","major_comments":[{"comment":"The load-bearing claim that semantic abduction enables a 'principled trade-off' between causal control and identity preservation rests on treating z as exogenous noise independent of pa in Equations (12)-(14). The paper explicitly states in the Limitations that 'we do not guarantee that pa⊥⊥z|x'. With the non-identifiable unconditional prior p(z)=N(0,I), there is no mechanism preventing z from encoding the intervened attribute. If z encodes 'eyeglasses' or 'smiling', holding z fixed during do(g) or do(s) can carry the original attribute into the counterfactual, simultaneously lowering LPIPS and lowering effectiveness. Table 3 is consistent with this alternative explanation: for do(g) at ω=2, p∅=0.1, semantic obtains F1(g)=96.86 vs. spatial 99.07 while LPIPS improves from 0.171 to 0.096. The manuscript therefore does not rule out the possibility that the reported identity gain is an artifact of attribute leakage rather than a genuine causal property. A concrete fix within scope would be to train a probe to predict pa from abducted z, to ablate by re-sampling z from the prior p(z) during prediction, or to condition the semantic prior on pa and re-run Table 3.","section":"§3.2 and Limitations"},{"comment":"Dynamic semantic abduction is presented as a headline contribution in Section 3.3, and the text claims it further improves preservation of backgrounds, hairstyle, skin tone, and facial structure. However, Table 3 contains no rows for 'Semantic w/ Dynamic Abd.', and Figure 5 only shows IDP as a function of η for dynamic abduction, without a matched static-semantic baseline in the same plot. The quantitative evidence for the dynamic claim is therefore limited to qualitative panels in Figure 3c. Please add a quantitative soundness table (composition, reversibility, IDP, effectiveness) for dynamic semantic abduction under matched ω, p∅, and η, including an η=0 no-update control.","section":"§4, Table 3 and Figure 3c/5"},{"comment":"The abstract and Section 4 characterize the effectiveness cost of semantic abduction as 'minor' or 'small'. For the smiling intervention at ω=2, p∅=0.1, semantic obtains F1(s)=94.93 vs. spatial F1(s)=99.09, a drop of 4.16 points on the primary effectiveness metric, while IDP improves from 0.139 to 0.066. This is a substantial cost on the direct intervention target and should be discussed as such; the current wording understates a result that is central to the claimed trade-off.","section":"Table 3, do(s) block"}],"minor_comments":[{"comment":"The expectation in the training objective is written as E_{x,c,t,ε,z} without specifying the distribution of z; please clarify that z is sampled from the variational posterior q_φ(z|e_φ(x)), since this matters for the correct interpretation of the objective.","section":"Equation (13)"},{"comment":"The notation x_{t-1}=h^{-1}_{t-1|C}(x_{t-2}) is inconsistent with Equation (7), where h^{-1}_{t|C} maps x_{t-1} to x_t, and with Algorithm 1, which takes the already-inverted trajectory as input. Please clarify whether x_{t-1} is recomputed from x_{t-2} or read from the stored inversion trajectory.","section":"Equation (20) and Algorithm 1"},{"comment":"The citation 'Anonymous, 2024' appears in the Introduction and in the reference list as a submission under review; a published proceedings paper should not cite an anonymous under-review manuscript, so this should be replaced with a citable source or removed.","section":"References"},{"comment":"Standard deviations are reported for the CelebA-HQ results in Table 3 but not for the Morpho-MNIST results in Tables 1 and 2, making it difficult to assess whether small differences, such as the composition values separating spatial and semantic mechanisms, are meaningful.","section":"Tables 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"This is a strong empirical paper, but the leakage concern is central and explicitly acknowledged by the authors; I would require a direct experiment on the content of z before accepting the causal interpretation. The missing quantitative table for dynamic semantic abduction is also easily fixable and should be included. The citation to an anonymous under-review submission should be cleaned up in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a well-executed empirical paper that adds a real twist to diffusion counterfactuals—semantic abduction via a variational encoder held fixed under intervention—and it is honest about the main caveat. The identity-preservation gain over spatial DDIM inversion is real on the reported metrics, but the mechanism might partly be leakage of the target attribute into the semantic latent, and the authors don't provide the test that would settle it.\n\nWhat's new: the combination of a semantic latent z with DDIM inversion for low-level u, plus per-timestep null-token optimization for dynamic abduction, is not in the cited DiffSCM, Komanduri et al., or Mokady et al. It's an incremental but useful integration. They evaluate on three datasets, with controlled Morpho-MNIST experiments on a known SCM showing the framework can capture complex causal effects, and they report standard deviations. Code is available.\n\nSoft spots: the main one is identifiability. They train z with a standard normal prior, and the Limitations explicitly say they don't guarantee pa⊥z|x. If z absorbs the intervened attribute, composition and reversibility improve because the intervention is partially undone. The lower F1 for semantic on CelebA-HQ do(g) is consistent with that. On Morpho-MNIST, the known SCM still shows high effectiveness, so the leakage story isn't universal, but it remains a live concern for the face results. The per-image tuning of eta in CTA weakens the claim of a principled trade-off; a fixed procedure would be stronger. The CelebA-HQ baseline (VCI without center-crop) is somewhat unfavorable, though the authors acknowledge it. The 'first' claim is a bit strong given Komanduri et al., but the specific mechanism is new.\n\nBottom line: this paper is for people working on diffusion-based causal reasoning or counterfactual editing; they'll find the method useful and the limitations transparent. It deserves serious peer review—the weaknesses are not load-bearing, but they should be addressed by the authors. I'd cite the method and bring it to our reading group.","headline":"Solid empirical contribution with a real but acknowledged identifiability caveat; worth reading for diffusion counterfactual work, though the central identity-preservation claim is not fully settled.","tokens_in":35205,"tokens_out":2383,"would_cite":true,"duration_ms":30865,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that diffusion-based image counterfactuals can preserve identity by abducting a semantic latent code that stays fixed under intervention, at a small cost in effectiveness.","keywords":["counterfactual image generation","deep structural causal models","diffusion models","semantic abduction","identity preservation","classifier-free guidance","causal inference","medical image editing"],"falsifier":"Train a probe to predict the intervened attribute from the abducted semantic code $z$ alone; if the probe recovers the attribute substantially above chance on images where the intervention changes only the parents, the semantic code is entangled with the parents and the method's core assumption fails.","tokens_in":34226,"feed_emoji":"🖼️","tokens_out":12829,"duration_ms":132697,"temperature":0.7,"pith_summary":"This paper tries to establish that diffusion models can edit images counterfactually without silently changing who or what is in the image, by upgrading the abduction step of the structural-causal-model recipe. Instead of only inverting the diffusion trajectory into a spatial noise field, the authors add a variational semantic encoder that extracts a high-level code $z$ from the image and hold that code fixed while the causal parents (the attributes being edited) are intervened upon. On Morpho-MNIST, CelebA-HQ, and EMBED mammograms, semantic abduction improves composition and reversibility---the standard identity-preservation measures---compared with spatial-only abduction at the same guidance settings, while losing only a few points of intervention effectiveness. A dynamic variant, which optimises per-timestep guidance tokens to align the counterfactual trajectory with the inverse trajectory, further preserves backgrounds, facial structure, and illumination. This matters because faithful, identity-preserving counterfactuals are what make fairness audits, medical artefact removal, and causal explanations trustworthy.","feed_headline":"Diffusion edits gain identity from a frozen semantic code","feed_subtitle":"Holding a high-level latent during intervention cuts identity drift while keeping edits causally effective.","key_machinery":"The load-bearing object is the decomposed exogenous noise $\\epsilon=(u,z)$ with the approximate posterior $p_S(\\epsilon|x,pa)\\approx q_\\phi(z|e_\\phi(x))\\,\\delta(u-h^{-1}_\\theta(x,(z,pa)))$, together with the amortised anti-causal guidance update $\\epsilon_\\theta(x_t,\\emptyset,t)+\\omega(\\epsilon_\\theta(x_t,\\tilde{c}_{sem},t)-\\epsilon_\\theta(x_t,\\emptyset,t))$. The spatial term $u$ encodes low-level structure via DDIM inversion; the semantic term $z$ is the decodable invariant content; dynamic abduction adds per-timestep guidance tokens $\\emptyset^*_t$ optimised by counterfactual trajectory alignment. The mechanism carries the argument because holding $z$ fixed is exactly what localises the edit to the intervened parents, while the sharpened anti-causal score enforces effectiveness.","core_discovery":"The central discovery is a clean split of diffusion exogenous noise into a spatial term and a semantic term, with the semantic term drawn from a learned amortised posterior and kept invariant under intervention. Formally, the image mechanism is written as $x := f_\\theta(\\epsilon, pa) \\approx h^\\omega_\\theta(u, (z, pa))$, where $u$ is obtained by DDIM inversion of $x$ given $(z, pa)$, and $z \\sim q_\\phi(z|e_\\phi(x))$ is a variational code learned end-to-end with a $\\beta$-KL regulariser. At intervention time the parents change from $pa$ to $\\tilde{pa}$ while $z$ stays fixed, so the generated counterfactual continues to share the high-level content of the original. The paper's headline evidence is Table 3: at $\\omega=2$, $p_\\emptyset=0.1$, the semantic mechanism reaches LPIPS 0.096 for eyeglasses against 0.171 for the spatial mechanism, with F1 effectiveness staying within a few points; Figure 3c shows that dynamic semantic abduction additionally preserves background, hairstyle, skin tone and facial structure.","pith_inferences":["Beyond the paper, the same semantic-abduction idea should be testable in text-guided latent diffusion: a fixed identity embedding could play the role of $z$, holding the subject stable while a prompt changes the attribute.","A direct probe experiment---train a classifier to predict the intervened attribute from $z$ alone on held-out images---would quantify how much of the identity-preservation result depends on the exogeneity assumption the paper leaves unverified.","The reported trade-off curves suggest that counterfactual benchmarks should report soundness as a function of $\\omega$ and $p_\\emptyset$ rather than at a single operating point, because a single point can rank methods almost arbitrarily.","Counterfactual trajectory alignment could transfer to image-conditional editing beyond null-text inversion, potentially improving reversibility in any guided diffusion editor that optimises per-step tokens."],"forward_implications":["At fixed guidance settings, semantic mechanisms match spatial mechanisms on intervention faithfulness (F1 or accuracy within a few points) while roughly halving perceptual identity loss, so the identity/effectiveness trade-off becomes a controllable operating point rather than a fixed property of diffusion.","Increasing the guidance scale $\\omega$ improves effectiveness but monotonically worsens composition and reversibility in both mechanisms, so users can choose where to sit on the curve.","Dynamic semantic abduction with counterfactual trajectory alignment improves preservation of background, hairstyle, skin tone, and facial structure; on EMBED it removes 95.16% of triangular and 91.69% of circular skin markers while holding density and cancer labels fixed.","Diffusion models tuned for diverse random sampling (large $\\omega$, small $p_\\emptyset$) turn out to be the wrong operating regime for counterfactual soundness, so L3-style causal evaluation should guide conditioning design."],"supporting_citations":[{"why":"Supplies the deep structural-causal-model recipe of abduction-action-prediction that the proposed diffusion mechanisms instantiate.","marker":"Pawlowski et al. (2020)"},{"why":"Provides the high-fidelity probabilistic-causal-model baseline (HVAE) that the paper extends and compares against.","marker":"De Sousa Ribeiro et al. (2023)"},{"why":"Defines composition, reversibility, and effectiveness as the axiomatic soundness metrics used for evaluation.","marker":"Monteiro et al. (2023)"},{"why":"Classifier-free guidance is reframed as amortised anti-causal guidance, the mechanism that enforces intervention faithfulness.","marker":"Ho & Salimans (2022)"},{"why":"DDIM provides the deterministic invertible transitions used for spatial abduction.","marker":"Song et al. (2020a)"},{"why":"The diffusion-autoencoder architecture supplies the semantic encoder/decoder design that the paper adapts and trains end-to-end.","marker":"Preechakul et al. (2022)"},{"why":"Null-text inversion motivates the counterfactual trajectory alignment used in dynamic semantic abduction.","marker":"Mokady et al. (2023)"},{"why":"The prior diffusion SCM baseline that the paper generalises to multiple causal parents and compares in Table 1.","marker":"Sanchez & Tsaftaris (2022)"}],"fun_headline_variants":["Semantic abduction keeps diffusion counterfactuals faithful","Frozen semantic latent preserves identity in diffusion edits","Diffusion counterfactuals: split noise, hold semantics","Keep the semantic code, change the cause: diffusion edits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that the semantic code $z$ carries only identity information, not the attribute being edited; if the code secretly contains the intervened attribute, holding it fixed will leak the original attribute value into the counterfactual.","fun_headline_variants_meta":{"raw":{"variants":["Semantic abduction keeps diffusion counterfactuals faithful","Frozen semantic latent preserves identity in diffusion edits","Diffusion counterfactuals: split noise, hold semantics","Keep the semantic code, change the cause: diffusion edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000503,"raw_usage":{"total_tokens":2446,"prompt_tokens":925,"completion_tokens":1521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":1458}},"tokens_in":541,"tokens_out":1521,"duration_ms":13399,"temperature":1.0,"reasoning_tokens":1458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:22:53.491647+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a probe to predict the intervened attribute from the abducted semantic code $z$ alone; if the probe recovers the attribute substantially above chance on images where the intervention changes only the parents, the semantic code is entangled with the parents and the method's core assumption fails.","supporting_citations":[{"cited_title":"Measuring axiomatic soundness of counterfactual image models","cited_arxiv_id":"2303.01274","evidence_quote":"Defines composition, reversibility, and effectiveness as the axiomatic soundness metrics used for evaluation."}],"review_version":1}