{"id":"3ec70288-9a05-493a-a1e2-2f594804629f","arxiv_id":"2507.19882","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"DiCap generates diffusion-based counterfactual images and uses them as hard negatives in contrastive prompt learning, reporting modest gains over existing prompt learning baselines on unseen classes.","lead":"This paper introduces a method that generates altered images, called counterfactuals, to train better text prompts for vision-language models like CLIP. It reports gains on image classification, retrieval, and visual question answering, especially for categories the model has not seen during training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.2's condition (3) is inconsistent with condition (2) and the paper's own SCM: any invertible g with g(x)⊥(y,n) forces x⊥(y,n), contradicting the causal model where y,n cause x.","rationale":"The paper's central value proposition is a theoretically grounded guarantee of identifiability and strict error bounds for counterfactual generation. The reader's CONDITIONAL verdict rests on condition (3) being unverified. My stress-test finds a stronger, internal inconsistency: condition (3) cannot hold for the paper's own causal model when combined with the required invertibility of g. This is not a matter of external consensus but of logical compatibility among the stated assumptions. A separate gap also exists in Corollary 3.4, where reconstruction error at the factual label does not bound error at the counterfactual label without an unstated Lipschitz or uniform-error assumption on h. The empirical recipe may still be useful, but the theoretical foundation advertised as the paper's main novelty is not supportable as written. The manuscript would need a major theoretical revision, so I recommend REJECT for the current version; a corrected theory could alter this verdict. The reader correctly identified condition (3) as load-bearing, so my agreement is partial: I agree on the location but find the situation is not merely an unverified assumption but a contradiction with the stated model.","tokens_in":15539,"tokens_out":9574,"duration_ms":95040,"concrete_test":"Analytical check: instantiate the minimal SCM from Section 3.2, e.g., x = (u, y, n) with u⊥(y,n) and non-constant independent y,n. For any invertible g, compute I(g(x);y,n); by invariance it equals I(x;y,n) = H(y)+H(n) > 0, so condition (3) fails. The decisive test is to exhibit an invertible g and a non-degenerate f with I(g(f(y,n,u)); y,n)=0; the data-processing inequality shows this is impossible unless f is constant in (y,n). This settles whether the theorem's premises are non-vacuous.","verdict_should_be":"REJECT","load_bearing_attack":"Section 3.2's data generation process states that the label y causes causal features and n causes non-causal features, so x is a non-trivial function of (y,n) in Eq. (4). Theorem 3.2 assumes condition (2) that g is invertible and condition (3) that g(x) ⊥ y,n. For any invertible g, mutual information is invariant: I(g(x); y,n) = I(x; y,n). If g(x) is independent of (y,n), then I(x; y,n) = 0, meaning x must be independent of (y,n). This contradicts the assumed SCM whenever f depends on y or n, which is exactly the setting the paper aims to model. The premise set of Theorem 3.2 is therefore satisfiable only in a degenerate causal model where labels and non-causal factors have no effect on images, making the identifiability conclusion vacuous for the problem addressed. The reader's concern is sharper than 'not verified': condition (3) cannot hold together with condition (2) under the paper's own causal assumptions. Thus the advertised theoretical guarantee in the abstract is internally unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DiCap, a prompt-learning framework that generates counterfactual images using classifier-guided diffusion with an anti-causal predictor and then uses these images as hard negatives in a contrastive loss to align prompts with causal features. The authors claim theoretical guarantees for identifiability and error bounds of the counterfactual generation, and report accuracy improvements over CoOp, CoCoOp, and CPL on image classification, image-text retrieval, and visual question answering, particularly on unseen classes.","tokens_in":15788,"tokens_out":6793,"duration_ms":59950,"significance":"If correct, the method would be a significant contribution because it couples diffusion-based counterfactual generation with prompt learning and demonstrates consistent gains across three tasks. The empirical study is broad and the reported results are uniformly positive. However, the central theoretical claim is not merely unproven; it rests on an internally inconsistent set of assumptions, so the advertised guarantee does not hold. The paper also omits the derivations promised by its 'rigorous theoretical' framing, and the connection between the theory and the actual algorithm is not established. These issues affect the core contribution of the manuscript rather than its presentation.","major_comments":[{"comment":"Condition (3) of Theorem 3.2 is inconsistent with the structural causal model in Eq. (4) and Figure 3. For any invertible function g, mutual information is invariant, so I(g(x); y, n) = I(x; y, n). If g(x) is independent of (y, n), then I(x; y, n) = 0, which implies x is independent of (y, n). This contradicts Eq. (4) whenever f depends on y or n, which is exactly the setting the paper aims to model. The premise set is satisfiable only in the degenerate case where f is constant in (y, n), making the counterfactual outcome equal to the factual outcome and the identifiability conclusion vacuous. Consequently, Corollaries 3.3 and 3.4 and the abstract's claim that the approach 'guarantees the identifiability of counterfactual outcomes' are not supported.","section":"Section 3.4, Theorem 3.2"},{"comment":"No proofs or derivations are provided for Theorem 3.2 or for Corollaries 3.3 and 3.4. The manuscript states the results and asserts that they follow from rigorous mathematical reasoning, but the actual arguments are absent. This is not a presentation issue: the central theoretical contribution of the paper consists of these statements, and without proofs or precise references the claims cannot be verified by a reader.","section":"Section 3.4"},{"comment":"The error bound in Corollary 3.4 concerns the reconstruction pair (g, h) with h(g(x), y, n) = x, but the algorithm actually generates counterfactuals through the classifier-guided sampling procedure in Eq. (2), and the relation between that procedure and the function h is never established. In addition, the distance d(·,·) used in the reconstruction-error assumption is not defined, and no continuity or Lipschitz property of h is stated to transfer a bound on d(h(g(x), y, n), x) to d(h(g(x), y_cf, n), x_cf). The claimed error transfer is therefore not a formal consequence of the stated assumptions.","section":"Section 3.4 vs. Section 3.3"},{"comment":"The paper claims that the generated images satisfy 'minimal sufficiency' according to Definition 3.1, but the only quantitative evaluation uses the CLD metric borrowed from Sanchez and Tsaftaris, which measures a predictor's confidence change and not the distance constraint in Eq. (3). No experiment checks whether the generated counterfactual is the smallest perturbation that changes the label. Thus the central property that the method is designed to guarantee is not actually validated.","section":"Section 4.3.2, Definition 3.1"}],"minor_comments":[{"comment":"The tables report accuracies and percentage improvements inline in a way that is difficult to parse, for example 'CoOp 97.93 8.04 92.82 1.87 …'; using separate columns or a delta format would improve readability.","section":"Tables 1 and 2"},{"comment":"The counterfactual loss L_cf simplifies to a sum of log(1 + exp((v_cf_i - v_i) · g(w_{y_i})/τ)) after cancellation of the −log exp(v_i · g(w)/τ) terms, which is a softplus of the score difference rather than the stated two-term contrastive objective; please clarify the intended formula and its derivation.","section":"Equation (7)"},{"comment":"The pseudocode does not initialize the accumulators L_basic and L_cf, and the loop body updates these variables inside the for-loop while the total loss is computed after the loop, making the intended optimization procedure ambiguous.","section":"Algorithm 1"},{"comment":"The method name is spelled inconsistently as both 'DiCap' and 'Dicap'; please standardize the spelling.","section":"Throughout"},{"comment":"The notation 'do(y := y_cf)' is non-standard; in Pearl's do-calculus the operation should be written do(Y = y_cf), and the label variable Y should be defined consistently with Section 3.2.","section":"Corollary 3.3"},{"comment":"Reference [65] is cited as the source of the data generation process in Figure 3, but the cited paper is about contrastive learning inverting the data generating process; please verify the citation and provide the appropriate source for the graph structure.","section":"Reference [65]"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft: the ACM template still contains placeholder conference metadata, and some tables and equations are poorly formatted. More importantly, the theoretical guarantee advertised in the abstract is contradicted by the assumptions of Theorem 3.2 itself, and no proofs are supplied for the stated theorem and corollaries. Because this theoretical claim is the paper's main novelty, the inconsistency cannot be fixed by a local edit. If the authors were to resubmit an empirical-only version that drops the identifiability and error-bound claims, the positive results across three tasks might be worth evaluating, but the current framing makes the manuscript unsuitable for publication in its present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the empirical recipe is worth a look, but the theoretical guarantees do not survive contact with the paper's own model. Condition (3) of Theorem 3.2 cannot hold alongside condition (2) and Eq. (4). If g is invertible, I(g(x); y,n) = I(x; y,n); so g(x)⊥(y,n) forces x⊥(y,n). But Eq. (4) says y and n cause x. The theorem's assumptions are only satisfiable in a degenerate model where y and n have no effect on x, making the identifiability conclusion vacuous and Corollary 3.4's error bound unsupported. The paper also states Theorem 3.2 without a proof—just a claim of 'rigorous mathematical reasoning'—so the abstract's guarantee is overclaimed.\n\nWhat is actually new: using classifier-guided diffusion counterfactuals as hard negatives in contrastive prompt learning, with the second-closest predicted class as the counterfactual label. That is a clean idea, and the experiments cover image classification, retrieval, and VQA with consistent gains on unseen classes. The baselines are standard (CoOp, CoCoOp, CPL), and the comparison looks honest.\n\nThe soft spots beyond the theory: tables report averages over five runs but no error bars; no code release is mentioned; the 'closest as counterfactual' ablation only compares to random sampling, so the benefit of similarity-based selection is suggestive but not deeply analyzed. The diffusion guidance largely follows Sanchez and Tsaftaris [51], which is acknowledged.\n\nMy take: strip the broken theory and this is a reasonable empirical contribution. A serious referee should demand either a correct proof of Theorem 3.2 (likely impossible under the current conditions) or a revised text that removes the unsupported identifiability and error-bound claims. The method itself deserves peer review—the idea of training prompts against diffusion-generated counterfactuals is worth testing—but the paper needs substantial revision before acceptance. Send it to review, and tell the authors the theory section is the problem.","headline":"A useful empirical recipe for contrastive prompt learning with diffusion counterfactuals, but the paper's advertised identifiability guarantee is built on a theorem whose assumptions contradict its own data-generating model.","tokens_in":16301,"tokens_out":4516,"would_cite":true,"duration_ms":43047,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that diffusion-generated counterfactual images, used as hard negatives in contrastive prompt learning, align prompts with causal image features and improve accuracy on unseen classes.","keywords":["prompt learning","counterfactual generation","diffusion models","causal invariance","contrastive learning","vision-language models","identifiability","error bounds"],"falsifier":"On a synthetic dataset with known structural equations, compute the true counterfactual $f(y_{cf},\\boldsymbol{n},\\boldsymbol{u}_x)$ for each sample, run DiCap, and compare the observed counterfactual error to the reconstruction error; if the observed error exceeds the reconstruction error for any sample, Corollary 3.4's bound is false.","tokens_in":15359,"feed_emoji":"🎯","tokens_out":7958,"duration_ms":71280,"temperature":0.7,"pith_summary":"Prompt learning for vision-language models usually tunes text prompts while keeping encoders frozen, but such prompts can latch onto spurious correlations and fail on unseen classes. This paper proposes DiCap, a pipeline that generates counterfactual images with a classifier-guided diffusion model steered by an anti-causal predictor, then trains prompts against those images as hard negatives. The central claim is that these counterfactuals satisfy a minimal-sufficiency criterion, so only label-relevant features change, and that the counterfactual estimation error is bounded by the reconstruction error of the diffusion process. Under stated invertibility and independence conditions the counterfactual outcome is identifiable, and the paper reports consistent gains over CLIP and prior causal prompt-learning baselines, including an average 3.87 percentage-point improvement on unseen image-classification classes.","feed_headline":"Diffusion counterfactuals steer prompts to causal features","feed_subtitle":"Generated counterfactuals as hard negatives lift unseen-class accuracy across three vision tasks.","key_machinery":"The load-bearing mechanism is the pairing of an invertible encoding $g(\\boldsymbol{x})$ with an anti-causal predictor gradient that drives diffusion sampling. The paper models images as $\\boldsymbol{x}=f(y,\\boldsymbol{n},\\boldsymbol{u}_x)$, with label $y$ causing causal features and $\\boldsymbol{n}$ causing non-causal ones; Theorem 3.2 states conditions, including $g(\\boldsymbol{x})\\perp y,\\boldsymbol{n}$, under which the encoding is recoverable, so that replacing $y$ by the counterfactual label $y_{cf}$ in the decoder produces the true counterfactual. In practice this is implemented by a DDIM-style forward pass that produces structured noise as abduction, gradient guidance from the anti-causal predictor as action, and a dual contrastive loss with the counterfactual as hardest negative as estimation. This mechanism is what turns a generic diffusion model into a counterfactual generator with a stated error bound.","core_discovery":"On the paper's own terms, the discovery is that a classifier-guided diffusion model, when guided by the gradient of an anti-causal predictor $\\nabla_{\\boldsymbol{x}}p_{\\phi}(y_{cf}\\mid \\boldsymbol{x})$, generates counterfactual images that are minimally sufficient: the smallest perturbation that changes the label while preserving non-causal factors. Theorem 3.2 gives invertibility and independence conditions under which the recovered latent is identifiable, and Corollary 3.4 states that if the reconstruction error is at most $\\delta$, the counterfactual estimation error is at most $\\delta$. Training the tunable prompt with a dual contrastive loss, using the counterfactual as the hardest negative, then aligns the prompt embedding with causal features and yields the reported generalization gains on unseen classes.","pith_inferences":["A sharper test of the causal mechanism would hold the prompt learner fixed and vary only the anti-causal predictor's training data; if a predictor trained purely on non-causal features still improves prompts, the benefit is not exclusively causal.","The error-bound argument suggests a cheap diagnostic: on any dataset with a known generative model, compute reconstruction error and counterfactual error, and if the ratio exceeds one, one of Theorem 3.2's independence or invertibility conditions is violated.","The same pipeline could be extended to multimodal prompts by feeding both image and text into the diffusion guidance, which the authors note as future work but do not develop.","If the minimal-sufficiency claim transfers to medical imaging, counterfactual explanations of this kind could reveal which visual features drive a diagnosis, but the paper's own caution about responsible deployment applies."],"forward_implications":["Reconstruction error on factual images can serve as a certificate for counterfactual quality, removing the need for ground-truth counterfactual labels during training.","Seen-class-only training in this framework yields improved unseen-class accuracy, reducing the annotation burden for new categories.","The same counterfactual-hardest contrastive objective transfers across image classification, image-text retrieval, and visual question answering from a single prompt learner.","Automatically choosing the closest alternative class as the counterfactual label produces harder negatives and better prompts than random class selection.","Shorter prompts combined with counterfactual negatives behave as a regularizer, avoiding spurious task-specific cues."],"supporting_citations":[{"why":"Supplies the classifier-guided diffusion recipe for counterfactual generation that DiCap generalizes by explicitly modeling non-causal factors.","marker":"[51]"},{"why":"Established classifier-guided diffusion sampling, the gradient-guidance mechanism the paper adapts for anti-causal prediction.","marker":"[9]"},{"why":"Provides the denoising diffusion probabilistic model formulation whose reverse process underlies the counterfactual estimation step.","marker":"[16]"},{"why":"Gives the DDIM ordinary-differential-equation view used for the abduction step and structured noise generation.","marker":"[54]"},{"why":"Defines the CLIP contrastive pretraining objective and frozen encoders that the prompt learner builds on.","marker":"[48]"},{"why":"Defines the conditional prompt-learning baseline that DiCap extends and compares against.","marker":"[63]"},{"why":"Provides the prior causal counterfactual prompt-learning method CPL, the main baseline DiCap claims to outperform.","marker":"[15]"},{"why":"Motivates the choice of the closest alternative class as the hardest negative sample in contrastive learning.","marker":"[50]"},{"why":"Establishes the anti-causal direction of learning, justifying the use of label-to-image gradients in sampling.","marker":"[52]"}],"fun_headline_variants":["Diffusion counterfactuals make prompts causally invariant","DiCap diffusion counterfactuals for causal prompt learning","Minimally sufficient counterfactuals improve prompt generalization","Causal prompt learning via diffusion-generated counterfactuals","Diffusion counterfactuals steer prompts to invariant features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument's load-bearing assumption is that the latent code a pretrained denoiser recovers from an image is statistically independent of the image's label and its non-causal background factors, and the paper does not demonstrate that any real pretrained model satisfies this condition.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion counterfactuals make prompts causally invariant","DiCap diffusion counterfactuals for causal prompt learning","Minimally sufficient counterfactuals improve prompt generalization","Causal prompt learning via diffusion-generated counterfactuals","Diffusion counterfactuals steer prompts to invariant features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000541,"raw_usage":{"total_tokens":2568,"prompt_tokens":897,"completion_tokens":1671,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":1595}},"tokens_in":513,"tokens_out":1671,"duration_ms":12642,"temperature":1.0,"reasoning_tokens":1595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:51:16.302669+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a synthetic dataset with known structural equations, compute the true counterfactual $f(y_{cf},\\boldsymbol{n},\\boldsymbol{u}_x)$ for each sample, run DiCap, and compare the observed counterfactual error to the reconstruction error; if the observed error exceeds the reconstruction error for any sample, Corollary 3.4's bound is false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the classifier-guided diffusion recipe for counterfactual generation that DiCap generalizes by explicitly modeling non-causal factors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prior causal counterfactual prompt-learning method CPL, the main baseline DiCap claims to outperform."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the anti-causal direction of learning, justifying the use of label-to-image gradients in sampling."}],"review_version":1}