Pith. sign in

REVIEW 4 major objections 6 minor 63 references

Video classifiers can be explained with counterfactual videos whose only changed part is the deciding motion cue.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:17 UTC pith:7MTXZLPR

load-bearing objection Novel video CFE framework via I2V diffusion latent optimization, but the minimal-edit claim is not enforced by the objective. the 4 major comments →

arxiv 2511.20295 v2 pith:7MTXZLPR submitted 2025-11-25 cs.CV

Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanations

Chao Wang , Chengan Che , Xinyue Chen , Sophia Tsoka , Luis C. Garcia-Peraza-Herrera This is my paper
classification cs.CV
keywords counterfactual explanationsvideo classificationdiffusion modelsimage-to-video generationlatent optimizationspatiotemporal featuresmodel interpretabilityspurious features
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper's goal is to bring counterfactual explanations—minimal, meaningful changes to an input that flip a model's verdict—from images to video. It claims that a video classifier can be explained by generating a near-identical video, starting from the same first frame, in which only the motion or facial/body cues the classifier actually uses are altered, so the prediction moves to a chosen target class. Image-based explanation methods, the authors argue, cannot do this because they operate in the late, texture-forming stages of diffusion and leave motion untouched. The proposed framework searches the initial latent of a first-frame-conditioned image-to-video diffusion model, guided only by the classifier's own loss, and the paper shows the resulting videos satisfy validity, proximity, actionability, realism, and spatiotemporal consistency while also surfacing a spurious dodge cue in a robust action classifier.

Core claim

The central claim is that a useful video counterfactual can be produced by optimizing a single initial noise latent in a deterministic image-to-video diffusion generator. The optimization has two stages: an inversion stage tunes the latent so the generated video reconstructs the original input and anchors the search nearby; a generation stage then minimizes the target classifier's cross-entropy with the target class together with a translation-invariant style loss. The paper argues this yields minimal, semantically meaningful edits that preserve identity and scene while changing dynamic features, and it validates the claim on synthetic motion, facial emotion, and human action datasets, inclu

What carries the argument

The load-bearing object is the initial latent noise of an image-to-video (I2V) latent diffusion model conditioned on the original video's first frame. Because the generator is deterministic given that noise, optimizing it by backpropagating gradients from the target classifier and a Gram-matrix style loss moves the decoded video toward the target class while keeping it near the input's appearance; the style loss is translation-invariant, so it preserves global style without blocking the motion edits that must change. The two-stage scheme (inversion, then counterfactual generation) and progressive increase of denoising steps carry the search from an anchor near the original video to a nearby

Load-bearing premise

The load-bearing premise is that the style regularizer (the Gram-matrix term in the method's objective) keeps generated videos close to the original video's appearance without suppressing the motion changes needed to flip the classifier; this is tested in only one ablation with one hand-picked weight, and the paper's own limitations section restricts the claim to short clips and domain-matched generators.

What would settle it

Sweep the style-loss weight across several orders of magnitude on the synthetic motion dataset: if at every weight that preserves visual realism the flip rate drops or the edited direction no longer tracks the target class, the regularizer assumption fails. Alternatively, if re-running the same optimization with different random seeds yields valid flips but inconsistent semantic edits, the minimal-and-meaningful reading is underdetermined.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Video classifiers can be audited by asking why class A rather than class B and inspecting which movements the generated video changes.
  • Counterfactual generation doubles as a debugging tool: the paper's dodge-as-kicking case shows it can expose spurious cues in a robust action classifier.
  • Naively adapting image-based classifier-guidance to videos is not enough; frame-level and mid-noise variants fail to keep temporal consistency and realism.
  • Because guidance comes only from the target classifier and the original video, the explanations reflect the classifier's own decision logic rather than text-prompt priors.
  • The authors' own scope limits—about four-second clips, roughly two hours per explanation, and the need for a domain-matched generator—define where the claim is currently testable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The translation-invariant style regularizer may under-explain classifiers that rely on absolute position or composition, since those cues are precisely what the regularizer permits to change; this is a testable boundary of the claim.
  • The need to fine-tune a separate generator per domain means explanation availability depends on generator coverage; a failure to produce a plausible counterfactual could reflect generator limits rather than classifier behavior.
  • The reported metric gap—noise-based attacks score well on SSIM/LPIPS—suggests that a semantic, human-aligned spatiotemporal metric is required before minimality can be compared across methods.
  • If the latent-search idea generalizes, the same mechanism could probe video models for protected-attribute cues by targeting different counterfactual labels.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes BTTF, an optimization framework for generating video counterfactual explanations (CFEs). It uses an image-to-video (I2V) latent diffusion model (Wan-I2V) as the generator and optimizes the initial latent z_T in two stages: first, an inversion stage minimizes an L1 reconstruction loss to anchor the latent near the encoded input; second, a CFE stage minimizes the target classifier's cross-entropy toward a target class plus a style regularizer based on per-frame RGB Gram matrices. A progressive schedule increases the number of denoising steps from 1 to 15 to mitigate gradient vanishing. The method is evaluated on three target classifiers (motion, emotion, action) and compared with PGD attacks and adapted classifier-guidance baselines. The paper claims the generated videos are valid, minimal, realistic, and spatiotemporally consistent, and reports a case study identifying a spurious 'dodge' feature in an NTU action classifier.

Significance. If the central claims hold, BTTF would be a meaningful first step in an underexplored area: it demonstrates that I2V diffusion latents can be optimized with classifier gradients to produce realistic videos that flip video classifiers, and the spurious-feature discovery in Fig. 8 is a compelling application. The paper is transparent about hyperparameters and provides algorithm pseudocode; it also honestly acknowledges the lack of standard metrics. However, the paper does not currently prove the 'minimality' part of its CFE definition, and the evaluation is largely qualitative. Because the contribution is timely and the deficiencies are addressable, I view this as a revision rather than a rejection.

major comments (4)
  1. [Eq. (2), Algorithm 1, Table S1] Eq. (2), Algorithm 1, Table S1: L_C has no proximity term. The inversion loss (Eq. 1) only initializes z_T; during K_C=100 Stage-2 steps z_T is updated by L_C alone and can drift. For Shape-Moving, Table S1 sets K_I=0, so the pure-motion demo (Fig. 3) uses no inversion. L_S is a per-frame RGB Gram difference, constraining color covariance, not motion or semantics. Thus 'minimal, localized edits' (Sec. 1 criterion ii; Sec. 4.2) are not enforced. As Sec. 4.4 notes, SSIM/LPIPS can be high for meaningless noise. Add a proximity term or quantitative minimality evidence.
  2. [Sec. 4.2-4.5, Table 2] The evidence for 'concrete insights into the classifier's decision-making mechanism' is mostly qualitative (Figs. 3-8). Table 2 reports no error bars, sample sizes, or statistical tests; the only method comparison in the main text is PGD, which the paper itself shows is not explanatory. The supplementary CG baselines are a useful start, but no human evaluation of usefulness or minimality is provided. I recognize the lack of standard metrics (Sec. 5), but the strength of the claims exceeds the evidence. Please add variance over videos/initializations and at least a small user study, or restrict conclusions to proof-of-concept.
  3. [Fig. 6, Table S1] The ablations for inversion and style loss are based on a single hand-picked example each, with no quantitative measure of proximity or realism beyond reported SSIM/LPIPS values. The style coefficient λ=1e5 and the progressive schedule N=15 are fixed without sensitivity analysis. Since L_S is intended to keep the output on the input manifold, a sweep of λ and a quantitative comparison over multiple videos is needed to support these design choices.
  4. [Abstract and Sec. 3.2] The statement that 'both optimization processes are guided solely by the target classifier' is inaccurate; Stage 1 uses Eq. (1) and Stage 2 includes L_S. More importantly, the generator is LoRA-fine-tuned on the same training distribution as the target classifier (Sec. 4.1), so the generated edits are shaped by the generator's prior as well as by classifier gradients. The paper should qualify the 'faithful' claim or provide evidence that the edits are attributable to the classifier (e.g., compare with a generator fine-tuned on a disjoint distribution).
minor comments (6)
  1. [Abstract / Sec. 3.2] The statement that 'both optimization processes are guided solely by the target classifier' is contradicted by the use of reconstruction loss Eq. (1) and style loss Eq. (2). Please rephrase.
  2. [Fig. 2] The caption calls the target classifier a 'black box', but Algorithm 1 backpropagates through it to update z_T. Clarify whether white-box gradient access is required.
  3. [References] Reference [15] appears to be a docking/scoring paper in medicinal chemistry, not the diffusion GLIDE paper; the citation seems to be a placeholder error.
  4. [Fig. 7 caption] Typo: 'perutrba-tions' should be 'perturbations'.
  5. [Tables 2 and S3] Report standard deviations and the number of videos used; currently only means are given.
  6. [Sec. 4.1] Hyperparameters λ=1e5, N=15, K_I/K_C are given without sensitivity analysis; at least a coarse sweep would help.

Circularity Check

0 steps flagged

No significant circularity: the counterfactual validity is an optimization target, and the proximity/realism concerns are empirical limitations rather than definitional reductions.

full rationale

The paper's central derivation is an optimization: BTTF directly optimizes the initial latent z_T using the classifier's cross-entropy loss toward a target class (Eq. 2), so 'validity' is the objective being optimized rather than an independent prediction. This is the standard definition of a counterfactual, not a circular reduction. The claimed 'proximity' is supported by the inversion stage (Eq. 1), but the inversion loss appears only in Stage 1 and is absent from the Stage 2 objective; the final proximity is therefore an empirical property (tested qualitatively in Fig. 6), not an identity forced by construction. This is a possible correctness/robustness gap, not a circularity. The diffusion model is fine-tuned on same-domain data, but that is an external generative prior and does not reduce to the classifier's outputs; the 'guided solely by the target classifier' wording is an overstatement, but again not circular. No load-bearing self-citations or imported uniqueness theorems appear in the manuscript; all cited methods are external. The paper itself acknowledges that SSIM/LPIPS are inadequate (Secs. 4.4 and 5), which is a limitation of the evaluation, not evidence of circular reasoning. Overall, I find no step where a claimed result is equivalent to its input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method depends on several hand-chosen hyperparameters and domain assumptions about the diffusion model's latent space. No new physical entities are introduced.

free parameters (4)
  • Style loss coefficient λ = 1e5
    Hand-chosen regularization weight balancing classifier loss and style preservation in Eq. (2).
  • Max denoising steps N = 15
    Chosen to control computation and convergence; progressive schedule from 1 to N.
  • Inversion iterations KI = 0 for M-swin, 40 for E-swin and A-swinR
    Per-dataset choices; skipping inversion for the synthetic dataset suggests it is not always required.
  • CFE generation iterations KC = 100
    Fixed number of optimization steps for all datasets.
axioms (5)
  • domain assumption The I2V diffusion model Wan-I2V produces a deterministic mapping from initial noise latent z_T and first frame to a spatiotemporally coherent video.
    Relies on the property of Wan-I2V; no guarantee that the latent space is well-behaved for optimization.
  • domain assumption Optimizing z_T in the VAE latent space yields semantically meaningful, smooth video edits.
    Central to the method; if the latent space is not semantically structured, edits become meaningless noise.
  • domain assumption The target classifier's gradients backpropagated through the diffusion model are informative for generating valid counterfactuals.
    The method assumes classifier guidance works for video diffusion as it does for image diffusion.
  • domain assumption Fine-tuning the diffusion model on the domain of the target classifier is necessary and sufficient for preserving realism.
    They claim this adaptation is required; no theoretical justification.
  • domain assumption The style loss (Gram matrix difference) preserves the input video manifold without preventing motion editing.
    Demonstrated only in a single qualitative ablation; not theoretically grounded.

pith-pipeline@v1.3.0-alltime-deepseek · 141 in / 7476 out tokens · 148262 ms · 2026-08-03T20:17:41.320662+00:00 · methodology

0 comments
read the original abstract

Counterfactual explanations (CFEs) are minimal and semantically meaningful modifications of the input of a model that alter the model predictions. They highlight the decisive features the model relies on, providing contrastive interpretations for classifiers. State-of-the-art visual counterfactual explanation methods have primarily focused on interpreting image classifiers, leaving the domain of video models relatively underexplored. For the video CFEs to be useful, they have to be physically plausible, temporally coherent, and exhibit smooth motion trajectories. Existing CFE image-based methods, designed to explain image classifiers, lack the capacity to generate temporally coherent, smooth and physically plausible video CFEs. To address this, we propose Back To The Feature (BTTF), an optimization framework that generates video CFEs. Our method introduces two novel features, 1) an optimization scheme to retrieve the initial latent noise conditioned by the first frame of the input video, 2) a two-stage optimization strategy to enable the search for counterfactual videos in the vicinity of the input video. Both optimization processes are guided solely by the target classifier, ensuring the explanation is faithful. To accelerate convergence, we also introduce a progressive optimization strategy that incrementally increases the number of denoising steps. Extensive experiments on video datasets such as Shape-Moving (motion classification), MEAD (emotion classification), and NTU RGB+D (action classification) show that our BTTF effectively generates valid, visually similar and realistic counterfactual videos that provide concrete insights into the classifier's decision-making mechanism.

Figures

Figures reproduced from arXiv: 2511.20295 by Chao Wang, Chengan Che, Luis C. Garcia-Peraza-Herrera, Sophia Tsoka, Xinyue Chen.

Figure 1
Figure 1. Figure 1: Video counterfactual explanations with BTTF. The top row shows an input video, where the facial expression is predicted as “Angry” with 98% confidence by the target video classifier E-swin. To answer “why angry not sad?”, our method BTTF (middle row) introduces minimal and semantically mean￾ingful changes to the input video, resulting in the alteration of the model’s prediction to “Sad”. Similarly, to answ… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the BTTF optimization framework for video CFEs. In the first stage for inversion, the initial latent input zT , which is initially sampled from the Gaussian distribution, is optimized by the backpropagated gradients from the reconstruction loss LI between the noise-free latent zˆ0 and the original input video latent zi. In the second stage for CFE generation, zˆ0 is further decoded by the V… view at source ↗
Figure 3
Figure 3. Figure 3: CFE videos generated by BTTF for the target motion classifier M-swin trained on Shape-Moving. BTTF changes M￾swin’s prediction on the original input video from “Up” to target motion classes “Left”, “Down” and “Right”, respectively, demon￾strating that BTTF is capable of precisely editing pure dynamic features (here, movement directions) to produce CFE videos. proaches, UVCE [3] adopts an optimization-guide… view at source ↗
Figure 4
Figure 4. Figure 4: CFE videos generated by BTTF for the target emotion classifier E-swin trained on MEAD. BTTF alters E-swin’s prediction on the original input video from “Neutral” to target motion classes “Fear”, “Contempt” and “Disgust”, respectively. The results indicate the strong capacity of BTTF in editing emotion features in a semantically meaningful way. latent zˆ0, which constitutes the prediction for the following … view at source ↗
Figure 5
Figure 5. Figure 5: Video CFE generated by BTTF for the target action classifier A-swinR trained on NTU RGB+D. BTTF flips A-swinR’s prediction on the original input video from “Hand waving” to target action classes “Taking a selfie”, “Pointing” and “Staggering”, respec￾tively. The results demonstrate the edits of BTTF for human actions are physically plausible. NTU RGB+D, we train the model using PGD adversarial robust traini… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study. Without inversion, CFE video contains some unnecessary changes (stepping rightward in the bottom left row), validating the effectiveness of inversion in enhancing the proximity of CFE videos. Without style loss, CFE video suffers severe quality degradation, validating the effectiveness of style loss regularization in maintaining the realism of CFE videos. inal video. Video style regularizat… view at source ↗
Figure 8
Figure 8. Figure 8: Spurious features detection by BTTF. The first row is original videos, while the second row is generated CFE videos by BTTF with the target class “kicking”. Inside CFE videos, there are no actual kicking actions but someone moving backward (red boxes). That is because the target classifier A-swinR mistakenly recognizes the dodge movement of the other person as “kicking”, indicating the great potential of B… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 11 linked inside Pith

  1. [1]

    Cosmos world foun- dation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foun- dation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025. 4

    Pith/arXiv arXiv 2025

  2. [2]

    Diffusion visual counterfactual explana- tions.Advances in Neural Information Processing Systems, 35:364–377, 2022

    Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explana- tions.Advances in Neural Information Processing Systems, 35:364–377, 2022. 2, 3, 4

    2022

  3. [3]

    Dig-in: Diffusion guidance for investigating networks- uncovering classifier differences neuron visualisations and visual counterfactual explanations

    Maximilian Augustin, Yannic Neuhaus, and Matthias Hein. Dig-in: Diffusion guidance for investigating networks- uncovering classifier differences neuron visualisations and visual counterfactual explanations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11093–11103, 2024. 3, 4

    2024

  4. [4]

    Sparse visual counterfac- tual explanations in image space

    Valentyn Boreiko, Maximilian Augustin, Francesco Croce, Philipp Berens, and Matthias Hein. Sparse visual counterfac- tual explanations in image space. InDAGM German Confer- ence on Pattern Recognition, pages 133–148. Springer, 2022. 2, 3

    2022

  5. [5]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. Inpro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017. 3

    2017

  6. [6]

    A frank-wolfe framework for efficient and effective adver- sarial attacks

    Jinghui Chen, Dongruo Zhou, Jinfeng Yi, and Quanquan Gu. A frank-wolfe framework for efficient and effective adver- sarial attacks. InProceedings of the AAAI conference on artificial intelligence, pages 3486–3494, 2020. 3

    2020

  7. [7]

    Learning temporal coherence via self- supervision for gan-based video generation.ACM Transac- tions on Graphics (TOG), 39(4):75–1, 2020

    Mengyu Chu, You Xie, Jonas Mayer, Laura Leal-Taixé, and Nils Thuerey. Learning temporal coherence via self- supervision for gan-based video generation.ACM Transac- tions on Graphics (TOG), 39(4):75–1, 2020. 2

    2020

  8. [8]

    Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 3, 1

    2021

  9. [9]

    Relative State

    Hugh Everett. "Relative State" Formulation of Quantum Me- chanics.Reviews of Modern Physics, 29(3):454–462, 1957. 2

    1957

  10. [10]

    Robust physical-world attacks on deep learning visual classification

    Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 1625–1634, 2018. 1

    2018

  11. [11]

    Latent diffusion counterfactual explanations.arXiv preprint arXiv:2310.06668, 2023

    Karim Farid, Simon Schrodi, Max Argus, and Thomas Brox. Latent diffusion counterfactual explanations.arXiv preprint arXiv:2310.06668, 2023. 2, 3, 4

    Pith/arXiv arXiv 2023

  12. [12]

    Tex- ture synthesis using convolutional neural networks.Ad- vances in neural information processing systems, 28, 2015

    Leon Gatys, Alexander S Ecker, and Matthias Bethge. Tex- ture synthesis using convolutional neural networks.Ad- vances in neural information processing systems, 28, 2015. 5

    2015

  13. [13]

    A neural algorithm of artistic style.arXiv preprint arXiv:1508.06576, 2015

    Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style.arXiv preprint arXiv:1508.06576, 2015. 5

    Pith/arXiv arXiv 2015

  14. [14]

    Counterfactual explanations and how to find them: literature review and benchmarking.Data Mining and Knowledge Discovery, 38(5):2770–2824, 2024

    Riccardo Guidotti. Counterfactual explanations and how to find them: literature review and benchmarking.Data Mining and Knowledge Discovery, 38(5):2770–2824, 2024. 1, 2

    2024

  15. [15]

    Glide: a new approach for rapid, accurate dock- ing and scoring

    Thomas A Halgren, Robert B Murphy, Richard A Friesner, Hege S Beard, Leah L Frye, W Thomas Pollard, and Jay L Banks. Glide: a new approach for rapid, accurate dock- ing and scoring. 2. enrichment factors in database screening. Journal of medicinal chemistry, 47(7):1750–1759, 2004. 3

    2004

  16. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  17. [17]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems, 30, 2017. 3

    2017

  18. [18]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 4, 1

    2020

  19. [19]

    An introduction to flow matching and diffusion models.arXiv preprint arXiv:2506.02070, 2025

    Peter Holderrieth and Ezra Erives. An introduction to flow matching and diffusion models.arXiv preprint arXiv:2506.02070, 2025. 4

    arXiv 2025

  20. [20]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022. 6

    2022

  21. [21]

    Steex: steering counter- factual explanations with semantics

    Paul Jacob, Éloi Zablocki, Hedi Ben-Younes, Mickaël Chen, Patrick Pérez, and Matthieu Cord. Steex: steering counter- factual explanations with semantics. InEuropean Confer- ence on Computer Vision, pages 387–403. Springer, 2022. 3

    2022

  22. [22]

    Diffu- sion models for counterfactual explanations

    Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Diffu- sion models for counterfactual explanations. InProceedings of the Asian conference on computer vision, pages 858–876,

  23. [23]

    Ad- versarial counterfactual visual explanations

    Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Ad- versarial counterfactual visual explanations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 16425–16435, 2023. 3

    2023

  24. [24]

    Text- to-image models for counterfactual explanations: a black- box approach

    Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Text- to-image models for counterfactual explanations: a black- box approach. InProceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision, pages 4757– 4767, 2024. 2

    2024

  25. [25]

    3d convolu- tional neural networks for human action recognition.IEEE transactions on pattern analysis and machine intelligence, 35(1):221–231, 2012

    Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 3d convolu- tional neural networks for human action recognition.IEEE transactions on pattern analysis and machine intelligence, 35(1):221–231, 2012. 2

    2012

  26. [26]

    Multimodal explanations by predicting coun- terfactuality in videos

    Atsushi Kanehira, Kentaro Takemoto, Sho Inayoshi, and Tat- suya Harada. Multimodal explanations by predicting coun- terfactuality in videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8594–8602, 2019. 4

    2019

  27. [27]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15954–15964, 2023. 3

    2023

  28. [28]

    Cycle-consistent counter- factuals by latent transformations

    Saeed Khorram and Li Fuxin. Cycle-consistent counter- factuals by latent transformations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10203–10212, 2022. 3

    2022

  29. [29]

    From identifiable causal representations to control- lable counterfactual generation: A survey on causal genera- tive modeling.arXiv preprint arXiv:2310.11011, 2023

    Aneesh Komanduri, Xintao Wu, Yongkai Wu, and Feng Chen. From identifiable causal representations to control- lable counterfactual generation: A survey on causal genera- tive modeling.arXiv preprint arXiv:2310.11011, 2023. 2

    Pith/arXiv arXiv 2023

  30. [30]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. 4

    Pith/arXiv arXiv 2024

  31. [31]

    Imagenet classification with deep convolutional neural net- works.Advances in neural information processing systems, 25, 2012

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works.Advances in neural information processing systems, 25, 2012. 3

    2012

  32. [32]

    On space-time interest points.International journal of computer vision, 64(2):107–123, 2005

    Ivan Laptev. On space-time interest points.International journal of computer vision, 64(2):107–123, 2005. 2

    2005

  33. [33]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 4

    Pith/arXiv arXiv 2022

  34. [34]

    Structure matters: Tackling the semantic discrepancy in diffusion models for image inpainting

    Haipeng Liu, Yang Wang, Biao Qian, Meng Wang, and Yong Rui. Structure matters: Tackling the semantic discrepancy in diffusion models for image inpainting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8038–8047, 2024. 3

    2024

  35. [35]

    Video swin transformer

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202–3211, 2022. 5, 2

    2022

  36. [36]

    Zero-shot model diagnosis

    Jinqi Luo, Zhaoning Wang, Chen Henry Wu, Dong Huang, and Fernando De la Torre. Zero-shot model diagnosis. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 11631–11640, 2023. 3

    2023

  37. [37]

    Towards deep learn- ing models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017. 3, 6, 7

    Pith/arXiv arXiv 2017

  38. [38]

    Understanding the latent space of diffusion models through the lens of riemannian geometry

    Yong-Hyun Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Understanding the latent space of diffusion models through the lens of riemannian geometry. Advances in Neural Information Processing Systems, 36: 24129–24142, 2023. 3

    2023

  39. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 4195–4205,

  40. [40]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 4

    2021

  41. [41]

    D’artagnan: Counterfactual video genera- tion

    Hadrien Reynaud, Athanasios Vlontzos, Mischa Dom- browski, Ciarán Gilligan Lee, Arian Beqiri, Paul Leeson, and Bernhard Kainz. D’artagnan: Counterfactual video genera- tion. InInternational Conference on Medical Image Com- puting and Computer-Assisted Intervention, pages 599–609. Springer, 2022. 2

    2022

  42. [42]

    Beyond trivial counterfactual explanations with diverse valuable explanations

    Pau Rodriguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Charlin, and David Vazquez. Beyond trivial counterfactual explanations with diverse valuable explanations. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1056–1065, 2021. 3

    2021

  43. [43]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 4

    2022

  44. [44]

    Image synthesis with a single (robust) classifier.Advances in Neu- ral Information Processing Systems, 32, 2019

    Shibani Santurkar, Andrew Ilyas, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with a single (robust) classifier.Advances in Neu- ral Information Processing Systems, 32, 2019. 3

    2019

  45. [45]

    Latent Diffusion Counterfactual Explanations.Springer, 15297 LNCS:295–311, 2025

    Simon Schrodi, Karim Farid, Max Argus, and Thomas Brox. Latent Diffusion Counterfactual Explanations.Springer, 15297 LNCS:295–311, 2025. 3

    2025

  46. [46]

    Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis

    Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1010–1019, 2016. 5, 2

    2016

  47. [47]

    Re- thinking visual counterfactual explanations through region constraint

    Bartlomiej Sobieski, Jakub Grzywaczewski, Bartłomiej Sadlej, Matthew Tivnan, and Przemyslaw Biecek. Re- thinking visual counterfactual explanations through region constraint. InThe Thirteenth International Conference on Learning Representations, 2024. 2, 3

    2024

  48. [48]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 1

    Pith/arXiv arXiv 2010

  49. [49]

    Generative modeling by esti- mating gradients of the data distribution.Advances in neural information processing systems, 32, 2019

    Yang Song and Stefano Ermon. Generative modeling by esti- mating gradients of the data distribution.Advances in neural information processing systems, 32, 2019. 4, 1

    2019

  50. [50]

    Improved techniques for training score-based generative models.Advances in neural information processing systems, 33:12438–12448, 2020

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models.Advances in neural information processing systems, 33:12438–12448, 2020. 4, 1

    2020

  51. [51]

    Rethinking the inception archi- tecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 2818–2826, 2016. 3

    2016

  52. [52]

    To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018. 3

    Pith/arXiv arXiv 2018

  53. [53]

    Understanding physical dynamics with counterfactual world modeling

    Rahul Venkatesh, Honglin Chen, Kevin Feigelis, Daniel M Bear, Khaled Jedoui, Klemen Kotar, Felix Binder, Wan- hee Lee, Sherry Liu, Kevin A Smith, et al. Understanding physical dynamics with counterfactual world modeling. In European Conference on Computer Vision, pages 368–387. Springer, 2024. 2

    2024

  54. [54]

    Dickerson, and Keegan E

    Sahil Verma, John P. Dickerson, and Keegan E. Hines. Counterfactual explanations for machine learning: A review. ArXiv, abs/2010.10596, 2020. 2

    Pith/arXiv arXiv 2010

  55. [55]

    Coun- terfactual explanations without opening the black box: Au- tomated decisions and the gdpr.Harv

    Sandra Wachter, Brent Mittelstadt, and Chris Russell. Coun- terfactual explanations without opening the black box: Au- tomated decisions and the gdpr.Harv. JL & Tech., 31:841,

  56. [56]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. 4

    Pith/arXiv arXiv 2025

  57. [57]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation

    Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. Mead: A large-scale audio-visual dataset for emotional talking-face generation. InEuropean conference on com- puter vision, pages 700–717. Springer, 2020. 5, 2

    2020

  58. [58]

    Video- to-video synthesis

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video- to-video synthesis. InProceedings of the 32nd Interna- tional Conference on Neural Information Processing Sys- tems, pages 1152–1164, 2018. 2

    2018

  59. [59]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 2

    2004

  60. [60]

    Fast diffusion-based counterfactuals for shortcut removal and generation

    Nina Weng, Paraskevas Pegios, Eike Petersen, Aasa Feragen, and Siavash Bigdeli. Fast diffusion-based counterfactuals for shortcut removal and generation. InEuropean Conference on Computer Vision, pages 338–357. Springer, 2024. 1, 2, 3

    2024

  61. [61]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7623–7633, 2023. 3

    2023

  62. [62]

    Celebv-text: A large-scale facial text-video dataset

    Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Wei- dong Cai, and Wayne Wu. Celebv-text: A large-scale facial text-video dataset. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 14805–14814, 2023. 6

    2023

  63. [63]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 3

    2018