Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures

Gautam Ranka; Paras Chopra

REVIEW 4 major objections 4 minor 14 references

Denoising models build internal representations of visual illusions that never reach the pixels they generate.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:53 UTC pith:QXV5BPMW

load-bearing objection The internal-encoding half is solid and worth publishing; the 'perceptual phantom' headline overreaches — the injection assay's only positive control fails and the within-manifold test shows a small but nonzero shift. the 4 major comments →

arxiv 2607.17138 v1 pith:QXV5BPMW submitted 2026-07-19 cs.CV

Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures

Gautam Ranka , Paras Chopra This is my paper

classification cs.CV

keywords visual illusionsdenoising diffusion modelsinternal representationsperceptual phantomsmechanistic interpretabilityU-Net bottleneckdiffusion transformerspsychophysical alignment

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Denoising models—networks trained to remove noise and reconstruct natural images—develop internal representations of human visual illusions even though those representations never influence the images they generate. The paper shows this across six denoising models and three architecture families, and demonstrates that the denoising objective, not the architecture, is the main driver. On luminance-based illusion stimuli, the internal activations track a validated psychophysical model of human brightness perception and scale monotonically with illusion strength. The authors call the output-inert representations 'perceptual phantoms' and present them as the first such internal-output dissociation characterized for perceptual representations in generative vision.

Core claim

The paper claims that denoising models encode visual illusions internally at specific layers and channels, independent of architecture. This encoding is causally active in internal processing—ablating illusion-sensitive channels at the bottleneck reduces the downstream internal signal by 43.8%—yet injecting those same representations into the generation pipeline produces no measurable pixel shift in any tested architecture. The authors term this dissociation a 'perceptual phantom': a representation that is involved in internal computation but invisible to any output-based evaluation. They also report that the effect is driven primarily by the denoising objective rather than the network archi

What carries the argument

The central object is the 'perceptual phantom': an internal representation that is detectable and causally active inside the network yet leaves no trace in generated pixels. The argument rests on three measurement tools: (1) layer-resolved activation probing that compares the differential activation between two physically identical but perceptually different image regions; (2) internal-to-internal channel ablation, which zeroes illusion-sensitive channels at one bottleneck layer and measures the effect at another, avoiding the output-level blind spot of standard causal tracing; and (3) phantom-injection tests that patch illusion activations into a clean denoising trajectory and measure pixel

Load-bearing premise

The central claim depends on the assumption that the injection pipeline can detect activation-to-pixel transfer at all, yet the paper's own positive control (a +2 SD brightness offset) also produces no pixel shift, so the phantom null may reflect a blind probe rather than a genuine absence.

What would settle it

Run the same single-step activation-injection pipeline with a control that carries a known physical change (e.g., adding +2 standard deviations of brightness to the target region). The paper's Appendix AI reports this control also produces zero pixel shift; if a sensitive injection method can move pixels for such a control but not for illusion channels, the phantom claim survives; if no control ever moves pixels, the null is uninformative and the phantom conclusion collapses. Alternatively, a psychophysical test asking humans to judge generated images after illusion-channel amplification would

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If true, a model can represent a perceptual property without ever expressing it in behavior, so output-level benchmarks for human-likeness are incomplete.
The denoising objective—not the specific network architecture—is what produces human-like illusion sensitivity; discriminative models with the same attention mechanism show 2.5× weaker effects.
Illusion-sensitive activations track a validated psychophysical model of human brightness perception (Spearman ρ ≥ 0.70) and scale monotonically with parametric illusion strength, suggesting graded internal encoding.
Channel ablation shows these representations are causally involved in internal processing (43.8% signal reduction), localizing the effect to the U-Net bottleneck rather than skip-connection pathways.
The perceptual-phantom property replicates across pixel-space DDPMs, latent diffusion U-Nets, and a diffusion transformer, so it is not a U-Net artifact.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Inference: If perceptual phantoms are general, then evaluating generative models by their outputs alone systematically undercounts what they have learned about perception; internal-probe benchmarks would be needed alongside behavioral ones.
Inference: The finding connects to the broader amnesic-probing phenomenon in language models; the paper's attenuation gradient suggests a graded dissociation rather than a binary one, which might be a common organizational principle in deep networks.
Inference: A testable extension is to use a chromatic psychophysical model (the paper notes FLODOG only covers luminance) to see whether color-illusion activations also track human perception and whether those too are phantoms.
Inference: The mechanistic story—bottleneck localization and DiT self-attention/LayerNorm absorption—predicts that architectural changes that weaken self-correction (e.g., removing LayerNorm or attention) should make illusion representations output-visible; that is a concrete experiment the paper did not run.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The internal-encoding half is solid and worth publishing; the 'perceptual phantom' headline overreaches — the injection assay's only positive control fails and the within-manifold test shows a small but nonzero shift.

read the letter

You should know two things before reading. The descriptive half is real and worth taking seriously: across pixel-space DDPMs, a latent U-Net, and DiT, denoising models develop activation patterns that discriminate illusory from physically matched regions, concentrated near the bottleneck; and the DiT-versus-ViT comparison is the cleanest evidence to date that the denoising objective, not the attention architecture, drives the effect. The other half — the 'perceptual phantom' claim that these representations are active internally but invisible at the output — is not established by the evidence, and the paper's own controls are what sink it.

Credit where it's due. The observational work is disciplined: real-counterpart comparisons, random-image geometry baselines, pixel-shuffle controls, permutation tests over the 246 layer-timestep grid, and an ablation that beats a 500-set random-channel null by 8x and survives leave-one-out cross-validation with expected shrinkage. They even refused to use flipped or synthetic duplicates to inflate N, which would have bought them an 89% bigger ablation number. That is better practice than most of what I see.

Now the soft spots, in size order. The phantom null depends on injection pipelines that cannot be shown to detect anything. Appendix AI is the positive injection control and it fails: a +2 SD brightness offset injected through the single-step pipeline also gives shift_frac ~ 0. The authors read this as confirming the phantom; the correct reading is that the single-step assay transmits nothing, so its null is uninformative. Appendix AK makes it worse: within-manifold zero-ablation of the all_sig channels gives shift_frac +0.011 with CI [+0.001, +0.025] — by their own criterion that is a measurable shift, opposite to the predicted direction, and one random control nearly excludes zero too. Between that and Appendix AJ showing the cross-domain injections are off-manifold (cosine ~0.58), 'no measurable pixel shift across all tested architectures' is overreach. What the MSE read-only asymmetry does support is the weaker, still interesting claim that illusion channels are less output-coupled than the average channel. That should be the headline.

Two smaller issues. The FLODOG rho >= 0.70 and the near-perfect dose-response correlations are selected from the best of many layer-timestep combinations; the per-image rho at the a-priori layer is 0.33 with CI including zero, so those numbers need correction or pre-registration. And the training-versus-architecture claim is softer than advertised given one random DiT seed matches the trained effect — they admit this.

Bottom line: interpretability and perception people should read this, and a serious editor should send it out. The descriptive contribution is publishable; the phantom framing needs either a positive control that actually works in the same pipeline or a reframe to 'reduced output coupling,' and the main text has to confront Appendix AK's contradiction.

Referee Report

4 major / 4 minor

Summary. The paper probes internal activations of denoising and discriminative models on visual illusion stimuli (GVIL and Pyllusion) and reports three main claims: (1) denoising models develop illusion-sensitive representations at specific internal layers across architectures, while discriminative models show weaker effects; (2) the encoding is causally involved in internal processing, as shown by channel ablation; and (3) the encoding is 'output-inert' — injecting these representations into the generation pipeline produces no measurable pixel shift, a phenomenon the authors term 'perceptual phantoms.' The evidence includes layer-resolved Cohen's d maps, real-counterpart and random-image controls, permutation tests, bootstrap CIs, FLODOG psychophysical correlations, dose-response curves, channel-ablation with cross-validation, and injection null results across four architectures.

Significance. If the phantom claim held, this would be a notable contribution: the first demonstration of an internal-output dissociation for perceptual representations in generative vision models, with cross-architecture replication and a proposed attenuation gradient. The paper is also commendable for including multiple controls (real-counterpart, random-pixel baseline, permutation tests, random-initialization), reporting bootstrap CIs at small N, performing cross-validation for the ablation analysis, and releasing code. However, the central novelty — the 'perceptual phantom' — rests on a null injection result whose only positive control fails, and several headline effect sizes are selected post hoc over layers/timesteps/metrics. These issues are fixable but currently undermine the strongest claim and temper the architecture-generalization conclusion.

major comments (4)

[§3.5, Appendix AI, Appendix AK] The phantom claim is underdetermined by a failed positive control. Appendix AI reports that injecting a +2 SD brightness offset into the illusion region via the single-step DDIM pipeline also yields shift_frac ≈ 0. This means the assay cannot detect even a large, pixel-relevant perturbation; the null for illusion activations is therefore equally consistent with an insensitive measurement pipeline as with a true 'phantom.' The within-manifold ablation (Appendix AK) does not supply the missing control: the all_sig zero-ablation gives a small CI-excluding-zero positive shift (opposite to the phantom direction), and one random control also nearly excludes zero. The MSE read-only comparison is a different intervention (ablation) and does not calibrate the injection protocol. Please provide a positive control on the exact injection protocol — e.g., a feature known to transfer to pixels — or re
[§4.1, §4.4, Appendix AO, Appendix L] The headline 'peak illusion sensitivity' comparisons (Fig. 6A) use maxima over layers and timesteps selected per model, while the a-priori test statistics do not survive the paper's own multiple-hypothesis correction. At mid_attn_0, t=50, the raw permutation p=0.028 (Appendix AO) would not survive FDR across 246 combinations, and Appendix L states that the a-priori layers do not survive FDR at α=0.05 for the FLODOG correlations; the overall permutation p<0.0001 is driven by t=900, a timestep the paper excludes as confounded (Appendix B). The claim that the denoising objective, not architecture, drives the effect thus rests on unadjusted maxima. Report FDR-adjusted p-values for each headline and show the full distribution over layers/timesteps, not just the selected peak.
[§4.2, Figure 3, Appendices U, L] The abstract states that activations 'track a validated psychophysical model (Spearman ρ ≥ 0.70),' but these values come from post-hoc selected layers and timesteps on programmatic stimuli. At the a-priori mid_attn_0, t=50, the per-image FLODOG correlation is ρ=0.331 with a CI that includes zero (Appendix U), and the aggregate GVIL correlation is near zero. Because many layer×timestep combinations were presumably searched, the reported ρ=0.78 and 0.70 need a selection correction or a pre-specified layer/timestep. Please report how many combinations were tested and the FDR-adjusted significance; otherwise the strong psychophysical alignment claim is not yet supported.
[§4.4, Figure 6C, Appendix AD] The DiT-XL/2 vs ViT-B/16 comparison is presented as a clean test of training objective ('same attention mechanism'), but the models differ in patch size (2 vs 16), parameter count (675M vs 86M), conditioning (class-conditioned with adaLN vs plain ViT), and training data details. Appendix AD further shows that a randomly initialized DiT seed (seed 0, |d|=0.741) matches the trained model's peak (0.711), so the architecture-vs-objective decomposition is not clean. Either control for these factors (e.g., a classification-trained DiT) or soften the 'denoising objective more important than architecture' claim to reflect the confounds.

minor comments (4)

[§3.5, Appendix R] The shift_frac equation contains a typo ('shif t_f rac') and the notation ∆_X, ∆_Xr, ϵ is not defined consistently with the main text. Please define all terms and use a single notation for the illusion-region differential.
[Appendix U] The 'Note on attention d=0.952' is confusing: a headline effect that is 'not bootstrappable from the L2-proxy CSVs' is presented as real, but no CI or permutation p-value is given. Either provide a valid uncertainty estimate or remove the assertion.
[§4.1, Appendix AO] The text says the permutation test is performed 'with FDR,' but the p-values reported for the a-priori layers (0.028, 0.017) appear to be raw. Clarify whether these are FDR-adjusted and, if not, provide the adjusted values.
[Appendix T] The table header '100%<0.1?' is unclear. Specify what fraction of image pairs has shift_frac below 0.1 and how the 0.1 threshold was chosen.

Circularity Check

0 steps flagged

No significant circularity: the paper's claims are empirical measurements against public models and stimuli, with no fitted parameter renamed as prediction and no load-bearing self-citation.

full rationale

I walked the derivation chain and found no step where a claimed 'prediction' or 'first-principles result' reduces to its inputs by construction. The observational claims (layer-localized illusion sensitivity, FLODOG correlation, dose-response) are measured directly on public models and stimuli; no parameter is fitted and then reported as a prediction. The causal ablation claims use random-channel null distributions and are cross-validated, which mitigates the concern that channels are selected and evaluated on the same N=35 images. There are no self-citations by the authors, no imported uniqueness theorem, and no ansatz smuggled in via citation. The new term 'perceptual phantom' is an interpretation of a null injection result, not a definitional identity. The main weakness I weighed is Appendix AI: the single-step injection pipeline's positive control (+2 SD brightness offset) also yields shift_frac ≈ 0, so the phantom null may reflect measurement insensitivity rather than output-inert representations. This is a validity/control limitation, not circularity, and I note it here rather than in the steps. The paper also discloses its own limitations (small N, FLODOG domain restriction, seed variability, empirical rather than exhaustive phantom characterization), which further supports a non-circularity finding.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper does not introduce new physical entities. Its main postulate is the 'perceptual phantom' concept, which is a descriptive label for an internal-output dissociation rather than a new entity. The analysis relies on several domain assumptions about the stimuli, the psychophysical reference, and the probing protocol. The free parameters listed are analysis choices that materially affect the reported effect sizes.

free parameters (4)

Timestep set = t=50,150 (primary); t>700 excluded
Primary claims are based on t∈{50,150} because t>700 suffers a heavy-noise confound (App. B). This selection shapes the measured effect sizes.
Channel aggregation metric = max
'The max metric is primary for observational experiments' (§3.2) because it produces the strongest signal at N=35; the paper reports consistency across metrics, but the headline effect size depends on this choice.
Illusion-sensitive channel threshold = |d|≥0.5
Channels are classified as illusion-sensitive if |Cohen's d|≥0.5 (App. P, G); this threshold defines the ablation sets and the reported channel counts.
FLODOG/dose-response layer selection = e.g., up_1_attn_2, t=50
The layers/timesteps exhibiting ρ≥0.70 were identified from a larger search; the reported correlations are maxima of that search, not pre-registered.

axioms (4)

domain assumption GVIL images, after excluding flips, contain two physically identical regions that appear perceptually different to humans.
The entire Δ measure assumes illusion_a and illusion_b are physically matched and that the visual illusion is the only driver of differential activation. §3.1, Fig 1.
domain assumption FLODOG is a valid reference for human brightness perception on luminance stimuli.
Used as ground truth for SBC and Hermann grid; the paper notes it is not valid for chromatic stimuli (App. L).
domain assumption Noise schedule and single-step probing at t=50/150 produce activations representative of the models' normal processing.
The paper uses single forward pass at fixed timesteps; t>700 is excluded due to noise confound (App. B).
domain assumption Zero-ablation is an adequate proxy for removing a channel's causal contribution.
The paper acknowledges the known incompleteness of nullifying interventions and supplements with counterfactual patching (§3.5, App. AG, App. AI).

pith-pipeline@v1.3.0-alltime-deepseek · 21922 in / 10315 out tokens · 98760 ms · 2026-08-01T18:53:33.657587+00:00 · methodology

0 comments

read the original abstract

Deep neural networks trained on natural images are shown to produce outputs consistent with human observers for brightness illusions. While this phenomenon has been documented across architectures, all evidence, to date, is measured at the output level: restored pixels, decoded trajectories, or classification decisions. Whether these models actually represent illusions internally, and if so where and how, remains unknown. We show that denoising models develop illusion-sensitive representations at specific internal layers, across varied architectures. Specifically, we identify the layers and channels that discriminate illusory from physically matched control regions. We show that the denoising objective is a more important driver of the effect than the architecture. On domain-appropriate stimuli, these activations track a validated psychophysical model of human brightness perception (FLODOG; Spearman $\rho \geq 0.70$) and scale monotonically with parametric illusion strength. Leveraging these findings, we provide causal evidence via channel ablation showing that illusion-sensitive channels specifically and substantially affect the internal signal. Yet injecting these representations into the generation pipeline produces no measurable pixel shift across all tested architectures; we term such representations perceptual phantoms: active in internal processing yet invisible to any output-based evaluation. While related internal-output dissociations have been characterized in language models, this is the first such characterization for perceptual representations in denoising vision models.

Figures

Figures reproduced from arXiv: 2607.17138 by Gautam Ranka, Paras Chopra.

**Figure 1.** Figure 1: Illusion encoding traced through a denoising U-Net. (A) A GVIL color contrast illusion: UP: the two elliptical regions (illusion_a, red; illusion_b, blue) are physically the same color but appear different due to the context. DOWN: same image with illusive feature removed (B) Spatial activation maps at four layers tracing the signal through the network for the UP and DOWN images. ∆ denotes the raw mean act… view at source ↗

**Figure 2.** Figure 2: Layer-resolved illusion sensitivity. (A) Heatmap of |Cohen’s d| across all 41 U-Net layers and 6 timesteps (max metric, N=35 base color images). mid_attn_0 and up_1_attn_0 (largest attention effect) are highlighted. (B) Paired bar chart comparing ∆ill vs. ∆real at the three a-priori layers, showing 2.3–4.5× specificity to illusory context. The illusion effect at mid-block layers also substantially exceeds … view at source ↗

**Figure 3.** Figure 3: Psychophysical alignment of U-Net activations. (A) Per-pixel scatter plot of FLODOGpredicted brightness vs. U-Net activation magnitude within the illusion region for a representative SBC image, at up_1_attn_2, t=50 (ρ=0.78). (B) Top layers by Spearman ρ on programmatic SBC and Hermann-grid stimuli. 0 1 2 3 Illusion strength 1 0 1 2 3 4 5 Glass's threshold str=0.2 Spearman = 0.976 (A) Ebbinghaus = 0.976 Tr… view at source ↗

**Figure 4.** Figure 4: Monotonic dose-response to parametric illusion strength. (A) Ebbinghaus: Glass’s ∆ vs. illusion strength at up_1_attn_2, t=50. Signal is suppressed at low strengths (str <0.2) and rises monotonically above a threshold. (B) Ponzo: perfectly monotone dose-response at conv_norm_out, t=150 (C) Trained vs. random-initialization comparison at the trained model’s best layer. Untrained U-Nets also achieve high ρ ≈… view at source ↗

**Figure 5.** Figure 5: Percentage decrease in Reconstruction MSE on ablating specific channels vs random [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: Architecture generalization across denoising and discriminative models. (A) Peak |d| bar chart across all nine models, colored by family (denoising vs. discriminative). (B) DiT-XL/2 block profile: |d| across 28 transformer blocks, peaking at block 26. (C) ViT-B/16 vs. DiT-XL/2, 2.5× difference in illusion sensitivity. Mechanism We then try to analyze the effect mechanistically. For DiT, we trace injected a… view at source ↗

**Figure 7.** Figure 7: Causal evidence. (A) Channel ablation: percentage ∆ reduction at mid_attn_0 per ablation group. (B) Skip-connection dissociation: |d| comparison across bottleneck, skip-proxy layers, and decoder-entry layers. The illusion signal concentrates at the bottleneck, not in skip connections. within each level). All stimuli are 256×256 RGB, normalized to [−1, 1] to match model input conventions. O Per Model Peak a… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

14 extracted references · 1 canonical work pages

[1]

Predictive coding feedback results in perceived illusory contours in a recurrent neural network.arXiv preprint arXiv:2102.01955,

Zhaoyang Bai, Mitsuo Nakamura, and Hidehiko Kaneko. Predictive coding feedback results in perceived illusory contours in a recurrent neural network.arXiv preprint arXiv:2102.01955,

Pith/arXiv arXiv
[2]

This preserves the local target-object appearance while disrupting the far surround context

) and pasting back the original bounding-box crops from illusion_a andillusion_bat their original positions. This preserves the local target-object appearance while disrupting the far surround context. Since the boundary boxes contain local background pixels which are generally the illusion drivers, this control does not isolate the contribution of immedi...

2082
[5]

URL https://www.nature.com/articles/s41598-024-52423-7

doi: 10.1038/s41598-024-52423-7. URL https://www.nature.com/articles/s41598-024-52423-7. Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. InAdvances in Neural Information Processing Systems (NeurIPS),

work page doi:10.1038/s41598-024-52423-7
[7]

Designing and interpreting probes with control tasks

John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733– 2743, Hong Kong, China, November

2019
[9]

Mingi Kwon, Jeonghyo Jeong, and Youngjung Uh

arXiv:2309.16779. Mingi Kwon, Jeonghyo Jeong, and Youngjung Uh. Diffusion models already have a semantic latent space. InInternational Conference on Learning Representations (ICLR),

Pith/arXiv arXiv
[11]

All analyses use only the 35 base images (filter: ’flip’ not in name and not name.endswith(’_r’) )

12 A Dataset: N=35 Base Images The GVIL dataset [Zhang et al., 2023] contains 72 color images: 35 base images, 35 horizontally flipped versions, and 2 additional images. All analyses use only the 35 base images (filter: ’flip’ not in name and not name.endswith(’_r’) ). Flipped versions are excluded to prevent near- duplicate image pairs from inflating eff...

2023
[13]

[2023], vl-illusion/GVIL Pyllusion (parametric stimuli li- brary) MITRealityBending/Pyllusion Makowski et al

Asset License Source GVIL stimulus dataset Not specified by authors; used for aca- demic research only Zhang et al. [2023], vl-illusion/GVIL Pyllusion (parametric stimuli li- brary) MITRealityBending/Pyllusion Makowski et al

2023
[2007]

The CreativeML OpenRAIL-M license on Stable Diffusion 1.5 includes use-based behavioral restrictions (no harmful or deceptive use), which our diagnostic probing does not violate

Vision Research, 2007 License compliance notes.The CC BY-NC 4.0 license on DiT-XL/2 restricts use to non- commercial purposes, which our research-only use complies with. The CreativeML OpenRAIL-M license on Stable Diffusion 1.5 includes use-based behavioral restrictions (no harmful or deceptive use), which our diagnostic probing does not violate. The GVIL...

2007
[2019]

doi: 10.18653/v1/D19-1275

Association for Computational Linguistics. doi: 10.18653/v1/D19-1275. URLhttps://aclanthology.org/D19-1275/. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems (NeurIPS), volume 33, pages 6840–6851,

doi:10.18653/v1/d19-1275
[2021]

URL https://aclanthology

doi: 10.1162/tacl_a_00359. URL https://aclanthology. org/2021.tacl-1.10/. Kayson Fakhar, Shrey Dixit, Fatemeh Hadaeghi, Konrad P. Kording, and Claus C. Hilgetag. Downstream network transformations dissociate neural activity from causal functional contri- butions.Scientific Reports, 14:2103, January

doi:10.1162/tacl_a_00359 2021
[2022]

Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556,

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556,

Pith/arXiv arXiv
[2023]

arXiv:2304.14997. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations (ICLR),

Pith/arXiv arXiv
[2024]

Measuring the reliability of causal probing methods: Tradeoffs, limitations, and the plight of nullifying interventions

Marc Canby, Adam Davies, Chirag Rastogi, and Julia Hockenmaier. Measuring the reliability of causal probing methods: Tradeoffs, limitations, and the plight of nullifying interventions. In Interpretable AI: Past, Present and Future Workshop at NeurIPS 2024,

2024
[2025]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun

arXiv:2412.10122. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778,

Pith/arXiv arXiv

[1] [1]

Predictive coding feedback results in perceived illusory contours in a recurrent neural network.arXiv preprint arXiv:2102.01955,

Zhaoyang Bai, Mitsuo Nakamura, and Hidehiko Kaneko. Predictive coding feedback results in perceived illusory contours in a recurrent neural network.arXiv preprint arXiv:2102.01955,

Pith/arXiv arXiv

[2] [2]

This preserves the local target-object appearance while disrupting the far surround context

) and pasting back the original bounding-box crops from illusion_a andillusion_bat their original positions. This preserves the local target-object appearance while disrupting the far surround context. Since the boundary boxes contain local background pixels which are generally the illusion drivers, this control does not isolate the contribution of immedi...

2082

[3] [5]

URL https://www.nature.com/articles/s41598-024-52423-7

doi: 10.1038/s41598-024-52423-7. URL https://www.nature.com/articles/s41598-024-52423-7. Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. InAdvances in Neural Information Processing Systems (NeurIPS),

work page doi:10.1038/s41598-024-52423-7

[4] [7]

Designing and interpreting probes with control tasks

John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733– 2743, Hong Kong, China, November

2019

[5] [9]

Mingi Kwon, Jeonghyo Jeong, and Youngjung Uh

arXiv:2309.16779. Mingi Kwon, Jeonghyo Jeong, and Youngjung Uh. Diffusion models already have a semantic latent space. InInternational Conference on Learning Representations (ICLR),

Pith/arXiv arXiv

[6] [11]

All analyses use only the 35 base images (filter: ’flip’ not in name and not name.endswith(’_r’) )

12 A Dataset: N=35 Base Images The GVIL dataset [Zhang et al., 2023] contains 72 color images: 35 base images, 35 horizontally flipped versions, and 2 additional images. All analyses use only the 35 base images (filter: ’flip’ not in name and not name.endswith(’_r’) ). Flipped versions are excluded to prevent near- duplicate image pairs from inflating eff...

2023

[7] [13]

[2023], vl-illusion/GVIL Pyllusion (parametric stimuli li- brary) MITRealityBending/Pyllusion Makowski et al

Asset License Source GVIL stimulus dataset Not specified by authors; used for aca- demic research only Zhang et al. [2023], vl-illusion/GVIL Pyllusion (parametric stimuli li- brary) MITRealityBending/Pyllusion Makowski et al

2023

[8] [2007]

The CreativeML OpenRAIL-M license on Stable Diffusion 1.5 includes use-based behavioral restrictions (no harmful or deceptive use), which our diagnostic probing does not violate

Vision Research, 2007 License compliance notes.The CC BY-NC 4.0 license on DiT-XL/2 restricts use to non- commercial purposes, which our research-only use complies with. The CreativeML OpenRAIL-M license on Stable Diffusion 1.5 includes use-based behavioral restrictions (no harmful or deceptive use), which our diagnostic probing does not violate. The GVIL...

2007

[9] [2019]

doi: 10.18653/v1/D19-1275

Association for Computational Linguistics. doi: 10.18653/v1/D19-1275. URLhttps://aclanthology.org/D19-1275/. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems (NeurIPS), volume 33, pages 6840–6851,

doi:10.18653/v1/d19-1275

[10] [2021]

URL https://aclanthology

doi: 10.1162/tacl_a_00359. URL https://aclanthology. org/2021.tacl-1.10/. Kayson Fakhar, Shrey Dixit, Fatemeh Hadaeghi, Konrad P. Kording, and Claus C. Hilgetag. Downstream network transformations dissociate neural activity from causal functional contri- butions.Scientific Reports, 14:2103, January

doi:10.1162/tacl_a_00359 2021

[11] [2022]

Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556,

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556,

Pith/arXiv arXiv

[12] [2023]

arXiv:2304.14997. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations (ICLR),

Pith/arXiv arXiv

[13] [2024]

Measuring the reliability of causal probing methods: Tradeoffs, limitations, and the plight of nullifying interventions

Marc Canby, Adam Davies, Chirag Rastogi, and Julia Hockenmaier. Measuring the reliability of causal probing methods: Tradeoffs, limitations, and the plight of nullifying interventions. In Interpretable AI: Past, Present and Future Workshop at NeurIPS 2024,

2024

[14] [2025]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun

arXiv:2412.10122. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778,

Pith/arXiv arXiv