REVIEW 3 major objections 6 minor 19 references
I See, Therefore I Do: Estimating Causal Effects for Image Treatments
T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Seeing the treatment image improves causal effect estimates
desk verdict First image-treatment ITE benchmarks, but the zero-shot mechanism and the data-generation link between images and outcome embeddings are under-specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the NICE architecture plus its two-term objective. A covariate encoder $\Phi$ maps user features to a low-dimensional space; a treatment encoder $\Psi$ maps frozen image embeddings $\Lambda(I_t)$ from ResNet or VGG into the same space; the concatenation $[\Phi(x); \Psi(\Lambda(I_t))]$ is fed to one of $k$ treatment head networks $\pi_t$, one per treatment, to predict potential outcomes. The objective is $L = \alpha L_1 + \beta L_2$, where $L_1$ is the factual MSE and $L_2$ is the average Maximum Mean Discrepancy (MMD) between joint embeddings of every pair of treatment groups; MMD is a distance between distributions that acts as an Integral Probability Metric to reduce treatment-assignment bias. The semi-synthetic outcome equation $y_t^i = c\,\tilde y_t^i (x_i^\top z_t + x_i^\top z_{k+1})$ with treatment centroids $z_t$ completes the machinery: it defines the signal that the model must extract from image embeddings, and the experiment's realism rests on whether the posters actually delivered to the model carry that centroid signal.
What would settle it
Check whether the ResNet embeddings of the posters chosen as treatments equal the centroids $z_t$ used in Eq. (7). If they do not, retrain NICE on treatments whose embeddings are deliberately decoupled from $z_t$ while outcomes stay tied to $z_t$; if NICE's advantage over baselines disappears, the reported superiority depends on that alignment rather than on a generally useful architecture.
Extended reading notes
Core claim
The paper's central claim is that an ITE estimator which consumes image treatments through their embeddings — NICE — substantially outperforms existing methods that either ignore treatment attributes (TARNet) or were designed for graph/text treatments and adapted to images (GraphITE, SIN, CaML). In the proposed semi-synthetic framework, outcomes are generated by $y_t^i = c\,\tilde y_t^i \left(x_i^\top z_t + x_i^\top z_{k+1}\right)$, where $z_t$ are treatment centroids from poster embeddings, and NICE's job is to recover this relationship from the image it sees. Across $k=4,8,16$ treatments and several treatment-assignment bias settings, NICE achieves rooted PEHE values roughly 20–30 percent below the best baseline in the standard setting (e.g., 91.95 vs 127.74 for $k=4$ with NICE-ResNet against SIN), and similar advantages in the zero-shot evaluation where one treatment is held out during training. The paper further reports that the advantage persists whether the frozen embedding backbone is ResNet or VGG.
Load-bearing premise
The evaluation assumes the image shown to the model is the same object whose features generate the simulated outcome; if that link is not built into the data, the reported improvement could come from the data construction rather than from causal estimation.
Editorial extensions
If this is right
- If the reported results hold, ITE estimation for images becomes practical for personalization tasks such as thumbnail selection, product-image display, and content recommendation, where each treatment is a high-dimensional visual.
- The zero-shot results imply the model can produce effect estimates for treatment images never seen during training, which matters in settings where new images appear continuously.
- The performance gap between NICE and the baselines grows with the number of treatments, suggesting that image-aware treatment representations matter more as the treatment space becomes richer.
- Because NICE works with both ResNet and VGG embeddings, the architecture can be paired with more recent pre-trained visual encoders without changing the training procedure.
- The proposed semi-synthetic simulation framework gives future work a way to generate benchmark datasets for image-treatment causal effects, addressing the lack of real-world ground-truth counterfactual data.
Reading between the lines
- Editorially, the same architecture could in principle be carried over to video or multimodal treatments, since the treatment encoder only consumes a fixed-dimension embedding; NICE's behavior on such treatments is untested in this paper.
- The outcome model in Eq. (7) is linear in the treatment embedding, which may favor estimators that learn linear or nearly linear mappings from embeddings to outcomes; a non-linear outcome generator would test whether the advantage persists.
- A direct real-world check would be a randomized experiment on thumbnail or product-image selection where NICE's predicted individual effects are compared with observed engagement; this would separate genuine generalization from the semi-synthetic setup.
- The paper does not state whether the $k$ posters chosen as treatments have embeddings equal to the centroids $z_t$ used to generate outcomes; if they do not, the model cannot see the outcome-determining signal, so the reported superiority could be an artifact of the simulation rather than of NICE's design.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes NICE, a neural architecture for estimating individual treatment effects when treatments are images. NICE combines learned user-covariate representations with representations of pretrained image embeddings (ResNet or VGG) and feeds the concatenation into per-treatment head networks; training uses an MSE regression loss plus an MMD-based balancing loss. To evaluate the method, the authors introduce a semi-synthetic data-generation procedure based on PosterLens in which potential outcomes depend on user covariates and treatment-specific centroid embeddings, with treatment assignment skewed by a parameter kappa. Experiments compare NICE against TARNet, GraphITE, SIN, and CaML on rooted PEHE for 4, 8, and 16 treatments, under standard, biased-assignment, and zero-shot settings; NICE reports substantially lower PEHE in all tables.
Significance. If the empirical claims hold, the paper addresses an underexplored and practically relevant setting: ITE estimation with image treatments. The proposed architecture is simple and the experimental comparisons are broad, covering multiple treatment counts, two embedding backbones, and several confounding strengths; the performance gaps over the baselines are large and consistent. The main value would be a reproducible semi-synthetic benchmark for image treatments and a demonstration that treatment-image representations can improve causal-effect estimates. However, the paper's central claims currently rest on two unstated or underspecified elements: the link between the generated outcome centroids and the actual treatment images, and the mechanism by which per-treatment-head models produce zero-shot predictions. The paper does not provide code or a precise data-generation recipe, so as written the results are not fully verifiable.
major comments (3)
- [Data Simulation, Eq. (7)] The potential outcome is defined as y_t_i = c * ytilde_t_i * (x_i^T z_t + x_i^T z_{k+1}), but the manuscript never states that the k treatment posters I_t are selected so that their embedding Lambda(I_t) equals the centroid z_t used in the outcome equation. The model observes only Lambda(I_t), not z_t. If Lambda(I_t) differs from z_t, then the outcome-relevant term x_i^T z_t is not a function of the image shown to the model, and the image is causally inert in the data-generation process. In the standard (non-zero-shot) setting, per-treatment heads could then fit the treatment-conditional linear map from x to y using the treatment identity alone, reducing the comparison to categorical multi-treatment ITE rather than image-treatment ITE. Please state explicitly how the k treatment images are chosen relative to the centroids z_t, and if no such link is intended, redesign the simulation so that the outcome depends on the image actually provided to the model.
- [Proposed Model, Eq. (3); Table 2] The architecture in Eq. (3) defines the prediction for treatment t through a dedicated head pi_t, with heads only for t in {1,...,k}. Table 2 reports zero-shot PEHE for a held-out treatment z whose samples are not seen during training, but the paper does not describe how NICE (or the adapted baselines) computes yhat_i^z for this unseen treatment. Without a concrete mechanism, such as using the shared representation Psi(Lambda(I_z)) in a common outcome module or a meta-learning step that maps treatment embeddings to head parameters, the zero-shot results in Table 2 are not reproducible and cannot be attributed to the proposed architecture. Please add the precise zero-shot inference procedure and state which parameters are updated or frozen.
- [Data Simulation and Problem Formulation, Eqs. (1), (2), (7)] The generated outcomes include an idiosyncratic noise factor ytilde_t_i ~ N(mu_t, sigma_t), so the conditional expectation in Eq. (1) is E[y_t_i | x_i] = c * mu_t * (x_i^T z_t + x_i^T z_{k+1}). However, the paper appears to evaluate PEHE by comparing predictions against the realized noisy y_t_i values, calling them ground truth. If so, the evaluation target is not the ITE defined in Eq. (1) but a noisy individual draw, and the reported PEHE includes irreducible noise that is not a function of x. Please clarify how the true tau^{a,b}(x_i) in Eq. (2) is computed from the generated data; if the realized outcomes are used, either average over multiple draws of ytilde or set ytilde to its mean so that the ground truth matches the estimand.
minor comments (6)
- [Problem Formulation, Eq. (2)] The sentence 'where hat_tau(.) represents the estimated PEHEs produced by a model' should say 'estimated ITEs', since PEHE is the evaluation metric rather than the quantity being estimated.
- [Problem Formulation, Eq. (2)] The summation upper index for b is written as b = 0 to a - 1, but treatments are indexed from 1 to k; this should presumably be b = 1 to a - 1 to match the binom(k,2) normalization.
- [Algorithm 1, line 6] The notation 'MMD ({Phi; Psi(Lambda))}t=a' is missing a closing brace and the displayed index range for b is garbled; please clean up the pseudocode so it matches Eq. (5).
- [Proposed Model, Assumptions 1-2] The paper invokes strong ignorability, which assumes all confounders are in x, but then motivates image treatments by arguing that they can account for hidden confounders not in x. These statements are in tension; please reconcile the motivation with Assumption 1.
- [Experiments, Tables 1-4] The paper reports means and standard deviations over 10 iterations but does not provide statistical significance tests. Given the large reported gaps this is not fatal, but paired tests across seeds would strengthen the claim that NICE 'significantly outperforms' the baselines.
- [Data Simulation, Eq. (8)] In the treatment assignment probability, the subscript in exp(kappa_i y_t_i) appears to use the user index i instead of the treatment index; this should probably be exp(kappa_t y_t_i) to match the description of kappa as a treatment-specific parameter.
Circularity Check
No circular derivation: NICE's objective and evaluation are self-contained; only a non-load-bearing self-citation prevents a score of 0.
full rationale
The derivation is not circular. NICE minimizes alpha*L1 + beta*L2, where L1 is MSE on observed outcomes and L2 is an MMD regularizer; neither is defined in terms of the PEHE metric or the data-generation centroids z_t. Outcomes in Eq. (7) are generated as functions of x_i and z_t, and the model must learn the map from (x_i, I_t) to y; the outcome is not a training target that trivially equals the prediction. The zero-shot PEHE in Eq. (9) is only an evaluation metric, not a training loss, so it cannot force the reported results. The sole self-citation, Thorat et al. 2023, appears in the literature survey listing prior multi-treatment methods and is not load-bearing for NICE's architecture, losses, or empirical claims. The concern that Lambda(I_t) may differ from z_t is a construct-validity caveat, not a circularity: it could make the benchmark uninformative, but it does not make the estimates equal to the inputs by construction. No equation reduces to an earlier one and no fitted parameter is renamed as a prediction. Hence no significant circularity.
Assumptions & free parameters
free parameters (5)
- alpha =
0.5 or 1.0 via grid search
- beta =
0.5
- c =
5
- mu_t, sigma_t =
drawn from N(0.45, 0.15) and N(0.1, 0.05)
- kappa_k =
10, 50, or 100
assumptions (4)
- domain assumption Strong ignorability: unconfoundedness and positivity (Assumptions 1 and 2) hold.
- domain assumption Pre-trained image embeddings from ResNet or VGG capture treatment-relevant information for outcomes.
- ad hoc to paper Potential outcomes follow the multiplicative model y_t_i = c * ytilde_t_i * (x_i^T z_t + x_i^T z_{k+1}).
- ad hoc to paper Treatment images correspond to the centroids z_t used in the outcome equation, implied but not stated.
Cite this review
Pith. "Pith review of I See, Therefore I Do: Estimating Causal Effects for Image Treatments." pith.science (2026). https://pith.science/paper/BCDKYSDA
@misc{pith2026241206810,
author = {Pith},
title = {Pith review of: I See, Therefore I Do: Estimating Causal Effects for Image Treatments},
year = {2026},
howpublished = {\url{https://pith.science/paper/BCDKYSDA}},
note = {Machine review of arXiv:2412.06810}
}
read the original abstract
Causal effect estimation under observational studies is challenging due to the lack of ground truth data and treatment assignment bias. Though various methods exist in literature for addressing this problem, most of them ignore multi-dimensional treatment information by considering it as scalar, either continuous or discrete. Recently, certain works have demonstrated the utility of this rich yet complex treatment information into the estimation process, resulting in better causal effect estimation. However, these works have been demonstrated on either graphs or textual treatments. There is a notable gap in existing literature in addressing higher dimensional data such as images that has a wide variety of applications. In this work, we propose a model named NICE (Network for Image treatments Causal effect Estimation), for estimating individual causal effects when treatments are images. NICE demonstrates an effective way to use the rich multidimensional information present in image treatments that helps in obtaining improved causal effect estimates. To evaluate the performance of NICE, we propose a novel semi-synthetic data simulation framework that generates potential outcomes when images serve as treatments. Empirical results on these datasets, under various setups including the zero-shot case, demonstrate that NICE significantly outperforms existing models that incorporate treatment information for causal effect estimation.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
- [3]
-
[4]
Guo, R.; Li, J.; and Liu, H. 2020. Learning individual causal effects from networked observational data. In Proceedings of the 13th international conference on web search and data mining, 232--240
work page 2020
-
[5]
Harada, S.; and Kashima, H. 2021. Graphite: Estimating individual effects of graph-structured treatments. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 659--668
work page 2021
-
[6]
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778
2016
-
[7]
Johansson, F.; Shalit, U.; and Sontag, D. 2016. Learning representations for counterfactual inference. In International conference on machine learning, 3020--3029. PMLR
work page 2016
-
[8]
Kaddour, J.; Zhu, Y.; Liu, Q.; Kusner, M. J.; and Silva, R. 2021. Causal effect inference for structured treatments. Advances in Neural Information Processing Systems, 34: 24841--24854
work page 2021
Show all 19 references
-
[9]
Nilforoshan, H.; Moor, M.; Roohani, Y.; Chen, Y.; S urina, A.; Yasunaga, M.; Oblak, S.; and Leskovec, J. 2023. Zero-shot causal learning. Advances in Neural Information Processing Systems, 36: 6862--6901
2023
-
[10]
Pearl, J. 2017. Detecting Latent Heterogeneity. Sociological Methods & Research, 46(3): 370--389
2017
-
[11]
Rubin, D. B. 2005. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469): 322--331
2005
-
[12]
M.; and Karlen, W
Schwab, P.; Linhardt, L.; Bauer, S.; Buhmann, J. M.; and Karlen, W. 2020. Learning counterfactual representations for estimating individual dose-response curves. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 5612--5619
2020
-
[13]
Schwab, P.; Linhardt, L.; and Karlen, W. 2018. Perfect match: A simple method for learning representations for counterfactual inference with neural networks. arXiv preprint arXiv:1810.00656
2018 arXiv
-
[14]
D.; and Sontag, D
Shalit, U.; Johansson, F. D.; and Sontag, D. 2017. Estimating individual treatment effect: generalization bounds and algorithms. In International conference on machine learning, 3076--3085. PMLR
2017
-
[15]
Shpitser, I.; and Pearl, J. 2012. Identification of conditional interventional distributions. arXiv preprint arXiv:1206.6876
2012 arXiv
-
[16]
Simonyan, K.; and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
2014 arXiv
-
[17]
K.; Fukumizu, K.; Gretton, A.; Sch \"o lkopf, B.; and Lanckriet, G
Sriperumbudur, B. K.; Fukumizu, K.; Gretton, A.; Sch \"o lkopf, B.; and Lanckriet, G. R. 2012. On the empirical estimation of integral probability metrics
2012
-
[18]
Thorat, A.; Kolla, R.; Pedanekar, N.; and Onoe, N. 2023. Estimation of individual causal effects in network setup for multiple treatments. arXiv preprint arXiv:2312.11573
2023 arXiv
-
[19]
Yoon, J.; Jordon, J.; and Van Der Schaar, M. 2018. GANITE: Estimation of individualized treatment effects using generative adversarial nets. In International conference on learning representations
2018
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.