{"id":"ca2970b0-bd1e-4c39-9676-1afad19fa800","arxiv_id":"2502.09663","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DiffEx builds a classifier-aware latent space with a diffusion model, finds contrastive directions in it, and ranks them to produce visual explanations that reveal cellular phenotype changes.","lead":"DiffEx is a method that uses diffusion models to find image features that push a classifier's decision, turning a black-box classifier into a set of visual explanations. It can reveal subtle cell changes in microscopy images, which may help researchers see how drugs or diseases alter cells.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Concatenating the classifier score into the semantic code lets direction models shift the score itself, so DiffEx's attribute explanations may be class-conditional generation artifacts rather than latent-space discoveries.","rationale":"The reader's CONDITIONAL verdict is appropriate: the central claim is visually plausible but not quantitatively established. My concern is more specific than the reader's weakest assumption: rather than relying on latent-space entanglement, the method has a direct, unexamined route to bypass the encoder latent space altogether by modifying the classifier-score component of zsem. Because zsem includes the classifier score and D_k (Eq. 4) is defined on zsem, the ranking step (which selects directions by score change) could preferentially select directions that simply move the score. If so, the method reduces to class-conditional editing and the reported biological phenotypes are not independent discoveries. The concrete test would settle this by measuring the score-subspace component of the ranked directions and testing whether the method survives removal of the score channel. If the test clears, the paper's evidence is materially stronger; if not, the central claim is invalid. I therefore keep the reader's CONDITIONAL verdict unchanged, with the score-subspace check as a required condition.","tokens_in":12097,"tokens_out":7276,"duration_ms":85541,"concrete_test":"Re-run the direction-learning stage on BBBC021 with the same hyperparameters and record, for each of the top-3 ranked directions, the normalized projection of MLP1(zsem)/||MLP1(zsem)|| onto the classifier-score subspace of zsem. If the squared projection norm exceeds 0.1 for any top direction, the explanation is confounded. As a control, retrain direction models with the score component held fixed (or removed from zsem) and check whether classifier-relevant attributes are still discovered; if they are not, the original results are attributable to the score shortcut.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §3.1, zsem is formed by concatenating the classifier score with the encoder latent code and is used to condition the DDIM. In §3.2, the direction model D_k in Eq. 4 operates on this full zsem vector: D_k(z, α) = z + α MLP1(z)/||MLP1(z)||. Nothing constrains MLP1 from changing the classifier-score component of zsem. The ranking in §3.3 selects directions by the change they induce in the classifier score, so any direction that moves the score component will be preferentially retained. The visual changes in Figs. 2, 4, and 5 could then be produced by the diffusion model's class-conditional generation (conditioned on an altered score), not by an attribute direction in the encoder's semantic latent space. This would make the central claim circular: DiffEx would be showing the classifier the score it already output, rather than explaining the visual attributes that drive its decision. The paper reports no ablation or analysis of how much of each discovered direction lies in the score subspace, so the concern is unresolved as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"DiffEx proposes to explain image classifiers by learning a set of interpretable directions in a semantic latent space built with a diffusion autoencoder. The semantic code concatenates an encoder embedding with the classifier score and is used to condition a DDIM, while a KL classifier loss encourages reconstructions to preserve the score. Direction models are then trained with a contrastive loss and a decorrelation regularizer, and directions are ranked by the average classifier-score shift they induce. The authors demonstrate the method on FFHQ gender classification and on two fluorescence microscopy datasets (BBBC021 and Golgi), claiming that the top directions replicate known treatment phenotypes such as cytoplasm loss, nuclear count reduction, and Golgi scattering.","tokens_in":12307,"tokens_out":7258,"duration_ms":86554,"significance":"If the central claims hold, DiffEx would be a useful interpretability tool: it converts a black-box classifier into a ranked set of visually editable attribute directions and can potentially surface subtle phenotypes in microscopy data. The idea of making the latent space classifier-aware and then discovering directions with contrastive learning is a reasonable extension of DiffAE and NoiseCLR. However, the current evidence is mostly qualitative: phenotype replication is asserted from selected figures, disentanglement is not measured, and the classifier-score concatenation introduces a confounding channel that is never analyzed. With additional ablations and quantitative evaluation on generated images, the contribution could be significant for both interpretability and bioimage analysis; as written, the load-bearing claims are not yet established.","major_comments":[{"comment":"The concatenation of the classifier score into zsem creates a confound that is not addressed. The direction model Dk(z, α) in Eq. (4) is free to move the score coordinate of zsem, and the ranking procedure in §3.3 explicitly retains directions whose average classifier-score change is large. Since the DDIM is conditioned on zsem, changing the score coordinate alone can produce target-class-like appearances; the selected directions may therefore be score-conditional sampling artifacts rather than attribute directions in the encoder's semantic latent space. The authors should report, for each discovered direction, the proportion of its norm that lies in the score coordinate(s), or ablate by projecting MLP1(z) orthogonal to the score subspace, or compare against a baseline that changes only the score coordinate. Without one of these analyses, the explanation claim is not established.","section":"§3.1, Eq. (2), Eq. (4), §3.3"},{"comment":"The phenotype-replication claims are supported only by visual inspection of generated images. Fig. 6 reports CellProfiler measurements on real images only, so it confirms that the two conditions differ in the expected way but does not show that DiffEx's shifted images exhibit those differences. The authors should measure the same morphological features on the generated/shifted images and report, for each direction, whether the feature moves in the claimed direction and by how much relative to the real-condition gap. Without such measurements, statements such as 'Direction 1 eliminates the cytoplasm' or 'the Golgi apparatus becomes more scattered' remain qualitative assertions.","section":"§4.3, Figs. 4–6"},{"comment":"The claim that the discovered directions are 'well disentangled and separated' is not backed by a quantitative metric. Equation (8) is a ranking loss over feature differences computed by MLP2; it may separate the feature representations used in the contrastive objective, but it does not by itself ensure that shifting along one discovered direction leaves the attributes associated with other directions invariant. The authors should provide a quantitative disentanglement evaluation, for example by measuring attribute classifier outputs or pixel-level feature changes during traversal along each direction and computing pairwise interference.","section":"§3.2, Eq. (8), §4.3"},{"comment":"Several central hyperparameters and implementation details are missing: λ1, λ2, α, the number of directions N, the ranking threshold, the number of images in the ranking sample, and the architecture/training details of the two MLPs. The ranking pseudo-code is deferred to Supplementary B, but the supplementary material is not included with the manuscript. Since the ranking threshold determines which directions are retained and α controls the edit magnitude, the experiments cannot be reproduced or fully assessed as presented.","section":"§3.2–§3.3; Supplementary B"},{"comment":"The denominator of Eq. (8) contains only negative-pair terms, making the loss -log(pos/neg) rather than a standard InfoNCE loss whose denominator also includes positive pairs. Please clarify whether this is intentional and define the exact sets of positive and negative pairs in both words and equations, including the role of the self-pair (j = i) and the treatment of terms with l = k.","section":"§3.2, Eq. (8)"}],"minor_comments":[{"comment":"In Eq. (10), the text says 'where λ1 is a hyperparameter', but the expression contains λ2; please correct this to λ2 (or state the intended weighting).","section":"End of §3.2"},{"comment":"The comparison section refers to 'In Fig. 4, we further evaluate...' but the GCD comparison appears in Fig. 8; also, the sentence about CellProfiler contains 'As shown in Fig.' with a missing figure number.","section":"§4.4"},{"comment":"The direction-specific network is written as MLP1 without a subscript k; please use MLP1^k (or similar) to make explicit that each direction model has its own parameters.","section":"Eq. (4)"},{"comment":"KID is computed on fluorescence microscopy images using an ImageNet-pretrained Inception network, which may not be well suited to this domain; please report confidence intervals and consider domain-specific embeddings or FID variants.","section":"Table 2"},{"comment":"Several citations are malformed, including 'et al, 2006', 'et al., 2022', and 'et al., 2010', which should be given proper author names; in addition, the supplementary material (Sections A and B) is referenced but not included.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready for publication in its current form: the core idea is promising, but the evaluation is largely qualitative, the score-concatenation confound needs a targeted ablation, and the paper contains multiple missing figures and references that suggest a hasty submission. I recommend major revision with a request for the missing quantitative analyses and a complete, self-contained supplementary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DiffEx is a plausible new pipeline that should be reviewed, but the current write-up doesn't yet rule out the possibility that its 'directions' are just the classifier score being moved through the concatenated latent code.\n\nThe genuinely new bit is the combination of a DiffAE-style semantic encoder, a contrastive loss to separate directions, and a ranking by classifier shift. Conditioning the latent on the classifier's probability vector (Eq. 2) makes the space classifier-aware, and the contrastive objective in Eq. 8 is a reasonable way to encourage direction separation. On the empirical side, the face results (lipstick, hairstyle) and the Golgi scattering are visually compelling and consistent with known biology. The comparison to GCD is a good idea and shows the method can produce cleaner counterfactuals, at least on the examples shown.\n\nThe soft spot is the one the stress-test flags, and it's genuine. zsem is the concatenation of the encoder output and the classifier score. The direction model in Eq. 4 operates on that full vector, so MLP1 can change the score component directly. The ranking in Section 3.3 then selects precisely those directions that move the score. Nothing in the paper measures how much of each direction's effect sits in the score subspace, so the visual changes could be the diffusion model generating a class-consistent image rather than an attribute-level shift in the encoder's semantic space. That's not a fatal flaw for a counterfactual explanation—it's still showing what flips the classifier—but it undercuts the stronger claim that the method 'identifies' separable attributes and 'discovers' phenotypes. The paper needs to either ablate the score component or quantify its contribution.\n\nThe other issues are more standard for a preprint: evaluation is almost entirely qualitative. The CellProfiler numbers in Fig. 6 are on real images, not on the generated ones, so they don't validate that the generated shifts reproduce the phenotype. No code or data is public, and several hyperparameters (lambda_1, lambda_2, alpha, N, the ranking threshold) are unspecified. There are also small errors—'where lambda_1 is a hyperparameter' after Eq. 10 should be lambda_2, and 'BBC021' in Table 1—that suggest a rushed copyedit.\n\nIf I were the editor, I'd send this to peer review. The idea is worth serious consideration and the flaws are the kind that can be addressed with an ablation and some quantitative evaluation. A referee should push for those, but the paper shouldn't be desk-rejected.","headline":"DiffEx is a promising classifier-explanation pipeline, but the current version doesn't rule out that its discovered directions are partly artifacts of moving the classifier score through the concatenated latent code.","tokens_in":12833,"tokens_out":3086,"would_cite":false,"duration_ms":31998,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DiffEx generates visually interpretable attributes that explain a classifier and reveal microscopic cellular variations between conditions.","keywords":["diffusion models","classifier explainability","latent space directions","contrastive learning","cellular phenotype discovery","microscopy image analysis","counterfactual explanations","semantic latent space"],"falsifier":"Apply the top-ranked direction from the treated-versus-untreated classifier to a held-out set of untreated images and measure the target phenotype (e.g., cytoplasm area, nuclei count, Golgi area) automatically: the claim fails if the direction does not change the measured phenotype in the expected direction for most images, or if it also changes unrelated attributes.","tokens_in":11912,"feed_emoji":"🔬","tokens_out":6650,"duration_ms":68783,"temperature":0.7,"pith_summary":"DiffEx is a method for explaining what a trained image classifier sees, by generating a ranked set of visual attributes that push the classifier from one class to another. The paper claims that these attributes are not just heatmaps but editable image directions: shifting an image's latent code along a learned direction changes one interpretable property at a time, such as lipstick, hairstyle, cytoplasm loss, or Golgi scattering. On two microscopy datasets, the top-ranked directions replicate known treatment phenotypes, including cell death and cytoskeletal disruption on BBBC021 and Nocodazole-induced Golgi scattering on the Golgi dataset. If correct, the method turns a black-box classifier into a phenotyping tool that could suggest new biomarkers and treatment effects without hand-crafted image features.","feed_headline":"DiffEx turns classifier decisions into editable visual directions","feed_subtitle":"The method recovers known drug effects, pointing toward automated phenotype discovery.","key_machinery":"The load-bearing object is the classifier-aware semantic code $z_{\\mathrm{sem}}$, formed by concatenating an encoder's latent vector with the classifier's prediction score and used to condition the DDIM denoiser, which makes the latent space carry classifier-relevant attributes. Direction discovery is carried by MLP-based direction models $D_k(z,\\alpha)=z+\\alpha\\,\\mathrm{MLP}_1(z)/\\|\\mathrm{MLP}_1(z)\\|$, whose edits are separated by a contrastive loss (Eq. 8) and decorrelated by a covariance regularization term (Eq. 9), then ranked by average change in classification score. The mechanism's job is to turn the classifier's decision surface into a short list of global, one-at-a-time attribute edits.","core_discovery":"The central claim is that a classifier can be explained by training a diffusion autoencoder whose semantic latent code includes the classifier's score, then learning a set of direction models $D_k(z,\\alpha)=z+\\alpha\\,\\mathrm{MLP}_1(z)/\\|\\mathrm{MLP}_1(z)\\|$ that shift the latent code along normalized directions, with a contrastive loss that makes edits from the same direction attract and edits from different directions repel. The discovered directions are ranked by how much they change the classifier's probability, and applying a direction to an image produces a counterfactual that visibly exhibits a single attribute. The paper shows directions for gender classification (lipstick, hairstyle length) and, on biological data, directions that replicate the phenotypes induced by the tested compounds: cytoplasm disappearance, nuclei reduction, red-channel intensity changes, and nuclei clustering for Latrunculin B, and Golgi apparatus scattering for Nocodazole. The ranked directions are meant to be the classifier's own decision attributes, made visible and editable.","pith_inferences":["Because ranking removes images already explained, the number of useful directions is bounded by the diversity of the image sample, which may explain why the Golgi experiment collapsed onto a single phenotype.","Deployed prospectively on an uncharacterized drug screen, the top-ranked directions would generate candidate biomarkers that could be tested with targeted stains or single-cell assays.","Making the direction models conditional on the starting class, or on multiple classifier scores, might separate additional attributes in datasets where directions currently collapse."],"forward_implications":["Applied to treated versus untreated microscopy images, the top-ranked directions recover multiple known drug-induced phenotypes, including cytoplasm loss, nuclei reduction, and red-channel intensity changes on BBBC021.","On the Golgi dataset, the identified direction reproduces Nocodazole-induced Golgi scattering, and the reverse direction reproduces aggregation.","On the FFHQ gender classifier, directions correspond to recognizable attributes such as lipstick and hairstyle length, showing the method works on natural images as well.","Because reconstruction keeps classifier accuracy high, the edited images are not adversarial artifacts but preserve the features the classifier relies on.","The ranking procedure yields a short list of directions that most affect the classifier, rather than requiring the user to inspect all latent dimensions."],"supporting_citations":[{"why":"Supplies the diffusion-autoencoder architecture whose semantic latent code DiffEx extends with classifier scores.","marker":"Preechakul et al., 2022"},{"why":"Shows that diffusion models have discoverable semantic latent directions, motivating the search in the encoder's latent space.","marker":"Kwon et al., 2023"},{"why":"Provides the contrastive-learning framework and MLP direction model for discovering interpretable latent directions.","marker":"Y¨uksel et al., 2021"},{"why":"Adapts contrastive direction discovery to diffusion models, the template DiffEx follows for its own latent space.","marker":"Dalva & Yanardag, 2024"},{"why":"Introduces the classifier-KL loss used to keep generated images aligned with the explained classifier's scores.","marker":"Lang et al., 2021b"},{"why":"Provides the covariance-decorrelation regularization used to separate learned directions.","marker":"Bardes et al., 2022"},{"why":"Serves as the diffusion-based counterfactual-direction baseline compared on natural and biological images.","marker":"Sobieski & Biecek, 2024"},{"why":"Contributes the BBBC021 treated-versus-untreated microscopy dataset used to validate phenotype discovery.","marker":"et al., 2010"},{"why":"Contributes the FFHQ face dataset used to demonstrate natural-image explanations.","marker":"Karras et al., 2019"}],"fun_headline_variants":["DiffEx edits images to reveal classifier's cell cues","DiffEx turns classifier decisions into visual counterfactuals","DiffEx exposes the phenotypes behind classifier calls","DiffEx: editable directions explain a classifier's choices","DiffEx finds microscopic variations via classifier edits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that a single shift of the latent code moves the same visual attribute in every image, so a direction learned from some images will edit the same attribute in others.","fun_headline_variants_meta":{"raw":{"variants":["DiffEx edits images to reveal classifier's cell cues","DiffEx turns classifier decisions into visual counterfactuals","DiffEx exposes the phenotypes behind classifier calls","DiffEx: editable directions explain a classifier's choices","DiffEx finds microscopic variations via classifier edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1433,"prompt_tokens":913,"completion_tokens":520,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":448}},"tokens_in":529,"tokens_out":520,"duration_ms":5775,"temperature":1.0,"reasoning_tokens":448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T05:23:17.673638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the top-ranked direction from the treated-versus-untreated classifier to a held-out set of untreated images and measure the target phenotype (e.g., cytoplasm area, nuclei count, Golgi area) automatically: the claim fails if the direction does not change the measured phenotype in the expected direction for most images, or if it also changes unrelated attributes.","supporting_citations":[{"cited_title":"and Yanardag, P","cited_arxiv_id":null,"evidence_quote":"Adapts contrastive direction discovery to diffusion models, the template DiffEx follows for its own latent space."},{"cited_title":"VICReg : Variance-invariance-covariance regularization for self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Provides the covariance-decorrelation regularization used to separate learned directions."},{"cited_title":"A style-based generator architecture for generative adversarial networks","cited_arxiv_id":null,"evidence_quote":"Contributes the FFHQ face dataset used to demonstrate natural-image explanations."}],"review_version":1}