{"id":"4f51608f-ee2d-467b-a5ae-312005ab14d6","arxiv_id":"2509.23926","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An unsupervised method, EDDP, jointly learns encoding-decoding direction pairs for concepts in CNN latent spaces, recovering interpretable and influential concepts without labels, validated on synthetic and real data.","lead":"An unsupervised method learns pairs of latent-space directions that read and write concept information in deep vision networks, and shows these direction pairs align with interpretable concepts and influence predictions. The approach enables model explanations, counterfactuals, and bias correction without retraining, which matters for auditing deployed vision systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"§4.3 signal-vector estimator is unbiased only if concept factors are independent within positive subsamples; this is asserted, not tested, and the Deep Dream validation cannot detect the resulting bias.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing point: the signal-vector estimator in §4.3 assumes independence of concept factors after positive subsampling, and this is not established. My stress-test confirms that this assumption is genuinely load-bearing. The synthetic experiment in §6 is constructed so that the independence holds, so it cannot validate the method for real data. The real-world faithfulness evaluation via dreaming directions is also not an independent ground-truth check: it optimizes the same decoding direction that the method learned, so it primarily tests consistency between w and ŝ, not whether ŝ equals the true encoding direction. If the independence assumption fails, the central claim (c) is unsupported, and the applications based on encoding directions (RCAV influence, counterfactuals, model correction) inherit that risk. However, the concern is addressable with a targeted synthetic test, and the paper already frames its conclusions conditionally; therefore the reader's CONDITIONAL verdict remains appropriate and no verdict change is needed.","tokens_in":57631,"tokens_out":4152,"duration_ms":70621,"concrete_test":"Re-run the synthetic experiment of §6.2 with the independence assumption deliberately violated: keep Eq. (3), but sample the latent factors so that within the positive set of concept 0, corr(αp,0, αp,1) = corr(αp,0, αp,2) = 0.5 (e.g. via a Gaussian copula), while preserving the marginal distributions used in §6. Measure the cosine similarity between learned ŝ_i and ground-truth s_i. If the similarity stays near 1.0, the estimator is robust and the concern is resolved; if it degrades substantially, the encoding-direction half of the central claim is not supported for real data, where such correlations are the norm. A complementary analytical check: compute the bias term Σ_{j≠i} s_j cov(αj,αi|positive i)/var(αi|positive i) from Broden annotations using the learned detectors as α estimates.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 extends Eq. (1) to the multi-concept model xp = Sαp + Dβp + c. The estimator ŝ_i = cov[xp, αp,i]/var[αp,i] is unbiased only if, in the subsample of patches where concept i is present, αp,i is uncorrelated with all other concept factors αp,j and all distractor coefficients βp,f. The paper claims this 'can be considered independent by assumption' after removing the detection biases, but natural concepts are not independent within positive examples: patches containing 'car' are more likely to contain 'road' and less likely to contain 'sky' even after conditioning on the presence of 'car'. If these residual correlations are nonzero, the estimator converges to s_i plus a weighted sum of the other signal directions and distractor directions; the bias is proportional to the cross-covariances. L_fso only encourages w_i⊥ŝ_j; it does not remove correlation in α, and URA is an indirect heuristic. The real-data faithfulness test (§7.2) fits a 'dreaming direction' from features that are being optimized to increase w_i^T x; this direction is not an independent ground truth and will be consistent with any signal vector lying near the span of w_i and the model's Jacobian. Thus the central claim (c) — that signal vectors faithfully estimate encoding directions on real data — rests on an unverified independence assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Encoding-Decoding Direction Pairs (EDDP), an unsupervised method to recover, for concepts encoded as directions in a deep network's latent space, both a decoding direction (filter/concept detector) and an encoding direction (signal vector). The decoding directions are obtained by directional clustering with sparsity losses, the encoding directions by a covariance-based signal-vector estimator applied to concept-positive subsamples, and the search is guided by a novel Uncertainty Region Alignment (URA) loss that aligns the uncertainty region of the concept detectors with that of the network. The method is validated on synthetic data with ground-truth direction pairs, on several CNN architectures (ResNet18, ResNet50, EfficientNet, Inception-v3, VGG16) with interpretability and influence metrics, and through applications including global sensitivity analysis, local Concept Contribution Maps, counterfactual explanations, and model correction on a toy watermark task.","tokens_in":58006,"tokens_out":4086,"duration_ms":36368,"significance":"If fully validated, this would be a useful contribution to unsupervised concept discovery in vision networks: it explicitly separates decoding from encoding directions, avoids feature-reconstruction objectives, and leverages network weights via URA. The paper's strengths include a controlled synthetic experiment with perfect recovery, a broad evaluation across five architectures, careful ablations, and a concrete model-correction demonstration where the learned signal vector agrees with a supervised Pattern-CAV (cosine 0.99). However, the real-world faithfulness claim for signal vectors rests on an independence assumption that is asserted rather than tested, and on a dreaming-direction validation that is partly circular. These issues are load-bearing for the central claim (c), so the current evidence is not yet conclusive.","major_comments":[{"comment":"The unbiasedness of the signal-vector estimator ŝ_i = cov[x_p, α_{p,i}]/var[α_{p,i}] requires that, within the subsample of patches containing concept i, α_{p,i} is uncorrelated with all other concept factors α_{p,j} and all distractor coefficients β_{p,f}. The paper states that these 'can be considered independent by assumption' after removing detection biases, but this is not generally true for natural concepts (e.g., patches containing 'car' are more likely to contain 'road' and less likely to contain 'sky'). L_fso encourages w_i ⊥ ŝ_j but does not remove correlations among the α factors, and URA is an indirect heuristic. This is load-bearing for claim (c). Please provide either an empirical test of the residual correlations on real features, or an alternative validation of signal-vector fidelity that does not rely on the learned decoding direction (e.g., compare with supervised Patte","section":"Section 4.3, Eq. (1)"},{"comment":"The 'dreaming direction' faithfulness check is partly circular. The direction is obtained by maximizing the learned decoding direction w_i, then fitting a line to the feature evolution during that optimization. Any signal vector lying near the span of w_i and the model's Jacobian will appear consistent with this direction, so high cosine similarity does not independently confirm that ŝ_i is the true encoding direction. The histograms (e.g., Figs. 11–12) report that ~90% of directions exceed 0.7, but no comparison is made to random directions or to directions learned by an independent method. This weakens claim (c) for real data. I recommend adding a non-circular validation, such as the supervised Pattern-CAV comparison from Section 7.11.4 applied to multiple concepts, or a baseline using random or shuffled directions to calibrate the cosine-similarity threshold.","section":"Section 7.2, Figures 9–13"},{"comment":"The influence metric I1 shows EDDP variants ranked last in most cases (e.g., ResNet18: I1=0.58–0.63 vs. PCA 0.73–0.74; EfficientNet: 0.94 vs. PCA 0.95). The paper interprets this as an interpretability–influence trade-off, but this is not established. Since the title and abstract emphasize 'concepts of influence', the lower I1 should be discussed more carefully or matched against the significant-direction metrics (SDC/SCDP) where EDDP-C improves with L_cur. Without this, the claim that the method recovers 'concepts of influence' in the unsupervised comparison is only weakly supported.","section":"Section 7.5, Tables 3–7"},{"comment":"The model-correction experiment is a convincing demonstration for a single, artificially injected concept, and the cosine similarity of 0.99 to the supervised Pattern-CAV is encouraging. However, this is a toy setup where the watermark is a low-dimensional, highly localized artifact. It does not resolve the concern that real-world concepts are correlated and high-dimensional. Please clarify how this evidence generalizes to the broader claim (c).","section":"Section 7.11.5 and Eq. (33)"}],"minor_comments":[{"comment":"Typo: 'In constrast' should be 'In contrast'. Also the caption lists four items but the numbering is confusing; please renumber or use clearer labels.","section":"Figure 8 caption"},{"comment":"Typo: 'EDPP' should be 'EDDP' in the description of the basis change for PCA and EDDP.","section":"Section 7.1"},{"comment":"The notation mixes a_{p,i}, α_{p,i}, and a_p,i. Please standardize to α_{p,i} throughout.","section":"Section 4.3"},{"comment":"The URA loss L_ur is defined with a negative sign: -E[H(f+(X'))]. Since maximizing entropy should reduce uncertainty, the sign convention should be stated explicitly in the text to avoid confusion.","section":"Section 4.4"},{"comment":"The comparison with CBE /w Luur is informative, but the notation 'Ortho' is not defined in the table; please add a footnote explaining the columns.","section":"Section A.11, Tables 24–25"}],"recommendation":"major_revision","confidential_remarks":"The paper is substantial and the synthetic/toy validations are strong, but the real-data faithfulness claim rests on an untested independence assumption and a partly circular dreaming-direction validation. I would support acceptance after the authors provide an independent, non-circular validation of signal-vector fidelity (e.g., multi-concept supervised Pattern-CAV comparison) or explicitly narrow the claimed scope of claim (c). The influence results also deserve a more careful treatment, but they are not fatal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this if you want a solid, practical take on unsupervised concept discovery in CNNs. The method (EDDP) learns encoding-decoding direction pairs in an end-to-end, unsupervised way, extending the authors' earlier UIBE/CBE work by dropping orthogonality constraints, adding new losses, estimating signal vectors, and introducing Uncertainty Region Alignment. That last piece—aligning the network's uncertain-prediction region with the concept detectors' ambiguous-decision region—is the most original move. The synthetic experiments are convincing: they recover ground-truth direction pairs with high cosine similarity, and the ablation shows the filter-signal orthogonality loss matters. The real-world evaluation is also unusually thorough: five architectures, multiple baselines, interpretability and influence metrics, and concrete applications from global explanations to a model-correction demo that actually fixes a watermark bias. That toy correction is a nice independent confirmation of the signal vector in one real case.\n\nThe main soft spot is exactly where the stress-test pokes. Section 4.3 estimates the signal vector from the covariance of positive subsamples, and the proof of unbiasedness requires that concept factors be independent within that subsample. The paper says this holds 'by assumption' after removing detection biases. That is asserted, not established. In natural images, concepts within positive examples are often correlated—cars co-occur with roads, skies are anti-correlated with indoor objects—so the estimator can pick up cross-concept contaminations. The Filter-Signal Orthogonality loss reduces filter leakage, but it does not remove correlation in the latent factors. The Deep Dream validation cannot rescue this: it maximizes the learned decoding direction, so the dreaming direction is not an independent ground truth and will likely align with a biased signal vector. This does not sink the method—the synthetic results and the watermark experiment show the estimator can be faithful in controlled settings—but it means claim (c) on real data is weaker than the paper lets on.\n\nTwo smaller issues: no code is released (a real handicap for a method this multi-step), and the comparisons to PCA/NMF use thresholds those methods were not designed for, which flatters EDDP in the interpretability tables. The authors acknowledge some of this in the limitations.\n\nOverall: this is a genuine contribution to tooling for interpretability, not a new scientific phenomenon. It deserves a serious referee and, after revision, could be a useful resource. I would not cite it in my own next paper until the independence question is at least tested with correlated synthetic concepts or a more independent real-world validation.","headline":"Useful, well-built unsupervised concept-direction tool with a genuine but addressable gap: the real-world encoding-direction estimate leans on an untested independence assumption.","tokens_in":58446,"tokens_out":1125,"would_cite":false,"duration_ms":13261,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An unsupervised method recovers the direction pairs that let deep vision networks encode and decode concepts, enabling concept attribution, counterfactual explanations, and model correction without labels.","keywords":["encoding-decoding direction pairs","concept detectors","signal vectors","directional clustering","uncertainty region alignment","concept interpretability","model correction","linear representation hypothesis"],"falsifier":"Construct a synthetic or semi-natural dataset where the positive patches of a concept always also contain a second concept (so the two latent factors are perfectly correlated in the subsampled set). If EDDP's signal vector for one of those concepts deviates substantially from its ground-truth encoding direction (cosine similarity well below the near-perfect values reported for the independent-factor synthetic experiment), the subsampling independence assumption is violated and the encoding-direction half of the central claim fails.","tokens_in":57512,"feed_emoji":"🧠","tokens_out":2786,"duration_ms":28881,"temperature":0.7,"pith_summary":"The paper argues that the latent mechanism by which deep vision networks write (encode) and read (decode) concept information into their representations can be recovered in an unsupervised way. For each concept, the method learns a decoding direction—a linear filter that acts as a concept detector—and an encoding direction (a signal vector) that estimates how the concept is written into embeddings. The decoding directions come from directional clustering of activations, the signal vectors from a probabilistic covariance estimator with a key subsampling step, and a novel Uncertainty Region Alignment loss steers the search toward directions the network actually uses. On synthetic data the method recovers ground-truth direction pairs; on real CNNs the learned decoding directions are highly monosemantic and often more interpretable than prior unsupervised baselines, while the signal vectors align with activation-maximization 'dreaming' directions, supporting their faithfulness. If correct, this gives annotation-free access to the concepts a model relies on, opening routes to global and local explanations and to interventions that change or correct predictions.","feed_headline":"Direction pairs reveal how vision networks encode concepts","feed_subtitle":"Unsupervised method finds concept detectors and signal vectors, enabling counterfactuals and model correction.","key_machinery":"The central objects are the encoding-decoding direction pair: a concept detector (a linear filter w_i plus bias b_i forming a binary classifier that reads the concept's latent factor) and a signal vector s_i (an estimate of the concept's encoding direction). The detector is learned via directional clustering with sparsity and maximum-activation losses, plus new interpretability losses (Focal Sparsity Loss, Excessively Active Classifier Loss) and a Self-Weighted Reduction soft-max aggregation. The signal vector is estimated from a multi-concept signal-distractor data model using the covariance formula s_i = cov[x_p, z_p,i] / var[z_p,i], crucially subsampling to patches that contain the concep","core_discovery":"Each concept in a deep vision network can be characterized by an encoding-decoding direction pair: an encoding direction (signal vector) that writes the concept's latent factor into a patch embedding, and a decoding direction (filter) that reads the factor back via an inner product. The paper's central claim is that these pairs can be learned jointly and unsupervised from unlabeled activations by (1) finding decoding directions through directional clustering that enforces sparsity in a soft-binary concept space, (2) estimating encoding directions with a subsampled covariance formula that, among patches containing the concept, treats latent factors of different concepts as independent, and (3","pith_inferences":["The observed interpretability-influence tradeoff (more monosemantic directions often scored lower on average sensitivity) hints that a fully faithful account of a model's decision strategy may require sacrificing human-friendly monosemanticity in favor of more abstract, multi-concept directions—an editorial inference beyond the paper's conclusions.","The subsampling independence assumption is most likely to break when concepts are strongly correlated or mutually exclusive within the positive subset (e.g., 'car' patches almost always contain 'road'); the paper only tests synthetic data with independent concept factors, so the method's real-world fidelity may degrade exactly in such common, entangled concept groups.","The method's reliance on directional clustering of activations suggests it could transfer to vision transformers or even non-image modalities whenever the linear representation hypothesis holds, but the paper does not test these settings and this remains an open extension.","The paper's model-correction example points toward a general recipe: discover a spurious concept's encoding direction unsupervisedly, then subtract it from representations during inference; because this requires no retraining, it could scale to auditing large pre-trained models, though the safety of such interventions appears unanalyzed."],"forward_implications":["If the recovered direction pairs are faithful, model predictions can be explained in terms of the concepts the model actually uses, without needing concept annotations or additional trained heads.","The encoding direction enables concept sensitivity testing (e.g., with RCAV) to identify which concepts the network relies on for each class, enabling global model-behavior diagrams.","The decoding direction and signal vector together support local, spatially-aware Concept Contribution Maps that break down a prediction logit into per-concept, per-patch contributions.","Interventions become possible: the signal value of a concept can be overwritten to produce counterfactual explanations (e.g., replacing 'road' with 'building' changes the prediction as expected).","The method can identify and suppress unintended biases: in a toy model, the watermark concept was detected with IoU 0.85, its signal vector matched a supervised Pattern-CAV with cosine similarity 0.99, and suppressing it recovered most of the poisoned-set accuracy.","The paper argues this provides many of the benefits of Concept Bottleneck Models without training additional components or accessing annotations."],"fun_headline_variants":["Encoding-decoding direction pairs expose deep vision concepts","Unsupervised method finds how vision nets encode concepts","Direction pairs unlock concept read/write in deep nets","Deep vision concepts read and written via direction pairs","How deep vision networks encode concepts: direction pairs"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The signal-vector estimate is valid only if, among patches that contain a concept, that concept's latent factor is independent of other concepts' latent factors and of distractor noise; the paper asserts this independence after subsampling rather than verifying it on real data, and it further assumes that the learned filters become approximately orthogonal to other signal directions and to the unestimated distractor subspace.","fun_headline_variants_meta":{"raw":{"variants":["Encoding-decoding direction pairs expose deep vision concepts","Unsupervised method finds how vision nets encode concepts","Direction pairs unlock concept read/write in deep nets","Deep vision concepts read and written via direction pairs","How deep vision networks encode concepts: direction pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000573,"raw_usage":{"total_tokens":2571,"prompt_tokens":798,"completion_tokens":1773,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":1714}},"tokens_in":542,"tokens_out":1773,"duration_ms":13343,"temperature":1.0,"reasoning_tokens":1714,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T14:37:24.489686+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a synthetic or semi-natural dataset where the positive patches of a concept always also contain a second concept (so the two latent factors are perfectly correlated in the subsampled set). If EDDP's signal vector for one of those concepts deviates substantially from its ground-truth encoding direction (cosine similarity well below the near-perfect values reported for the independent-factor synthetic experiment), the subsampling independence assumption is violated and the encoding-direction half of the central claim fails.","supporting_citations":[],"review_version":1}