{"id":"ac582f0b-adbf-401e-bd5f-ed09ef783677","arxiv_id":"2412.15698","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Concept boundary vectors are derived from the boundary between latent concept clusters, and the paper reports they capture semantic relationships better than concept activation vectors.","lead":"This paper introduces concept boundary vectors, a new way to extract interpretable directions from a neural network's latent space by using the boundary between concepts as the starting point. The authors report that, across several measures on MNIST and CIFAR10, these vectors outperform the standard concept activation vectors at capturing the meaning of a concept relationship.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CBV-vs-CAV logit-influence and adversarial comparisons may be a norm-scaling artifact: CAVs are never normalized to unit length, while CBVs are unit vectors by construction.","rationale":"The reader's verdict is CONDITIONAL, and I agree that the paper should not be accepted as-is; however, the most load-bearing issue is not the homogeneity of boundary normal vectors, but the normalization confound in the primary comparison. The logit-influence metric is directional and scale-dependent; if one vector has smaller Euclidean norm, it will score lower regardless of direction. CBVs are unit vectors by construction, while CAVs are unnormalized classifier weights. This is a concrete, testable flaw that, if confirmed, would directly undermine the main experimental support for the paper's central claim. The concept algebra experiment (Section 3.4) provides scale-invariant evidence, partially mitigating the concern, and the paper does state in Section 3.5 that CBV-based perturbations generalize to the full cluster, which is relevant. But the normalization issue is not addressed anywhere, and the absence of released code makes verification impossible. I therefore maintain CONDITIONAL status but with an explicit additional requirement: the authors must normalize CAVs (and CBVs) before computing scale-dependent metrics and report both raw and normalized results. The reader's homogeneity concern is also valid but secondary; it would become important only after the normalization issue is resolved, because a poorly-aligned CBV might still outperform a scale-disadvantaged CAV.","tokens_in":12345,"tokens_out":5585,"duration_ms":52282,"concrete_test":"Re-run the Section 3.2 logit-influence experiment and the Section 3.5 adversarial-construction experiment with all CAVs normalized to unit L2 norm before computing logit influence (Eq. 1) and before perturbing activations; do the same for CBVs if they are not already unit vectors. Compare the resulting distributions in Figures 2 and 8. If the CBV advantage shrinks or disappears, the central claim is an artifact of vector scale.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that concept boundary vectors (CBVs) are more effective than concept activation vectors (CAVs) rests heavily on the logit influence experiment (Section 3.2) and the adversarial construction experiment (Section 3.5). In both, the metric depends on the scale of the concept vector. Equation (1) computes a directional derivative of the logit along v; this scales linearly with ||v||. Similarly, the adversarial perturbation in Section 3.5 requires a latent perturbation aligned with v, and the input-space distance to reclassification decreases as ||v|| increases. CBVs are optimized to maximize average cosine similarity to the unit boundary normals N± (Section 2.2, footnote 3), so they are effectively unit vectors. CAVs are obtained as the weight vector of a logistic regression classifier (Section 3.1) with no stated normalization. Without normalization, a CAV with smaller norm will mechanically yield lower logit influence and require larger input perturbations. Figures 2 and 8, which appear to show CBV superiority, could therefore reflect nothing more than the relative magnitudes of the two vectors. The paper does not report normalized CAVs anywhere in the main text or appendix, and the code is not yet released for verification. This concern is distinct from the boundary-normal homogeneity issue raised by the reader: even if N± are perfectly homogeneous, the comparison is still scale-confounded. The concept algebra experiment (Section 3.4) uses cosine similarity and is scale-invariant, so it provides some independent support; however, the logit-influence and adversarial experiments are the primary quantitative evidence for the headline claim, and their validity as stated is doubtful.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces concept boundary vectors (CBVs), a concept-vector construction that extracts pairs of latent activations lying on the boundary between two concepts, forms unit boundary-normal vectors from these pairs, and optimizes a single unit vector to maximize average cosine similarity to them. The authors compare CBVs against concept activation vectors (CAVs) on a CNN trained on MNIST and a vision transformer fine-tuned on CIFAR-10, using logit influence, concept algebra, adversarial reclassification, and topological analyses of concept clusters and boundaries. The central claim is that CBVs are more effective than CAVs at encoding the relationship between concepts and better capture the geometry of the latent boundary.","tokens_in":12691,"tokens_out":3961,"duration_ms":34323,"significance":"If the empirical claims hold, CBVs would be a useful new tool for model interpretability, and the paper's use of topological data analysis to check the homogeneity and linear-separability assumptions underlying concept vectors is a valuable direction. The paper also identifies a clear limitation of CAVs and proposes a geometrically motivated alternative. However, the current evidence is not yet convincing because the main comparative experiments are confounded by vector-norm scaling, lack statistical rigor, and some evaluations reuse the same boundary points used in construction. The significance of the contribution therefore depends on whether these confounds can be addressed with additional experiments.","major_comments":[{"comment":"The logit influence metric in Eq. (1) and the adversarial-perturbation magnitude in Section 3.5 both scale linearly with the Euclidean norm of the concept vector. CBVs are unit vectors by construction, whereas CAVs are unnormalized logistic-regression weight vectors. Without normalizing CAVs, the observed superiority of CBVs in Figures 2 and 8 could simply reflect the relative norms of the two vectors. Please rerun these experiments with CAVs normalized to unit norm (or with norms matched), and report the norms of the unnormalized CAVs so readers can assess the confound.","section":"§3.2, Eq. (1) and §3.5"},{"comment":"The statement that the negative correlation between logit influence and boundary complexity is 'significant at the 5% level' is not backed by any statistical details: no test name, p-value, confidence interval, or sample size is reported. Since logit influence is scale-dependent and CAVs are unnormalized, this correlation may also be confounded by vector norms. Please provide the full statistical analysis and repeat the correlation with normalized concept vectors.","section":"§4.1, Figure 11"},{"comment":"The CBV optimization objective maximizes average cosine similarity to the set of boundary normal vectors N±, which presupposes that N± is sufficiently homogeneous and approximately unimodal. The paper tests homogeneity for the concept clusters (assumption A2) in Section 4.2, but never tests the homogeneity of N± itself. If the boundary normals are multimodal or highly dispersed, the optimized single vector may not align with any local boundary, undermining the method's rationale. Please add a dispersion or clustering analysis of N± (e.g., circular variance, silhouette scores, or persistent homology of N±) and discuss the results.","section":"§2.2 and §4.2"},{"comment":"The adversarial-reclassification experiment in Figure 7 evaluates on the same boundary pairs P± that were used to construct the CBV, making this part of the evaluation in-sample and potentially circular. Although Figure 8 extends the analysis to the entire cluster of latent activations, the paper should clearly separate the in-sample boundary-point results from the out-of-sample full-cluster results, and report the out-of-sample performance on non-boundary points separately.","section":"§3.5"},{"comment":"Several load-bearing comparisons lack error bars and significance tests. The claims that CBVs have 'greater' logit influence, are 'more successful' in concept algebra, and require 'lower amplitude' adversarial perturbations are made without quantifying run-to-run variability from CAV training and CBV optimization. Please provide repeated-run statistics (e.g., mean ± std over multiple seeds) and appropriate significance tests for these comparisons.","section":"Figures 2, 6, 8, and 11"}],"minor_comments":[{"comment":"There are typos, e.g., 'commmonly' in the Introduction and 'extententangled' in Section 3.3; a careful proofread is needed.","section":"Abstract and §1"},{"comment":"The code is stated to be available 'here' but no URL is visible in the manuscript; please include a working link or supplementary material.","section":"Footnote 4 and §7.3"},{"comment":"The finding that CBVs are more similar to boundary normal vectors than CAVs is expected by construction, since CBVs are explicitly optimized for that similarity; consider presenting this only as a sanity check of the optimization objective, not as evidence of representational quality.","section":"§7.3, Figure 16"},{"comment":"The consistency-error formula appears to use v^{ℓ1}_± in both terms of the norm; if the second term is meant to use v^{ℓ2}_±, please correct the typo.","section":"§7.3.1"},{"comment":"The experimental details do not report hyperparameters for the CBV optimization (e.g., learning rate, number of optimization steps, initialization) or for Algorithm 1 beyond the statement that 10,000 optimizations are run; please list these values for reproducibility.","section":"§3.1 and §2.2"}],"recommendation":"major_revision","confidential_remarks":"The norm-scaling confound is the most serious issue and, if confirmed, would invalidate the central comparative claim as currently presented. However, it is fixable by normalizing CAVs and rerunning the experiments, so I do not recommend rejection at this stage. The lack of code and incomplete statistical reporting are also significant but addressable. I encourage the editor to seek the revised version with these analyses before making a final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe new thing here is a concept vector built by first finding nearest-neighbor pairs across two concept clusters and then fitting a unit vector to maximize cosine similarity to those boundary normals. That construction is not in the CAV/CAR literature, and the paper is honest that it is a variation on that program. The topological analysis of assumption A2 (concept homogeneity) is a genuine plus; using TARDIS Euclidicity to probe whether a single direction can capture a concept is more than most concept-vector papers do.\n\nThe problem is the headline comparison. Logit influence (eq. 1) is a directional derivative along v, so it scales linearly with ||v||. The adversarial perturbation experiment has the same dependence. CBVs are unit vectors by construction; CAVs are logistic-regression weight vectors with no stated normalization. Figures 2 and 8 therefore may show nothing more than a norm difference. The stress-test note is correct, and the paper does not report normalized CAVs anywhere. This is not a minor issue; it is the main quantitative evidence for the central claim.\n\nOther soft spots: Figure 16 is tautological (CBV is optimized against N±, so it is not evidence). Section 3.5 uses boundary pairs both to construct and evaluate, although Figure 8 extends to the whole cluster, which partly mitigates overfitting. Figures 2, 6, 8, 11 lack error bars and test statistics; the 'significant at the 5% level' in Section 4.1 is not backed by numbers. The boundary-normal homogeneity assumption is never directly tested, only the cluster homogeneity. Code is not actually released despite the footnote.\n\nCredit where due: the concept algebra experiment (Section 3.4) uses cosine similarity and is scale-invariant, and it gives independent evidence that CBVs behave better. The ViT consistency analysis is interesting and not scale-confounded. So the method may be genuinely better; the paper just has not shown it with the current comparisons.\n\nThis is a subfield-level contribution for interpretability researchers, and it deserves a serious referee -- the construction is sound enough to warrant heavy-revision peer review, not desk reject. For my own work, I would not cite it until the norm issue is resolved and code is out. Bring it to reading group? Maybe, since the scale confound is a good teaching example, but the paper itself is not ready.","headline":"The new construction is plausible, but the headline CBV-vs-CAV comparisons are confounded by vector norm, and the independent evidence does not yet rescue the central claim.","tokens_in":13184,"tokens_out":2498,"would_cite":false,"duration_ms":22251,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces concept boundary vectors, constructed from the boundary between latent representations of concepts, and shows empirically that they encode concept-concept relationships more effectively than concept activation vectors.","keywords":["concept boundary vectors","concept activation vectors","latent space geometry","interpretability","logit influence","concept algebra","topological data analysis","vision transformer"],"falsifier":"Take a concept pair whose boundary normal vectors split into two or more well-separated clusters with distinctly different directions. If the homogeneity assumption fails, the single optimized boundary vector will align poorly with each cluster, and its logit influence on the target class should drop to the level of (or below) a concept activation vector. Concretely, cluster $N_{\\pm}$ by cosine similarity into two groups, construct per-group boundary vectors, and compare their logit influences against the single global boundary vector; a large gap would falsify the claim that one boundary vector suffices.","tokens_in":12167,"feed_emoji":"🧠","tokens_out":6174,"duration_ms":45861,"temperature":0.7,"pith_summary":"This paper introduces concept boundary vectors, a construction that derives a concept's direction in a neural network's latent space from the boundary between two concepts' latent activations. Rather than training a classifier to separate positive and negative examples, the method pairs up mutually nearest activations across the two sets, normalizes their differences into boundary normal vectors, and optimizes a single vector to have maximum average cosine similarity with those normals. The paper claims this boundary-derived vector encodes the concept-concept relationship more faithfully than concept activation vectors, and tests the claim on an MNIST classifier and a CIFAR-10 vision transformer using logit influence, concept algebra, and adversarial perturbation experiments. A sympathetic reader would care because concept vectors are a standard interpretability device, so a more faithful construction could make model explanations more trustworthy.","feed_headline":"Boundary vectors beat activation vectors at encoding concepts","feed_subtitle":"Deriving concept vectors from boundary geometry improves logit influence, concept algebra, and reclassification.","key_machinery":"The load-bearing object is the set of boundary normal vectors $N_{\\pm}$, obtained from mutually nearest-neighbor pairs $(a_+, a_-)$ across the positive and negative concept activation sets, normalized to unit length. A single vector is then optimized to maximize the average cosine similarity with $N_{\\pm}$, and that optimized vector is the concept boundary vector. This construction directly respects the geometry of the separating boundary, whereas a concept activation vector is the weight vector of a logistic regression classifier and is agnostic to where the boundary points actually lie. The paper also relies on the logit influence metric, the average directional derivative of the target-class logit along the concept vector, as the main quantitative measure of how well a concept vector represents a concept relationship.","core_discovery":"The central claim is that the geometry of the boundary between latent concept clusters is a major carrier of semantic relationship, and that a vector explicitly aligned with boundary normals captures that relationship better than the weight vector of a separating linear classifier. Concept boundary vectors are defined by first constructing the set $N_{\\pm}$ of unit vectors pointing from a negative activation to its paired positive activation at the boundary, then optimizing a single vector $v_{\\pm}$ to maximize the average cosine similarity to $N_{\\pm}$. Empirically, these vectors produce higher logit influence on the target class and lower influence on the source class, satisfy concept algebra $v_{1,2} + v_{2,3} \\approx v_{1,3}$ more often, and generate smaller adversarial perturbations that still flip the model's classification. The paper further shows that logit influence decreases as the topological complexity of the boundary increases, and that concept vectors are more effective when concept activations are internally homogeneous.","pith_inferences":["Because the paper validates homogeneity for concept clusters but never for the boundary normal vectors themselves, a direct extension is to fit a mixture of boundary vectors per concept pair when $N_{\\pm}$ is multimodal; this would test whether a single vector is always sufficient.","The same boundary-normal construction could be applied to unsupervised feature dictionaries, aligning sparse autoencoder features to boundary normals rather than to activation directions, potentially yielding more interpretable feature-concept correspondences.","The negative correlation between logit influence and boundary complexity hints that training with margin or boundary-sharpening losses could make latent spaces more concept-vector-friendly, a consequence the paper leaves implicit.","The boundary-pairing step is quadratic in the number of activations, so scaling to larger concept sets would require approximate nearest-neighbor search; which boundary pairs are kept would then affect the resulting vector."],"forward_implications":["Concept boundary vectors give a higher logit influence on the target class and a stronger suppression of the source class than concept activation vectors on nearly all MNIST digit pairs.","Linear concept algebra succeeds more often with concept boundary vectors, meaning the vector sum $v_{1,2} + v_{2,3}$ lands closer to $v_{1,3}$ than it does for activation vectors.","Adversarial inputs built from concept boundary vectors need smaller perturbations to reclassify boundary latents, and that improvement extends to the whole concept cluster, not just the boundary points.","Across layers of a vision transformer, concept boundary vectors are more consistent between layers than concept activation vectors, with consistency error peaking at the middle layers.","Topological boundary complexity is negatively correlated with concept vector effectiveness, supporting the paper's claim that boundary geometry matters for concept representations."],"supporting_citations":[{"why":"Defines concept activation vectors and the TCAV score that the paper uses as its baseline and extends to the logit influence metric.","marker":"[10]"},{"why":"Introduces concept activation regions and motivates geometry-aware concept constructions, the direct predecessor the paper builds on.","marker":"[15]"},{"why":"Supplies the topological data analysis method used to measure boundary complexity and relate it to logit influence.","marker":"[16]"},{"why":"Formulates the linear representation hypothesis, which is assumption A1 for concept vector constructions.","marker":"[17]"},{"why":"Provides the consistency error metric and the methodology for translating latent perturbations into input space.","marker":"[27]"},{"why":"Supplies the Euclidicity measure used to test the homogeneity assumption A2 on concept activations.","marker":"[28]"}],"fun_headline_variants":["Boundary geometry yields better concept vectors","Concept boundary vectors outperform activation vectors","Boundary normals capture concepts more effectively","New concept vectors derived from boundary normals","Boundary-based concept vectors improve concept encoding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The construction assumes that the boundary normal vectors between two concepts are homogeneous enough that one single vector, chosen to maximize average cosine similarity, can faithfully represent the concept relationship.","fun_headline_variants_meta":{"raw":{"variants":["Boundary geometry yields better concept vectors","Concept boundary vectors outperform activation vectors","Boundary normals capture concepts more effectively","New concept vectors derived from boundary normals","Boundary-based concept vectors improve concept encoding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000526,"raw_usage":{"total_tokens":2483,"prompt_tokens":836,"completion_tokens":1647,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":452,"completion_tokens_details":{"reasoning_tokens":1584}},"tokens_in":452,"tokens_out":1647,"duration_ms":11404,"temperature":1.0,"reasoning_tokens":1584,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:09:31.793910+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a concept pair whose boundary normal vectors split into two or more well-separated clusters with distinctly different directions. If the homogeneity assumption fails, the single optimized boundary vector will align poorly with each cluster, and its logit influence on the target class should drop to the level of (or below) a concept activation vector. Concretely, cluster $N_{\\pm}$ by cosine similarity into two groups, construct per-group boundary vectors, and compare their logit influences against the single global boundary vector; a large gap would falsify the claim that one boundary vector suffices.","supporting_citations":[{"cited_title":"Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCA V), June","cited_arxiv_id":null,"evidence_quote":"Defines concept activation vectors and the TCAV score that the paper uses as its baseline and extends to the logit influence metric."},{"cited_title":"Topological Data Analysis of Decision Boundaries with Application to Model Selection","cited_arxiv_id":"1805.09949","evidence_quote":"Supplies the topological data analysis method used to measure boundary complexity and relate it to logit influence."},{"cited_title":"Topological Singularity Detection at Multiple Scales","cited_arxiv_id":"2210.00069","evidence_quote":"Supplies the Euclidicity measure used to test the homogeneity assumption A2 on concept activations."}],"review_version":1}