{"id":"2f8a12b4-aa24-476f-a7bd-1793b23e7387","arxiv_id":"2411.16895","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new XAI method clusters a network's most-confused labels into a WordNet-labeled hierarchy, and its experiments suggest larger models form more human-aligned concepts.","lead":"This paper proposes an explainable AI method that builds a family tree of the categories a neural network confuses, based on the top few labels it assigns to each image. The authors argue this tree reveals the concepts a network has learned, and that larger models form more human-like hierarchies than efficient ones.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NMA's 'latent concepts' are built only from correctly classified test images, so the inferred hierarchy may reflect selection bias, calibration artifacts, or dataset co-occurrence rather than the network's actual decision process; this unvalidated proxy is the central weak point.","rationale":"The reader's weakest assumption correctly identifies the top-K softmax probabilities as a potentially unfaithful proxy for internal concept organization. My stress-test sharpens this into a concrete, falsifiable flaw: the implementation departs from the formal definition by using only correctly classified images, and the robustness conclusion is disconnected from any experiment. This supports the reader's CONDITIONAL verdict rather than changing it, because the issue is an unvalidated design choice and missing controls, not an internal contradiction that invalidates the method outright. The proposed concrete test would settle whether the selection filter is responsible for the reported hierarchies, and a secondary comparison with internal activations would address the proxy validity more directly. My disagreement with the reader is only partial: I emphasize the correct-only sampling discrepancy and the untested robustness inference, which are more specific than the broad statement about softmax fidelity. A revised paper that fixes the sampling, adds precision and error bars, and includes a white-box sanity check would move toward ACCEPT; without these, CONDITIONAL remains the appropriate verdict.","tokens_in":12898,"tokens_out":4526,"duration_ms":47573,"concrete_test":"For ResNet50 on the same 41-label ImageNet subset, recompute the connections graph and hierarchical clustering using (a) all test images, as Section 3.1 formally defines, and (b) only correctly classified images, as Section 4 implements. Compare the resulting dendrograms at the level of the clusters described in Section 5.1 (around 'distance 15') using adjusted Rand index or cophenetic correlation. If cluster memberships differ materially between (a) and (b), the inferred 'latent concepts' are an artifact of the correct-only selection, undermining the claim that NMA reveals a stable latent hierarchy. If stable, compare the softmax-based dendrogram with one built from cosine distances of penultimate-layer activations to test whether softmax confusions actually reflect the network's internal structure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that NMA reveals a hierarchy of logical concepts from a network's latent decision-making without inspecting its internals. The entire hierarchy is derived from the confusion matrix pi,j defined in Section 3.1, where pi,j is the average probability of label j over images of label i. However, the experiments (Section 4, Implementation) state: 'For each correctly classified test image, a NMA was performed and the appropriate weighted edge was updated in the connections graph.' This replaces the formal definition with a conditional average over correct predictions only. Excluding misclassified images removes the very cases where the model's label competition may be most diagnostic, and the resulting dendrogram is therefore dependent on an arbitrary filtering choice that the method section does not justify. Additionally, even with this filtering, no validation shows that softmax confusions track internal representations; the same hierarchy could arise from low-level visual similarity (e.g., background objects, texture), from softmax miscalibration, or from dataset biases such as class co-occurrence. The robustness claim in the abstract is never tested experimentally: the paper contains no adversarial attacks, no perturbation analysis, and no comparison with internal activations, so the leap from dendrogram structure to 'price of explainability and robustness' is speculative.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Near-Misses Analysis (NMA), a black-box explanation method that constructs a hierarchical clustering of class labels from the top-K softmax probabilities of a trained image classifier on test images. The resulting dendrogram is interpreted as the model's latent concept hierarchy, and clusters are assigned human-readable names via WordNet hypernyms. A query explanation is a path from the predicted label to the root, and a score measures overlap between the machine-generated concept set and a WordNet-based human annotation. The method is applied to ResNet50, EfficientNetB0, VGG16, and MobileNetV2 on ImageNet (via miniImageNet) and CIFAR100. The authors report architecture-specific dendrograms and scores, and conclude that more complex architectures form more human-compatible concepts, suggesting a trade-off between efficiency and explainability/robustness.","tokens_in":13118,"tokens_out":5755,"duration_ms":51682,"significance":"If the central claim were validated, NMA would offer a lightweight, black-box window into the conceptual organization learned by a classifier, complementary to white-box methods like TCAV. The method is simple, requires only output probabilities, and ships with code and a UI, which are strengths. However, the paper currently provides no evidence that softmax confusions correspond to internal representations, and the robustness conclusion is entirely speculative. The evaluation metric is recall-only and derived from the same WordNet resource used for naming, so the quantitative results largely measure WordNet consistency. The paper's significance therefore hinges on additional validation that is currently absent.","major_comments":[{"comment":"The definition of p_{i,j} in Section 3.1 averages over all images with ground-truth label i, but the experimental protocol in Section 4 states that NMA is performed 'for each correctly classified test image' and only those edges are updated. This conditional average over correct predictions is a different quantity and will change the graph weights whenever the model misclassifies some images of class i; the paper neither justifies this filtering nor analyzes its effect. Because the entire hierarchy is derived from these weights, the reported dendrograms depend on an arbitrary and undocumented choice.","section":"§3.1 and §4"},{"comment":"The abstract and Section 3 claim that NMA reveals concepts 'inferred from the latent decision-making process' of the network, yet Section 6 explicitly defers white-box testing to future work. There is no comparison with internal activations, no probing, and no ablation showing that the dendrograms are not reproducible from low-level visual similarity or dataset co-occurrence. To support the central interpretability claim, the authors should include at least one of: (a) a comparison with hierarchies built from penultimate-layer activation similarities, (b) an ablation on a model with randomized final layers, or (c) a controlled dataset experiment demonstrating that the inferred clusters track the network's learned concept groups.","section":"§1, §3, and §6"},{"comment":"The abstract and Section 6 assert that efficient architectures 'pay the price of explainability and robustness in terms of concepts generation,' but no robustness experiment is performed. Section 5.1 contains the speculative statement that 'an adversarial image might easily cause it to confuse one object with another.' The paper should either include adversarial attack or perturbation experiments to support this claim, or explicitly remove the robustness claim from the abstract and conclusions.","section":"§1, §5.1, and §6"},{"comment":"The explanation score φ is defined as |S_i ∩ U_i| / |S_i|, which is recall-only and never penalizes extra concepts in the machine-generated explanation. Moreover, both S_i and U_i are derived from WordNet hypernyms: the 'human annotation' is not an independent human judgment but the same lexical hierarchy used for the automatic concept naming. The score therefore measures consistency with WordNet rather than explainability to users. A proper evaluation would include a precision term, a comparison against a baseline clustering (e.g., a random label dendrogram or a static WordNet tree), or a human study.","section":"§3.4, Eq. (1)"},{"comment":"The method has several free parameters—K=3, t=10^{-6}, the hierarchical linkage criterion, and the hand-picked 41-label subset—and no sensitivity analysis is reported. In particular, with t=10^{-6}, the 'otherwise ∞' case in the edge weight definition in Section 3.1 is unlikely to be triggered for any pair of labels on ImageNet, making the threshold effectively irrelevant; the authors should report the distribution of p_{i,j} values and show that the architectural differences in the dendrograms are stable under reasonable variation of K and t.","section":"§3.1, §4, and §5"}],"minor_comments":[{"comment":"The author line contains typos: 'A vivit Levy' and 'Collee of Engineering.Design.Art' should be corrected.","section":"Authors and affiliations"},{"comment":"The statement that 'The test set used for creating NNs explanations on ImageNet is the miniImageNet benchmark dataset' is ambiguous because miniImageNet is a few-shot learning benchmark, not a standard ImageNet test set; clarify its source, size, and how it was used as a test set.","section":"§4"},{"comment":"The 'Depth' column should define what is being counted (e.g., number of Keras layers, trainable layers, or total layers), and the 'Size (MB)' measurement should be applied consistently across models.","section":"Table 1"},{"comment":"Interpretations such as 'container ship is clustered with air transport probably due to its large size' are post-hoc and unsupported by quantitative evidence; the authors should either provide image statistics or attention maps to support such claims or present them as hypotheses.","section":"§5.1"},{"comment":"Equations (1) and (2) are not numbered in the text and the notation for S_i and U_i is introduced only in prose; please number the equations and clarify whether the 'human annotated explanations' are WordNet hypernym paths or independently collected human judgments.","section":"§3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a borderline major-revision/reject case. The NMA idea is simple, novel, and potentially useful for black-box explanation, and the authors provide code and a UI. However, the central claim that the method reveals latent decision-making concepts is unsupported by internal validation, the robustness conclusion is entirely speculative, and the evaluation metric is recall-only and circular with respect to WordNet. I recommend requesting a major revision with additional experiments rather than rejecting, because the identified weaknesses are addressable: the authors could add activation-based comparisons, adversarial attacks, sensitivity analyses, and non-circular baselines within the scope of the paper. Please also ask the authors to clarify the provenance and role of the miniImageNet test set."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper builds a label hierarchy from top-K softmax confusions and names clusters with WordNet hypernyms. The idea is simple, the implementation is clean, and the code is available. But the abstract's claims about revealing latent concepts and paying a robustness price go beyond what the experiments support. The most concrete problem is that the formal definition of the connection graph (Section 3.1) uses all test images, while the implementation (Section 4) only updates edges for correctly classified images. That filtering is arbitrary and could materially change the resulting dendrogram. It is not justified anywhere in the paper.\n\nWhat's genuinely useful: the paper demonstrates that different architectures produce qualitatively different dendrograms on the same test set, and the discussion of those differences is thoughtful. The authors also show some honesty by noting when a cluster is likely caused by a dataset artifact (broccoli inside a wok). The shared code and demo video are a plus.\n\nThe soft spots are real and several are load-bearing. First, there is no validation that softmax confusions track internal representations; the hierarchy could as easily reflect visual similarity or dataset co-occurrence. The paper even provides an example of that. Second, the robustness claim is pure speculation—no adversarial attacks, no perturbation analysis, nothing. Third, the evaluation uses a hand-picked subset of 41 labels, a recall-only score (intersection over human hypernyms, with no precision), and no error bars or significance tests. Since both the machine naming and the human ground truth use WordNet, the score mostly measures consistency with WordNet rather than any independent notion of explainability. Fourth, the use of miniImageNet as the test set for ImageNet-trained models is an odd choice that is not discussed.\n\nWho is this for? Practitioners wanting a quick black-box visualization of label relationships might find the tool useful as an exploratory aid. As a research contribution, however, it is a plausible starting point rather than a demonstrated method. It deserves a serious referee because the idea is accessible and the architecture comparison is a useful hook, but it needs major revision: fix the correct-class-only discrepancy, validate against internal representations or tone down the claim, report precision alongside recall, and test robustness or drop it.\n\nMy recommendation: send it to peer review, but brace the authors for heavy revision. The core method is not novel, but the application direction has potential.","headline":"A clear, reproducible confusion-based concept hierarchy that overclaims its ability to reveal latent concepts; the robustness conclusion is not supported by the experiments.","tokens_in":13635,"tokens_out":3874,"would_cite":false,"duration_ms":39661,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the near-miss labels a classifier almost chooses reveal a hierarchy of abstract concepts the network has learned, and that efficient architectures achieve similar accuracy at the cost of less human-aligned, less…","keywords":["near-miss analysis","explainable AI","concept hierarchy","hierarchical clustering","neural network interpretability","model efficiency","ImageNet","CIFAR100"],"falsifier":"Train the same architectures on ImageNet with labels randomly permuted inside each WordNet coarse class so that near misses are semantically meaningless, then run NMA: if the dendrogram still produces clean human concepts at comparable distances, the hierarchy is an artifact of dataset statistics rather than a trace of the network's latent decision-making.","tokens_in":12643,"feed_emoji":"🧠","tokens_out":7594,"duration_ms":66993,"temperature":0.7,"pith_summary":"This paper proposes that the labels a classifier almost chooses—the top few probabilities below its first choice—form a readable map of how the network organizes what it knows. Averaging those near-miss probabilities over correctly classified test images produces a weighted graph over labels, which is then turned into a dendrogram and named with WordNet hypernyms. On ImageNet and CIFAR100, four architectures yield markedly different hierarchies. The paper's central empirical claim is that accuracy alone hides a trade-off: the compact EfficientNet and MobileNet reach top-5 accuracy close to ResNet but form less human-aligned, more visually accidental concepts, while the shallow, parameter-heavy VGG competes with deep ResNet. If correct, this gives a black-box window into concept formation and a quantitative score for comparing models without touching their internals.","feed_headline":"Near misses expose the concept tree hidden inside a neural net","feed_subtitle":"Clustering the labels a classifier almost picks reveals how architectures differ in concept formation, and what efficiency costs.","key_machinery":"The central object is the K-near-miss set: the top K labels in the sorted output probability vector of a classifier, plus a cutoff variant that keeps all labels with probability above a threshold $t$. For each pair of labels $(i,j)$, the method averages $P(y=j\\mid X=x)$ over all correctly classified test images of label $i$, calling this $p_{i,j}$, and uses its complement $1-p_{i,j}$ as an edge weight in a connections graph when $p_{i,j}\\ge t$. Floyd-Warshall all-pairs shortest paths turn the graph into a metric, a hierarchical clustering routine turns the metric into a dendrogram, and a WordNet lowest-common-hypernym lookup names each cluster. The mechanism reads concept structure off output probabilities alone, with no access to weights, activations, or gradients.","core_discovery":"The paper's core claim is that the ranked probability vector of a classifier, and especially the top few labels it does not choose, is a trace of the concepts the network has formed. Averaging these near misses over all correctly classified test images yields a weighted connections graph; shortest paths on that graph define a metric, hierarchical clustering converts the metric into a dendrogram, and naming each cluster by its lowest common WordNet hypernym turns the dendrogram into sentences such as 'the black swan is part of water birds.' The paper reports that four architectures queried on the same images form markedly different hierarchies: ResNet's aligns best with human hypernym structure, VGG matches ResNet on animals and even separates cats from dogs more cleanly, while EfficientNet and MobileNet form visually or accidentally based clusters (for example grouping broccoli with woks because of a training image). The authors take these differences as evidence that efficient networks 'pay the price' of explainability and robustness in concept generation, and that the common assumption favoring deep, thin architectures over wide, shallow ones may be wrong.","pith_inferences":["The method's sensitivity to dataset artifacts—EfficientNet's broccoli-in-wok cluster—suggests it could be used to audit training data for spurious correlations, an application the paper does not develop.","The same near-miss graph could be inverted for adversarial purposes: the dendrogram identifies which labels are conceptually close, so an attacker could push an image toward a neighboring cluster and expect high confusion.","A natural validation step, absent from the paper, would be to test whether the dendrogram aligns with layer-by-layer internal representations, which would turn NMA into a white-box probe for locating where concepts are formed."],"forward_implications":["Different architectures will generally exhibit different concept dendrograms on the same data, so NMA can be used as an architecture-comparison tool before deployment.","A model can match another in top-5 accuracy yet be less explainable and more fragile: in the reported experiments, ResNet scored 0.89, VGG 0.82, EfficientNet 0.61, and MobileNet 0.52 on the proposed explanatory measurement.","Concept quality does not track depth alone; the shallow but parameter-heavy VGG outperformed the deeper EfficientNet and MobileNet, suggesting that thin fully-connected layers may limit abstraction.","The score $\\phi(M)$ gives a quantitative, human-anchored proxy for explainability, allowing models trained on the same label set to be ranked on concept alignment."],"supporting_citations":[{"why":"Supplies the concept-based XAI framing (TCAV) that this paper extends from single concepts to a learned hierarchy of concepts.","marker":"Kim et al. 2018"},{"why":"Establishes that structured and concept learning improves adversarial robustness, the link the paper invokes to argue efficient models' weaker concepts carry a robustness price.","marker":"Galloway, Taylor, and Moussa 2018"},{"why":"Defines the EfficientNet architecture and the scaling philosophy that motivates the comparison of efficient versus large models.","marker":"Tan and Le 2019"},{"why":"Provides the common assumption that lower layers detect visual features while upper layers assemble concepts, which the VGG result is used to challenge.","marker":"Bau et al. 2017"},{"why":"Motivates the value of hierarchical concept learning by showing its superiority in few-shot tasks.","marker":"Mao et al. 2019"},{"why":"Supplies the miniImageNet benchmark used as the ImageNet test sample for building all explanations.","marker":"Fifty et al. 2023"}],"fun_headline_variants":["Near misses expose the hidden concept tree of AI models","What a neural net almost chooses reveals its inner concepts","Efficient networks pay an explainability price in concept learning","Near-miss analysis lays bare the concepts a network forms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the average softmax probabilities of near-miss labels, taken only from correctly classified test images, faithfully represent the network's internal concept organization rather than visual similarity, dataset bias, or calibration artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Near misses expose the hidden concept tree of AI models","What a neural net almost chooses reveals its inner concepts","Efficient networks pay an explainability price in concept learning","Near-miss analysis lays bare the concepts a network forms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000951,"raw_usage":{"total_tokens":4031,"prompt_tokens":893,"completion_tokens":3138,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":3073}},"tokens_in":509,"tokens_out":3138,"duration_ms":22501,"temperature":1.0,"reasoning_tokens":3073,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:45:21.289242+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architectures on ImageNet with labels randomly permuted inside each WordNet coarse class so that near misses are semantically meaningless, then run NMA: if the dendrogram still produces clean human concepts at comparable distances, the hierarchy is an artifact of dataset statistics rather than a trace of the network's latent decision-making.","supporting_citations":[{"cited_title":"Predicting Adversarial Examples with High Confidence","cited_arxiv_id":"1802.04457","evidence_quote":"Establishes that structured and concept learning improves adversarial robustness, the link the paper invokes to argue efficient models' weaker concepts carry a robustness price."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the EfficientNet architecture and the scaling philosophy that motivates the comparison of efficient versus large models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the common assumption that lower layers detect visual features while upper layers assemble concepts, which the VGG result is used to challenge."},{"cited_title":"B.; and Wu, J","cited_arxiv_id":null,"evidence_quote":"Motivates the value of hierarchical concept learning by showing its superiority in few-shot tasks."}],"review_version":1}