{"id":"3916f8b9-d95d-448d-96b8-12e0403f5c59","arxiv_id":"2505.20293","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ECO-Concept uses slot attention plus LLM-based comprehensibility feedback to learn explainable text concepts without concept annotations.","lead":"This paper introduces ECO-Concept, a system that finds human-readable concepts in text without needing any labeled concept examples, then uses large language models to judge and improve how understandable those concepts are. It reports that the discovered concepts are more comprehensible to people than those from earlier concept-based methods while keeping classification accuracy close to black-box models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The LLM highlight target in Eq. (9) is validated on the model's own concepts after training, so the causal role of LLM feedback in improving human comprehensibility is not yet isolated; a human-rationale or random-highlight control is needed.","rationale":"The reader's weakest assumption identified the LLM highlight matrix S in Eq. (9) as load-bearing, and I agree that this is the critical premise. I partially agree because I would sharpen the concern in two ways. First, Appendix B validates LLM highlights on the model's own post-training concepts, which makes the validation partially self-confirming: the model was optimized to match those highlights, so human agreement with them does not prove they are good targets for other models or for the base model before enhancement. Second, the mechanized 'comprehensibility' improvement may be partly driven by the rule in Section 3.4 that zeroes S for concepts the LLM cannot summarize, effectively suppressing concepts rather than making them more understandable; the paper's metrics and human studies do not separate these two effects. I give credit for the genuine human studies, especially forward simulatability, which is a behavioral measure that is harder to game than subjective agreement. However, forward simulatability is run on only two datasets and lacks a control condition that replaces the LLM target with a non-semantic baseline. The proposed test with human rationales and random highlights would directly settle whether the specific content of S matters, or whether any target that forces sparse, simplified attention would produce comparable human ratings. I would keep the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT, because the concern is real but the paper has enough independent human evidence to remain plausible pending the control experiment.","tokens_in":17367,"tokens_out":6193,"duration_ms":72556,"concrete_test":"Collect token-level human rationales on a held-out sample of concepts from CEBaB and Beer (e.g., 50 concepts, 3 independent annotators, with inter-annotator agreement reported). From the same Base model, train three variants identical except for the target S in Eq. (9): (a) original LLM highlights, (b) human rationales, (c) random binary highlights with the same density as (a). Then run the paper's forward simulatability and subjective comprehensibility ratings with new participants on held-out concepts. If (a) is not significantly better than (c), or if (b) significantly outperforms (a), the specific LLM target is not the driver of comprehensibility and the central claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Eq. (9), where the comprehensibility loss matches slot attention A to LLM-generated highlights S, and sets S to zero for concepts the LLM deems semantically meaningless. The load-bearing assumption is that S is a valid training target for human comprehensibility. Appendix B does not establish this: it only measures human agreement with LLM summaries and highlights on the model's own final concepts, which have been trained to match those same LLM outputs. High agreement can therefore reflect self-consistency in the training loop rather than the validity of S as a human-comprehensibility signal. The quantitative concept metrics in Table 2 (Semantics, Distinctiveness, Consistency) are likewise produced with LLM summarization and evaluation, so the main automatic evidence is partly circular with the training signal. The one non-circular result, forward simulatability, covers only Beer and AGnews, and no control separates the effect of genuinely reshaping concepts from the effect of suppressing ambiguous concepts (S=0 for 'meaningless' concepts), which can inflate apparent comprehensibility by deleting hard cases. Thus the paper has not yet isolated that LLM feedback, rather than the suppression mechanism or additional fine-tuning, causes the human-rated comprehensibility gains.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ECO-Concept, an intrinsically interpretable framework for text classification that discovers concepts without any concept annotations. It uses a slot-attention concept extractor to obtain M concept prototypes, regularized by consistency and distinctiveness losses. A second stage evaluates concept comprehensibility with an LLM that summarizes the top-activated exemplars per concept and highlights concept-related segments; the MSE between the model's slot attention and the LLM highlight matrix is used as a comprehensibility loss to fine-tune the model. Experiments on seven text datasets compare ECO-Concept with black-box, supervised, and unsupervised baselines, and report classification accuracy/F1, LLM-based concept quality metrics (Semantics, Distinctiveness, Consistency), and human studies (intruder detection, subjective ratings, forward simulatability).","tokens_in":17713,"tokens_out":3188,"duration_ms":32295,"significance":"If the central claim holds, the paper makes a valuable contribution by incorporating human-comprehensibility feedback into the training loop of an unsupervised concept discovery method, without needing concept annotations. The design is coherent, and the human evaluations (intruder detection, subjective ratings, forward simulatability) are a genuine attempt to measure what the title promises. The paper also ships a project page with code, includes parameter sensitivity and robustness analyses, and attempts to control for additional fine-tuning in the ablation of §4.4. However, the LLM-in-the-loop design creates a circularity concern that must be addressed before the claim that LLM feedback improves human comprehensibility can be accepted.","major_comments":[{"comment":"The central claim that the LLM-based comprehensibility loss makes concepts more human-understandable is partially circular. The loss trains the model to match LLM-generated highlights S, and then the Semantics and Distinctiveness metrics in Table 2 are obtained by summarizing and evaluating concepts with the same (or closely related) LLM. Appendix B validates the LLM as a human proxy by asking humans to rate agreement with LLM summaries and highlights on the model's own final concepts, which have themselves been trained to match those LLM outputs; high agreement can therefore reflect self-consistency in the training loop rather than the validity of S as a target for human comprehensibility. I ask for a control that breaks this loop, for example training with human-annotated rationales or with random highlights of the same density, and reporting whether the human-rated comprehensibility gains persist. Without such a control, the paper has not isolated that LLM feedback—rather than the suppression mechanism in Eq. (9) or merely longer fine-tuning—causes the gains.","section":"§3.4, Eq. (9), §4.3, Appendix B"},{"comment":"The paper states in §4.2 that ECO-Concept shows \"significant improvements\" over unsupervised baselines with pairwise t-tests at a 95% confidence level, but it does not report the t-statistics, p-values, degrees of freedom, or any multiple-comparison correction. Tables 1 and 2 report only point estimates without standard deviations or confidence intervals, which is insufficient to support the significance claim for a method whose metrics (especially Semantics and Distinctiveness) are produced by a stochastic LLM. Since the human studies in Tables 3-4 also report only point accuracies, the statistical evidence for the method's superiority is incomplete.","section":"§4.2, Tables 1 and 2"},{"comment":"The ablation does not isolate the causal role of the LLM feedback. In Eq. (9), concepts judged semantically meaningless by the LLM are assigned S=0, which drives their activations toward zero; suppressing ambiguous concepts could by itself raise the Semantics and Distinctiveness scores by removing hard cases. The current comparison with the w/o L_com variant controls for additional fine-tuning, but it does not control for the suppression mechanism or for the information added by the positive highlights (S=1 tokens). A control that fine-tunes with only the suppression term (or with random highlight targets of the same density) is needed to attribute the human-rated comprehensibility gains to the LLM-generated highlighting specifically.","section":"§4.4, Fig. 4, Eq. (9)"}],"minor_comments":[{"comment":"The word \"adaption\" should be \"adoption\" or \"adaptation\".","section":"Abstract"},{"comment":"The notation k(k-1) in the denominators of the consistency and distinctiveness losses is unclear because k is only defined in the surrounding text; the summation ranges over pairs (u_m, u'_m) and pairs of average features, which the equations do not make explicit.","section":"§3.2, Eqs. (3)-(4)"},{"comment":"When |B_m ∩ D_high| is zero for some concept m, the denominator in Eq. (9) would be zero; the paper should state how such concepts are handled.","section":"§3.4, Eq. (9)"},{"comment":"The criterion for deciding that a concept's meaning has \"remained unchanged\" during the iterative re-summarization is not specified; this could matter for reproducibility.","section":"§3.4, Training Strategies"},{"comment":"The human evaluation results are reported as point estimates without confidence intervals or significance tests across participants; given the small sample sizes (42 and 24 participants), the spread should be reported, especially for the forward simulatability claim in Table 4.","section":"§4.3, Tables 3-4, Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about circularity is valid and lands on a load-bearing point: the LLM serves both as the training target in Eq. (9) and as the evaluator in Table 2, and Appendix B does not break that loop. I would like the authors to add the missing control (e.g., random-highlight or suppression-only fine-tuning) and to report full statistics for Tables 1-4. The paper is otherwise a well-executed study with genuine human evaluations, and with these additions it could become a solid contribution to the unsupervised concept-based explanation literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, this paper is worth a serious look. The central idea—use LLM comprehensibility judgments as a training signal to reshape slot-attention concepts in text—is new to me and it works about as well as claimed. The authors put in real effort on evaluation: seven tasks, multiple baselines, and three kinds of human studies including forward simulatability. The forward simulatability gains on Beer and AGnews are meaningful despite the small sample. The ablation w/o L_com is the right control: it shows that the improvement isn't just from extra fine-tuning. I believe the core claim that LLM-guided enhancement improves human comprehensibility is plausible and largely supported.\n\nThe soft spots are real but not fatal. First, Tables 1–3 are point estimates with no error bars or confidence intervals, and the pairwise t-tests named in §4.2 are never shown. That's fixable but should be asked for. No code is released either. Second, the reviewer's circularity concern has teeth: the same kind of LLM serves as teacher for the highlight target S in Eq. (9) and as the scorer for Semantics/Distinctiveness in Table 2. The human intruder detection and ratings break the loop, but the automatic metrics could still be inflated. Third, the stress-test note about S=0 for 'meaningless' concepts is on target—I haven't seen a control that separates 'making concepts clearer' from 'deleting hard concepts.' A random-highlight target or a human-rationale-based target would settle this.\n\nOne thing I'd push back on: the claim that LLM highlights are validated as a human proxy via Appendix B is indeed too weak, but the human studies later in the paper do show that the final concepts are more comprehensible. So I wouldn't call the central argument circular; I'd call the mechanism-understanding incomplete.\n\nWho is this for? Anyone working on concept-based interpretability or LLM-guided representation learning. It deserves peer review—send it to a careful referee with a request for code, error bars, and the proposed controls. My verdict: conditional accept at best, but definitely not a desk reject.","headline":"A genuinely novel LLM-in-the-loop concept discovery method with credible human studies, but the causal role of the LLM feedback is not fully isolated and statistical reporting is thin.","tokens_in":18206,"tokens_out":2091,"would_cite":true,"duration_ms":21384,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that using LLM feedback as a training loss makes automatically discovered text concepts more comprehensible to humans while keeping accuracy on par with black-box models.","keywords":["concept-based explanations","unsupervised concept discovery","slot attention","LLM as human proxy","text classification","interpretability","comprehensibility loss","explainable AI"],"falsifier":"An experiment that would settle it: recruit humans to rate or simulate two models trained identically except that one uses $\\mathcal{L}_{\\mathrm{com}}$ and the other does not (or uses random highlights). If the $\\mathcal{L}_{\\mathrm{com}}$ model's concepts are not rated as more comprehensible or do not improve simulation accuracy, the central claim fails. Also, if replacing the GPT-4o highlights with a deliberately poor highlighter still improves human ratings, then the improvement is not caused by LLM alignment.","tokens_in":17174,"feed_emoji":"🧠","tokens_out":5518,"duration_ms":54250,"temperature":0.7,"pith_summary":"ECO-Concept aims to build text classifiers that explain their own predictions through concepts discovered from data, with no human-written concept labels. The paper claims that concept comprehensibility can be made an explicit training objective: a large language model summarizes each discovered concept and highlights which tokens belong to it, and the model is fine-tuned to minimize the gap between those highlights and its own slot attention. On seven classification datasets, the resulting concepts are meant to be both task-useful and human-understandable, matching or exceeding black-box accuracy while beating existing unsupervised and post-hoc concept methods in human evaluations. The value of the claim is that interpretability no longer has to be justified after the fact or paid for with performance.","feed_headline":"LLM feedback makes text AI concepts human-readable","feed_subtitle":"New method matches black-box accuracy while beating concept baselines in human studies on seven tasks.","key_machinery":"The central object is the ECO-Concept feedback loop joining a slot-attention concept extractor with an LLM concept evaluator. Slot attention makes up to $M$ trainable prototypes compete to explain encoded tokens, producing a concept attention matrix $A$ and concept features $U$. A linear classifier predicts from total concept activations, creating a concept bottleneck. The evaluator selects highly activated exemplars, prompts an LLM to summarize each concept and highlight related tokens (matrix $S$), and defines the comprehensibility loss $\\mathcal{L}_{\\mathrm{com}} = \\frac{1}{M} \\sum_m \\beta_m \\, \\mathrm{MSE}(A_{m,:}, S_{m,:})$, with $\\beta_m$ encoding concept importance. This loss, together with consistency and distinctiveness regularizers, carries the argument: it converts \"human-understandable\" into a differentiable objective.","core_discovery":"The central claim is that comprehensibility is not a byproduct of concept learning but a trainable signal. ECO-Concept extracts concepts via slot attention over text tokens, turns the summed attention into a concept bottleneck for classification, and then uses LLM-generated concept summaries and token highlightings as a pseudo-ground truth. The comprehensibility loss $\\mathcal{L}_{\\mathrm{com}}$ compares the model's concept attention matrix to the LLM highlight matrix, weighted by each concept's importance (average activation times classifier weight magnitude), and this loss is added to cross-entropy, consistency, and distinctiveness losses. The paper reports that across CEBaB, Beer, Hotel, IMDB, AGnews, Twitter, and SciCite, this procedure yields classification accuracy at or above black-box and supervised concept models and concept evaluation scores higher than Cockatiel, Concept-Shap, and ProtoTEx, and that human subjects detect intruders, rate quality, and simulate model outputs better with ECO-Concept explanations.","pith_inferences":["A direct test the paper does not run: compare human-rated comprehensibility of the same model with $\\mathcal{L}_{\\mathrm{com}}$ on versus off in a randomized A/B study. If the loss only reshapes concepts without raising human agreement or simulation accuracy, the mechanism would not be responsible for the reported gains.","The LLM-as-proxy idea could generalize beyond ECO-Concept: any concept-based explainer that produces token-level or segment-level activations could be fine-tuned with the same summarization-highlighting loop, including prototype networks or vision concept models.","The paper's fixed concept count and API-cost constraint suggest an obvious extension: adaptive concept discovery (add or merge concepts during training) combined with cheaper open LLM evaluators would determine whether the comprehensibility gains persist when the evaluator is weaker.","The importance weighting $\\beta_m$ means the framework trades off comprehensibility of low-importance concepts; an implicit consequence is that the model may be allowed to keep obscure low-importance concepts if they help accuracy."],"forward_implications":["Any text classification dataset without concept annotations gets built-in, global concept explanations; the four annotation-free datasets in the paper are direct evidence.","Because the LLM is consulted only during training, deployed inference runs entirely on the local RoBERTa-based model, incurring no extra API cost at prediction time.","The framework's concept bottleneck keeps accuracy competitive with black-box classifiers, so the interpretability gain does not require a large accuracy trade-off.","The comprehensibility enhancement stage improves concept semantics, distinctiveness, and consistency over the base slot-attention model, and it can be applied after a first phase of regular concept training."],"supporting_citations":[{"why":"Supplies slot attention, the mechanism the concept extractor adapts from object-centric learning to text concepts.","marker":"Locatello et al., 2020"},{"why":"Supplies the auto-interpretability paradigm of using LLMs to explain neurons and predict activations, which the concept evaluator adapts to concepts.","marker":"Bills et al., 2023"},{"why":"Extends LLM-based interpretability to large-scale feature extraction; cited as the prior basis for LLMs as human proxies.","marker":"Templeton et al., 2024"},{"why":"Defines the concept bottleneck architecture that ECO-Concept uses for its linear classifier on total concept activations.","marker":"Koh et al., 2020"},{"why":"Prior slot-attention concept learning in vision that ECO-Concept builds on for text.","marker":"Wang et al., 2023"},{"why":"Another prior object-centric concept learning work that motivates the slot-attention concept extractor.","marker":"Hong et al., 2024"},{"why":"Unsupervised self-explaining baseline (SelfExplain) that ECO-Concept compares against.","marker":"Rajagopal et al., 2021"},{"why":"Unsupervised prototype-based baseline (ProtoTEx) used for concept and human evaluation comparisons.","marker":"Das et al., 2022"},{"why":"Post-hoc concept method (Cockatiel) used as a comparison baseline.","marker":"Jourdan et al., 2023"},{"why":"Post-hoc concept method (Concept-Shap) used as a comparison baseline.","marker":"Yeh et al., 2020"}],"fun_headline_variants":["LLM feedback loop turns AI concepts human-readable","Unsupervised concept discovery with LLM quality checks","ECO-Concept: LLM guidance makes concepts comprehensible","AI explanations get clearer via LLM-scored concepts","No labels needed: LLM helps AI concepts make sense"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that what the LLM highlights as concept-related tokens is a trustworthy target for what humans find comprehensible, and that pushing slot attention toward those highlights makes explanations better rather than merely different.","fun_headline_variants_meta":{"raw":{"variants":["LLM feedback loop turns AI concepts human-readable","Unsupervised concept discovery with LLM quality checks","ECO-Concept: LLM guidance makes concepts comprehensible","AI explanations get clearer via LLM-scored concepts","No labels needed: LLM helps AI concepts make sense"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000449,"raw_usage":{"total_tokens":2245,"prompt_tokens":905,"completion_tokens":1340,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1263}},"tokens_in":521,"tokens_out":1340,"duration_ms":11350,"temperature":1.0,"reasoning_tokens":1263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:55:07.033665+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that would settle it: recruit humans to rate or simulate two models trained identically except that one uses $\\mathcal{L}_{\\mathrm{com}}$ and the other does not (or uses random highlights). If the $\\mathcal{L}_{\\mathrm{com}}$ model's concepts are not rated as more comprehensible or do not improve simulation accuracy, the central claim fails. Also, if replacing the GPT-4o highlights with a deliberately poor highlighter still improves human ratings, then the improvement is not caused by LLM alignment.","supporting_citations":[],"review_version":1}