{"id":"215f33e9-dfde-483f-9742-324bd52872b8","arxiv_id":"2412.07992","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"CB-LLMs add a concept bottleneck layer to large language models, achieving black-box-level classification and generation quality while enabling concept detection, unlearning, and steering.","lead":"Researchers built a framework, CB-LLMs, that adds human-interpretable concept layers to large language models, allowing text classification and generation to be explained and controlled by named concepts. The work is a candidate path toward safer, more steerable LLMs because it replaces some post-hoc explanations with built-in ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pseudo-concept labels used to supervise the bottleneck are constructed with the ground-truth class label (Eq. 2), so neuron faithfulness may reflect class membership rather than the named concept; the central interpretability claim is not yet established.","rationale":"The paper's central contribution is intrinsic interpretability: neurons in the bottleneck are supposed to correspond to human-understandable concepts, and this is what enables faithful explanations, steering, and safety claims. The most load-bearing assumption is therefore that the automatically generated concept scores are faithful proxies for human concept presence. The reader's weakest_assumption correctly identifies this. My stress-test sharpens it with a specific mechanism: ACC in Eq. (2) injects the ground-truth class label directly into the training targets by zeroing out all cross-class concept scores. Consequently, the CBL is trained to reproduce class-conditioned concept patterns, and the subsequent human faithfulness evaluation filters out disconfirming evidence because top-activated samples come from the same class as the concept. This is not an accusation of fraud; it is a structural property of the training pipeline. The paper does provide meaningful supporting evidence: released code, large-scale experiments, human studies, and an adversarial disentanglement module that plausibly improves steerability. The generation perplexity numbers are also worse than the black-box on three of four datasets, but the interpretability claim is more central than the exact generation-quality parity claim. Because the concern is specific, empirically testable, and addressable by a human-annotation validation study, the appropriate verdict remains CONDITIONAL rather than REJECT or ACCEPT. My read does not move the reader's verdict, hence UNCHANGED.","tokens_in":23869,"tokens_out":2913,"duration_ms":33994,"concrete_test":"Run a human-concept validation study on 500 YelpP test reviews: have annotators label the presence or absence of 50 concepts, including concepts from both the positive and negative classes (e.g., 'Overpriced' appearing in a positive review). Compare CB-LLM neuron activations for each concept against human presence labels using AUC or calibration error. If neurons achieve high discriminative accuracy for cross-class concept occurrences, ACC is not the source of the alignment; if activations track class labels rather than human concept presence (e.g., high activation whenever the review is negative, regardless of the concept), the faithfulness claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CB-LLMs provide faithful, human-understandable explanations because bottleneck neurons align with named concepts. That claim stands or falls on the quality of the concept labels used to train the CBL. In Section 3.1 Step 3, Automatic Concept Correction (ACC, Eq. 2) sets every concept score to zero unless the concept belongs to the sample's ground-truth class. The CBL is then trained to reproduce these class-conditional targets. This means the training signal for each neuron is not 'does this concept appear in the text?' but 'does this concept appear in a text of this class?' A neuron can therefore learn a class-membership detector that happens to correlate with the concept name, without genuinely representing the concept's semantics. The human evaluation in Task 1 only inspects top-activated samples for each neuron, and because the training target zeroes out cross-class concepts, those top samples are almost guaranteed to come from the neuron's associated class; negative evidence is filtered out by construction. The generation setting compounds this: Section 4.2 uses dataset class labels directly as concept labels, so 'concept detection' is a restated classification objective, not evidence of independent concept understanding. The paper does include a partial validation of one concept ('overpriced') with an NLI model, but that covers a single concept and does not address the general label-fidelity problem. If these pseudo-labels are not faithful proxies for human concepts, the interpretability, faithfulness, and steerability claims lose their foundation, even though the classification accuracy numbers are internally consistent.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Concept Bottleneck Large Language Models (CB-LLMs), a framework that inserts a human-interpretable concept bottleneck layer into pretrained LLMs for both text classification and text generation. For classification, concepts are generated with ChatGPT, scored by sentence-embedding similarity, corrected using ground-truth class labels (ACC), and used to train a bottleneck layer whose activations feed a sparse linear classifier. For generation, a hybrid design combines the concept bottleneck with an unsupervised layer, trained with an adversarial disentanglement objective to keep concept information out of the unsupervised path, enabling concept detection, steering, and toxicity reduction. Experiments on SST2, YelpP, AGnews, and DBpedia report classification accuracy near or above fine-tuned black-box models, human faithfulness ratings above a TBM&C3M baseline, and generation accuracy, steerability, and perplexity close to a fine-tuned Llama3-8B. The paper claims to be the first CBM that scales to large text classification and text generation while providing intrinsic interpretability.","tokens_in":24241,"tokens_out":4626,"duration_ms":46745,"significance":"If the central interpretability claim is valid, this is a significant step toward intrinsically interpretable LLMs. The method is fully automatic, avoids per-sample LLM labeling at inference, and scales to datasets with hundreds of thousands of examples. The paper also contributes a novel adversarial disentanglement mechanism for hybrid CBMs in generation, and it provides multiple case studies (concept unlearning, toxicity detection and steering) with concrete validation of one unlearning behavior via an NLI model. The authors released code, and the appendix contains substantial additional results across backbones and datasets. The main weakness is that the evidence for the faithfulness of the learned neurons is incomplete: the generation concept-detection metric is circular, and the ACC target construction may allow neurons to encode class membership rather than semantic concepts. These issues are load-bearing because the paper's headline contribution is 'faithful interpretability.'","major_comments":[{"comment":"The reported 'concept detection accuracy' in the generation setting is circular. The concept labels y_c used in Eq. (5) are simply the dataset class labels (as stated in §4.2, e.g., world, sport, business, and technology news for AGnews), and the CBL is trained with cross-entropy to predict these labels. The 'Accuracy' row in Table 5 is therefore the test accuracy of the same classifier that defines the concept-detection objective; high numbers only show that the CBL memorizes the training objective, not that the neurons detect concepts in a way that is independent of the class label. This metric does not support the conclusion that 'the interpretable neurons behave as expected.' Please evaluate concept detection on concepts that are not identical to the class labels, or provide human-annotated concept-presence labels on held-out examples to measure alignment.","section":"§4.2, Eq. (5), Table 5"},{"comment":"Automatic Concept Correction (ACC) zeros out the concept score for every concept whose associated class differs from the sample's ground-truth class. As a result, the training target for each neuron is class-conditional: a neuron can achieve high similarity in Eq. (3) by learning a class-membership detector rather than the semantic content of its named concept. The human faithfulness study in Task 1 (Appendix A.7) only presents the top-activated samples for each neuron; since cross-class samples have zero target by construction, those top samples are almost guaranteed to come from the neuron's own class, so high ratings in Table 3 do not rule out class-membership coding. The NLI check for the 'overpriced' concept in §3.3 is encouraging but covers one concept only. To support the central interpretability claim, the paper should test neurons on samples from other classes (e.g., how strongly a 'sports' neuron activates on business or world news), or validate a random sample of neurons against concept-level human labels.","section":"§3.1, Eq. (2)"},{"comment":"The TBM&C3M baseline is re-implemented with only 1,000 labeled samples per dataset (labeled by Llama3-8B-Instruct), because the original methods' per-sample LLM labeling is cost-prohibitive. This means the accuracy and faithfulness comparisons in Tables 2-4 are against a heavily data-restricted approximation of TBM and C3M, not against the methods as originally proposed. The text's claim that CB-LLMs 'significantly outperforming TBM&C3M' (Section 3.1, Step 3) is therefore not established on equal footing. Please train the baseline on the full training sets, or at least on matched subsets with the same number of labeled examples as used for CB-LLMs, and clearly state the training data size in the table captions.","section":"§3.2 Setup, Table 2, Tables 3-4"},{"comment":"The conclusion that CB-LLMs 'achieve performance nearly on par with black-box LLMs (within 1% gap)' is not supported for generation quality. The perplexity values in Table 5 show large differences: 116.22 vs 84.70 on SST2 (about 37% worse) and 13.03 vs 6.62 on YelpP (about 97% worse), with AGnews also worse (18.25 vs 12.52). The text dismisses these differences by saying 'a small difference in perplexity would not affect generation quality,' but these are not small differences. Please report confidence intervals or a statistical test for perplexity, and revise the generation-quality claim accordingly; the accuracy and steerability results can still stand, but the 'nearly on par' phrasing should be qualified to reflect the actual generation quality gap.","section":"Table 5, §4.2 'Generation Quality', §5"}],"minor_comments":[{"comment":"The heading 'Efficency' contains a typo; it should be 'Efficiency.'","section":"§3.2"},{"comment":"The human study description says a random baseline was evaluated for Task 1, but the random baseline numbers are not reported in Table 3; please include them for context, since they are essential for interpreting the absolute activation-faithfulness ratings.","section":"Appendix A.7, Table 3"},{"comment":"The text says TBM requires multiple queries to GPT-4 for each text sample, but the re-implementation used Llama3-8B-Instruct on 1,000 samples; please clarify that the quoted cost and the implemented baseline are different.","section":"Section 2, Related Work"},{"comment":"The symbol '∥' is used for concatenation but is not defined in the text; please define it the first time it appears.","section":"Eq. (6)"},{"comment":"The 'cos cubed' similarity is mentioned but not defined; please provide the formula or a citation to the source (Oikarinen et al., 2023).","section":"Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published as an ICLR 2025 conference paper, but the journal version should still address the circularity in the generation concept-detection metric and the class-conditional nature of ACC, because these directly affect the validity of the interpretability claims. The TBM&C3M baseline issue is also important: the comparison is against a 1,000-sample reimplementation, which is not the original method. These are fixable with additional experiments and revised claims, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely useful systems paper, and the interpretability claim is weaker than the abstract suggests. The stress-test note is right about ACC, and the paper's own numbers undercut the \"within 1%\" generation-quality claim. Still, the engineering is real and the code is out, so it deserves referee time.\n\nWhat's new: the first concept-bottleneck framework that scales to large text classification (500k+ samples) and to autoregressive generation, using a hybrid CBL with an unsupervised layer plus adversarial disentanglement. The ACS labeling with sentence embeddings is cheap and practical, and the adversarial module clearly improves steerability over the no-ADV ablation. Classification results are solid: accuracy matches black-box models on four benchmarks with multiple backbones, and the MTurk study shows people prefer its explanations over the TBM/C3M reimplementation. The concept-unlearning case study with NLI validation is the right kind of check.\n\nSoft spots: the stress-test concern lands. In Eq. 2, ACC zeros out every concept score unless the concept belongs to the sample's ground-truth class, so the CBL learns class-conditional scores rather than \"does this concept appear in the text.\" The top-activated-sample faithfulness test is then partly circular, because negative evidence is filtered by construction. The NLI check covers only one concept. This does not kill the classification engineering, but it means the faithful-interpretability claim is not established.\n\nIn the generation setting, the concept labels are just the dataset class labels, so the \"concept detection\" accuracy in Table 5 is classification accuracy, not independent evidence of concept understanding. Also, the perplexity table contradicts the text: SST2 CB-LLM is 116.22 versus 84.70 for the black-box model, and even the no-ADV version is 59.19. That is not \"similar.\" It is a minor blow to the generation-quality story, not to the steerability story.\n\nMinor issue: TBM&C3M is a reimplementation on only 1,000 samples, not the original pipeline. That is a defensible cost decision, but the comparison should be labeled as such.\n\nWho this is for: people building interpretable or steerable LLM pipelines, especially for safety tooling. The code and recipes are useful even if the faithfulness claims need tightening. I would accept this for peer review and ask for revisions on label fidelity, perplexity reporting, and claim softening. I would cite it as the scalable CBM-for-LLM reference.","headline":"Real engineering contribution with a weaker-than-advertised interpretability claim; worth reviewing, but the class-conditioned pseudo-labels and contradictory perplexity numbers need addressing.","tokens_in":24772,"tokens_out":2155,"would_cite":true,"duration_ms":24104,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Inserting a concept bottleneck layer into pretrained LLMs gives near-black-box classification and generation quality while exposing interpretable, steerable concepts.","keywords":["concept bottleneck models","interpretable machine learning","large language models","text classification","text generation","concept steering","concept unlearning","toxicity reduction"],"falsifier":"Swap the ChatGPT concept lists between two classes before training a CB-LLM, keeping everything else identical; if the resulting model still achieves the same accuracy and human raters still find its neuron explanations equally faithful, then the concept names are not semantically load-bearing and the interpretability claim fails. A more direct test would replace the automatically generated concept scores with human-annotated concept labels on the same dataset and check whether neuron-concept alignment and explanation faithfulness improve.","tokens_in":23695,"feed_emoji":"🧠","tokens_out":7289,"duration_ms":66571,"temperature":0.7,"pith_summary":"This paper proposes Concept Bottleneck Large Language Models (CB-LLMs), a way to turn a pretrained black-box LLM into an inherently interpretable model by inserting a human-interpretable concept bottleneck layer before the prediction layer. For text classification, the authors claim CB-LLMs match or slightly beat fine-tuned black-box models on SST2, Yelp, AGnews, and DBpedia while producing explicit concept-based explanations that human raters judge more faithful than prior interpretable-text models. For text generation, they claim a hybrid design with an adversarially disentangled unsupervised layer lets CB-LLMs detect concepts, steer generation toward a chosen concept, and reduce toxicity, with generation quality and perplexity close to a fine-tuned black-box Llama3-8B. The paper concludes that interpretability need not come with the usual accuracy cost: the reported gap to black-box LLMs is within 1 percent, and the interpretable neurons enable concept unlearning and controllable responses.","feed_headline":"Concept-bottleneck LLMs match black-box quality within 1 percent","feed_subtitle":"A concept layer lets users detect, steer, and unlearn concepts without sacrificing performance.","key_machinery":"The load-bearing object is the Concept Bottleneck Layer (CBL): a linear layer whose k neurons are each trained to reproduce a named concept score, so that each neuron's activation has a human-readable meaning and its contribution to the final prediction is a linear weight times the neuron activation. Concept scores are produced by Automatic Concept Scoring (ACS), which uses cosine similarity between a sentence embedding of the text and embeddings of ChatGPT-generated concept phrases, and by Automatic Concept Correction (ACC), which uses the ground-truth class label to zero out scores for concepts belonging to other classes. For generation, the machinery adds an unsupervised parallel layer and an adversarial training module: a linear classifier tries to predict concepts from the unsupervised layer's features while the unsupervised layer is trained to maximize entropy of that classifier, forcing concept information to live only in the interpretable CBL and making neuron interventions effective.","core_discovery":"The central claim is that a pretrained LLM can be made intrinsically interpretable without sacrificing task performance by training a Concept Bottleneck Layer (CBL) whose neurons are forced to align with automatically generated, human-readable concepts. Concept lists come from a single ChatGPT query per class; each text's concept scores are computed as cosine similarities between the text embedding and concept embeddings; an Automatic Concept Correction step zeroes out scores whose concepts belong to a different class than the sample's ground-truth label; and the backbone plus CBL are trained to reproduce these corrected scores before a sparse linear layer maps the ReLU'd activations to class logits. In the generation setting, the paper claims the same CBL structure plus an unsupervised parallel layer trained adversarially to remove concept information yields a model whose interpretable neurons can detect the topic or toxicity of a prompt, whose generation can be steered by setting neuron activations, and whose output quality is essentially on par with a black-box fine-tuned model. On the four classification datasets, CB-LLMs with ACC reach 0.94-0.99 accuracy, matching or occasionally exceeding the fine-tuned black-box baseline; in generation, concept detection accuracy is within 1 percent of a directly fine-tuned concept classifier, steerability scores range from 0.76 to 0.95, and perplexity is comparable to the black-box model.","pith_inferences":["If the concept labels are derived from class-level ChatGPT lists and sentence-embedding similarities, then for richer or open-ended generation tasks the 'concepts' are likely to be coarse and surface-level; extending the approach to concept sets built from domain ontologies or human annotation would test whether the interpretability holds beyond class-like categories.","The adversarial disentanglement mechanism is a general recipe: it could be applied to any hybrid bottleneck architecture to prevent an unsupervised residual path from silently carrying the information that the interpretable path is supposed to own, and its success suggests information leakage is the main threat to bottleneck faithfulness.","A natural next experiment is to measure whether the concept bottleneck's explanations support human debugging in practice, e.g., whether users who can intervene on neurons achieve higher task accuracy than users of a black-box model with post-hoc explanations, which the paper does not directly test."],"forward_implications":["CB-LLMs (classification) reach the accuracy of fine-tuned black-box models on SST2, Yelp, AGnews, and DBpedia, with ACC closing the gap by about 3.5 percent on average.","CB-LLMs (generation) can detect the concept of a prompt from its neuron activations, with accuracy within 1 percent of a directly fine-tuned concept classifier.","Setting a target concept neuron to a high activation steers generation toward that concept; steerability scores range from 0.76 to 0.95, far above a model trained without adversarial disentanglement.","The framework supports concept unlearning: deactivating a neuron or removing its weights shifts predictions in the intended direction, e.g., 79 percent of reviews that flipped after unlearning 'overpriced' strongly entail that concept.","Toxicity detection in a chatbot variant reaches 0.9996 accuracy, and users can steer responses between benign and toxic by intervening on four interpretable neurons."],"supporting_citations":[{"why":"introduces the concept bottleneck model structure that the paper adapts to LLMs, supplying the core architecture.","marker":"[5]"},{"why":"the previous interpretable text classification approach (TBM) whose scalability limits motivate this work and which serves as a baseline.","marker":"[8]"},{"why":"the C3M baseline that combines human-annotated and ChatGPT concepts for text classification and is compared against.","marker":"[17]"},{"why":"label-free CBM work that supplies the cos-cubed similarity objective used for training the bottleneck layer.","marker":"[10]"},{"why":"ChatGPT used to generate per-class concept lists, the source of the concept vocabulary the bottleneck neurons are trained to match.","marker":"[11]"},{"why":"supplies the all-mpnet-base-v2 sentence embedding model used in Automatic Concept Scoring.","marker":"[18]"},{"why":"Llama3 model used as backbone for generation experiments and as the black-box baseline.","marker":"[1]"}],"fun_headline_variants":["CB-LLMs: concept-level interpretability without accuracy loss","See inside the LLM: concept neurons for transparent AI","Steer, detect, unlearn: concept bottleneck LLMs in action","Interpretable LLMs that match black-box performance","Concept bottleneck LLMs: control and transparency for AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's interpretability rests on the assumption that ChatGPT-generated concept lists and text-similarity scores from an embedding model faithfully capture the human concepts that actually drive each text; if these scores misrepresent human concepts, the named neurons are not truly aligned with their labels and the faithfulness, detection, and steering claims collapse.","fun_headline_variants_meta":{"raw":{"variants":["CB-LLMs: concept-level interpretability without accuracy loss","See inside the LLM: concept neurons for transparent AI","Steer, detect, unlearn: concept bottleneck LLMs in action","Interpretable LLMs that match black-box performance","Concept bottleneck LLMs: control and transparency for AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000257,"raw_usage":{"total_tokens":1604,"prompt_tokens":996,"completion_tokens":608,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":526}},"tokens_in":612,"tokens_out":608,"duration_ms":6099,"temperature":1.0,"reasoning_tokens":526,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:20:12.447400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Swap the ChatGPT concept lists between two classes before training a CB-LLM, keeping everything else identical; if the resulting model still achieves the same accuracy and human raters still find its neuron explanations equally faithful, then the concept names are not semantically load-bearing and the interpretability claim fails. A more direct test would replace the automatically generated concept scores with human-annotated concept labels on the same dataset and check whether neuron-concept alignment and explanation faithfulness improve.","supporting_citations":[],"review_version":1}