{"id":"6960c0e9-9ece-4ba7-91c7-4badcc17124a","arxiv_id":"2608.07594","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Training a language model with a built-in concept bottleneck preserves compute-optimal scaling and yields interpretability metrics that improve with scale, demonstrated on an 8B causal diffusion model.","lead":"Interpretability is normally added after training; this paper builds it into the training objective of an 8B language model and shows the model stays competitive while its outputs can be traced to human-understandable concepts. It matters because it suggests explanation and control can be designed in from the start, rather than reverse-engineered, which could change how future AI systems are audited.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Atlas concept semantic cards are human-validated for only ~100 of 33,732 concepts and the annotator only at a permissive LLM-judged threshold, so the 'human-understandable' core of the central claim is not yet established.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the Atlas concept library is only lightly human-validated, and the annotator is judged by an LLM at a permissive threshold. I agree that this is the most fundamental threat. The paper's inherent-interpretability definition requires that semantic cards describe what the attributed variables actually encode; if most concepts are not human-recognizable, then concept attribution, steering, and the interpretability metrics are all anchored to labels that do not correspond to human understanding. The scaling-law analysis, which the reader also questions, is important but secondary: even a clean scaling result would not rescue the central claim if the 'interpretable' interface is not semantically faithful. The paper is honest about the limitation, stating that the human study 'does not certify every concept,' and the small validation sample is a missing support rather than a demonstrated failure. A larger human study on both the concept library and the trained model's token-level alignment would settle the question. Since the reader's recommendation of conditional acceptance already incorporates this concern and asks for addressable fixes, my stress-test does not change the verdict; it sharpens the specific validation needed.","tokens_in":49575,"tokens_out":9821,"duration_ms":100726,"concrete_test":"Run a pre-registered human study on a random sample of 1,000 Atlas concepts, not only the nine largest taxonomy branches, using the same Phase 1 and Phase 2 protocol, and additionally present 200 randomly selected Steerling-8B concept embeddings' top-20 tokens to the same annotators for blind fit against the concept's semantic card. Require e.g. >= 80% of Phase-2 Atlas labels with mean fit >= 4 and >= 75% of model token lists rated as matching the label; if these thresholds are not met, the semantic-faithfulness condition of Definition 3.3 is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that model representations become 'aligned with human-understandable concepts' rests on Definition 3.3's semantic-faithfulness conditions (Interpretation and Coverage). The evidence for Interpretation is thin. Section 4.4's Phase 1 samples 100 concepts, and Phase 2 only 34 concepts, out of 33,732; the paper explicitly says this 'does not certify every concept.' Phase 1 also finds that only 55% of human-written name attempts from lifted-word evidence are rated >= 4, so a substantial minority of concepts may not be independently recoverable. The Stage 3 annotator, which supplies chunk-level supervision for the concept loss, is evaluated on held-out chunks by an LLM judge: mean per-chunk rating 2.94 on a 1-5 scale, with 95.5% of chunks at the permissive '>= 2' threshold; human ratings cover only 50 chunks. If annotations are frequently weak or wrong, the concept bottleneck is trained to align with labels that do not correspond to what humans understand, and the concept attribution interface inherits those faulty semantics. Finally, the Known Concept Alignment metric, the one not directly supervised, is scored by an LLM judge on the model's top tokens; no human judges those model-derived token lists. The validation chain in Section 4.4 ends at Atlas labels, not at the trained model's concept embeddings, leaving a gap between 'human-endorsed label' and 'model embedding points at human-recognizable tokens.' This is a missing support rather than an internal inconsistency, but without it the paper's headline claim about interpretability scaling is not about human interpretability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that interpretability can be designed into language-model pretraining as a fixed rather than growing cost, and that the resulting models become more interpretable as they scale. It formalizes inherent interpretability through five conditions, builds the Atlas concept library (33,732 concepts derived from roughly 14 million tags), introduces a causal-diffusion backbone with an additive concept bottleneck, and reports IsoFLOP sweeps over autoregressive, causal-diffusion, and their +Concept variants. Scaling fits are used to predict the validation loss and four interpretability metrics of the 8B Steerling model; the validation-loss prediction is within 0.11 nats. The paper also describes a 1.2T-token pretraining run followed by 150B-token mid-training with an interleaved steering phase, and compares Steerling-8B with open peer models.","tokens_in":49967,"tokens_out":7190,"duration_ms":67336,"significance":"If the central claims held, the paper would be a substantial contribution: it would provide a recipe-level argument that human-understandable concept bottlenecks can survive at pretraining scale, with attribution and steering as native operations rather than post-hoc estimates. The main empirical asset is the careful IsoFLOP methodology and the successful validation-loss extrapolation to Steerling-8B; the exact additive logit decomposition and the honest reporting of pretraining failures in Sections 9.4 and 10.4 are also strengths. However, the interpretability-scaling claim currently rests on metrics that are partly the optimized losses themselves, and the human-meaningfulness evidence covers a tiny fraction of the concept library. If the authors add non-circular metrics or ablations and broaden the human validation, the contribution would be significant; as presented, the magnitude of the claimed result exceeds the evidential support.","major_comments":[{"comment":"The claim that 'all interpretability metrics improve with compute' (Section 8.3.2) is weakened by the fact that three of the four metrics are the training losses themselves or direct consequences of loss weights: Concept Loss is Eq. (10), Concept Independence Loss is Eq. (14), and Concept Contribution is pushed toward 1 by the reconstruction loss (Eq. 12) and residual dropout p_epsilon. The scaling fits therefore partly describe optimization of the objective rather than an emergent property of larger models. Known Concept Alignment is the only metric not directly supervised, and it is also the metric closest to the paper's semantic-faithfulness claim. I recommend adding at least one non-loss-based behavioral metric (e.g., the steering benchmark of Section 10.2.4) across the small-scale sweep, or ablating the interpretability losses to show that favorable scaling is not simply inherited from the loss schedule.","section":"§7, §8.3 (Eqs. 10, 12, 14)"},{"comment":"Condition 4 of Definition 3.3 (Interpretation) and the abstract's 'human-understandable concepts' are supported by a human study covering only 100 concepts in Phase 1 and 34 concepts in Phase 2 out of 33,732, with the Stage 3 annotator validated on 50 human-rated chunks and an LLM judge at a permissive 'at least 2' threshold (95.5% per-chunk). The paper explicitly states that this 'does not certify every concept.' Since the concept bottleneck is trained with positive-only labels from this annotator, a substantial minority of non-human-meaningful or misannotated concepts could break the semantic faithfulness of the concept attribution interface while leaving all reported numbers unchanged. A larger stratified sample with a coverage bound on the fraction of concepts that fail human recognition, or an explicit restriction of the central claim to the validated subset, is needed.","section":"§4.4"},{"comment":"The 'small, fixed per-backbone offset' in compute-optimal exponents is not yet established. In the AR pair the exponent shift is +0.093 (0.528 to 0.621) and the bootstrap intervals do not overlap with the baseline; in the CDLM pair the shift is -0.129 (0.632 to 0.503) and the intervals overlap. The sign change across backbones is compatible only with a per-backbone statement, not a universal 'interpretability tax' story. More importantly, L_infinity moves downward in both pairs (1.857 to 1.193; 2.658 to 1.942) with wide confidence intervals (e.g., AR+Concept L_infinity 90% CI [-0.703, +0.394]), so the joint fit's offset is not identified at the small scales used. A paired-difference bootstrap on the implied loss offset at the deployment compute budget would be a more direct test of the claim.","section":"§8.2.2, Table 6"},{"comment":"The metric extrapolations are presented as 'within tight bounds' without prediction intervals. For the compute-only fits, R-squared ranges from 0.49 to 0.75 and several asymptotes e are pinned to natural bounds (e.g., Concept Loss e = 0.000 with CI [-0.002, +0.000]; Concept Contribution e = 1.000; Known Concept Alignment e = 5.000). From Table 7, CDLM+Concept Concept Contribution is predicted 0.915 versus actual 0.876, and Known Concept Alignment is predicted 3.87 versus actual 3.77; the differences are small, but so are the natural ranges, and a trivial predictor that always outputs the nearest bound would produce comparable errors for some metrics. Please provide prediction intervals for the extrapolation (not only parameter CIs) and report the error of a constant-at-bound baseline.","section":"§8.3, Tables 7–8"}],"minor_comments":[{"comment":"The sentence 'each concept is reprsentation as a direction' contains a typo; it should read 'represented'.","section":"§1"},{"comment":"The Phase 2 sample is described as 34 concepts from 8 annotators, but Table 4 reports n = 205 for the Atlas label and n = 402 for human labels; the relationship between these counts and the number of concepts is not explained in the main text and should be clarified.","section":"§4.4, Table 4"},{"comment":"The Huber loss is applied to log-residuals with delta = 10^{-3}; this is an unusually small threshold for Huber loss and the choice deserves a one-sentence justification or a sensitivity check.","section":"§8.2.1, Eq. (28)"},{"comment":"The masking-curriculum description says the moving Gaussian center 'rose from 0.2 to 0.8' while Section 9.2 says the curriculum shifts 'from low to high masking'; the text later refers to 'reaching 0.5 on its climb to its endpoint of 0.8.' Please align these statements and specify whether the center or the mean of the Gaussian is being reported.","section":"§9.4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is ambitious and the engineering effort is substantial, but the title-level claims generalize beyond the evidence. The main revision-level needs are: (1) make the interpretability-scaling analysis non-circular by adding metrics that are not optimized losses, and (2) expand or carefully scope the human-meaningfulness validation. I do not see grounds for rejection, but the current evidence does not support the strongest formulations in the abstract and Section 8."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core empirical result is real and worth engaging with: adding a concept bottleneck shifts compute-optimal scaling exponents by a fixed per-backbone offset rather than a growing tax, and three of four interpretability metrics extrapolate predictably to the 8B model. The 0.11-nat validation-loss prediction from small-scale fits is a genuine external validation, and the Causal Diffusion attention variant plus the Atlas pipeline are substantial engineering contributions. The paper is honest about several of its own limitations, which helps. The soft spots are where the reader and stress-test note land. First, the human validation of Atlas covers only about 100 of 33,732 concepts in Phase 1 and 34 in Phase 2. The paper explicitly says this does not certify every concept, but the abstract's phrasing 'aligned with human-understandable concepts' leans on that certification. A minority of concepts (17% flagged as incoherent or noisy) may not be human-recoverable, and the Stage 3 annotator is only LLM-judged at a permissive threshold. This is missing support rather than a fatal flaw, but it is the load-bearing part of the headline claim. Second, three of the four interpretability metrics are direct training losses or close derivatives. Concept loss and independence loss are optimized, and concept contribution is shaped by residual dropout and reconstruction loss. Only known concept alignment is not directly supervised, and it is scored by an LLM judge. That makes the 'interpretability scales' result interesting but partly circular; the metrics are not independent of the objective. The paper acknowledges some of this, but the framing in the abstract is more confident than the evidence. Third, the peer comparison is presented more favorably than the numbers support. Steerling-8B is trained on 1.35T tokens versus peers trained on 2-16x more compute, but it trails the best peers on most benchmarks. 'Within approximately 10%' is a fair summary, but 'competitively performant' overstates it. Also, the model needed dedicated steering training during mid-training before it responded reliably to concept injection, which slightly undercuts the 'inherent' controllability claim. The scaling-law analysis itself is careful: the joint Chinchilla fit, the bootstrap intervals, and the acknowledgment that some asymptotes pin to natural bounds are all good practice. The paper deserves a serious referee. For a reading group, it would generate good discussion about what counts as interpretability and how to validate it at scale. I would cite the scaling-law result, but I would not cite the human-interpretability claim without checking the underlying data once it is released. Recommendation: send to peer review. The issues are addressable - release artifacts, extend human validation, add post-hoc baselines, and soften the abstract - and the core empirical finding is worth publishing after those revisions.","headline":"Impressive scaling-law analysis and real engineering, but the human-interpretability claim rests on a thin validation sample and partly circular metrics.","tokens_in":840,"tokens_out":1115,"would_cite":true,"duration_ms":22303,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scaling interpretable LMs costs a fixed offset, not a growing tax","keywords":["inherent interpretability","concept bottleneck","scaling laws","diffusion language models","concept attribution","model steering","concept annotation","compute-optimal scaling"],"falsifier":"Two checks would settle the central claim: (1) rerun the human nameability and label-fit study on a random sample of 1,000 Atlas concepts instead of the 100 and 34 used here; if Atlas labels stop beating taxonomy or embedding distractors under blind comparison, the human-meaningfulness premise fails. (2) Extend the IsoFLOP sweep for a +Concept family to 10x the largest compute budget; if the validation-loss gap to the unconstrained backbone widens beyond the fitted fixed offset, or if concept loss and independence loss stop improving, the 'fixed offset, improves with scale' claim fails.","tokens_in":49367,"feed_emoji":"🧭","tokens_out":8871,"duration_ms":69650,"temperature":0.7,"pith_summary":"This paper challenges the assumption that interpretability is a tax on capability: the standard view is that you train an opaque model and then try to explain it afterwards, paying some capability price for the explanation. Instead, the authors build interpretability into pretraining as a design constraint—a concept bottleneck between the transformer backbone and the output head, supervised by a large automatically-built library of human-meaningful concepts—and measure what it costs. Across autoregressive and causal-diffusion backbones, over three orders of magnitude of compute, they find that adding the concept module shifts the compute-optimal scaling exponents by a small, fixed per-backbone offset, so the penalty does not grow with scale. At the same time, the model's concept predictions, known/unknown disentanglement, concept contribution to predictions, and alignment of concept embeddings with human labels all improve as compute increases. The resulting 8B model, Steerling-8B, remains within roughly 10% of open peers trained on 2–16x more compute, supporting the paper's claim that a model can be both interpretable and competitively performant.","feed_headline":"Scaling interpretable LMs costs a fixed offset, not a growing tax","feed_subtitle":"With interpretability designed into training, an 8B model stays within 10% of peers trained on up to 16x more compute.","key_machinery":"The load-bearing object is the concept module, an additive concept bottleneck inserted between the transformer backbone and the linear language-modeling head. It decomposes the hidden state as h ≈ k̂ + û + ε, with k̂ a weighted sum of supervised 'known' concept embeddings, û a weighted sum of unsupervised 'unknown' embeddings, and ε an explicit residual; because the head is linear, every output logit decomposes exactly into per-concept contributions, making concept attribution and steering closed-form operations. Two supporting mechanisms carry the argument: Causal Diffusion, a block-causal attention pattern (bidirectional within blocks, causal across them) that keeps diffusion-style training while preserving autoregressive KV caching, and the masked-diffusion objective's [MASK] token, which acts as a trained in-distribution absence baseline for input attribution. The third pillar is Atlas, an automated pipeline that turns millions of documents into a canonical library of 33,732 human-meaningful concepts and a chunk-level annotator, providing the semantic supervision the bottleneck needs. The scaling analysis then fits compute-optimal power laws and an irreducible-loss form, showing a fixed per-backbone offset and favorable interpretability scaling.","core_discovery":"The paper's central claim is that interpretability does not have to be purchased after training at a growing capability cost. Training with an explicit, additive concept bottleneck—where the hidden state is decomposed as a sum of known-concept, unknown-concept, and residual contributions before the language-modeling head—shifts the compute-optimal scaling exponents by a small, fixed per-backbone offset rather than a penalty that compounds with scale. The evidence is an IsoFLOP sweep across four families (autoregressive, causal diffusion, and each with the concept module) over three orders of magnitude of compute, plus a joint Chinchilla-form fit that predicts the validation loss of the deployed 8B model within 0.11 nats. All four interpretability metrics improve with compute: concept loss declines, known and unknown representations become more linearly independent, the fraction of prediction routed through concepts rises, and concept embeddings become better aligned with their human-assigned labels. The authors interpret this as showing that models do not become harder to understand as they scale; they become easier, and the fixed offset means interpretability can be included in the training recipe without giving up the scaling frontier.","pith_inferences":["Editorial extension: if the fixed-offset law holds beyond 8B, interpretability-by-design becomes a practical default for safety-critical or regulated deployments, since the one-time capability cost does not compound with scale.","Editorial extension: the scaling results depend on the concept library being held fixed; if the concept library itself were scaled or updated during training, the observed offset and exponent shifts could change, suggesting a testable variant where library size is a controlled variable.","Editorial extension: the paper's interpretability metrics all measure the concept module's own behavior; an independent test—e.g., steering toward concepts outside the Atlas library and measuring whether generation moves accordingly—would probe whether the claimed alignment generalizes beyond the supervised vocabulary."],"forward_implications":["If adding an interpretable concept bottleneck only shifts scaling exponents by a fixed per-backbone offset, then the compute-optimal frontier itself is preserved; the same scaling-law machinery used to plan opaque models can be used to plan interpretable ones.","Because all four interpretability metrics obey predictable power laws (three within tight bounds, one in the favorable direction), interpretability properties can be extrapolated before a large run, much like validation loss.","The trained [MASK] baseline and exact additive decomposition make the model's attribution and steering operations closed-form rather than estimated, so diagnosis and correction do not require extra fitted explanation machinery.","An inherently interpretable model can compete with opaque peers trained on 2–16x more compute, which directly contradicts the premise that interpretability must be exchanged for capability."],"supporting_citations":[{"why":"Supplies the IsoFLOP, power-law, and Chinchilla-form methodology used to measure the compute-optimal scaling offset.","marker":"Hoffmann et al. (2022)"},{"why":"Provides the masked-diffusion ELBO estimator and scaling baseline that the causal-diffusion family is compared against.","marker":"von Rütte et al. (2025)"},{"why":"Gives the masked-diffusion scaling exponents and the block-by-block sampling result that motivates the Causal Diffusion design.","marker":"Nie et al. (2025)"},{"why":"Introduces the block-diffusion attention pattern from which the paper's block-causal backbone is derived.","marker":"Arriola et al. (2025)"},{"why":"Provides the original concept bottleneck model formulation and the teacher-forcing technique the training dynamics extend.","marker":"Koh et al. (2020)"},{"why":"Extends concept bottlenecks to generative models with an unknown-concept channel, the direct precursor of the concept module.","marker":"Ismail et al. (2024)"},{"why":"Supplies the Nemotron-CC-HQ dataset that dominates the pretraining and mid-training mixtures for Steerling-8B.","marker":"Su et al. (2025)"},{"why":"Provides the irreducible-loss scaling form used to fit and extrapolate the interpretability metrics.","marker":"Gao et al. (2025)"},{"why":"Defines the steering benchmark protocol used to evaluate whether steering training improves concept activation and generation quality.","marker":"Wu et al. (2025)"}],"fun_headline_variants":["Interpretability scales with capability, not against it","Designing interpretability into training: fixed cost, not a growing tax","Steerling-8B shows interpretability improves with scale, at fixed offset","Interpretability as a design constraint scales with LMs, not against them","Fixed offset, not growing tax: interpretability scales with LMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire interpretability story rests on the Atlas concept library being genuinely meaningful to humans and on the chunk-level annotations being accurate enough to supervise the concept module; if many of the 33,732 concepts are not human-recognizable or the annotator frequently mislabels chunks, the claimed semantic faithfulness collapses.","fun_headline_variants_meta":{"raw":{"variants":["Interpretability scales with capability, not against it","Designing interpretability into training: fixed cost, not a growing tax","Steerling-8B shows interpretability improves with scale, at fixed offset","Interpretability as a design constraint scales with LMs, not against them","Fixed offset, not growing tax: interpretability scales with LMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000241,"raw_usage":{"total_tokens":1547,"prompt_tokens":997,"completion_tokens":550,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":458}},"tokens_in":613,"tokens_out":550,"duration_ms":3985,"temperature":1.0,"reasoning_tokens":458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:31:20.832502+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two checks would settle the central claim: (1) rerun the human nameability and label-fit study on a random sample of 1,000 Atlas concepts instead of the 100 and 34 used here; if Atlas labels stop beating taxonomy or embedding distractors under blind comparison, the human-meaningfulness premise fails. (2) Extend the IsoFLOP sweep for a +Concept family to 10x the largest compute budget; if the validation-loss gap to the unconstrained backbone widens beyond the fitted fixed offset, or if concept loss and independence loss stop improving, the 'fixed offset, improves with scale' claim fails.","supporting_citations":[],"review_version":1}