{"id":"bd771ade-5e2c-43a0-a096-1ae6f103ddea","arxiv_id":"2507.12990","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Training a small secondary sparse autoencoder on the reconstruction error of a pretrained SAE improves domain-specific reconstruction and language-model perplexity without hurting general performance.","lead":"Sparse autoencoders (SAEs) are tools for peeking inside language models, but they often miss rare, domain-specific concepts. This paper adds a small second SAE that learns what the first one got wrong on a specific domain, like chemistry or Russian, and combines both at inference time to capture those missing features.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported domain gains lack a mismatched-residual control, so they do not yet distinguish domain-specific feature learning from a generic additive capacity effect.","rationale":"Reader's weakest assumption (residual error must be learnable, sparse, and non-interfering) is closely related but not identical to the concern here. The paper gives partial evidence for learnability and non-interference: Figure 2 shows lower cosine overlap with base features than the comparators, and Table 10 reports interpretability scores above the base SAE. What is missing is the sharper control that separates 'domain-matched residual structure' from 'any extra additive module helps.' Because the residual encoder consumes x and always activates k features, the module is never truly dormant; the paper measures matched-domain and general-domain behavior but never an intentionally mismatched residual. This is a load-bearing gap for the mechanistic-interpretability framing: if a mismatched residual produces similar gains, the method is still a valid reconstruction booster, but the title-level claim of teaching old SAEs domain tricks is unsupported. The absence of LLM CE for the capacity-matched baselines is a smaller but related omission. These issues do not overturn the paper; the method is coherent, the ablations on sparsity and training tokens are informative, and the qualitative feature analyses provide some support. The conditional verdict remains appropriate, with the specific missing control added as a condition.","tokens_in":11165,"tokens_out":7879,"duration_ms":104779,"concrete_test":"On the UN Debates domain, train three residual SAEs of identical size (1024 features, k=5) and token budget: (1) on UN Debates residuals (current), (2) on Chemistry residuals, (3) on a shuffled/randomly permuted residual target. Evaluate all three added to the frozen base SAE on the UN Debates test set. If (2) or (3) recovers more than ~20% of (1)'s EV or LLM-CE improvement over the base SAE, the domain-specificity claim is not supported. As a secondary check, report LLM CE for the Extended SAE and SAE Stitching baselines at L0 matched to SAE Boost; comparable CE gains would weaken the claimed advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SAE Boost's central claim is that the residual SAE captures domain-specific structure in e = x - x_hat. In every reported experiment the residual is trained and evaluated on the same domain (Table 1), so the EV improvement is partly an in-distribution fitting artifact; any residual model trained to minimize ||e - e_hat||^2 on that domain will improve combined reconstruction by construction. The paper does not provide a control showing the benefit requires domain-matched residual structure. This matters because the residual encoder takes x, not e, and batch-topk forces exactly k=5 active features on every input; there is no domain gate, so an off-domain or mismatched residual SAE still emits a nonzero correction. If a residual trained on Chemistry (or on a shuffled-residual target) improves UN Debates EV/LLM-CE by a substantial fraction of the matched residual's gain, then the 'complementary domain features' interpretation is unsupported and the method reduces to a generic capacity patch. The baseline tables (3, 6, 7) also omit LLM CE, so the downstream metric that motivates the method is only compared against the no-adaptation baseline, not against Extended SAE or SAE Stitching at matched sparsity. Section 4.2.5's under-training result (up to 31% general EV loss with fewer than 100M tokens) reinforces that the residual module's behavior is not automatically safe or domain-selective.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SAE Boost, a residual learning method for adapting pretrained sparse autoencoders (SAEs) to specialized domains. A secondary residual SAE is trained, on domain-specific activations, to reconstruct the reconstruction error of a frozen base SAE; at inference the two SAE outputs are summed. Experiments on Qwen-2.5-7B and Llama-3.1-8B report improvements in explained variance and LLM cross-entropy on Chemistry, Russian, and UN Debates data, with small changes in general-domain performance. The paper also compares against Extended SAE, SAE Stitching, and full fine-tuning, and provides feature-level interpretability analyses.","tokens_in":11400,"tokens_out":3086,"duration_ms":37317,"significance":"If the central claims hold, the method is a useful and inexpensive tool for domain-targeted SAE enhancement: it is modular, requires no retraining of the base SAE, and the additive inference scheme is simple. The paper's strongest evidence is that a converged residual SAE can be added with little measured change in general-domain EV, and that 1024 residual features lead to notable EV and LLM-CE gains on several domains. The interpretability analysis (Tables 10 and 11) is a genuine attempt to show that the residual features are semantically meaningful. However, several load-bearing empirical claims are not fully supported: there is no control separating domain-matched residual structure from generic additive capacity, baseline comparisons omit the downstream LLM-CE metric, and the paper's own appendix contradicts the claim that SAE Boost has the best domain EV in all domains. These issues affect the interpretation of the method rather than the internal consistency of the training objective.","major_comments":[{"comment":"The statement that SAE Boost 'achieves the best domain-specific performance' (Section 4.2.3) is contradicted by the paper's own appendix: Table 6 shows Extended SAE reaches 0.726 Chemistry EV versus 0.716 for SAE Boost, and Table 7 shows Extended SAE reaches 0.741 Russian EV versus 0.725 for SAE Boost. The claim should be revised to per-domain comparisons, and the abstract's general superiority claim should be tempered. This is load-bearing because the method's advantage over Extended SAE currently rests on sparsity and general-domain EV, not on universally higher domain EV.","section":"Section 4.2.3, Tables 3, 6, 7"},{"comment":"The experiments lack a mismatched-residual control. Because the residual SAE is trained and evaluated on the same domain and its objective is to minimize ||e - e_hat||^2, the combined reconstruction's EV improvement is partly by construction; any residual model fit to the training-domain residuals would improve combined EV on that domain. Since the residual encoder takes x (not e) and batch-topk forces k=5 active features on every input, there is no domain gate, so a residual trained on a different domain or on shuffled residuals would still emit nonzero corrections. A control that trains the residual on Chemistry or on shuffled residuals and evaluates it on UN Debates is necessary to support the interpretation that the gains come from domain-specific complementary features rather than from a generic additive capacity increase.","section":"Section 3.1 and Section 4.2.1, Tables 1 and 3"},{"comment":"The baseline comparisons omit LLM cross-entropy. The method is motivated by downstream LLM performance, and Table 1 shows large LLM-CE improvements relative to the no-adaptation baseline, but Tables 3, 6, and 7 report only EV and L0 for Extended SAE, SAE Stitching, and full fine-tuning. Without LLM CE at matched sparsity, the claim that SAE Boost is the best balance of domain adaptation and general preservation is incomplete, especially for Chemistry and Russian where Extended SAE has higher domain EV.","section":"Tables 3, 6, and 7"},{"comment":"The claim that incorporating a residual SAE has 'minimal impact' on general performance is conditional on sufficient training. Section 4.2.5 reports that residual SAEs trained on fewer than 100M tokens reduce general EV by up to 31%, and Figure 4 shows a steep training curve. Since Table 2 reports only fully trained 1B-token residuals, the paper should state this condition prominently and provide the training-duration threshold more precisely, or the 'without interfering' claim will be overstated. In addition, all experiments appear to be single runs without error bars, so the small differences in Table 2 (e.g., -0.28%, +0.60%) cannot be distinguished from noise.","section":"Section 4.2.5 and Table 2"}],"minor_comments":[{"comment":"The base-model names are inconsistent: the paper uses 'Qwen' versus 'Qwen-2.5-7B-Base' and 'LLama' versus 'Llama Scope' (e.g., Table 2 caption). Please standardize.","section":"Throughout"},{"comment":"The sentence 'To support our interpretability claims. While Figure 4 provides anecdotal support...' is grammatically incomplete and should be rewritten; Table 10's evaluation should also state the number of runs used to compute the reported standard deviations.","section":"Section 4.2.6 and Appendix A.4"},{"comment":"The Pareto-frontier figure does not specify what the other plotted points correspond to (e.g., which baselines or mixtures); a legend or caption entry is needed for reproducibility.","section":"Figure 3"},{"comment":"The residual SAE is described as having no decoder bias 'to ensure it contributes only when meaningful domain-specific features are detected,' but this mechanism is not empirically isolated from simply reducing the model's capacity; a short ablation showing the effect of including the decoder bias would clarify the claim.","section":"Section 3.1"},{"comment":"Reference formatting is inconsistent (e.g., 'Henderson*' with an asterisk, and some entries use arXiv identifiers without a URL). Please align with the venue's style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is plausible and the non-interference result is empirically interesting, but the absence of a mismatched-residual control and the contradiction between the abstract's 'best' claim and Tables 6 and 7 are substantial. With additional controls, error bars, and per-domain claims, the paper could become acceptable; as it stands, the evidence supports an incremental improvement over Extended SAE rather than a clear superiority claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SAE Boost is a clean, useful idea: keep the base SAE frozen, train a small secondary SAE on the reconstruction residuals from a domain corpus, and sum the two reconstructions at inference. As far as I know nobody has applied this exact boosting/residual scheme to SAE domain adaptation before, and the paper shows it works across three domains and two base models, with mostly modest sparsity overhead. The authors also do something right that many SAE papers skip: they compare against extended dictionaries, SAE stitching, and full fine-tuning, report general-domain non-interference, and include a training-dynamics section warning that undertrained residuals can hurt general performance by up to 31% EV. The interpretability validation with detection/fuzzing scores is a nice touch. The soft spots are real but not fatal. First, there are no error bars anywhere; everything is single-run, so the magnitude of the claimed improvements is uncertain. Second, the best method framing is overstated: in the appendix, Extended SAE gets higher domain EV on Chemistry (0.726 vs 0.716) and Russian (0.741 vs 0.725). SAE Boost's advantage is specifically the better sparsity/EV trade-off, which is legitimate, but the paper should say that plainly instead of claiming superiority outright. Also, LLM cross-entropy is only reported for SAE Boost versus the no-adaptation baseline; the baselines in Tables 3, 6, and 7 have no LLM CE column, so the downstream metric that motivates the method is never compared across adaptation methods. The bigger issue is the one the stress test flags: there is no mismatched-residual control. The residual is always trained and evaluated on the same domain, so the combined reconstruction improvement is partly guaranteed by the training objective. Since the residual encoder takes x and always emits k=5 active features regardless of the input, a residual trained on Chemistry or on shuffled residuals might improve UN Debates performance almost as much. If that were true, the complementary domain-specific features interpretation collapses into additive capacity patch. The under-training result already shows the residual is not automatically domain-selective, which makes this control necessary. The method would still be practically useful either way, but the mechanistic-interpretability claim needs the control to stand. Who is this for? People working on SAE deployment who want cheap, modular domain adaptation. It deserves a serious referee; the central experiment is reproducible in principle and the gap I describe is fixable with one additional control. If I were editing, I would send it out with a request for that control and for LLM CE on the baselines.","headline":"SAE Boost is a clean, practical residual-learning trick for adapting SAEs to new domains; the experiments are solid, but the missing mismatched-residual control leaves the 'complementary domain features' interpretation underdetermined.","tokens_in":788,"tokens_out":828,"would_cite":true,"duration_ms":29107,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small residual SAE trained on a frozen SAE's reconstruction errors adds domain-specific features to LLM interpretability by simple addition during inference, improving domain explained variance and cross-entropy while leaving general…","keywords":["sparse autoencoders","mechanistic interpretability","residual learning","domain adaptation","feature blindness","LLM interpretability","explained variance"],"falsifier":"Train a residual SAE on residuals computed from the same domain activations after shuffling the token order within each activation, so any learned structure is destroyed; if the shuffled residual SAE still improves domain explained variance when added to the base SAE, the improvement is not caused by capturing meaningful domain residuals. A complementary check is to compare SAE Boost against a single SAE with the same total dictionary size trained directly on the domain data: if the larger single SAE matches or beats the boosted model, the residual decomposition itself is not the source of the gain.","tokens_in":10935,"feed_emoji":"🧩","tokens_out":12771,"duration_ms":121031,"temperature":0.7,"pith_summary":"The paper claims that an SAE's blindness to domain-specific features can be cured by training a small second SAE on the reconstruction error of the first, then adding the two outputs at inference. Each domain gets its own tiny residual SAE, trained only on that domain's texts, so the base model is never modified and general-domain performance stays essentially unchanged. Across chemistry, Russian, UN debates, and several other languages, the paper reports relative improvements in explained variance and reductions in LLM cross-entropy on the target domain, with under one percent change on general text. The residual SAE uses only five active features per input, so the fix is cheap, modular, and easily combined across domains.","feed_headline":"Small residual SAE adds missing domain features without retraining","feed_subtitle":"Training it on reconstruction error lifts domain explained variance by up to 59 percent while general tasks stay flat.","key_machinery":"The load-bearing object is a residual SAE, called SAE Boost: a sparse autoencoder trained with the usual reconstruction-plus-sparsity loss but with the target replaced by the residual $e = x - \\hat{x}$ of a frozen base SAE, and with the decoder bias removed so it contributes only when domain features are detected. The identity carrying the argument is the additive decomposition $x = \\hat{x} + e$; at inference it extends to $x \\approx \\hat{x} + \\sum_{i=1}^N \\hat{e}^{(i)}$ for $N$ domain-specific residual SAEs. This turns domain adaptation into a sum of modular corrections rather than a change to the base features.","core_discovery":"The central claim is that a general SAE's feature blindness can be repaired additively rather than by retraining. For a frozen base SAE with reconstruction $\\hat{x}$ of an activation $x$, the paper defines the residual $e = x - \\hat{x}$ and trains a second, smaller sparse autoencoder to map $x$ to an approximation $\\hat{e}$ of $e$, using the standard SAE loss $\\|e - \\hat{e}\\|_2^2 + \\lambda L_{reg}$. Inference then uses $x \\approx \\hat{x} + \\hat{e}$, and when several domain-specific residual SAEs are trained, their corrections are simply summed: $x \\approx \\hat{x} + \\sum_i \\hat{e}^{(i)}$. Because the residual SAE has no decoder bias and is trained only on domain residuals, the paper argues it learns complementary, not competing, features; it reports higher explained variance, lower LLM cross-entropy on chemistry, Russian, and UN-debates data, along with under one percent general-domain change, and it shows the learned features are interpretable domain concepts.","pith_inferences":["Inference: the same residual trick could be applied iteratively, training a third SAE on the residual of the combined two-model output, turning the approach into a genuine boosting sequence whose later stages capture ever rarer features; the paper only demonstrates one residual stage.","Inference: the low cosine similarity between residual and base features suggests a testable stopping rule, namely to train a residual SAE until the maximum-cosine-similarity distribution stabilizes, as a signal that it has converged to complementary rather than redundant features.","Inference: because the combined model is a sum, each domain-specific residual SAE can be ablated independently, giving a way to attribute which features drive the domain cross-entropy gain and to measure interference directly."],"forward_implications":["Domain-specific SAE interpretability can be added on demand: one trains only a small residual SAE on the target domain's reconstruction errors, leaving the base SAE frozen.","Multiple domain-specific residual SAEs can be combined by summing their outputs during inference, so corrections for chemistry, law, languages, or other corpora can be stacked without retraining the general model.","Because the residual SAE uses five active features against the base SAE's fifty, the added compute and sparsity overhead are small.","The method improves both reconstruction quality and LLM cross-entropy, meaning the recovered residual features carry information the language model actually uses for next-token prediction.","The reported under-one-percent change in general-domain explained variance means the base SAE's existing features are not displaced, so the technique is compatible with continued use of pretrained SAEs."],"supporting_citations":[{"why":"Establishes the sparse autoencoder dictionary-learning decomposition and the jumpReLU-type sparsity objective that the base SAE is built on.","marker":"Bricken et al., 2023"},{"why":"Supplies the scaling and evaluation methodology for SAEs, including explained variance, that the paper uses to measure domain gains.","marker":"Gao et al., 2024"},{"why":"Documents feature blindness and the conclusion that extremely large dictionaries are needed for rare concepts, which motivates the residual approach.","marker":"Templeton et al., 2024"},{"why":"Shows that domain adaptation of SAEs is currently done by retraining on specialized data, the expensive baseline this paper avoids.","marker":"Muhamed et al., 2024"},{"why":"Introduces SAE stitching and the distinction between novel and reconstruction latents, providing a direct comparison baseline.","marker":"Leask et al., 2025"},{"why":"Provides the batch-topk activation used for sparsity enforcement in training the base and residual SAEs.","marker":"Bussmann et al., 2024"},{"why":"Supplies the publicly released pretrained SAE on a second base LLM that the paper uses to test whether the method transfers across models.","marker":"He et al., 2024"},{"why":"Provides the general-domain Fineweb corpus and the multilingual Fineweb-2 datasets used for training and evaluation.","marker":"Penedo et al., 2024"}],"fun_headline_variants":["Boosting SAEs with residual learners captures missed domain features","Residual SAE: small add-on recovers domain features, no retraining","Teach old SAEs new domain tricks with boosted residual features","Residual learning boosts SAE interpretability for new domains","Small residual SAE recovers domain features without retraining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the reconstruction error $e = x - \\hat{x}$ of the base SAE on domain text is a learnable, sparse structure rather than noise, and that the residual SAE's features do not collide with the base features during additive combination.","fun_headline_variants_meta":{"raw":{"variants":["Boosting SAEs with residual learners captures missed domain features","Residual SAE: small add-on recovers domain features, no retraining","Teach old SAEs new domain tricks with boosted residual features","Residual learning boosts SAE interpretability for new domains","Small residual SAE recovers domain features without retraining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000775,"raw_usage":{"total_tokens":3412,"prompt_tokens":909,"completion_tokens":2503,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":2417}},"tokens_in":525,"tokens_out":2503,"duration_ms":18489,"temperature":1.0,"reasoning_tokens":2417,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:33:07.630473+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a residual SAE on residuals computed from the same domain activations after shuffling the token order within each activation, so any learned structure is destroyed; if the shuffled residual SAE still improves domain explained variance when added to the base SAE, the improvement is not caused by capturing meaningful domain residuals. A complementary check is to compare SAE Boost against a single SAE with the same total dictionary size trained directly on the domain data: if the larger single SAE matches or beats the boosted model, the residual decomposition itself is not the source of the gain.","supporting_citations":[],"review_version":1}