{"id":"cd3c2396-21cb-411e-94a7-ef8f9cc64d39","arxiv_id":"2606.18322","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SAE feature clamping fails to fully eliminate target behaviors because constrained residual optimization recovers the pre-intervention output while preserving post-intervention SAE feature values, demonstrated across TPP, unlearning, IOI, and refusal-steering tasks with 95.8% recovery in the safety ","lead":"The paper shows that clamping specific SAE features to suppress behaviors such as model refusal can be bypassed by finding small residual-stream perturbations that keep the clamped features fixed yet restore the original output. Smart readers should care because this exposes a gap in current feature-level safety techniques for large language models.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Encoder-orthogonal updates and Jacobian may not fully isolate recovery from partial intervention reversal","rationale":"The reader's weakest_assumption directly identifies the same constraint-sufficiency issue. Because the full text is now stated to be available yet the reported numbers still rely on those constraints without an explicit tightness proof or ablation, the UNVERDICTED status and LOW confidence remain appropriate; no stronger objection or clearance is warranted from the given material.","tokens_in":1842,"tokens_out":324,"duration_ms":26273,"concrete_test":"Recompute the refusal-steering recovery trajectories both with and without the encoder-orthogonal projection (or exact Jacobian); if the behavior-recovery rate and defended-feature drift remain statistically indistinguishable, the constraint does not isolate genuine recovery.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that post-intervention recovery demonstrates a genuine gap between SAE-feature control and behavioral control. This rests on the encoder-orthogonal constraint (single-layer) and feature-map Jacobian (cross-layer) ensuring that optimized residual perturbations cannot simply reverse the original clamp while holding the targeted feature activation fixed. If the encoder is nonlinear or the Jacobian is an approximation, these constraints can be satisfied by perturbations that still move the model along an effective reversal path in the original activation space; the reported recovery (e.g., 95.8 % on refusal) would then reflect incomplete isolation rather than an independent behavioral route. The abstract invokes this premise in the threat-model and recovery-formulation paragraphs, but the sufficiency of the constraints is the least-secured step for the headline conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that SAE-based feature interventions are unreliable for suppressing model behaviors because post-intervention recovery of the original behavior remains possible via constrained optimization of residual perturbations. Recovery is formulated to preserve the post-clamp values of the targeted SAE features while restoring pre-intervention behavior; encoder-orthogonal updates (single-layer) and the feature-map Jacobian (cross-layer) are used to rule out simple reversal of the intervention. Experiments on TPP, unlearning, IOI, and refusal steering report high recovery rates (e.g., 95.8% on valid refusal samples) with low defended-feature drift (0.131), and attribution analysis localizes recovery to the SAE reconstruction residual.","tokens_in":1967,"tokens_out":537,"duration_ms":28885,"significance":"If the central empirical result holds under the stated constraints, the work identifies a substantive gap between SAE-feature control and behavioral control. This is relevant for safety-critical applications that rely on feature clamping, and the constrained-optimization plus attribution approach supplies a concrete, falsifiable test for whether an SAE decomposition is behaviorally complete. The low-drift recovery numbers and cross-task consistency would strengthen the case that current SAE interventions leave exploitable residual routes.","major_comments":[{"comment":"Threat-model and recovery-formulation paragraphs: the encoder-orthogonal updates and feature-map Jacobian are presented as sufficient to ensure that optimized residuals cannot achieve recovery simply by partially reversing the original clamp. Because the SAE encoder is nonlinear, it is not shown that satisfaction of these (linear) constraints precludes effective reversal trajectories in the original activation space; if such trajectories exist, the reported recovery rates would not demonstrate an independent behavioral route. This premise is load-bearing for the headline conclusion.","section":"Threat model and recovery formulation"}],"minor_comments":[{"comment":"The abstract states that recovery occurs 'while keeping defended-feature relative drift to 0.131'; the precise definition of relative drift and the baseline against which it is measured should be stated explicitly in the methods section.","section":"Abstract / Methods"},{"comment":"The recovery-path attribution analysis is described only at a high level; a short paragraph or figure caption clarifying how the attribution isolates the reconstruction residual versus other components would improve reproducibility.","section":"Results"}],"recommendation":"major_revision","confidential_remarks":"The reader's stress-test note correctly flags that the sufficiency of the orthogonality/Jacobian constraints is the least-secured step; the manuscript would benefit from either a formal argument or additional ablation showing that the constraints cannot be satisfied by reversal-like perturbations."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their careful reading and for identifying this important subtlety in our threat-model formulation. We respond to the single major comment below.","responses":[{"response":"The referee correctly notes that the encoder is nonlinear and that our auxiliary linear constructions (encoder-orthogonal updates, feature-map Jacobian) do not by themselves constitute a complete proof against all conceivable reversal trajectories. However, the primary and nonlinear constraint in the optimization is the explicit requirement that the targeted SAE feature values—i.e., the direct output of the nonlinear encoder—remain exactly at their post-clamp levels. This constraint is enforced at every step and is verified after optimization by the reported low relative drift (0.131). Any trajectory that reversed the original clamp would necessarily change those feature values and would therefore be rejected by the optimizer. The linear constructions are used only to identify feasible search directions that satisfy the nonlinear feature-preservation constraint; they are not claimed to be a standalone guarantee. We will add a clarifying paragraph in the revised threat-model section that distinguishes the auxiliary linear methods from the primary nonlinear feature-value constraint and will report an additional diagnostic that measures how much the optimization would have to violate the feature constraint to achieve full reversal.","revision_made":"partial","referee_comment":"[Threat model and recovery formulation] Threat-model and recovery-formulation paragraphs: the encoder-orthogonal updates and feature-map Jacobian are presented as sufficient to ensure that optimized residuals cannot achieve recovery simply by partially reversing the original clamp. Because the SAE encoder is nonlinear, it is not shown that satisfaction of these (linear) constraints precludes effective reversal trajectories in the original activation space; if such trajectories exist, the reported recovery rates would not demonstrate an independent behavioral route. This premise is load-bearing for the headline conclusion."}],"tokens_in":1473,"tokens_out":377,"duration_ms":32056,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that clamping an SAE feature does not reliably eliminate the associated behavior. The authors set up a recovery optimization that starts from the clamped state and finds small residual changes to bring the original output back while keeping the clamped feature activation unchanged. They add encoder-orthogonal updates for single-layer cases and a feature-map Jacobian for cross-layer cases to make reversal harder. On refusal steering they report 95.8 % recovery on valid samples with only 0.131 relative drift on the defended feature, and they trace the recovery path to the SAE reconstruction residual.\n\nWhat stands out is the concrete formulation and the multi-task stress test. The work directly challenges the assumption in recent safety papers that feature-level clamps give behavioral control. The attribution to the unexplained residual is a useful pointer.\n\nThe soft spot is whether the orthogonality and Jacobian constraints actually isolate genuine recovery. If the encoder is nonlinear, an update that satisfies the reported conditions can still move the model along an effective reversal direction in activation space. The abstract does not show that the constraints were validated against this possibility, and the optimization details, data splits, and statistical controls are not visible here. The 95.8 % figure is large enough that any leakage would change the interpretation.\n\nThis is relevant for anyone building SAE-based monitors or steering systems. It raises a structural question worth checking rather than a minor implementation issue. I would send it to peer review so the methods can be examined in full.","headline":"The paper shows you can often recover suppressed behavior after SAE clamping by optimizing residuals while holding the target feature fixed, but the constraints meant to block simple reversal look under-tested.","tokens_in":2487,"tokens_out":375,"would_cite":false,"duration_ms":18017,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Clamping SAE features does not guarantee suppression of model behaviors, as residual perturbations can recover the original outputs while preserving the clamped features.","keywords":["Sparse Autoencoders","Model Interventions","Feature Clamping","Residual Stream","Post-Intervention Recovery","Refusal Steering","Safety Interventions"],"falsifier":"An experiment across the same tasks where no residual perturbation recovers the original behavior while keeping the clamped SAE features fixed and the intervention active throughout generation.","tokens_in":2703,"feed_emoji":"","tokens_out":619,"duration_ms":23622,"temperature":0.7,"pith_summary":"The paper establishes that SAE-based interventions, which clamp specific features to block harmful behaviors, can be circumvented through a recovery process. It formulates post-intervention recovery as an optimization problem that finds residual perturbations restoring pre-intervention behavior without changing the targeted SAE feature values. This recovery succeeds even when the intervention remains active during optimization and generation, using constraints like encoder-orthogonal updates to ensure it is not mere reversal. Experiments on tasks including refusal steering show high recovery rates with low feature drift. The results indicate that SAE features capture only part of the mechanism for a behavior, leaving the reconstruction residual as a path for recovery.","feed_headline":"SAE feature clamps allow 95.8% behavior recovery in refusal tasks","feed_subtitle":"Residual optimization restores outputs while holding clamped features fixed, showing feature control does not equal behavioral control.","key_machinery":"Post-intervention recovery formulated as a constrained residual-space optimization problem that restores pre-intervention behavior while enforcing preservation of targeted SAE feature values via encoder-orthogonal updates or feature-map Jacobian.","core_discovery":"SAE feature clamping supports causal intervention but leaves a recoverable failure mode: starting from the post-intervention residual state, constrained optimization can recover pre-intervention behavior while preserving the post-intervention values of the targeted SAE features, even under active intervention, with recovery localizing to the SAE reconstruction residual.","pith_inferences":["Safety methods relying on single SAE features may need to address the reconstruction residual explicitly to close the recovery path.","Behaviors could be distributed such that multiple SAE features or their interactions must be controlled together.","Alternative decomposition techniques beyond standard SAEs might reduce the size of the unexplained residual."],"forward_implications":["In refusal steering, recovery reaches 95.8 percent on valid samples with defended-feature relative drift at 0.131.","Recovery is possible in TPP, unlearning, IOI, and refusal steering settings.","Recovery localizes to the component of the residual unexplained by the SAE.","The gap between feature control and behavioral control holds under strong threat models with active intervention."],"fun_headline_variants":["SAE feature clamps do not prevent behavior recovery in refusals","Recovery optimization revives behavior despite active feature clamps","Suppressed behavior returns post SAE intervention via residuals","Feature clamps leave SAE reconstruction residual for recovery","Behavior recovery achievable while keeping clamped features unchanged"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The encoder-orthogonal updates for single-layer cases and the feature-map Jacobian for cross-layer cases prevent recovery from simply undoing the intervention rather than finding an alternative path.","fun_headline_variants_meta":{"raw":{"variants":["SAE feature clamps do not prevent behavior recovery in refusals","Recovery optimization revives behavior despite active feature clamps","Suppressed behavior returns post SAE intervention via residuals","Feature clamps leave SAE reconstruction residual for recovery","Behavior recovery achievable while keeping clamped features unchanged"]},"model":"grok-4.3","cost_usd":0.00553,"raw_usage":{"total_tokens":2681,"prompt_tokens":723,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":55299500,"prompt_tokens_details":{"text_tokens":723,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1888,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":723,"tokens_out":70,"duration_ms":22227,"temperature":1.0,"reasoning_tokens":1888,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T01:30:48.505408+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment across the same tasks where no residual perturbation recovers the original behavior while keeping the clamped SAE features fixed and the intervention active throughout generation.","supporting_citations":[],"review_version":1}