{"id":"1b744f04-2f2f-4a0a-b0b5-e5e691529d85","arxiv_id":"2605.28639","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Prohibited concepts remain recoverable from hidden states, influence attention routing, and shape generations in transformers under instruction-based suppression.","lead":"The paper finds that instructing transformer language models to suppress prohibited concepts leaves those concepts highly recoverable in hidden representations, still shaping attention patterns and downstream output despite clean lexical behavior. A smart generalist should read it to understand why current instruction-based safety methods may fail to change what models internally represent.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Probing classifiers, attention analysis, and semantic controls may introduce artifacts unrelated to suppression","rationale":"The reader's weakest assumption directly identifies the isolation of suppression effects as the key uncertainty. Given that the provided information is limited to the abstract, this remains the single most load-bearing methodological concern; no other internal inconsistency or unsupported assumption can be diagnosed from the available text.","tokens_in":1591,"tokens_out":306,"duration_ms":16429,"concrete_test":"Re-train the probing classifiers exclusively on hidden states from the suppression-prompt condition (holding out a validation set from the same condition) and re-evaluate recoverability accuracy; if accuracy drops below the reported levels or becomes indistinguishable from a matched non-prohibited control concept, the original results are likely driven by classifier artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that representational probing recovers the prohibited concept specifically because of (or despite) suppression, that attention routing is influenced by the concept under suppression instructions, and that semantic leakage occurs beyond lexical avoidance. This rests on the assumption that the chosen probing classifiers, head selection criteria, pooling strategies, and indirect semantic controls do not themselves create or amplify apparent recoverability and influence. If the classifiers are trained on data containing the target concept outside the suppression regime, or if the indirect controls fail to match all semantic dimensions except the suppression instruction, the measured effects could arise from general model knowledge or experimental design rather than the white-bear phenomenon itself.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper investigates whether instruction-based suppression of prohibited concepts in transformer language models eliminates their internal representations or merely blocks lexical output. Through representational probing, attention pattern analysis, and semantic leakage experiments on downstream generations, the authors claim that prohibited concepts remain highly recoverable from hidden states, continue to shape attention routing, and measurably influence generated text despite successful lexical avoidance. These effects are reported to persist across pooling strategies, indirect semantic controls, and multiple model families, indicating a gap between behavioral suppression and representational alignment.","tokens_in":1718,"tokens_out":592,"duration_ms":32256,"significance":"If the central empirical claims hold after methodological clarification, the work would be significant for AI alignment research by demonstrating that common suppression techniques are superficial at the representational level. The multi-model evaluation and use of indirect controls represent strengths that could help distinguish suppression-specific effects from general model knowledge. However, the absence of quantitative effect sizes, error bars, or explicit validation metrics in the provided abstract limits immediate assessment of practical impact.","major_comments":[{"comment":"§3 (Representational Probing): The probing classifiers appear to be trained on data containing the target concept outside the suppression regime. This setup risks attributing recoverability to general pretraining knowledge rather than persistence under suppression instructions; a within-regime training control or explicit comparison to non-suppressed baselines is needed to support the claim that suppression fails to reduce internal representations.","section":"§3"},{"comment":"§4.1 (Indirect Semantic Controls): No quantitative validation (e.g., cosine similarity thresholds, embedding distances, or human ratings) is provided to confirm that the indirect controls match all semantic dimensions except the suppression instruction. Without this, observed semantic leakage in generations could arise from residual semantic overlap rather than the attentional white bear effect.","section":"§4.1"},{"comment":"§5 (Attention Analysis): The criteria for identifying heads that 'continue to influence attention routing' under suppression are unspecified. If heads were selected post-hoc based on the presence of the effect, this introduces selection bias that undermines the claim of continued influence independent of experimental design choices.","section":"§5"}],"minor_comments":[{"comment":"The abstract states effects 'persist across pooling strategies' but does not list the specific strategies or report per-strategy statistics; adding a table with these breakdowns would improve clarity.","section":null},{"comment":"Figure captions for attention visualizations should explicitly state the suppression prompt template and model variant used in each panel.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript's focus on AI safety/alignment may align better with conference venues (ACL, NeurIPS) than traditional journals, but this does not affect the technical assessment."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive feedback on our manuscript. We address each of the major comments below, providing clarifications and indicating where revisions will be made to strengthen the paper.","responses":[{"response":"We appreciate this point. Our experiments train probes on hidden states extracted from inputs that include the suppression instruction, and we compare these to probes trained on non-suppressed inputs to show the persistence of the concept under suppression. To further address the concern about general pretraining knowledge, we will incorporate a within-regime training control in the revised manuscript, training probes exclusively on suppressed data and testing on held-out suppressed examples.","revision_made":"yes","referee_comment":"[§3] §3 (Representational Probing): The probing classifiers appear to be trained on data containing the target concept outside the suppression regime. This setup risks attributing recoverability to general pretraining knowledge rather than persistence under suppression instructions; a within-regime training control or explicit comparison to non-suppressed baselines is needed to support the claim that suppression fails to reduce internal representations."},{"response":"We agree that additional quantitative validation would enhance the robustness of our indirect semantic controls. In the original manuscript, the controls were constructed to differ only in the suppression aspect based on semantic similarity measures, but we did not report explicit thresholds. We will add cosine similarity thresholds, embedding distance metrics, and details on how the controls were validated in the revised version.","revision_made":"yes","referee_comment":"[§4.1] §4.1 (Indirect Semantic Controls): No quantitative validation (e.g., cosine similarity thresholds, embedding distances, or human ratings) is provided to confirm that the indirect controls match all semantic dimensions except the suppression instruction. Without this, observed semantic leakage in generations could arise from residual semantic overlap rather than the attentional white bear effect."},{"response":"The head selection was performed using a pre-specified criterion based on attention weight differences across conditions, applied consistently before examining the effect on generations. This was not post-hoc. We will explicitly state the selection criteria and thresholds in the methods section of the revised manuscript to clarify this process and mitigate concerns about selection bias.","revision_made":"yes","referee_comment":"[§5] §5 (Attention Analysis): The criteria for identifying heads that 'continue to influence attention routing' under suppression are unspecified. If heads were selected post-hoc based on the presence of the effect, this introduces selection bias that undermines the claim of continued influence independent of experimental design choices."}],"tokens_in":1303,"tokens_out":549,"duration_ms":41282,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is that the authors test whether telling a model not to generate something actually removes the idea from its hidden states or just blocks the words. They report that the concepts stay recoverable via probes, keep affecting attention patterns, and still influence generations even when the model avoids the forbidden terms.\n\nWhat is new is the specific package: they combine representational probing, attention analysis, and semantic leakage checks on instruction-based suppression, and they run it across multiple model families while varying pooling and using indirect semantic controls. That setup tries to show the effect is not just lexical avoidance.\n\nThe work does a reasonable job of attempting controls and testing robustness across models, which is a fair way to look for a consistent gap between behavior and representation.\n\nThe soft spots are clear from the abstract. There are no quantitative results, effect sizes, error bars, or exclusion criteria reported, so it is impossible to tell how strong or reliable the effects are. The stress-test point about possible artifacts from the probing classifiers, head selection, or control choices lands because the abstract gives no information on how the probes were trained or whether the indirect controls matched all relevant dimensions. If those choices introduce their own signals, the claimed white-bear effect could be overstated.\n\nThis is for researchers working on AI safety and alignment evaluations who want to know whether behavioral suppression actually changes internal states. A reader focused on representation-versus-behavior gaps would get some value from the experimental framing if the full methods and numbers hold up.\n\nI would send it to peer review. The question is relevant and the approach is straightforward enough to be worth referee time, even though the current write-up is too thin on data to assess the claims.","headline":"The paper claims suppression instructions leave prohibited concepts intact in LLM internals and still shape attention and output, but the abstract alone gives no numbers or method details to judge if that's supported.","tokens_in":2180,"tokens_out":423,"would_cite":false,"duration_ms":23301,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Prohibited concepts remain internally active in language models even when their output is suppressed by instructions.","keywords":["language models","instruction suppression","representational probing","attention analysis","semantic leakage","prohibited content","model alignment"],"falsifier":"An experiment in which prohibited concepts become unrecoverable from hidden representations under suppression, show no measurable influence on attention routing, and produce no semantic leakage in generations while still achieving lexical avoidance.","tokens_in":2486,"feed_emoji":"🧠","tokens_out":569,"duration_ms":34744,"temperature":0.7,"pith_summary":"The paper examines whether telling language models to avoid prohibited content actually removes those ideas from their thinking or merely stops them from saying the words. Researchers used tests to read out hidden information from the models, look at how attention is directed, and see if forbidden ideas leak into other parts of the responses. They discovered that the concepts stay easy to detect inside the model, keep affecting where the model pays attention, and still influence what the model generates even when direct mentions are avoided. This holds true no matter how the information is pooled or which models are tested. The result points to a mismatch between what the model does on the surface and what it represents underneath.","feed_headline":"Prohibited concepts stay recoverable inside suppressed language models","feed_subtitle":"Hidden states and attention patterns show instruction-based avoidance leaves internal activity intact across models.","key_machinery":"The attentional white bear effect, in which instruction-based suppression of a concept keeps it active in hidden states and attention patterns rather than removing it.","core_discovery":"Prohibited concepts remain highly recoverable from hidden representations under suppression, continue to influence attention routing, and measurably shape downstream generations despite successful lexical avoidance. These effects persist across pooling strategies, indirect semantic controls, and multiple model families, exposing a fundamental gap between behavioral and representational alignment.","pith_inferences":["Safety techniques relying only on output instructions may leave internal concept processing intact.","Direct interventions on hidden representations could be needed to achieve fuller suppression.","The same internal persistence might occur in non-text modalities or other model types."],"forward_implications":["Prohibited concepts can be recovered from model internals despite output suppression.","Attention mechanisms continue to route based on suppressed concepts.","Downstream text generation is influenced by these concepts in non-lexical ways.","The effects are consistent across different analysis methods and model types."],"fun_headline_variants":["Suppressed concepts stay active in model hidden representations","Instruction avoidance leaves concepts in transformer internals","White bear effect shows in attention under content suppression","Representational probes recover prohibited concepts despite suppression"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The methods for probing representations, analyzing attention, and checking semantic leakage isolate the effects of suppression without introducing their own artifacts or biases.","fun_headline_variants_meta":{"raw":{"variants":["Suppressed concepts stay active in model hidden representations","Instruction avoidance leaves concepts in transformer internals","White bear effect shows in attention under content suppression","Representational probes recover prohibited concepts despite suppression"]},"model":"grok-4.3","cost_usd":0.003557,"raw_usage":{"total_tokens":1782,"prompt_tokens":503,"num_sources_used":0,"completion_tokens":54,"cost_in_usd_ticks":35574500,"prompt_tokens_details":{"text_tokens":503,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1225,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":503,"tokens_out":54,"duration_ms":13781,"temperature":1.0,"reasoning_tokens":1225,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T12:40:55.522390+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment in which prohibited concepts become unrecoverable from hidden representations under suppression, show no measurable influence on attention routing, and produce no semantic leakage in generations while still achieving lexical avoidance.","supporting_citations":[],"review_version":1}