{"id":"06345610-058f-4a46-bb96-0986d90c0ac8","arxiv_id":"2505.06262","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Dialz is a modular Python toolkit for computing, applying, and visualizing steering vectors in open-source LLMs, demonstrated on stereotype mitigation and hallucination visualization.","lead":"Dialz is a new Python library that makes it easier for researchers to study and control the internal 'steering vectors' of open-source AI language models. It bundles dataset creation, vector computation, scoring, and visualization, and the authors show it can reduce stereotyped outputs in a demonstration.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Steering evaluation conflates stereotype mitigation with refusal; Figure 1's steered output is a refusal, so Table 1's reported reduction may not demonstrate the claimed capability.","rationale":"The reader identified LLM-as-a-judge validity as the weakest assumption. My concern is adjacent but more specific: the judge is not the only problem, because the intervention itself appears to change response format from generation to refusal, and the judge cannot distinguish a low-stereotype engaged answer from a refusal. Figure 1 provides direct evidence that the race-steering vector produces a refusal-style output, so this is not speculative. The toolkit contribution may still be useful, and the software itself is not undermined, but the empirical demonstration in Section 4.1 needs an additional control before the paper can claim that steering reduces stereotypical outputs. The reader's CONDITIONAL verdict remains appropriate, so I would not change it; the condition should include adding refusal/helpfulness analysis and, ideally, releasing the 150 eval prompts and human validation of a sample.","tokens_in":11057,"tokens_out":3212,"duration_ms":45072,"concrete_test":"Annotate all 450 outputs from Section 4.1 (baseline, prompting, steered) for whether each response engages with the prompt versus refuses or deflects, using two independent human annotators with disagreements adjudicated. Then recompute the mean GPT-4o stereotypicality separately for engaged outputs only. If the steered condition shows a substantially higher refusal rate than the prompting condition and its advantage over prompting disappears or shrinks materially among engaged-only outputs, then Table 1 should be reinterpreted as showing that steering reduces stereotypicality by inducing refusal, not by mitigating stereotypes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim of Section 4.1 is that steering vectors 'reduce stereotypicality more effectively than prompting alone' (Table 1). The load-bearing assumption is that a lower GPT-4o stereotypicality score corresponds to less stereotypical content while still responding to the prompt. This is not controlled. The sole visual example of the race-steering intervention (Figure 1) shows the steered output does not continue the story; it produces a refusal-style response ('This is not an appropriate or ethical way to portray any person...'). Such outputs will receive uniformly low stereotypicality ratings from any judge, yet they demonstrate non-engagement rather than stereotype mitigation. Section 4.1 reports no refusal rate, no answer-appropriateness measure, and no human evaluation of content quality for the 450 generated outputs. The comparative claim in Table 1 is therefore confounded: the steering condition may appear superior largely because it makes the model refuse, and the 'significant drop' may be an artifact of the evaluation metric rather than evidence that steering reduces stereotyping in otherwise helpful responses.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Dialz, an open-source Python toolkit for steering-vector research. The toolkit provides modules for creating or loading contrastive-pair datasets, computing steering vectors by PCA or mean difference, applying them during generation, scoring token-level activation alignment, and visualizing alignment heatmaps. The authors compare Dialz with existing libraries and report a demonstration in which steering vectors reduce GPT-4o-judged stereotypicality more effectively than prompting alone on Mistral 7B Instruct (Section 4.1), plus qualitative layer-wise visualizations for hallucination-related activations (Section 4.2). The central claim is that Dialz makes steering-vector experimentation easier and that the steering intervention itself reduces stereotypical outputs.","tokens_in":11153,"tokens_out":3163,"duration_ms":44645,"significance":"If properly validated, Dialz fills a real gap: existing tools such as repeng and steering-vectors focus on constructing and applying steering vectors, while Dialz additionally bundles dataset management, scoring, and visualization functionality. The paper ships an open-source package with documentation and tutorials, and the proposed workflow is coherent and likely useful to the interpretability community. However, the current empirical evidence for the flagship safety claim is too weak to support the conclusions as written. The refusal confound in Section 4.1, the reliance on an unvalidated LLM judge, and the absence of significance testing mean that the paper overstates what is demonstrated.","major_comments":[{"comment":"The comparison between prompting and steering is confounded by refusal behavior. Figure 1, the only qualitative example of the race-steering intervention, shows that the steered output does not continue the story but instead produces a refusal-style response (\"This is not an appropriate or ethical way to portray any person...\"). Such outputs receive low stereotypicality scores from any judge while not demonstrating that the model produces non-stereotypical, on-task content. Section 4.1 reports no refusal rate, no answer-appropriateness score, and no human evaluation of output quality for the 450 generated outputs. The claim that steering vectors reduce stereotypicality \"more effectively than prompting alone\" is therefore not supported by the reported data; the authors should measure refusal/engagement rates and, ideally, have human annotators judge both stereotypicality and task compliance.","section":"Section 4.1, Table 1, Figure 1"},{"comment":"The evaluation relies entirely on GPT-4o as an LLM judge, and the Limitations section concedes that this metric \"may not always reflect human judgment.\" Since all quantitative evidence for the stereotype-mitigation claim in Table 1 comes from this judge, the authors should provide at least a calibration study on a subsample of outputs with human ratings, or otherwise demonstrate that GPT-4o's stereotypicality scores correlate with human judgments for the specific intervention being tested. Without such validation, the reported reductions could be an artifact of judge bias rather than genuine stereotype mitigation.","section":"Section 4.1 and Limitations"},{"comment":"The paper describes the stereotypicality drop as \"significant,\" but no statistical test is reported. Table 1 gives means and standard deviations across five runs, with no per-category sample sizes, no effect sizes, and no correction for multiple comparisons. A permutation or bootstrap test on the 150 prompts (or a mixed-effects model with judge variance) is needed to support the significance claim. This is a load-bearing issue because the Conclusions rest on this \"significant drop.\"","section":"Section 4.1, Table 1, Conclusions"},{"comment":"The experimental setup uses a single model (Mistral 7B Instruct v0.1) and 150 prompts generated by DeepSeek R1, but the actual prompt list is not provided; Appendix C only shows the meta-prompt used to generate them. This limits reproducibility and makes it difficult to assess how representative or adversarial the prompts are. Please release the full prompt set, generated outputs, and judge ratings, or at least a substantial sample, so that the reported results can be independently checked.","section":"Section 4.1 and Appendix C"}],"minor_comments":[{"comment":"The abstract has an unlinked footnote marker \"1\" after \"reliable AI systems\" that points to the GitHub URL; this should be referenced in the text near the first mention of the repository.","section":"Abstract and Introduction"},{"comment":"Equation (3) defines wMD_l,t, but the following sentence refers to vMD_l,t; please unify the notation.","section":"Section 3.3, Equation (3)"},{"comment":"The notation \"|D|M i=1\" is unclear; it should be written as an explicit concatenation over the dataset index i so that the dimensions of X_l,t are unambiguous.","section":"Section 3.3, Equation (1)"},{"comment":"The hallucination visualization discussion is qualitative and anecdotal; please clarify in the text that this is an illustrative demonstration rather than a quantitative evaluation of hallucination detection.","section":"Section 4.2"},{"comment":"The code snippet uses display(html), which requires a Jupyter environment; a brief note indicating this dependency would help users running the example as a plain script.","section":"Section 3.5"}],"recommendation":"major_revision","confidential_remarks":"The software contribution is genuine and likely useful, but the empirical section overclaims. I would like to see the authors address the refusal confound and strengthen the evaluation before the paper appears in its current form; otherwise the safety claims will not withstand scrutiny. The paper may be better framed as a systems/demo contribution with clearly labeled illustrative experiments if the authors prefer not to run a full human evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: this is a software paper. Dialz packages dataset loading, vector training (PCA and mean difference), activation scoring, and visualization into one pip-installable library. That fills a real gap: repeng and steering-vectors don't have the datasets and scoring/visualization modules. The code appears to be released with docs and tutorials. If the library works as advertised, it saves researchers real time.\n\nThe empirical section is where the soft spots are. Section 4.1 claims steering reduces stereotypicality more than prompting. But the only qualitative example of the race steering (Figure 1) shows the model refusing to continue: \"This is not an appropriate or ethical way to portray any person...\" A refusal gets a low stereotypicality score from GPT-4o by construction, but it is not stereotype mitigation; it is non-engagement. Table 1 reports no refusal rate, no answer-appropriateness check, no human evaluation, and uses one model and 150 auto-generated prompts. So the comparative claim is partly confounded. The authors do acknowledge in Limitations that the LLM judge may not reflect human judgment, but that doesn't address the refusal issue. This needs a straightforward fix: report refusal rates, filter refusals, or evaluate only completions that actually engage with the prompt; add a human sample.\n\nOther things: the hallucination visualization is qualitative but does what it claims, and the paper doesn't overclaim there. The methods are not new, and the authors correctly cite Zou et al. and Turner et al. The citation pattern looks fine, no self-citation abuse. The prompt list is not released, which would help reproducibility. Minor: the abstract's \"paving the way for safer, more transparent...\" is boilerplate but runs slightly ahead of what is shown.\n\nMy take: the toolkit is the contribution. The safety demo is suggestive at best. I'd send this to peer review with a request for revision focused on de-confounding the stereotype experiment and releasing the evaluation prompts. It deserves referee time, but acceptance should hinge on the fix.\n\nBest.","headline":"A useful steering-vector toolkit whose main demo is partly undercut by a refusal confound; the library is the contribution, not the safety results.","tokens_in":11731,"tokens_out":1303,"would_cite":true,"duration_ms":16280,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dialz packages activation steering end-to-end and shows that steering vectors cut LLM stereotypicality more than prompting alone.","keywords":["steering vectors","activation engineering","contrastive pairs","LLM interpretability","stereotype mitigation","PCA","mean difference","token-level visualization"],"falsifier":"Re-run the 150-prompt benchmark from Section 4.1 with human stereotypicality ratings or with a behavioral probe such as StereoSet bias scores and lexical stereotype lists, and check whether steering still beats prompting; if human ratings show no advantage for steering over prompting, the judge-based result is called into question.","tokens_in":10791,"feed_emoji":"🧭","tokens_out":5749,"duration_ms":62955,"temperature":0.7,"pith_summary":"Dialz is an open-source Python library that packages the full workflow of activation steering: creating contrastive prompt datasets, computing steering vectors with PCA or mean difference, applying them during generation, scoring how strongly inputs align with a vector, and visualizing token-level alignment. The paper's empirical claim is that steering vectors computed from StereoSet sentence pairs reduce stereotypical output more than prompting alone: on Mistral 7B Instruct, GPT-4o-rated stereotypicality drops from 7.1 to 2.2 for race, 6.5 to 4.3 for gender, and 6.3 to 3.2 for religion, while a fairness prompt only reaches 5.0, 4.5, and 4.8. The authors also show that projecting activations onto a hallucination vector separates factual from fabricated statements at token level. A sympathetic reader should take away that steering vectors are a viable, interpretable complement to prompting for safety interventions, and that Dialz lowers the barrier to running such experiments.","feed_headline":"Steering vectors beat prompting at cutting stereotypes","feed_subtitle":"Dialz cuts GPT-4o-rated stereotypicality from 7.1 to 2.2 on race prompts with one activation direction.","key_machinery":"The central object is the steering vector: for a concept $t$ and layer $l$, one forms a matrix $X_{l,t}$ whose rows are hidden-state differences $h^+_{i,l} - h^-_{i,l}$ between positive and negative contrastive prompts, then takes either the first principal component of $X_{l,t}$ (the PCA method, following linear artificial tomography) or the mean of those differences. The vector is applied at inference time by adding a scalar multiple of it to the model's activations at selected layers. The Score module projects hidden states onto the vector with dot products (mean, final-token, or max-token) to quantify alignment, and the Visualize module maps those dot products onto token-level color gradients. This same machinery carries both the stereotype-mitigation experiment and the hallucination visualization.","core_discovery":"On the authors' own terms, the paper establishes that a single direction in activation space, computed from contrastive StereoSet pairs and applied with coefficient 1 to layers 10-19, reliably suppresses stereotypical completions in Mistral 7B Instruct. The reduction is consistent across race, gender, and religion categories and is larger than the reduction achieved by appending an explicit fairness instruction. The paper further reports that the same vector machinery, applied to a hallucination dataset, yields a direction whose per-token dot products clearly separate true statements from fabricated ones, with layer 18 of Llama 3.1 8B showing the strongest contrast. These results are presented as evidence that steering vectors can alter model behavior along targeted concepts and serve as a diagnostic tool for interpretability.","pith_inferences":["If part of the improvement comes from a generic refusal or disclaimer style that the judge rewards, pairing steering with human evaluation or behavioral stereotype tests would clarify whether actual stereotyping is removed.","Because steering vectors are computed per layer, the same toolkit could test whether different safety concepts localize to distinct layers; the paper's layer-18 observation for hallucination suggests such layer specificity exists.","A testable extension is combining multiple concept vectors, for instance race, gender, and religion, into an ensemble and measuring bias reduction on a held-out prompt set, along the lines of the authors' prior work on steering ensembles."],"forward_implications":["Steering vectors give a parameter-free, inference-time intervention that outperforms a fairness prompt on the tested model and judge, suggesting they can complement prompt engineering for safety.","Researchers can create or load contrastive datasets and reuse the same pipeline across concepts such as sycophancy, refusal, hallucination, and stereotypes, making prior results easier to replicate and extend.","Token-level visualization provides a diagnostic: words carrying false content light up against the hallucination vector, which can aid failure analysis in generation.","The library's design turns steering-vector research into a modular workflow rather than one-off scripts, accelerating iteration on dataset choice, layer selection, and steering coefficient."],"supporting_citations":[{"why":"Supplies the PCA / linear-artificial-tomography method that Dialz uses to compute steering vectors from contrastive activation differences.","marker":"Zou et al. (2023)"},{"why":"Introduced contrastive activation addition and the single-pair steering concept that the toolkit operationalizes.","marker":"Turner et al. (2023)"},{"why":"Provided the sycophancy, hallucination, refusal, and related contrastive datasets loaded by Dialz, plus the steering methodology it builds on.","marker":"Rimsky et al. (2024)"},{"why":"StereoSet is the source of the race, gender, and religion sentence pairs used to train the stereotype steering vectors in Section 4.1.","marker":"Nadeem et al. (2021)"},{"why":"Defines Mistral 7B Instruct v0.1, the model on which the stereotype-reduction benchmark is run.","marker":"Jiang et al. (2023)"},{"why":"Generated the 150 stereotype-eliciting prompts used to build the evaluation benchmark.","marker":"DeepSeek-AI (2025)"},{"why":"Provides Llama 3.1 8B Instruct, the model used for the hallucination visualization experiments.","marker":"AI@Meta (2024)"}],"fun_headline_variants":["Dialz: steer LLMs with one activation direction","Steering vectors cut stereotype bias in open LLMs","Dialz: one vector reduces race stereotype score","Activation direction lowers bias in LLM outputs","Dialz: Python toolkit for concept steering in LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, one the authors concede in their Limitations section, is that GPT-4o's 1-10 stereotypicality ratings capture actual harmful stereotyping; if the judge instead rewards a refusal style, the headline reductions could overstate the safety gain.","fun_headline_variants_meta":{"raw":{"variants":["Dialz: steer LLMs with one activation direction","Steering vectors cut stereotype bias in open LLMs","Dialz: one vector reduces race stereotype score","Activation direction lowers bias in LLM outputs","Dialz: Python toolkit for concept steering in LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000966,"raw_usage":{"total_tokens":4070,"prompt_tokens":862,"completion_tokens":3208,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":3133}},"tokens_in":478,"tokens_out":3208,"duration_ms":34764,"temperature":1.0,"reasoning_tokens":3133,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T01:00:22.202078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 150-prompt benchmark from Section 4.1 with human stereotypicality ratings or with a behavioral probe such as StereoSet bias scores and lexical stereotype lists, and check whether steering still beats prompting; if human ratings show no advantage for steering over prompting, the judge-based result is called into question.","supporting_citations":[],"review_version":1}