Pith. sign in

REVIEW 3 major objections 5 minor 273 references

Policy Gradient Steering: Interventions from Behavioral Objectives

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Policy Gradient Steering turns a few scored trajectories into a removable activation vector that steers behavior where contrastive activation methods cannot.

desk verdict A clean and mostly honest paper that adds a real method to activation steering, but the chess composition claim is built on a KL-matching premise that is never verified at the scale used. read the letter →

arxiv 2607.27574 v1 pith:D5APOQ4L submitted 2026-07-30 cs.LG cs.MA

Yoann Poupart , Aurélie Beynier , Nicolas Maudet This is my paper · ORCID
classification cs.LGcs.MA
keywords policygradientsteeringactivationinference-timeinterventiontaskvectorsreinforcementlearningbehavioralcontrolcomposableinterventionsmulti-agentbehavior
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing activation-steering methods build interventions from contrasts between representations reached after outcomes diverge; the paper shows this construction fails to steer a simple two-route gridworld policy because the contrast encodes the consequence of a route choice rather than the action that caused it. Policy Gradient Steering (PGS) instead treats steering as a reinforcement-learning problem: it accumulates gradients of a temporary behavioral objective with respect to an activation layer, weighting each action's score gradient by its behavioral return, and stores the result as one additive offset vector. The vector is calibrated under an action-KL budget using the activation-space Fisher matrix, so it can be scaled, composed, and removed at inference time while the base policy stays frozen. Across the three settings - gridworld, chess puzzles, and competitive football - PGS shifts route preference where contrastive baselines do not, its tactical vectors compose constructively, and its team-behavior effects transfer across opponents. If correct, PGS gives a compact, reusable interface for post-training behavioral control in exactly the cases where outcome-based contrasts cannot assign credit.

What carries the argument

The central object is the PGS steering vector v_PGS = E_D[rho_t (G_t - b(o_t)) grad_h log pi(a_t|o_t)]: the likelihood-ratio (REINFORCE) gradient of a temporary behavioral return with respect to a chosen activation layer, averaged over a small batch of rollouts. This vector carries the argument by assigning credit to the actions that produced outcomes and by being state-independent - one offset added at every decision. The companion identity is the Fisher calibration alpha = sqrt(2 epsilon / (v_PGS^T F v_PGS)), which treats the intervention as a small offset in a locally quadratic basin of the action KL and sets its strength to a policy-change budget epsilon. Per-decision importance ratios r

What would settle it

After applying a PGS vector to a chess policy, remove it by setting alpha = 0 on the same held-out puzzles and check that the canonical-move likelihood returns to the unsteered value; if the effect persists after removal (for example because of stateful normalization in the forward pass), the intervention is not cleanly removable as claimed.

Watch

Extended reading notes

Core claim

The paper's central claim is that a policy-gradient estimator evaluated at a hidden activation - not at the policy weights - turns scalar trajectory feedback into a removable behavioral intervention that contrastive activation methods cannot reliably produce. PGS computes v_PGS = E_D[rho_t (G_t - b(o_t)) grad_h log pi(a_t|o_t)] over a small scored-trajectory batch, then applies h <- h + alpha v_PGS at inference. Because grad_h log pi(a|o) differs across actions even when their pre-action representations are identical, PGS credits the branching action, where the decision-local contrast used by contrastive activation addition is exactly zero. Steering strength alpha = sqrt(2 epsilon / (v_PGS^T

Load-bearing premise

The local quadratic (Fisher) approximation of the action KL is accurate enough at the finite steering strengths used, so the closed-form coefficient alpha = sqrt(2 epsilon / (v_PGS^T F v_PGS)) keeps all compared methods at the same actual policy-change budget.

Editorial extensions

If this is right

  • PGS can steer a policy when the deciding action and its alternative share the same pre-action representation, a setting where contrastive activation addition produces zero decision-local gradient and leaves behavior nearly unchanged.
  • Independently fitted PGS vectors compose constructively: in chess, summing fork, pin, and skewer vectors gives composed gains close to fine-tuning while preserving each objective's isolated improvement (minimum retention above one).
  • PGS team-behavior interventions transfer across opponents in football: a passing intervention fitted against one opponent increases completed passes when played against each of the other evaluated opponents.
  • PGS achieves these changes with a small stored artifact (1,152 scalars versus 663,552 for fine-tuning in the chess all-block comparison), supporting lightweight post-training control.
  • Because the base policy parameters are untouched, the same vector can be applied at different strengths, negated, or removed at inference time without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If PGS vectors compose as cleanly as the chess results suggest, activation-space behavioral arithmetic could extend beyond plain summation - e.g., subtracting one vector to suppress a behavior, or averaging vectors fitted on different data sources - without retraining; the paper does not test these operations.
  • The football results indicate steering effectiveness depends on which trajectories the vector is fitted on; fitting on mixed-opponent data could reduce variance and improve transfer, which the paper leaves as future work.
  • Because the Fisher calibration is a local quadratic approximation, one testable extension is replacing the closed-form alpha with a small empirical KL backtracking at larger budgets; if the measured KL then departs from the requested epsilon, composition results at high strength would need re-calibration.
  • PGS's action-level credit could apply to language models at the token level, where each token choice precedes a change in hidden state; if token-level PGS vectors compose across attributes, it would give an alternative to contrastive activation editing for LLM behavior control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Policy Gradient Steering (PGS), a method that constructs an additive activation-space steering vector by accumulating return-weighted policy gradients of the log-likelihood of observed actions, computed from a small set of scored trajectories. The intervention is applied as h ← h + αv at inference time, leaving the base policy frozen. The paper evaluates PGS in three settings: a two-route gridworld where it shows that contrastive activation steering (CAA, COAST, K-Steer) fails to control the route decision, while PGS succeeds; chess puzzles where independently fitted PGS vectors for forks, pins, and skewers are combined and reported to compose constructively; and Google Research Football where PGS interventions are claimed to alter team behavior and transfer across opponents. The paper positions PGS as a compact, composable, and removable behavioral intervention method.

Significance. If the empirical claims are substantiated, PGS is a conceptually clean and practically useful addition to the post-training adaptation toolbox: it assigns credit to the actions that produce outcomes rather than to post-outcome representations, and it does so with a single gradient step and a storage overhead far smaller than LoRA or fine-tuning. The gridworld failure analysis (Section 4.3, Eq. 15) is a nice controlled demonstration of why outcome-based contrasts can fail at a branching decision. The paper also ships code and per-seed diagnostics (Appendix E), which supports reproducibility. However, the significance is currently tempered by two load-bearing empirical gaps: the chess experiments do not verify that each method actually operates at the same isolated action-KL budget, and the football transfer result lacks per-opponent unsteered baselines and is statistically underpowered. These issues must be addressed before the central claims can be fully accepted.

major comments (3)
  1. [§5.2, Tables 4–5, Eq. (12), Appendix B.2, Appendix D] The matched-budget premise is unverified in the chess experiments. All methods set α via the local quadratic (Fisher) approximation α = sqrt(2ε/(v^T F v)) with ε=0.1, using curvature estimated at coefficients ±1e-3 (Appendix B.2). The paper reports only composed KL for the three-way composition, not the held-out isolated action KL for each independently fitted intervention. If higher-order KL terms are non-negligible at the applied scale, the actual isolated KL may differ across methods and across seeds, undermining the comparison of isolated gains in Table 4 and the composition-retention metric R_min in Eq. (17). The paper's own Section 7 acknowledges that the local Fisher approximation may be inaccurate at finite scale. The gridworld verification (Table 2) does not transfer automatically to the deeper LCZero policy. Please report isolated held-out action KL for each method and seed in
  2. [§6.2, Table 9] The cross-play transfer claim is not supported by the data as presented. The text states that 'each intervention increases mean completed passes relative to the unmodified policy across evaluation opponents,' but Table 9 contains only the PGS intervention counts and does not show the unmodified group_pressure policy's completed passes against each of the four opponents. Without per-opponent baselines, the reported numbers could reflect opponent-specific baseline differences rather than a transferable intervention effect. Please add the unsteered group_pressure per-opponent counts (e.g., as a Base row or separate table) so the reader can directly verify the claimed increase.
  3. [§6.2, Table 8] The football results are statistically weak due to very large standard deviations and small sample sizes (n=5). For example, PGS yields 29.80±16.04 completed passes and LoRA 32.00±17.68; the difference between the two is far within noise, yet the text says PGS yields 'a completed-pass count comparable to the strongest learned adaptation' and a 'pronounced shift' relative to base. The mean differences versus the base are large, but the high variance means that the relative ordering of methods is not reliable. Please provide confidence intervals or effect sizes, and temper the comparative claims accordingly. At minimum, state that differences among the top methods are not statistically separable at this sample size.
minor comments (5)
  1. [General] The paper uses 'sample standard deviation' throughout, but with n=5 the standard error is much smaller; consider reporting both or explicitly noting the small sample size in the main tables.
  2. [Table 5] The maia-1500 rows in Table 5 repeat Table 4 but with small numeric differences (e.g., PGS composed gain 0.805 vs 0.804, LoRA 0.804 vs 0.803). Please clarify whether these are the same runs or different seeds, and ensure consistency.
  3. [Algorithm 1 (line 2)] The algorithm says 'Estimate the baseline b(o_t)' while the text states an action-independent baseline; please align the notation.
  4. [Eq. (8) and Section 3.2] The importance ratio ρ_t is introduced after the main formula; consider defining it before Eq. (8) or noting that ρ_t=1 for on-policy data to avoid initial confusion.
  5. [Table 8] The 'Goal diff.' column for ReFT reports 0.00±0.00, which is suspicious; please verify that no goals occurred in any of the five matches for that condition.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: PGS results are measured on held-out data; the only self-citation is a non-load-bearing software dependency.

full rationale

The derivation chain is self-contained. PGS vectors are formed by Eq. 8 as return-weighted action-score gradients over a fixed batch, and alpha is set by Eq. 12 from the local Fisher curvature. The headline results (gridworld route shifts, chess gains and composition retention, football event counts) are measured on disjoint held-out data rather than derived from the fitted parameters. No reported equation reduces a result to its inputs by construction. The calibration formula is an approximation that the paper explicitly labels local, and Section 7 states that if the local Fisher approximation is inaccurate at finite scale, empirical action-KL backtracking could refine the closed-form coefficient. This is an acknowledged verification concern, not a definitional reduction. The only self-citation is the first author's LCZeroLens software (Poupart 2026, Appendix E.2), used as an implementation dependency for chess experiments; it does not supply a load-bearing mathematical or empirical premise for the method's validity. The chess experiments do not report isolated held-out action KL, but the matched-budget premise is a potential confound, not circularity, and composed KL is reported as a measured outcome. Therefore the central claim has independent empirical content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

PGS introduces no new physical or ontological entities; the task-vector/steering-vector is an algorithmic artifact, not a new entity requiring independent evidence. The free parameters are the intervention strength, KL budget, discount factor, and activation sites, all chosen by hand or validation. The axioms are standard policy-gradient machinery plus domain assumptions about reward specification and environment fidelity.

free parameters (4)
  • steering strength alpha = 0.42 in gridworld; derived from KL budget elsewhere
    Controls intervention magnitude; selected on validation or via local Fisher curvature. Results depend on it and the quadratic approximation may fail at larger alpha.
  • KL budget epsilon = 0.1 in gridworld and chess
    Hand-chosen policy-change budget used to set alpha in Eq. 12; all methods are compared at this budget.
  • discount factor gamma = not fixed in the main text; football sweep over 0.0-1.0
    Return definition in Eq. 6 depends on gamma; the football ablation shows sensitivity, so the unstated main value is a free choice.
  • activation site(s) = gridworld: h1_post+h2_post; chess: all six residual blocks
    Intervention effectiveness depends strongly on the activation site (Tables 14 and 17); sites are chosen by ablation/validation, not derived from first principles.
assumptions (4)
  • standard math Policy-gradient theorem / REINFORCE gives a valid direction when differentiating log-probability w.r.t. activations
    Eq. 8; valid for on-policy data, and the authors acknowledge that the off-policy state-distribution mismatch is not corrected.
  • domain assumption Local quadratic approximation of action KL as a function of activation offset
    Eq. 12 and Appendix D; if inaccurate at finite alpha, calibration and KL-matched comparisons degrade.
  • domain assumption A scalar behavioral reward faithfully expresses the intended intervention
    Section 3.1; chess uses canonical-move binary reward and football uses event counts; a misspecified reward would steer the wrong behavior.
  • domain assumption The frozen base policies and environments provide a meaningful testbed with correctly handled action spaces
    Sections 4-6; legal-move masking in chess and event detection in football are needed for valid likelihoods and behavior counts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Policy Gradient Steering: Interventions from Behavioral Objectives." pith.science (2026). https://pith.science/paper/D5APOQ4L

@misc{pith2026260727574,
  author       = {Pith},
  title        = {Pith review of: Policy Gradient Steering: Interventions from Behavioral Objectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5APOQ4L}},
  note         = {Machine review of arXiv:2607.27574}
}
read the original abstract

Activation steering has emerged in large language models as a lightweight alternative for dynamically changing a model's behavior at inference time. However, we show that existing steering methods fail to steer even a simple policy in a two-route gridworld environment. To address this limitation, we propose Policy Gradient Steering (PGS), which formulates steering as a reinforcement learning problem. PGS accumulates gradients of a temporary behavioral objective over a small set of rollouts or demonstrations to construct a removable task vector. We first demonstrate the calibration and reversibility of PGS in a two-route gridworld environment. Using chess puzzles, we then evaluate independently fitted PGS vectors both in isolation and in combination, finding that compatible tactical objectives accumulate constructively. Finally, in competitive football, we show that PGS can alter specific team behaviors and that its effects transfer across opponents. Together, these results show that policy gradients provide a natural interface for constructing temporary and composable behavioral adaptations across diverse decision-making domains.

Figures

Figures reproduced from arXiv: 2607.27574 by the authors.

Figure 1
Figure 1. Overview of the policy gradient steering method. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Two-route Gridworld used to isolate behavioral [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Examples of tactical motifs used to fit and compose [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Google Research Football environment used for the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Mean statewise action probabilities for the original [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 5
Figure 5. Figure 5: Mean statewise action probabilities for the original [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 7
Figure 7. Figure 7: Frozen critic values for flank_pass, grouped by the active player’s role. The distributions reveal a pronounced positional bias in the uncentered critic output. Behavior Definitions and Base Phenotype [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

273 extracted references · 3 canonical work pages

  1. [1]

    ACM Computing Surveys , year=

    Explainable Reinforcement Learning: A Survey and Comparative Review , author=. ACM Computing Surveys , year=

  2. [2]

    ArXiv , year=

    A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges , author=. ArXiv , year=

  3. [3]

    ACM Computing Surveys , year=

    Explainability in Deep Reinforcement Learning: A Review into Current Methods and Applications , author=. ACM Computing Surveys , year=

  4. [4]

    Diffusion for World Modeling: Visual Details Matter in Atari , author=

  5. [5]

    and Polani, D

    Klyubin, A.S. and Polani, D. and Nehaniv, C.L. , booktitle=. Empowerment: a universal agent-centric measure of control , year=

  6. [6]

    2022 , eprint=

    Parametrically Retargetable Decision-Makers Tend To Seek Power , author=. 2022 , eprint=

    2022

  7. [7]

    2022 , eprint=

    Learning Altruistic Behaviours in Reinforcement Learning without External Rewards , author=. 2022 , eprint=

    2022

  8. [8]

    2022 , issue_date =

    Amirkhani, Abdollah and Barshooi, Amir Hossein , title =. 2022 , issue_date =. doi:10.1007/s10462-021-10097-x , journal =

    work page doi:10.1007/s10462-021-10097-x 2022

Show all 273 references
  1. [9]

    2015 , eprint=

    Explaining and Harnessing Adversarial Examples , author=. 2015 , eprint=

    2015

  2. [10]

    2020 , eprint=

    BERT-ATTACK: Adversarial Attack Against BERT Using BERT , author=. 2020 , eprint=

    2020

  3. [11]

    2023 , eprint=

    Visual Adversarial Examples Jailbreak Aligned Large Language Models , author=. 2023 , eprint=

    2023

  4. [12]

    2023 , eprint=

    Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models , author=. 2023 , eprint=

    2023

  5. [13]

    2023 , eprint=

    Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation , author=. 2023 , eprint=

    2023

  6. [14]

    Advances in Neural Information Processing Systems , editor=

    Optimal Policies Tend To Seek Power , author=. Advances in Neural Information Processing Systems , editor=. 2021 , ignore=

    2021

  7. [15]

    2024 , eprint=

    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training , author=. 2024 , eprint=

    2024

  8. [16]

    2023 , eprint=

    Goal Misgeneralization in Deep Reinforcement Learning , author=. 2023 , eprint=

    2023

  9. [17]

    2023 , eprint=

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection , author=. 2023 , eprint=

    2023

  10. [18]

    Multi-Agent Security Workshop @ NeurIPS'23 , year=

    Second-order Jailbreaks: Generative Agents Successfully Manipulate Through an Intermediary , author=. Multi-Agent Security Workshop @ NeurIPS'23 , year=

  11. [19]

    Multi-Agent Security Workshop @ NeurIPS'23 , year=

    Language Agents as Hackers: Evaluating Cybersecurity Skills with Capture the Flag , author=. Multi-Agent Security Workshop @ NeurIPS'23 , year=

  12. [20]

    NeurIPS 2023 Foundation Models for Decision Making Workshop , year=

    Mitigating Generative Agent Social Dilemmas , author=. NeurIPS 2023 Foundation Models for Decision Making Workshop , year=

    2023

  13. [21]

    2023 , eprint=

    LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents , author=. 2023 , eprint=

    2023

  14. [22]

    2023 , eprint=

    FIND: A Function Description Benchmark for Evaluating Interpretability Methods , author=. 2023 , eprint=

    2023

  15. [23]

    2023 , eprint=

    AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation , author=. 2023 , eprint=

    2023

  16. [24]

    NNUE Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi , journal=

    Yu Nasu , year=. NNUE Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi , journal=

  17. [25]

    Artificial intelligence , volume=

    An analysis of alpha-beta pruning , author=. Artificial intelligence , volume=. 1975 , publisher=

    1975

  18. [26]

    Mastering the game of Go without human knowledge , url =

    Silver, David and Schrittwieser, Julian and Simonyan, Karen and Antonoglou, Ioannis and Huang, Aja and Guez, Arthur and Hubert, Thomas and Baker, Lucas and Lai, Matthew and Bolton, Adrian and Chen, Yutian and Lillicrap, Timothy and Hui, Fan and Sifre, Laurent and van den Dries...

  19. [27]

    Bandit Based Monte-Carlo Planning

    Kocsis, Levente and Szepesv \'a ri, Csaba. Bandit Based Monte-Carlo Planning. Machine Learning: ECML 2006. 2006

    2006

  20. [28]

    2023 , eprint=

    Mindstorms in Natural Language-Based Societies of Mind , author=. 2023 , eprint=

    2023

  21. [29]

    2024 , eprint=

    Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL , author=. 2024 , eprint=

    2024

  22. [30]

    2024 , eprint=

    Large Language Model based Multi-Agents: A Survey of Progress and Challenges , author=. 2024 , eprint=

    2024

  23. [31]

    2023 , eprint=

    A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment , author=. 2023 , eprint=

    2023

  24. [32]

    Albrecht and Filippos Christianos and Lukas Sch\"afer , title =

    Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer , title =. 2024 , url =

    2024

  25. [33]

    Acquisition of chess knowledge in

    Thomas McGrath and Andrei Kapishnikov and Nenad Toma. Acquisition of chess knowledge in. doi:10.1073/pnas.2206625119 , url =

    doi

  26. [34]

    Advances in Neural Information Processing Systems , volume=

    Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex , author=. Advances in Neural Information Processing Systems , volume=

  27. [35]

    arXiv preprint arXiv:2309.09702 , year=

    Information based explanation methods for deep learning agents--with applications on large open-source chess models , author=. arXiv preprint arXiv:2309.09702 , year=

    arXiv

  28. [36]

    IFAC-PapersOnLine , volume=

    Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Concepts , author=. IFAC-PapersOnLine , volume=. 2023 , publisher=

    2023

  29. [37]

    Unveiling concepts learned by a world-class chess-playing agent , author=

  30. [38]

    2023 , eprint=

    Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero , author=. 2023 , eprint=

    2023

  31. [39]

    Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems , pages =

    Rodriguez, Sebastian and Thangarajah, John , title =. Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems , pages =. 2024 , isbn =

    2024

  32. [40]

    Adaptive Agents and Multi-Agent Systems , year=

    Design Patterns for Explainable Agents (XAg) , author=. Adaptive Agents and Multi-Agent Systems , year=

  33. [41]

    2023 , eprint=

    Deep reinforcement learning from human preferences , author=. 2023 , eprint=

    2023

  34. [42]

    2023 , eprint=

    Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safety , author=. 2023 , eprint=

    2023

  35. [43]

    Practices for Governing Agentic AI Systems , author=

  36. [44]

    ArXiv , year=

    Reinforcement Learning and Machine ethics:a systematic review , author=. ArXiv , year=

  37. [45]

    ArXiv , year=

    Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective , author=. ArXiv , year=

  38. [46]

    2024 , eprint=

    Levels of AGI: Operationalizing Progress on the Path to AGI , author=. 2024 , eprint=

    2024

  39. [47]

    2019 , eprint=

    Safe Reinforcement Learning via Probabilistic Shields , author=. 2019 , eprint=

    2019

  40. [48]

    2023 , eprint=

    Safe Reinforcement Learning via Probabilistic Logic Shields , author=. 2023 , eprint=

    2023

  41. [49]

    Nature , year=

    Controllability of complex networks , author=. Nature , year=

  42. [50]

    SIAM Journal on Control and Optimization , volume =

    Rahmani, Amirreza and Ji, Meng and Mesbahi, Mehran and Egerstedt, Magnus , title =. SIAM Journal on Control and Optimization , volume =. 2009 , doi =

    2009

  43. [51]

    Robustness to Multi-Modal Environment Uncertainty in

    Aakriti Agrawal and Rohith Aralikatti and Yanchao Sun and Furong Huang , booktitle=. Robustness to Multi-Modal Environment Uncertainty in. 2023 , ignore=

    2023

  44. [52]

    Multi-Agent Security Workshop @ NeurIPS'23 , year=

    I See You! Robust Measurement of Adversarial Behavior , author=. Multi-Agent Security Workshop @ NeurIPS'23 , year=

  45. [53]

    ArXiv , year=

    Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop , author=. ArXiv , year=

  46. [54]

    RLC , year=

    Informed POMDP: Leveraging Additional Information in Model-Based RL , author=. RLC , year=

  47. [55]

    Autonomous Agents and Multi-Agent Systems , year=

    A survey of multi-agent deep reinforcement learning with communication , author=. Autonomous Agents and Multi-Agent Systems , year=

  48. [56]

    Sensors (Basel, Switzerland) , year=

    Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey , author=. Sensors (Basel, Switzerland) , year=

  49. [57]

    Artificial Intelligence Review , year=

    Multi-agent deep reinforcement learning: a survey , author=. Artificial Intelligence Review , year=

  50. [58]

    2023 , ignore=

    Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions , author=. 2023 , ignore=

    2023

  51. [59]

    Nature , year=

    Grandmaster level in StarCraft II using multi-agent reinforcement learning , author=. Nature , year=

  52. [60]

    AAMAS Workshops , year=

    Cooperative Multi-agent Control Using Deep Reinforcement Learning , author=. AAMAS Workshops , year=

  53. [61]

    ArXiv , year=

    Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning , author=. ArXiv , year=

  54. [62]

    ArXiv , year=

    Foundation Models for Decision Making: Problems, Methods, and Opportunities , author=. ArXiv , year=

  55. [63]

    ArXiv , year=

    A Generalist Agent , author=. ArXiv , year=

  56. [64]

    Vicinagearth , year=

    A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges , author=. Vicinagearth , year=

  57. [65]

    ArXiv , year=

    LLM Multi-Agent Systems: Challenges and Open Problems , author=. ArXiv , year=

  58. [66]

    CoRR , year=

    Policy Distillation , author=. CoRR , year=

  59. [67]

    International Conference on Machine Learning , year=

    Self-Supervised Interpretable End-to-End Learning via Latent Functional Modularity , author=. International Conference on Machine Learning , year=

  60. [68]

    ArXiv , year=

    Interpretable Brain-Inspired Representations Improve RL Performance on Visual Navigation Tasks , author=. ArXiv , year=

  61. [69]

    ArXiv , year=

    Clustered Policy Decision Ranking , author=. ArXiv , year=

  62. [70]

    International Joint Conference on Artificial Intelligence , year=

    The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations , author=. International Joint Conference on Artificial Intelligence , year=

  63. [71]

    Neural Information Processing Systems , year=

    Sanity Checks for Saliency Maps , author=. Neural Information Processing Systems , year=

  64. [72]

    A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing , author=. Inf. , year=

  65. [73]

    ArXiv , year=

    Evaluating Post-hoc Interpretability with Intrinsic Interpretability , author=. ArXiv , year=

  66. [74]

    ArXiv , year=

    Revisiting the robustness of post-hoc interpretability methods , author=. ArXiv , year=

  67. [75]

    ACM Computing Surveys , year=

    Post-hoc Interpretability for Neural NLP: A Survey , author=. ACM Computing Surveys , year=

  68. [76]

    ArXiv , year=

    Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges , author=. ArXiv , year=

  69. [77]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Interpretable by Design: Learning Predictors by Composing Interpretable Queries , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  70. [78]

    Science , year=

    Human-level performance in 3D multiplayer games with population-based reinforcement learning , author=. Science , year=

  71. [79]

    Explainability in Deep Reinforcement Learning , author=. Knowl. Based Syst. , year=

  72. [80]

    IUI Workshops , year=

    Interestingness Elements for Explainable Reinforcement Learning through Introspection , author=. IUI Workshops , year=

  73. [81]

    ArXiv , year=

    Visualizing and Understanding Atari Agents , author=. ArXiv , year=

  74. [82]

    International Conference on Artificial Intelligence and Statistics , year=

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author=. International Conference on Artificial Intelligence and Statistics , year=

  75. [83]

    Neural Information Processing Systems , year=

    Verifiable Reinforcement Learning via Policy Extraction , author=. Neural Information Processing Systems , year=

  76. [84]

    Machine Learning , year=

    Explainable reinforcement learning (XRL): a systematic literature review and taxonomy , author=. Machine Learning , year=

  77. [85]

    Nature Machine Intelligence , year=

    From attribution maps to human-understandable explanations through Concept Relevance Propagation , author=. Nature Machine Intelligence , year=

  78. [86]

    Proceedings of the National Academy of Sciences of the United States of America , year=

    Impossibility theorems for feature attribution , author=. Proceedings of the National Academy of Sciences of the United States of America , year=

  79. [87]

    ArXiv , year=

    Not All Language Model Features Are Linear , author=. ArXiv , year=

  80. [88]

    Neural Information Processing Systems , year=

    Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning , author=. Neural Information Processing Systems , year=

  81. [89]

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    Deep Residual Learning for Image Recognition , author=. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    2016

  82. [90]

    CoRR , year=

    Very Deep Convolutional Networks for Large-Scale Image Recognition , author=. CoRR , year=

  83. [91]

    2018 , ignore=

    Improving Language Understanding by Generative Pre-Training , author=. 2018 , ignore=

    2018

  84. [92]

    1988 , ignore=

    A Value for n-person Games , author=. 1988 , ignore=

    1988

  85. [93]

    ArXiv , year=

    Explaining nonlinear classification decisions with deep Taylor decomposition , author=. ArXiv , year=

  86. [94]

    ArXiv , year=

    Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing , author=. ArXiv , year=

  87. [95]

    ArXiv , year=

    Concept-based Explainable Artificial Intelligence: A Survey , author=. ArXiv , year=

  88. [96]

    2018 , eprint=

    Understanding intermediate layers using linear classifier probes , author=. 2018 , eprint=

    2018

  89. [97]

    2018 , eprint=

    Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) , author=. 2018 , eprint=

    2018

  90. [98]

    Philosophy & Technology , year=

    Fair, Transparent, and Accountable Algorithmic Decision-making Processes , author=. Philosophy & Technology , year=

  91. [99]

    Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , year=

    Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing , author=. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , year=

    2020

  92. [100]

    2023 , eprint=

    Representation Engineering: A Top-Down Approach to AI Transparency , author=. 2023 , eprint=

    2023

  93. [101]

    2024 , eprint=

    Understanding Addition in Transformers , author=. 2024 , eprint=

    2024

  94. [102]

    2023 , eprint=

    Towards Automated Circuit Discovery for Mechanistic Interpretability , author=. 2023 , eprint=

    2023

  95. [103]

    2023 , howpublished =

    Language models can explain neurons in language models , author=. 2023 , howpublished =

    2023

  96. [104]

    2023 , eprint=

    Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models , author=. 2023 , eprint=

    2023

  97. [105]

    2023 , eprint=

    Analyzing Transformers in Embedding Space , author=. 2023 , eprint=

    2023

  98. [106]

    2023 , eprint=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. 2023 , eprint=

    2023

  99. [107]

    2023 , eprint=

    Linear Representations of Sentiment in Large Language Models , author=. 2023 , eprint=

    2023

  100. [108]

    2023 , eprint=

    Observable Propagation: A Data-Efficient Approach to Uncover Feature Vectors in Transformers , author=. 2023 , eprint=

    2023

  101. [109]

    2020 , eprint=

    Finding and Removing Clever Hans: Using Explanation Methods to Debug and Improve Deep Models , author=. 2020 , eprint=

    2020

  102. [110]

    2023 , eprint=

    LEACE: Perfect linear concept erasure in closed form , author=. 2023 , eprint=

    2023

  103. [111]

    2023 , eprint=

    Steering Llama 2 via Contrastive Activation Addition , author=. 2023 , eprint=

    2023

  104. [112]

    ArXiv , year=

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , author=. ArXiv , year=

  105. [113]

    2024 , eprint=

    Evidence of Learned Look-Ahead in a Chess-Playing Neural Network , author=. 2024 , eprint=

    2024

  106. [114]

    ArXiv , year=

    Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents , author=. ArXiv , year=

  107. [115]

    International Journal of Computer Vision , year=

    Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , author=. International Journal of Computer Vision , year=

  108. [116]

    PLoS ONE , year=

    On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , author=. PLoS ONE , year=

  109. [117]

    Nature Machine Intelligence , year=

    Concept whitening for interpretable image recognition , author=. Nature Machine Intelligence , year=

  110. [118]

    AAAI Conference on Artificial Intelligence , year=

    From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space , author=. AAAI Conference on Artificial Intelligence , year=

  111. [119]

    CoRR , year=

    Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , author=. CoRR , year=

  112. [120]

    2023 IEEE/CVF International Conference on Computer Vision (ICCV) , year=

    Linear Spaces of Meanings: Compositional Structures in Vision-Language Models , author=. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , year=

    2023

  113. [121]

    Why Should I Trust You?

    “Why Should I Trust You?”: Explaining the Predictions of Any Classifier , author=. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , year=

  114. [122]

    Neural Information Processing Systems , year=

    A Unified Approach to Interpreting Model Predictions , author=. Neural Information Processing Systems , year=

  115. [123]

    ArXiv , year=

    Visualizing and Understanding Convolutional Networks , author=. ArXiv , year=

  116. [124]

    AAAI Conference on Artificial Intelligence , year=

    Anchors: High-Precision Model-Agnostic Explanations , author=. AAAI Conference on Artificial Intelligence , year=

  117. [125]

    Explaining by Removing: A Unified Framework for Model Explanation , author=. J. Mach. Learn. Res. , year=

  118. [126]

    2020 , ignore=

    Zoom In: An Introduction to Circuits , author=. 2020 , ignore=

    2020

  119. [127]

    ArXiv , year=

    AttnLRP: Attention-Aware Layer-wise Relevance Propagation for Transformers , author=. ArXiv , year=

  120. [128]

    International Journal of Computer Vision , year=

    Visualizing Deep Convolutional Neural Networks Using Natural Pre-images , author=. International Journal of Computer Vision , year=

  121. [129]

    Neural Information Processing Systems , year=

    Locating and Editing Factual Associations in GPT , author=. Neural Information Processing Systems , year=

  122. [130]

    Transformer Circuits Thread , volume=

    A mathematical framework for transformer circuits , author=. Transformer Circuits Thread , volume=

  123. [131]

    ArXiv , year=

    Backward Lens: Projecting Language Model Gradients into the Vocabulary Space , author=. ArXiv , year=

  124. [132]

    ArXiv , year=

    In-context Learning and Induction Heads , author=. ArXiv , year=

  125. [133]

    ArXiv , year=

    Transcoders Find Interpretable LLM Feature Circuits , author=. ArXiv , year=

  126. [134]

    ArXiv , year=

    AtP*: An efficient and scalable method for localizing LLM behaviour to components , author=. ArXiv , year=

  127. [135]

    ArXiv , year=

    Analyzing Transformers in Embedding Space , author=. ArXiv , year=

  128. [136]

    ArXiv , year=

    Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models , author=. ArXiv , year=

  129. [137]

    ArXiv , year=

    SmoothGrad: removing noise by adding noise , author=. ArXiv , year=

  130. [138]

    Conference on Empirical Methods in Natural Language Processing , year=

    Attention is not not Explanation , author=. Conference on Empirical Methods in Natural Language Processing , year=

  131. [139]

    ArXiv , year=

    Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations , author=. ArXiv , year=

  132. [140]

    Neural Information Processing Systems , year=

    SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning , author=. Neural Information Processing Systems , year=

  133. [141]

    ArXiv , year=

    Efficiently Quantifying Individual Agent Importance in Cooperative MARL , author=. ArXiv , year=

  134. [142]

    ArXiv , year=

    On Diagnostics for Understanding Agent Training Behaviour in Cooperative MARL , author=. ArXiv , year=

  135. [143]

    ECML/PKDD , year=

    MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning , author=. ECML/PKDD , year=

  136. [144]

    ArXiv , year=

    Concept Learning for Interpretable Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  137. [145]

    IEEE Computational Intelligence Magazine , year=

    Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning With Shapley Values , author=. IEEE Computational Intelligence Magazine , year=

  138. [146]

    ArXiv , year=

    Interpretability Illusions in the Generalization of Simplified Models , author=. ArXiv , year=

  139. [147]

    ArXiv , year=

    A Survey on Large Language Model based Autonomous Agents , author=. ArXiv , year=

  140. [148]

    ArXiv , year=

    Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches , author=. ArXiv , year=

  141. [149]

    Autonomous Agents and Multi-Agent Systems , year=

    A survey and critique of multiagent deep reinforcement learning , author=. Autonomous Agents and Multi-Agent Systems , year=

  142. [150]

    ArXiv , year=

    AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning , author=. ArXiv , year=

  143. [151]

    PLoS ONE , year=

    Multiagent cooperation and competition with deep reinforcement learning , author=. PLoS ONE , year=

  144. [152]

    Neural Information Processing Systems , year=

    The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games , author=. Neural Information Processing Systems , year=

  145. [153]

    ArXiv , year=

    Value-Decomposition Networks For Cooperative Multi-Agent Learning , author=. ArXiv , year=

  146. [154]

    ArXiv , year=

    QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  147. [155]

    ArXiv , year=

    Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , author=. ArXiv , year=

  148. [156]

    Neural Information Processing Systems , year=

    Attention is All you Need , author=. Neural Information Processing Systems , year=

  149. [157]

    Nature Communications , year=

    Unmasking Clever Hans predictors and assessing what machines really learn , author=. Nature Communications , year=

  150. [158]

    Statistics and Computing , year=

    A tutorial on spectral clustering , author=. Statistics and Computing , year=

  151. [159]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  152. [160]

    ArXiv , year=

    Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning , author=. ArXiv , year=

  153. [161]

    ArXiv , year=

    Dissecting Language Models: Machine Unlearning via Selective Pruning , author=. ArXiv , year=

  154. [162]

    Nature , year=

    La Nouvelle Psychologie Animale Clever Hans (the Horse of Mr Van Osten): a Contribution to Experimental Animal and Human Psychology , author=. Nature , year=

  155. [163]

    ArXiv , year=

    Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  156. [164]

    ArXiv , year=

    Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  157. [165]

    CoRR , year=

    Prioritized Experience Replay , author=. CoRR , year=

  158. [166]

    ArXiv , year=

    Mean Field Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  159. [167]

    International Conference on Machine Learning , year=

    Prioritized Level Replay , author=. International Conference on Machine Learning , year=

  160. [168]

    ArXiv , year=

    Hybrid Reward Architecture for Reinforcement Learning , author=. ArXiv , year=

  161. [169]

    2019 , ignore=

    Explainable Reinforcement Learning via Reward Decomposition , author=. 2019 , ignore=

    2019

  162. [170]

    ArXiv , year=

    Structured World Representations in Maze-Solving Transformers , author=. ArXiv , year=

  163. [171]

    ArXiv , year=

    Eliciting Latent Predictions from Transformers with the Tuned Lens , author=. ArXiv , year=

  164. [172]

    ArXiv , year=

    Gradient Routing: Masking Gradients to Localize Computation in Neural Networks , author=. ArXiv , year=

  165. [173]

    2024 , ignore=

    SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks , author=. 2024 , ignore=

    2024

  166. [174]

    ArXiv , year=

    Understanding and Controlling a Maze-Solving Policy Network , author=. ArXiv , year=

  167. [175]

    ArXiv , year=

    ProtoX: Explaining a Reinforcement Learning Agent via Prototyping , author=. ArXiv , year=

  168. [176]

    Environmetrics , year=

    Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , author=. Environmetrics , year=

  169. [177]

    ArXiv , year=

    Genie: Generative Interactive Environments , author=. ArXiv , year=

  170. [178]

    ArXiv , year=

    Mastering Diverse Domains through World Models , author=. ArXiv , year=

  171. [179]

    ArXiv , year=

    Understanding Addition in Transformers , author=. ArXiv , year=

  172. [180]

    ArXiv , year=

    Progress measures for grokking via mechanistic interpretability , author=. ArXiv , year=

  173. [181]

    ArXiv , year=

    Not Just a Black Box: Learning Important Features Through Propagating Activation Differences , author=. ArXiv , year=

  174. [182]

    xAI , year=

    Use Bag-of-Patterns Approach to Explore Learned Behaviors of Reinforcement Learning , author=. xAI , year=

  175. [183]

    ArXiv , year=

    RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations , author=. ArXiv , year=

  176. [184]

    ArXiv , year=

    An Interpretability Illusion for BERT , author=. ArXiv , year=

  177. [185]

    ArXiv , year=

    Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Small , author=. ArXiv , year=

  178. [186]

    2024 , ignore=

    Planning in a recurrent neural network that plays Sokoban , author=. 2024 , ignore=

    2024

  179. [187]

    ArXiv , year=

    Evidence of Learned Look-Ahead in a Chess-Playing Neural Network , author=. ArXiv , year=

  180. [188]

    2024 , ignore=

    Interpreting the Learned Model in MuZero Planning , author=. 2024 , ignore=

    2024

  181. [189]

    ArXiv , year=

    Predicting Future Actions of Reinforcement Learning Agents , author=. ArXiv , year=

  182. [190]

    2020 , ignore=

    Understanding RL vision , author=. 2020 , ignore=

    2020

  183. [191]

    2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , year=

    Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations , author=. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , year=

    2024

  184. [192]

    ArXiv , year=

    Interpretability Needs a New Paradigm , author=. ArXiv , year=

  185. [193]

    ArXiv , year=

    Interpretable by Design: Wrapper Boxes Combine Neural Performance with Faithful Explanations , author=. ArXiv , year=

  186. [194]

    Adaptive Agents and Multi-Agent Systems , year=

    Boosting Studies of Multi-Agent Reinforcement Learning on Google Research Football Environment: the Past, Present, and Future , author=. Adaptive Agents and Multi-Agent Systems , year=

  187. [195]

    Proceedings of the Thirty-Fourth

    Google Research Football: A Novel Reinforcement Learning Environment , author =. Proceedings of the Thirty-Fourth. 2020 , doi =

    2020

  188. [196]

    AAAI Conference on Artificial Intelligence , year=

    Controlling Large Language Models Through Concept Activation Vectors , author=. AAAI Conference on Artificial Intelligence , year=

  189. [197]

    2026 , url =

    Poupart, Yoann , title =. 2026 , url =

    2026

  190. [198]

    Science , year =

    Silver, David and Hubert, Thomas and Schrittwieser, Julian and Antonoglou, Ioannis and Lai, Matthew and Guez, Arthur and Lanctot, Marc and Sifre, Laurent and Kumaran, Dharshan and Graepel, Thore and Lillicrap, Timothy and Simonyan, Karen and Hassabis, Demis , title =. Science , year =

  191. [199]

    Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , year =

    McIlroy-Young, Reid and Sen, Siddhartha and Kleinberg, Jon and Anderson, Ashton , title =. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , year =

  192. [200]

    , month = oct, pages =

    Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien and Delangue, Clement and Moi, Anthony and Cistac, Perric and Ma, Clara and Jernite, Yacine and Plu, Julien and Xu, Canwen and Le Scao, Teven and Gugger, Sylvain and Drame, Mariama and Lhoest, Quentin and R...

  193. [201]

    doi:10.5281/zenodo.15831829 , url =

    The pandas development team , title =. doi:10.5281/zenodo.15831829 , url =

    doi

  194. [202]

    2004 , howpublished =

    Holger Krekel and Bruno Oliveira and Ronny Pfannschmidt and Floris Bruynooghe and Brianna Laugher and Florian Bruhin , title =. 2004 , howpublished =

    2004

  195. [203]

    2024 , howpublished =

    Jack Lindsey and Adly Templeton and Jonathan Marcus and Thomas Conerly and Joshua Batson and Christopher Olah , title =. 2024 , howpublished =

    2024

  196. [204]

    2022 , howpublished =

    Neel Nanda , title =. 2022 , howpublished =

    2022

  197. [205]

    and Haberland, Matt and Reddy, Tyler and Cournapeau, David and Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and Bright, Jonathan and

    Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and Haberland, Matt and Reddy, Tyler and Cournapeau, David and Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and Bright, Jonathan and. Nature Methods , year =

  198. [206]

    Scikit-learn: Machine Learning in Python , author=. J. Mach. Learn. Res. , year=

  199. [207]

    International Conference on Machine Learning , year=

    Axiomatic Attribution for Deep Networks , author=. International Conference on Machine Learning , year=

  200. [208]

    ArXiv , year=

    TorchRL: A data-driven decision-making library for PyTorch , author=. ArXiv , year=

  201. [209]

    Computing in Science & Engineering , year=

    Matplotlib: A 2D Graphics Environment , author=. Computing in Science & Engineering , year=

  202. [210]

    Nature , year=

    Array programming with NumPy , author=. Nature , year=. doi:10.1038/s41586-020-2649-2 , url=

    doi

  203. [211]

    The Eleventh International Conference on Learning Representations , year =

    Editing Models with Task Arithmetic , author =. The Eleventh International Conference on Learning Representations , year =

  204. [212]

    Neural Information Processing Systems , year=

    Investigating Gender Bias in Language Models Using Causal Mediation Analysis , author=. Neural Information Processing Systems , year=

  205. [213]

    RelP: Faithful and Efficient Circuit Discovery via Relevance Patching , author=

  206. [214]

    ArXiv , year=

    BenchMARL: Benchmarking Multi-Agent Reinforcement Learning , author=. ArXiv , year=

  207. [215]

    International Symposium on Distributed Autonomous Robotic Systems , year=

    VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning , author=. International Symposium on Distributed Autonomous Robotic Systems , year=

  208. [216]

    International Conference on Machine Learning , volume =

    Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning , author =. International Conference on Machine Learning , volume =. 2024 , url =

    2024

  209. [217]

    2025 , url=

    TDHook: A Lightweight Framework for Interpretability , author=. 2025 , url=

    2025

  210. [218]

    Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , year=

    PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation , author=. Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , year=

  211. [219]

    2019 , url =

    Omry Yadan , title =. 2019 , url =

    2019

  212. [220]

    Seaborn: Statistical Data Visualization , author=. J. Open Source Softw. , year=

  213. [221]

    Computational Linguistics , year=

    The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation Analysis , author=. Computational Linguistics , year=

  214. [222]

    International Conference on Artificial Neural Networks , year=

    MAT-DQN: Toward Interpretable Multi-agent Deep Reinforcement Learning for Coordinated Activities , author=. International Conference on Artificial Neural Networks , year=

  215. [223]

    2025 IEEE 49th Annual Computers, Software, and Applications Conference (COMPSAC) , year=

    Investigating Transferability in Multi-Agent Reinforcement Learning , author=. 2025 IEEE 49th Annual Computers, Software, and Applications Conference (COMPSAC) , year=

    2025

  216. [224]

    ArXiv , year=

    Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games , author=. ArXiv , year=

  217. [225]

    AAPEI@ECAI , year=

    Fairness-Driven Explainable Learning in Multi-Agent Reinforcement Learning , author=. AAPEI@ECAI , year=

  218. [226]

    Experiment Tracking with Weights and Biases , year =

  219. [227]

    , booktitle=

    Fey, Matthias and Lenssen, Jan E. , booktitle=. Fast Graph Representation Learning with

  220. [228]

    R. J. Williams. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning. 1992

    1992

  221. [229]

    , title =

    Degris, Thomas and White, Martha and Sutton, Richard S. , title =. International Conference on Machine Learning , pages =. 2012 , url =

    2012

  222. [230]

    and McAllester, David and Singh, Satinder and Mansour, Yishay , title =

    Sutton, Richard S. and McAllester, David and Singh, Satinder and Mansour, Yishay , title =. Advances in Neural Information Processing Systems , volume =. 1999 , url =

    1999

  223. [231]

    International Conference on Machine Learning , volume =

    Schulman, John and Levine, Sergey and Abbeel, Pieter and Jordan, Michael and Moritz, Philipp , title =. International Conference on Machine Learning , volume =. 2015 , url =

    2015

  224. [232]

    International Conference on Machine Learning , volume =

    Huang, Jiawei and Jiang, Nan , title =. International Conference on Machine Learning , volume =. 2020 , url =

    2020

  225. [233]

    Rupam , title =

    Tosatto, Samuele and Patterson, Andrew and White, Martha and Mahmood, A. Rupam , title =. International Conference on Machine Learning , volume =. 2022 , url =

    2022

  226. [234]

    Findings of the Association for Computational Linguistics:

    Oozeer, Narmeen Fatimah and Marks, Luke and Barez, Fazl and Abdullah, Amir , title =. Findings of the Association for Computational Linguistics:. 2025 , publisher =. doi:10.18653/v1/2025.findings-emnlp.1278 , url =

    2025 doi

  227. [235]

    2026 , eprint =

    Miao, Miranda Muqing and Kim, Subin and Yang, Brandon and Ungar, Lyle , title =. 2026 , eprint =

    2026

  228. [236]

    and Harada, Daishi and Russell, Stuart J

    Ng, Andrew Y. and Harada, Daishi and Russell, Stuart J. , title =. Proceedings of the Sixteenth International Conference on Machine Learning , pages =

  229. [237]

    Proceedings of UniReps: the Third Edition of the Workshop on Unifying Representations in Neural Models , series =

    On Task Vectors and Gradients , author =. Proceedings of UniReps: the Third Edition of the Workshop on Unifying Representations in Neural Models , series =. 2026 , publisher =

    2026

  230. [238]

    and Potts, Christopher , booktitle =

    Wu, Zhengxuan and Arora, Aryaman and Wang, Zheng and Geiger, Atticus and Jurafsky, Dan and Manning, Christopher D. and Potts, Christopher , booktitle =. 2024 , url =

    2024

  231. [239]

    Advances in Neural Information Processing Systems , year =

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author =. Advances in Neural Information Processing Systems , year =

  232. [240]

    Forty-third International Conference on Machine Learning , year =

    Preference Goal Tuning: Post-Training as Latent Control for Frozen Policies , author =. Forty-third International Conference on Machine Learning , year =

  233. [241]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics , year =

    Adaptive Helpfulness--Harmlessness Alignment with Preference Vectors , author =. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics , year =

  234. [242]

    The Fourteenth International Conference on Learning Representations , year =

    Steering Language Models with Weight Arithmetic , author =. The Fourteenth International Conference on Learning Representations , year =

  235. [243]

    Neural Computation , volume =

    Improving Generalization for Temporal Difference Learning: The Successor Representation , author =. Neural Computation , volume =. 1993 , doi =

    1993

  236. [244]

    International Conference on Machine Learning , volume =

    Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement , author =. International Conference on Machine Learning , volume =. 2018 , url =

    2018

  237. [245]

    International Conference on Learning Representations , year =

    Universal Successor Features Approximators , author =. International Conference on Learning Representations , year =

  238. [246]

    International Conference on Machine Learning , volume =

    Dynamic Weights in Multi-Objective Deep Reinforcement Learning , author =. International Conference on Machine Learning , volume =. 2019 , url =

    2019

  239. [247]

    Advances in Neural Information Processing Systems , volume =

    A Natural Policy Gradient , author =. Advances in Neural Information Processing Systems , volume =

  240. [248]

    and Soatto, Stefano and Perona, Pietro , booktitle =

    Achille, Alessandro and Lam, Michael and Tewari, Rahul and Ravichandran, Avinash and Maji, Subhransu and Fowlkes, Charless C. and Soatto, Stefano and Perona, Pietro , booktitle =

  241. [249]

    International Conference on Machine Learning , volume =

    Opponent Modeling in Deep Reinforcement Learning , author =. International Conference on Machine Learning , volume =. 2016 , url =

    2016

  242. [250]

    International Conference on Machine Learning , volume =

    Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning , author =. International Conference on Machine Learning , volume =. 2024 , url =

    2024

  243. [251]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  244. [252]

    International Conference on Machine Learning , pages=

    Self-conditioning pre-trained language models , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    2022

  245. [253]

    Steering Llama 2 via Contrastive Activation Addition

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander. Steering Llama 2 via Contrastive Activation Addition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 202...

    2024 doi

  246. [254]

    Activation Scaling for Steering and Interpreting Language Models

    Stoehr, Niklas and Du, Kevin and Sn bjarnarson, V \'e steinn and West, Robert and Cotterell, Ryan and Schein, Aaron. Activation Scaling for Steering and Interpreting Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2...

    2024 doi

  247. [255]

    arXiv preprint arXiv:2506.03292 , year=

    Hypersteer: Activation steering at scale with hypernetworks , author=. arXiv preprint arXiv:2506.03292 , year=

    arXiv

  248. [256]

    Advances in Neural Information Processing Systems , volume =

    Decision Transformer: Reinforcement Learning via Sequence Modeling , author =. Advances in Neural Information Processing Systems , volume =. 2021 , url =

    2021

  249. [257]

    Second Workshop on Aligning Reinforcement Learning Experimentalists and Theorists (

    Policy Gradient Guidance Enables Test Time Control , author =. Second Workshop on Aligning Reinforcement Learning Experimentalists and Theorists (. 2025 , url =

    2025

  250. [258]

    Advances in Neural Information Processing Systems , year =

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , author =. Advances in Neural Information Processing Systems , year =

  251. [259]

    2023 , eprint =

    Steering Language Models With Activation Engineering , author =. 2023 , eprint =

    2023

  252. [260]

    International Conference on Learning Representations , year =

    Plug and Play Language Models: A Simple Approach to Controlled Text Generation , author =. International Conference on Learning Representations , year =

  253. [261]

    2025 , eprint =

    Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space , author =. 2025 , eprint =

    2025

  254. [262]

    Advances in Neural Information Processing Systems , volume =

    Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning , author =. Advances in Neural Information Processing Systems , volume =. 2021 , url =

    2021

  255. [263]

    Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , pages =

    Pfeiffer, Jonas and Kamath, Aishwarya and R. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , pages =. 2021 , url =

    2021

  256. [264]

    The Thirteenth International Conference on Learning Representations , year =

    Improving Instruction-Following in Language Models through Activation Steering , author =. The Thirteenth International Conference on Learning Representations , year =

  257. [265]

    Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages =

    Scalena, Daniel and Sarti, Gabriele and Nissim, Malvina , title =. Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages =. 2024 , publisher =. doi:10.18653/v1/2024.blackboxnlp-1.34 , url =

    2024 doi

  258. [266]

    Compositional Steering of Large Language Models with Steering Tokens , booktitle =

    Radevski, Gorjan and Gashteovski, Kiril and Hong, Giwon and Lawrence, Carolin and Glava. Compositional Steering of Large Language Models with Steering Tokens , booktitle =. 2026 , publisher =. doi:10.18653/v1/2026.acl-long.1435 , url =

    2026 doi

  259. [267]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =. 2022 , url =

    2022

  260. [268]

    Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =

    Yu, Chao and Velu, Akash and Vinitsky, Eugene and Gao, Jiaxuan and Wang, Yu and Bayen, Alexandre and Wu, Yi , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    2022

  261. [269]

    2020 , eprint=

    Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning , author=. 2020 , eprint=

    2020

  262. [270]

    and Farquhar, Gregory and Afouras, Triantafyllos and Nardelli, Nantas and Whiteson, Shimon , title =

    Foerster, Jakob N. and Farquhar, Gregory and Afouras, Triantafyllos and Nardelli, Nantas and Whiteson, Shimon , title =. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and ...

    2018

  263. [271]

    2022 , eprint=

    Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checking , author=. 2022 , eprint=

    2022

  264. [272]

    2021 , eprint=

    Adversarial Policies: Attacking Deep Reinforcement Learning , author=. 2021 , eprint=

    2021

  265. [273]

    Machine Intelligence Research , volume=

    An Empirical Study on Google Research Football Multi-Agent Scenarios , author=. Machine Intelligence Research , volume=. 2024 , doi=

    2024

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.