Pith. sign in

REVIEW 3 major objections 5 minor 43 references

FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FOCUS claims that decomposing an LLM's expert personas into orthogonal, domain-specific vectors and gating them by context outperforms prompting, steering, and supervised fine-tuning on finance, legal, medical, and cross-domain benchmarks.

desk verdict A useful, cheap domain-adaptation recipe whose 'uncoupling' claim is stronger than the math supports; worth reviewing with a demand for more evidence. read the letter →

arxiv 2608.05611 v1 pith:5LMQQ7JB submitted 2026-08-06 cs.CL cs.LG

Guanyu Wang , Zidi Zhang , Xu Chu This is my paper · ORCID
classification cs.CLcs.LG
keywords personavectorsorthogonaldecompositionexpertgatingmoduledomainadaptationLLMpersonascross-domainreasoningactivationsteeringrepresentationengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FOCUS argues that expert personas inside LLMs are entangled: the finance persona, legal persona, and medical persona share a general-expert component and leak into one another. The paper's proposal is to extract each persona as an activation-difference vector, orthogonalize it against the general-expert direction, and train a small gating module that mixes the decoupled vectors according to the question's context. On financial, legal, medical, and cross-domain benchmarks, the two instruction-tuned base models fitted with FOCUS beat prompting, inference-time steering, standard fine-tuning, and several domain-specialized systems. The claim matters because it offers a parameter-efficient route to domain adaptation that keeps conflicting behavioral demands (cautious medicine, risk-aware finance) from contaminating each other.

What carries the argument

Persona vectors, defined as the direction in representation space that separates expert from non-expert behavior: the difference between mean residual-stream activations of responses to positive expert prompts and negative non-expert prompts at layer 20. The decoupling step is Gram-Schmidt: each candidate vector $v_k$ is projected out of the normalized general-expert vector $q_g$ via $v'_k = v_k - (q_g^\top v_k) q_g$ and renormalized to $q_k$. The gating module is a two-layer MLP on the FFN output $h_{ffn}$ that produces softmax weights $\alpha$ over the four vectors $q_g$, $q_f$, $q_l$, $q_m$; the weighted sum $r$ is added to the residual stream with a scaling factor $\gamma$. Stage-1 training adds the regularizer $\lambda\,\mathrm{CE}(\mathrm{onehot}(k),\alpha)$ to the language-model loss to force sparse single-domain activation; stage-2 training drops the regularizer so the model learns to compose personas for mixed-domain questions.

What would settle it

Measure the pairwise cosine similarity between the decoupled vectors $q_f$, $q_l$, $q_m$: if the values are far from zero, the orthogonalization has not removed cross-domain coupling and any accuracy gains must have another cause. A second check is to replace the three domain vectors with random vectors of the same norm, retrain the gating module, and see whether the accuracy advantage over SFT disappears; if it does not, the specific persona content is not load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that decoupling is what makes activation-based persona control work across domains. The paper reports that removing the orthogonalization step lowers accuracy on every benchmark family, and that the gating module learns the intended sparse patterns: essentially one-hot activation on single-domain tasks and a medical-plus-legal mixture on the legal-medical benchmark. In the reported numbers, Qwen-FOCUS reaches 79.04% on professional medicine, 52.47% on stock prediction, and 81.61% on MedEthicsQA, each the best or tied-best among the methods compared. That is the evidence for the paper's thesis that explicit, decoupled, and contextually gated persona vectors are a workable mechanism for domain specialization.

Load-bearing premise

The entire pipeline rests on the assumption that a single activation-difference vector per domain, computed at layer 20 and cleaned only of the general-expert direction, is a faithful representation of the domain behavior that matters for the test tasks, with the remaining cross-domain correlations small enough to ignore.

Editorial extensions

If this is right

  • FOCUS improves both Qwen-2.5-7B-Instruct and Llama-3-8B-Instruct, so the mechanism transfers across base models.
  • Removing the decoupling step hurts accuracy on all benchmark groups, identifying cross-domain coupling as the specific cause of the gain.
  • The gating weights are interpretable: single-domain tasks activate one vector, while the legal-medical benchmark activates both, showing the model can signal which expertise it is using.
  • The two-stage training schedule matters: dropping either stage lowers accuracy, so the sparse-alignment and adaptive-composition phases play complementary roles.
  • Persona injection must stay moderate; $\gamma=2$ gives the best professional-medicine accuracy, and larger $\gamma$ degrades performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the decoupling only removes the general-expert component; pairwise domain-domain overlaps are never measured, so the claim of 'uncoupled' personas is only partially established.
  • Editorial inference: the same architecture could be extended to more domains or to finer-grained expert splits (e.g., sub-specialties within medicine) by iterating the Gram-Schmidt process, since adding a vector only requires a new orthogonalization pass.
  • Editorial inference: because the persona vectors are extracted with one generator model and injected into other models, the approach implicitly assumes cross-model transferability of these directions; a cross-model transfer test would make that assumption explicit.
  • Editorial inference: the gating weights $\alpha$ could be used as a cheap, transparent probe for when a model believes a prompt is mixed-domain, provided their stability under prompt paraphrasing is verified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. FOCUS proposes a persona-vector editing method for domain adaptation. It extracts general and domain-specific expert persona vectors (finance, law, medicine) as activation differences between contrastive responses, removes the component parallel to the general expert vector, and adds a lightweight gating MLP after an FFN that combines the persona vectors with weights predicted from context. A two-stage fine-tuning schedule first enforces sparse, one-hot gating on single-domain data via a cross-entropy regularizer and then adaptively learns weight combinations on mixed-domain data. Experiments compare Qwen-2.5-7B-Instruct and Llama-3-8B-Instruct against prompting, inference-time steering, SFT, CAFT, preventatively steered models, and domain-specific LLMs on stock prediction, LegalBench subsets, MMLU professional medicine, MedEthicsQA, and CUAD. The paper reports accuracy improvements and ablates the gating module, two-stage training, the scaling factor gamma, and the decoupling step.

Significance. If the mutual decoupling of persona vectors were established, FOCUS would be a useful lightweight domain-adaptation method: the two-stage training recipe and sparse gating regularizer are clearly described, the evaluation spans four domains on public benchmarks, and the ablations are informative. The paper also ships an anonymous code release, which aids reproducibility. However, the central claim is currently supported only by single-run accuracy deltas and by an incomplete decoupling operation; the paper's value would be considerably higher after direct validation of persona-vector content and uncertainty quantification.

major comments (3)
  1. [§3.2, Eq. (2)] The decoupling step subtracts only the projection onto q_g, so the resulting q_f, q_l, q_m are each orthogonal to q_g but not necessarily to one another. The abstract and §3.2 call these 'decoupled orthogonal persona vectors,' but no pairwise cosine similarities or other direct measures of remaining coupling are reported. Table 4's 'w/o decoupling' ablation removes the q_g projection only; it therefore demonstrates some benefit of that projection but does not test whether mutual correlations among domain vectors are harmful. Please report the mutual cosine similarities (and ideally an ablation that orthogonalizes q_f, q_l, q_m against each other, e.g., full Gram-Schmidt over all four vectors) so that the central 'uncoupled' claim is actually validated.
  2. [§4.1, Table 1] All reported accuracies appear to come from a single run: no variance estimates, no multiple-seed averages, and no statistical tests are given. Several headline gains are 1–2 points (e.g., Qwen-FOCUS vs SFT on TTP: 89.09 vs 87.88; vs Preventative Steering on Stock Predict: 52.47 vs 52.33), so the claim that 'FOCUS consistently improves accuracy on most evaluation tasks' is not yet supported with uncertainty quantification. Please add at least three seeds with standard deviations or confidence intervals for the main comparisons and ablations.
  3. [§3.1, Fig. 3] The paper does not directly validate that q_f, q_l, q_m isolate finance, legal, and medical behavior. Fig. 3 shows the trained router's softmax weights, which could in principle route to any set of directions; it does not measure the content of the persona vectors. Section 3.1 fixes layer 20 'consistent with [8]' without testing sensitivity to layer, and the extraction uses only 5 contrastive prompt pairs and 20 questions per domain. Please add a direct evaluation of vector content (e.g., steering the base model with individual q_k and measuring domain-specific behavior changes, or reporting activation-probe accuracy for domain labels) and a layer-sensitivity analysis.
minor comments (5)
  1. [Fig. 1(b)] The caption states that the correct answer is D, but the model output shown ends with '###C'; please correct the inconsistency.
  2. [§3.3, Eq. (6)] The regularizer term CE(Onehot(k), alpha) uses a domain label k in {f, l, m}, while alpha has four components including alpha_g. Please clarify the target encoding, in particular whether alpha_g is explicitly pushed to zero during Stage 1 or left unconstrained.
  3. [References] References [4] and [5] are the same paper; please deduplicate and renumber.
  4. [Table captions] Table captions contain 'T able 1' and 'T able 4' spacing typos; please fix.
  5. [§4.2, Table 3] The gamma sweep is informative, but no sensitivity analysis is reported for the regularizer weight lambda (set to 0.2); a small sweep would strengthen the claim that the two-stage training is robust to this hyperparameter.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FOCUS is a self-contained training method evaluated on external benchmarks; the decoupling overclaim is a validation gap, not a circular reduction.

full rationale

No load-bearing circular step is present. Persona vectors are extracted from the target model's residual-stream activations via Eq. (1) using contrastive prompts; the decoupling step in Eq. (2) is a fixed linear projection removing only the general-expert direction, and the gating module is trained with explicit losses in Eqs. (6) and (7). The reported accuracies come from external public benchmarks (stock prediction from Koa et al., LegalBench, MMLU professional medicine, MedEthicsQA, CUAD) that are not outputs of the FOCUS pipeline and are not optimized by any fitted parameter. Self-citations to the authors' Domaino1s work [11] supply public training datasets and baseline models, but these are external artifacts rather than theorems that force the conclusion. The paper's claim that the persona vectors are 'decoupled orthogonal' is stronger than what Eq. (2) guarantees, since only orthogonality to q_g is enforced and mutual correlations among q_f, q_l, q_m are never measured; this is a correctness/validation gap and a fair target for empirical criticism, but it does not make any predicted result equivalent to an input by construction. The gating behavior shown in Fig. 3 is a direct result of the supervised one-hot regularizer in Stage 1, so it is a trained property, not a circular prediction. Overall the derivation chain is self-contained against the stated benchmarks, and the central accuracy claims stand or fall on the experiments rather than on definitional equivalence.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on four tuned choices (gamma, lambda, layer 20, extraction counts), three unvalidated domain assumptions about persona geometry, and a new gating module whose behavior is only demonstrated in-paper. No free parameter is fitted to the evaluation targets in a classical overfitting sense, but gamma is tuned on one of the evaluation tasks, which slightly increases the circularity burden.

free parameters (4)
  • gamma = 2
    Injection strength of the persona offset vector. Selected by scanning gamma in {0,1,2,3,5,10} on Professional Medicine (Table 3), then fixed for all tasks. This is a hyperparameter tuned on the evaluation domain.
  • lambda = 0.2
    Weight of the gated selection regularizer in Stage 1 (Section 3.3). Stated as a chosen hyperparameter with no sensitivity analysis.
  • layer l = 20
    Layer at which persona vectors are extracted and injected. Set "consistent with [8]" (Section 4, Implementation Details), not tuned on the target tasks.
  • number of contrastive prompt pairs and questions = 5 pairs, 20 questions per domain
    Choices in the extraction pipeline (Section 3.1) with no ablation; the pipeline may be sensitive to these.
assumptions (4)
  • domain assumption Activation differences between contrastively prompted responses capture a stable, domain-specific behavioral direction that generalizes across prompts and questions.
    Section 3.1 defines persona vectors as mean activation differences on 20 questions. The claim that this direction is a reusable expert persona is assumed, not validated beyond downstream accuracy.
  • domain assumption The subspace spanned by the general expert vector q_g contains the shared expert component, so removing it leaves pure domain vectors.
    Section 3.2 projects out only q_g. If finance, law, and medicine share other components not aligned with q_g, the decoupled vectors remain coupled.
  • domain assumption A linear offset in the residual stream at one layer (layer 20) is a sufficient intervention to change domain behavior.
    Sections 3.1 and 3.3. The paper relies on prior work [8] for this premise and does not test other layers for the gating setup.
  • domain assumption The selected benchmark accuracies measure the relevant domain expertise and are sensitive enough to reflect persona purity.
    Section 4, datasets: stock prediction, LegalBench, MMLU professional medicine. Gains of 1-3 points are reported without statistical testing.
invented entities (2)
  • Expert gating module (two-layer MLP on FFN output producing 4 gating weights)
    purpose: Adaptively selects how much of each decoupled persona vector to add to the residual stream at inference time.
    It is a new architectural component trained within the paper; its behavior is only demonstrated on the paper's own benchmarks, with no public model release or independent evaluation.
  • Decoupled domain persona vectors q_f, q_l, q_m
    purpose: Orthogonalized directions intended to represent pure finance, legal, and medical expertise.
    These are constructed in the paper. Their claimed purity is supported only by the accuracy tables, not by any external behavioral or interpretability test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities." pith.science (2026). https://pith.science/paper/5LMQQ7JB

@misc{pith2026260805611,
  author       = {Pith},
  title        = {Pith review of: FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5LMQQ7JB}},
  note         = {Machine review of arXiv:2608.05611}
}
read the original abstract

Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (\textbf{\underline{F}}ine-tuning with \textbf{\underline{O}}rthogonal \textbf{\underline{C}}ontrol for \textbf{\underline{U}}ncoupled persona\textbf{\underline{S}}). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at \href{https://anonymous.4open.science/r/openpersona-48F4}{this url}.

Figures

Figures reproduced from arXiv: 2608.05611 by the authors.

Figure 1
Figure 1. Performance of prompting LLMs to assume expert personas on financial, legal, and medical tasks. In this paper, we propose FOCUS (Fine-tuning with Orthogonal Control for Uncoupled personaS), a persona-vector editing framework designed to decouple and control domain-specific expert personas. FOCUS first uses an automated pipeline to extract expert persona vectors from LLMs, and then applies orthog￾onal decomposition t… view at source ↗
Figure 2
Figure 2. Our FOCUS algorithm workflow. (a) Extract general expert and single-domain expert persona vectors. (b) Decouple single-domain expert vectors from the general expert vector. (c) Conduct two-stage fine-tuning to train the expert gating module, where the first stage aligns single-domain expert personas and the second stage en￾hances cross-domain generalization. Specifically, for each domain, we generate 5 pairs of cont… view at source ↗
Figure 3
Figure 3. Distribution of the gating weight vector α. shown in [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 21 canonical work pages

  1. [8]

    arXiv preprint arXiv:2507.21509 (2025)

    Chen, R., Arditi, A., Sleight, H., Evans, O., Lindsey, J.: Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509 (2025)

    arXiv 2025

  2. [1]

    ArXivabs/2407.21783(2024),https: //api.semanticscholar.org/CorpusID:271571434

    AI@Meta: The llama 3 herd of models. ArXivabs/2407.21783(2024),https: //api.semanticscholar.org/CorpusID:271571434

    arXiv 2024

  3. [2]

    arXiv preprint arXiv:2212.08073 (2022)

    Bai, Y., Kadavath, S., Kundu, S., et al.: Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022)

    arXiv 2022

  4. [3]

    arXiv preprint arXiv:2502.17601 (2025)

    Bartoszcze, L., Munshi, S., Sukidi, B., Yen, J., Yang, Z., Williams-King, D., Le, L., Asuzu, K., Maple, C.: Representation engineering for large-language models: Survey and research challenges. arXiv preprint arXiv:2502.17601 (2025)

    arXiv 2025

  5. [4]

    Frontiers in Medicine11, 1495582 (2024)

    Bélisle-Pipon, J.C.: Why we need to be careful with llms in medicine. Frontiers in Medicine11, 1495582 (2024)

    work page 2024

  6. [5]

    Frontiers in medicine11, 1495582 (2024)

    Bélisle-Pipon, J.C.: Why we need to be careful with llms in medicine. Frontiers in medicine11, 1495582 (2024)

    work page 2024

  7. [6]

    Advances in Neural Infor- mation Processing Systems36, 66044–66063 (2023)

    Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., Biderman, S.: Leace: Perfect linear concept erasure in closed form. Advances in Neural Infor- mation Processing Systems36, 66044–66063 (2023)

    work page 2023

  8. [7]

    arXiv preprint arXiv:2507.16795 (2025)

    Casademunt,H.,Juang,C.,Karvonen,A.,etal.:Steeringout-of-distributiongener- alization with concept ablation fine-tuning. arXiv preprint arXiv:2507.16795 (2025)

    arXiv 2025

Show all 43 references
  1. [9]

    In: The Twelfth International Conference on Learning Representations (2023)

    Cheng, D., Huang, S., Wei, F.: Adapting large language models via reading com- prehension. In: The Twelfth International Conference on Learning Representations (2023)

    2023

  2. [10]

    arXiv preprint arXiv:2505.23558 (2025)

    Chu, X., Chen, X., Wang, G., Tan, Z., Huang, K., Lv, W., Mo, T., Li, W.: Qwen look again: Guiding vision-language reasoning models to re-attention visual infor- mation. arXiv preprint arXiv:2505.23558 (2025)

    2025 arXiv

  3. [11]

    In: Findings of ACL 2025 (Jul 2025)

    Chu, X., Tan, Z., Xue, H., Wang, G., et al.: Domaino1s: Guiding LLM reasoning for explainable answers in high-stakes domains. In: Findings of ACL 2025 (Jul 2025)

    2025

  4. [12]

    In: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Chu,X.,Wang,G.,Tan,Z.,Chen,X.,Li,Z.,Mo,T.,Li,W.:Towardsorderfairness: Mitigating llms order sensitivity through dual group advantage optimization. In: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 4790–4805...

    2026

  5. [13]

    arXiv preprint arXiv:2501.14427 (2025)

    Chu, X., Xue, H., Tan, Z., Wang, B., Mo, T., Li, W.: Graphsos: graph sam- pling and order selection to help llms understand graphs better. arXiv preprint arXiv:2501.14427 (2025)

    2025 arXiv

  6. [14]

    Journal of Machine Learning Research25(70), 1–53 (2024)

    Chung, H.W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al.: Scaling instruction-finetuned language models. Journal of Machine Learning Research25(70), 1–53 (2024)

    2024

  7. [15]

    arXiv preprint arXiv:2209.10652 (2022)

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield- Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al.: Toy models of superposition. arXiv preprint arXiv:2209.10652 (2022)

    2022 arXiv

  8. [16]

    Advances in neural information processing systems36, 44123–44279 (2023)

    Guha, N., Nyarko, J., Ho, D., et al.: Legalbench: A collaboratively built bench- mark for measuring legal reasoning in large language models. Advances in neural information processing systems36, 44123–44279 (2023)

    2023

  9. [17]

    arXiv preprint arXiv:2009.03300 (2020)

    Hendrycks, D., Burns, C., Basart, S., et al.: Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)

    2020 arXiv

  10. [18]

    arXiv preprint arXiv:2103.06268 (2021)

    Hendrycks, D., Burns, C., Chen, A., et al.: Cuad: An expert-annotated nlp dataset for legal contract review. arXiv preprint arXiv:2103.06268 (2021)

    2021 arXiv

  11. [19]

    In: Findings of NAACL 2024 (2024)

    Jiang, H., Zhang, X., Cao, X., et al.: PersonaLLM: Investigating the ability of large language models to express personality traits. In: Findings of NAACL 2024 (2024)

    2024

  12. [20]

    Applied Sciences11(14), 6421 (2021)

    Jin, D., Pan, E., Oufattole, N., et al.: What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences11(14), 6421 (2021)

    2021

  13. [21]

    Medium (April 2025),https://medium.com/@aadharshkannan/ self-supervised-persona-auto-fine-tuning-llms-af30fa3ff192

    Kannan, A.: Self-supervised persona (auto fine-tuning llms). Medium (April 2025),https://medium.com/@aadharshkannan/ self-supervised-persona-auto-fine-tuning-llms-af30fa3ff192

    2025

  14. [22]

    In: Proceedings of the ACM Web Conference

    Koa, K.J., Ma, Y., et al.: Learning to generate explainable stock predictions using self-reflective large language models. In: Proceedings of the ACM Web Conference

  15. [23]

    arXiv preprint arXiv:2402.10373 (2024)

    Labrak, Y., Bazoge, A., Morin, E., et al.: Biomistral: A collection of open- source pretrained large language models for medical domains. arXiv preprint arXiv:2402.10373 (2024)

    2024 arXiv

  16. [24]

    Advances in Neural Information Processing Systems36, 41451–41530 (2023)

    Li, K., Patel, O., Viégas, F., Pfister, H., Wattenberg, M.: Inference-time inter- vention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems36, 41451–41530 (2023)

    2023

  17. [25]

    Science China Technological Sciences68(8), 1820403 (2025)

    Liu, H., Lei, Y., Wu, Z.: Prosocial behavior in large language models: Value alignment and affective mechanisms. Science China Technological Sciences68(8), 1820403 (2025)

    2025

  18. [26]

    that’s no laughing matter

    Perrigo, B.: The new ai-powered bing is threatening users. that’s no laughing matter. TIME (February 2023),https://time.com/6256529/ bing-openai-chatgpt-danger-alignment/

    2023

  19. [27]

    Qwen-Team: Qwen2.5: A party of foundation models (September 2024),https: //qwenlm.github.io/blog/qwen2.5/

    2024

  20. [28]

    In: Proceedings of the 58th annual meeting of the association for computational linguistics

    Ravfogel,S.,Elazar,Y.,Gonen,H.,Twiton,M.,Goldberg,Y.:Nullitout:Guarding protected attributes by iterative nullspace projection. In: Proceedings of the 58th annual meeting of the association for computational linguistics. pp. 7237–7256 (2020)

    2020

  21. [29]

    arXiv preprint arXiv:2110.08207 (2021)

    Sanh, V., Webson, A., Raffel, C., Bach, S.H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T.L., Raja, A., et al.: Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 (2021)

    2021 arXiv

  22. [30]

    In: Proceedings of the Title Suppressed Due to Excessive Length 13 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Tan, Z., Chu, X., Wang, G., Li, Z., Li, W., Mo, T.: Rado: Reasoning audit-driven optimization for rigorous reasoning in high-stakes domains. In: Proceedings of the Title Suppressed Due to Excessive Length 13 64th Annual Meeting of the Association for Computational Linguistics ...

    2026

  23. [31]

    arXiv preprint arXiv:2509.09689 (2025)

    Thakur, H., Agrawal, E., Mukund, S.: Personas within parameters: Fine-tuning small language models with low-rank adapters to mimic user behaviors. arXiv preprint arXiv:2509.09689 (2025)

    2025 arXiv

  24. [32]

    arXiv preprint arXiv:2406.01171 (2024)

    Tseng, Y.M., Huang, Y.C., Hsiao, T.Y., et al.: Two tales of persona in llms: A survey of role-playing and personalization. arXiv preprint arXiv:2406.01171 (2024)

    2024 arXiv

  25. [33]

    arXiv preprint arXiv:2308.10248 (2023)

    Turner, A.M., Thiergart, L., Leech, G., Udell, D., Vazquez, J.J., Mini, U., MacDi- armid, M.: Steering language models with activation engineering. arXiv preprint arXiv:2308.10248 (2023)

    2023 arXiv

  26. [34]

    In: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Wang, G., Chu, X., Tan, Z., Chen, X., Mo, T., Li, W.: Muse: Multi-stage graph reasoning via vision-language models. In: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 10442–10462 (2026)

    2026

  27. [35]

    In: Proceedings of the 7th ACM International Conference on Multimedia in Asia

    Wang, G., Tan, Z., Chu, X., Chen, X., Mo, T., Li, W.: Learn concepts from multi- scale visual information for compositional zero-shot learning. In: Proceedings of the 7th ACM International Conference on Multimedia in Asia. pp. 1–8 (2025)

    2025

  28. [36]

    arXiv preprint arXiv:2109.01652 (2021)

    Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)

    2021 arXiv

  29. [37]

    arXiv preprint arXiv:2506.22808 (2025)

    Wei, J., Meng, Z., Xiao, Z., et al.: Medethicsqa: A comprehensive question answering benchmark for medical ethics evaluation of llms. arXiv preprint arXiv:2506.22808 (2025)

    2025 arXiv

  30. [38]

    arXiv preprint arXiv:2412.20138 (2024)

    Xiao, Y., Sun, E., Luo, D., Wang, W.: Tradingagents: Multi-agents llm financial trading framework. arXiv preprint arXiv:2412.20138 (2024)

    2024 arXiv

  31. [39]

    In: The Twelfth International Conference on Learning Representations (2023)

    Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q.V., Zhou, D., Chen, X.: Large lan- guage models as optimizers. In: The Twelfth International Conference on Learning Representations (2023)

    2023

  32. [40]

    arXiv preprint arXiv:2309.16042 (2023)

    Zhang, F., Nanda, N.: Towards best practices of activation patching in language models: Metrics and methods. arXiv preprint arXiv:2309.16042 (2023)

    2023 arXiv

  33. [41]

    Advances in Neural Information Processing Systems37, 26045– 26081 (2024)

    Zhang, K., Zeng, S., Hua, E., et al.: Ultramedical: Building specialized generalists in biomedicine. Advances in Neural Information Processing Systems37, 26045– 26081 (2024)

    2024

  34. [42]

    In: Findings of ACL 2024 (Aug 2024)

    Zhou, X., Lu, Y., Ma, R., et al.: Making harmful behaviors unlearnable for large language models. In: Findings of ACL 2024 (Aug 2024)

    2024

  35. [2024]

    4304–4315 (2024)

    pp. 4304–4315 (2024)

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.