The reviewed record of science sign in
Pith

arxiv: 2402.01618 · v1 · pith:CPM3GS46 · submitted 2024-02-02 · cs.CL

Style Vectors for Steering Generative Large Language Model

Kai Konen , Sophie Jentzsch , Diaoul\'e Diallo , Peer Sch\"utt , Oliver Bensch , Roxanne El Baff , Dominik Opitz , Tobias Hecking This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:CPM3GS46record.jsonopen to challenge →

classification cs.CL
keywords stylevectorsactivationsengineeringlanguagelargeresearchspecific
0
0 comments X
read the original abstract

This research explores strategies for steering the output of large language models (LLMs) towards specific styles, such as sentiment, emotion, or writing style, by adding style vectors to the activations of hidden layers during text generation. We show that style vectors can be simply computed from recorded layer activations for input texts in a specific style in contrast to more complex training-based approaches. Through a series of experiments, we demonstrate the effectiveness of activation engineering using such style vectors to influence the style of generated text in a nuanced and parameterisable way, distinguishing it from prompt engineering. The presented research constitutes a significant step towards developing more adaptive and effective AI-empowered interactive systems.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Distributed Interpretability and Control for Large Language Models

    cs.LG 2026-04 conditional novelty 4.0

    A distributed system for logit lens and steering vectors on multi-GPU LLMs achieves up to 7x lower activation memory and 41x higher throughput while producing monotonic output shifts with mean slope 0.702.