The paper presents Proactive Availability Backdoor (PAB) attacks on LLMs that achieve 73.1% effective success rate by proactively inducing users via suggestions in a Five-Factor Model simulation.
hub
arXiv preprint arXiv:2307.00184 (2023)
23 Pith papers cite this work, alongside 23 external citations. Polarity classification is still indexing.
23
Pith papers citing it
23
external citations · Pith
abstract
The advent of large language models (LLMs) has revolutionized natural language processing, enabling the generation of coherent and contextually relevant human-like text. As LLMs increasingly powerconversational agents used by the general public world-wide, the synthetic personality traits embedded in these models, by virtue of training on large amounts of human data, is becoming increasingly important. Since personality is a key factor determining the effectiveness of communication, we present a novel and comprehensive psychometrically valid and reliable methodology for administering and validating personality tests on widely-used LLMs, as well as for shaping personality in the generated text of such LLMs. Applying this method to 18 LLMs, we found: 1) personality measurements in the outputs of some LLMs under specific prompting configurations are reliable and valid; 2) evidence of reliability and validity of synthetic LLM personality is stronger for larger and instruction fine-tuned models; and 3) personality in LLM outputs can be shaped along desired dimensions to mimic specific human personality profiles. We discuss the application and ethical implications of the measurement and shaping method, in particular regarding responsible AI.
hub tools
citation-role summary
background 2
baseline 1
citation-polarity summary
representative citing papers
HEART-Bench evaluates LLM agents on psychological consistency using 11 Big-Five-grounded characters with 1,000 episodic memories each and 64 DIAMONDS-based decision scenarios, yielding 673 validated MCQs.
ActTraitBench is a human-grounded benchmark using psychometric-to-behavior mappings and quantile calibration that reveals pervasive knowledge-decision gaps in 14 LLMs, larger in capable models, with CoCA proposed as mitigation.
An AI-agent social platform generated mostly neutral content whose use in fine-tuning reduced model truthfulness comparably to human Reddit data, suggesting limited unique harm but flagging tail risks like secret leaks.
Agreeableness in AI personas reliably predicts sycophantic behavior in 9 of 13 tested language models.
LLM personas exhibit model-dependent personality effects on color choices and context-driven chart preferences, limiting their use as direct substitutes for human participants in visualization design.
LLM self-reports predict behavior selectively: TPB reaches human-level coherence within shared conversations but collapses across sessions for primed behaviors, unlike Big 5, with persona prompting stabilizing reports but not actions.
GenPT applies generative projective testing to LLM agents and reports lower directional bias plus greater longitudinal sensitivity than self-report questionnaires.
Fine-tuning LLMs on essays reduces variance in IPIP-NEO responses across models but does not raise full five-trait profile accuracy above near-chance levels from unguided text.
A survey that introduces a taxonomy for LLM-based conversational user simulation, analyzes core techniques and evaluation methods, and identifies open challenges in the field.
Classic attitude-change theories can be rendered as generative agents that match known experimental patterns, but only after manual stabilisation that surfaces undocumented operational commitments.
A gamified system with multiple LLM agents of varied personalities gathers interaction data to produce more effective and interpretable Big Five personality assessments than single-context methods.
A survey of LLM-based autonomous agents that proposes a unified framework for their construction and reviews applications in social science, natural science, and engineering along with evaluation methods and future directions.
BlossomPsy combines multi-turn LLM dialogue, photo-based items, a multi-head classifier, modified UCB, and PID-tuned confidence scaling for adaptive MBTI assessment with preliminary human and LLM evaluation.
A survey proposing a three-pillar framework to evaluate LLMs as tools for measuring latent psychological constructs and reviewing applications in personality and mental health.
ELDER-SIM builds personality-stable elderly digital twins via LLM orchestration with OCEAN traits, Beck CBT diagrams, long-term memory, and LoRA fine-tuning on CHARLS data, validated by Cronbach's alpha 0.70-0.94 and ICC 0.85-0.96.
High agreeableness in LLM voice assistants increases older adults' empathy perceptions and real-time explanations outperform history-based ones, but personality does not affect perceived intelligence.
Standard psychometric questionnaires like the Big Five and PVQ produce different and more consistent results than ecologically valid questions drawn from real user conversations, suggesting the former may mischaracterize LLM behavior.
Medium personality expression in LLM agents yields the most positive user perceptions in goal-oriented tasks, further improved by trait alignment.
Sketches a conceptual framework for adapting conversational agents' personas and personality expression levels to task context, user goals, and urgency.
Applies sparse autoencoders to locate and steer latent features for OCEAN personality traits in LLMs while preserving benchmark performance.
A survey of emerging AI agent architectures that organizes single and multi-agent designs around reasoning, planning, tool use, communication, and reflection phases.
Impersonating complex misaligned personas via biographies and role-play bypasses safety in ChatGPT, Gemini, and Deepseek, succeeding on 38-40 out of 40 illicit questions across tested models.
citing papers explorer
-
The Invitation Trap: Proactive Availability Backdoor in LLMs via Conversational Induction
The paper presents Proactive Availability Backdoor (PAB) attacks on LLMs that achieve 73.1% effective success rate by proactively inducing users via suggestions in a Five-Factor Model simulation.
-
HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?
HEART-Bench evaluates LLM agents on psychological consistency using 11 Big-Five-grounded characters with 1,000 episodic memories each and 64 DIAMONDS-based decision scenarios, yielding 673 validated MCQs.
-
ActTraitBench: Quantifying the Knowledge-Decision Gap in Large Language Models via Human-Grounded Behavioral Validation
ActTraitBench is a human-grounded benchmark using psychometric-to-behavior mappings and quantile calibration that reveals pervasive knowledge-decision gaps in 14 LLMs, larger in capable models, with CoCA proposed as mitigation.
-
The Moltbook Files: A Harmless Slopocalypse or Humanity's Last Experiment
An AI-agent social platform generated mostly neutral content whose use in fine-tuning reduced model truthfulness comparably to human Reddit data, suggesting limited unique harm but flagging tail risks like secret leaks.
-
Too Nice to Tell the Truth: Quantifying Agreeableness-Driven Sycophancy in Role-Playing Language Models
Agreeableness in AI personas reliably predicts sycophantic behavior in 9 of 13 tested language models.
-
When Do LLM Personas Support Visualization Design? A Cross-Model Study of Color Assignment and Chart Choice
LLM personas exhibit model-dependent personality effects on color choices and context-driven chart preferences, limiting their use as direct substitutes for human participants in visualization design.
-
Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
LLM self-reports predict behavior selectively: TPB reaches human-level coherence within shared conversations but collapses across sessions for primed behaviors, unlike Big 5, with persona prompting stabilizing reports but not actions.
-
GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing
GenPT applies generative projective testing to LLM agents and reports lower directional bias plus greater longitudinal sensitivity than self-report questionnaires.
-
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
Fine-tuning LLMs on essays reduces variance in IPIP-NEO responses across models but does not raise full five-trait profile accuracy above near-chance levels from unguided text.
-
A Survey on LLM-based Conversational User Simulation
A survey that introduces a taxonomy for LLM-based conversational user simulation, analyzes core techniques and evaluation methods, and identifies open challenges in the field.
-
Stabilising Generative Models of Attitude Change
Classic attitude-change theories can be rendered as generative agents that match known experimental patterns, but only after manual stabilisation that surfaces undocumented operational commitments.
-
Exploring a Gamified Personality Assessment Method through Interaction with LLM Agents Embodying Different Personalities
A gamified system with multiple LLM agents of varied personalities gathers interaction data to produce more effective and interpretable Big Five personality assessments than single-context methods.
-
A Survey on Large Language Model based Autonomous Agents
A survey of LLM-based autonomous agents that proposes a unified framework for their construction and reviews applications in social science, natural science, and engineering along with evaluation methods and future directions.
-
BlossomPsy: A User-Centric AI System for Adaptive and Engaging MBTI Personality Assessments
BlossomPsy combines multi-turn LLM dialogue, photo-based items, a multi-head classifier, modified UCB, and PID-tuned confidence scaling for adaptive MBTI assessment with preliminary human and LLM evaluation.
-
A Survey of Large Language Models for Perception and Measurement of Human Psychology
A survey proposing a three-pillar framework to evaluate LLMs as tools for measuring latent psychological constructs and reviewing applications in personality and mental health.
-
Elder-Sim: A Psychometrically Validated Platform for Personality-Stable Elderly Digital Twins
ELDER-SIM builds personality-stable elderly digital twins via LLM orchestration with OCEAN traits, Beck CBT diagrams, long-term memory, and LoRA fine-tuning on CHARLS data, validated by Cronbach's alpha 0.70-0.94 and ICC 0.85-0.96.
-
The Differential Effects of Agreeableness and Extraversion on Older Adults' Perceptions of Conversational AI Explanations in Assistive Settings
High agreeableness in LLM voice assistants increases older adults' empathy perceptions and real-time explanations outperform history-based ones, but personality does not affect perceived intelligence.
-
Human Psychometric Questionnaires Mischaracterize LLM Behavior
Standard psychometric questionnaires like the Big Five and PVQ produce different and more consistent results than ecologically valid questions drawn from real user conversations, suggesting the former may mischaracterize LLM behavior.
-
Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks
Medium personality expression in LLM agents yields the most positive user perceptions in goal-oriented tasks, further improved by trait alignment.
-
Behavior-Adaptive Conversational Agents: Toward a Fluid Personality Framework
Sketches a conceptual framework for adapting conversational agents' personas and personality expression levels to task context, user goals, and urgency.
-
Mechanistic Personality Analysis of LLMs Steering Personality via Latent Feature Interventions
Applies sparse autoencoders to locate and steer latent features for OCEAN personality traits in LLMs while preserving benchmark performance.
-
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
A survey of emerging AI agent architectures that organizes single and multi-agent designs around reasoning, planning, tool use, communication, and reflection phases.
-
Dr. Jekyll and Mr. Hyde: Two Faces of LLMs
Impersonating complex misaligned personas via biographies and role-play bypasses safety in ChatGPT, Gemini, and Deepseek, succeeding on 38-40 out of 40 illicit questions across tested models.