Pith. sign in

REVIEW 4 cited by

Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.01183 v1 pith:CLCSYDVK submitted 2023-06-01 cs.CL

Adithya V Ganesan , Yash Kumar Lal , August H{aa}kan Nilsson , H. Andrew Schwartz This is my paper
classification cs.CL
keywords gpt-3zero-shotperformancepersonalitysystematicclassificationclosehowever
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Very large language models (LLMs) perform extremely well on a spectrum of NLP tasks in a zero-shot setting. However, little is known about their performance on human-level NLP problems which rely on understanding psychological concepts, such as assessing personality traits. In this work, we investigate the zero-shot ability of GPT-3 to estimate the Big 5 personality traits from users' social media posts. Through a set of systematic experiments, we find that zero-shot GPT-3 performance is somewhat close to an existing pre-trained SotA for broad classification upon injecting knowledge about the trait in the prompts. However, when prompted to provide fine-grained classification, its performance drops to close to a simple most frequent class (MFC) baseline. We further analyze where GPT-3 performs better, as well as worse, than a pretrained lexical model, illustrating systematic errors that suggest ways to improve LLMs on human-level NLP tasks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Context-Value-Action Architecture for Value-Driven Large Language Model Agents

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    The Context-Value-Action architecture decouples reasoning from action in LLM agents via a human-data-trained Value Verifier, mitigating polarization and outperforming prompt-based methods on a large real-world benchmark.

  2. When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Code-generating LLMs lose 20 to 40 percentage points in pass rate when task descriptions are ambiguous, incomplete, or contradictory.

  3. LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    LLM framework converts facial action unit sequences to text, fuses with responses, and regresses to personality scores, reporting lower errors and higher correlations than baselines on AVI-6.

  4. Personality Prediction from Life Stories using Language Models

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A two-step model, sliding-window fine-tuned RoBERTa plus an attention GRU, predicts Big Five scores from life-narrative interviews with higher R2 than several transformer baselines.

Pith tools