Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation

Ashton Anderson; Difan Jiao; Jessica Bo; Seth Grief-Albert

REVIEW 4 major objections 4 minor 35 references

Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Base LLMs emulate people; tuned LLMs estimate them.

desk verdict Useful emulation/estimation distinction, but the base-vs-post-trained comparison needs matched scaffolds before the causal story holds. read the letter →

arxiv 2608.03044 v1 pith:SB4MG3NM submitted 2026-08-04 cs.CL cs.AI

Seth Grief-Albert , Jessica Bo , Difan Jiao , Ashton Anderson This is my paper · ORCID

classification cs.CLcs.AI

keywords opinionsimulationemulationvsestimationbasemodelspost-traineddemographicconditioningdistributionalalignmentpersonacollapseopen-responsegeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that conflicting results in LLM opinion simulation come from conflating two distinct tasks. In emulation, a model generates individual survey answers and the population distribution emerges by aggregation; in estimation, the model directly predicts the distribution. Across three matched base/post-trained model pairs on the Pew American Trends Panel, base models are stronger emulators—closer to human ground truth and better at preserving demographic structure—while post-trained models are stronger estimators. This suggests that simulation users should choose the substrate according to the output form required: base models for generating text, post-trained models for predicting distributions.

What carries the argument

The central object is the emulation–estimation distinction itself, operationalized through two evaluation pipelines: open-response generation with an LLM judge mapping free text to answer categories (to eliminate positional bias), and verbalized-distribution estimation in which the model outputs a JSON probability vector. The load-bearing comparison runs across matched base and post-trained model pairs, using total variation distance and Wasserstein distance as complementary fidelity metrics, plus Spearman correlation of model and human pairwise inter-group distances to test demographic structure. The proposed explanatory mechanism is the 'assistant persona bottleneck': post-training steers the base distribution into a single helpful-assistant persona that is good at direct estimation but compresses the diversity needed for faithful emulation.

What would settle it

Re-run the emulation evaluation with both model types under identical prompt scaffolds and identical sampling temperatures (for example, giving post-trained models the interviewer-participant dialogue with demographic conditioning as a preceding turn). If a post-trained model then matches or beats its base counterpart on Wasserstein distance to human ground truth, the paper's central claim of a base-model emulation advantage is falsified.

Watch

Extended reading notes

Core claim

On a benchmark of 59 four-option Pew survey items across seven demographic conditions, the authors find that base (pre-instruction-tuning) models generate individual responses whose aggregated distributions are closer to human ground truth on both total variation and Wasserstein distance in every model-condition comparison, and they better track the structure of demographic differences (Spearman rho 0.61–0.75 vs. 0.35–0.59 for post-trained models). Post-trained models give more accurate direct estimates of population distributions, with error decreasing at larger scale and the frontier reference model performing best. The authors interpret this as a tradeoff introduced by post-training: instruction tuning compresses output diversity (post-trained responses are 8.0–14.0 times more self-similar at the bigram level), so the model plays a single assistant persona simulating a demographic persona, whereas base models draw on the pre-training persona distribution directly. The practical conclusion is that emulation and estimation should be evaluated separately, and model choice should follow whether the task needs generated text or direct distribution prediction.

Load-bearing premise

The comparison assumes that the measured differences between base and post-trained models come from training status rather than from the different prompt scaffolds (dialogue vs. system-instruction) and sampling temperatures used for each model type; if post-trained models were re-run under the base models' dialogue scaffold, the emulation ranking could change.

Editorial extensions

If this is right

Model selection for human simulation becomes task-driven: use base models when the pipeline needs generated text (synthetic respondents, interactive agents), and post-trained models when it needs direct distributional estimates (polling-style predictions).
Conflicting prior results become explainable as a task mismatch: studies reporting good alignment typically evaluated base-model sampling, while studies reporting persona collapse and demographic insensitivity mostly tested post-trained models.
Emulation evaluations should use open-response generation with judge-based mapping to avoid the positional bias that confounds first-token probability extraction; the paper's first-token results corroborate the main finding.
Post-training is not uniformly better for simulation: it trades estimation accuracy against the output diversity required for faithful emulation, a tradeoff that should be measured and reported rather than assumed away.
If the claim holds, instruction-tuned models' exaggeration of inter-group differences (about 2x) relative to humans, versus base models' compression to about 70%, bears directly on any deployment of synthetic respondents for policy or market research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the emulation–estimation split may generalize beyond opinion surveys to any demographic-conditioned generation task, such as persona-driven dialogue, creative writing, or agent simulation, where base models' broader output distribution could better preserve population diversity than instruct-tuned models.
Editorial inference: a testable extension would fine-tune a post-trained model to restore output diversity (e.g., via diversity-promoting training or multi-persona decoding) and check whether its estimation advantage survives; if the tradeoff is controlled by a single capability knob, it could be tuned rather than accepted.
Editorial inference: because frontier base checkpoints are not released, the paper's scale trend for estimation cannot currently be tested for emulation at the largest closed models; open-weight frontier base models would provide the decisive data.
Editorial inference: the Spearman results suggest emulation quality is not only about aggregate closeness but about the geometry of inter-group differences; a practical extension is to use the pairwise distance ratio (compression vs. exaggeration) as a design diagnostic for synthetic populations.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Best to know: this paper gives a clean name to a real distinction — generating individual responses (emulation) vs directly predicting the population distribution (estimation) — and shows, across three matched model families, that base models are more faithful emulators while post-trained models are better estimators. That is a genuinely useful framing, and the open-response protocol is a sensible way to dodge positional bias. The paper is honest about its limits and does not overclaim on the mechanism.

What is solid: six models, three matched base/post-trained pairs, 59 Pew items, seven demographic conditions. The emulation advantage is consistent in every condition, and the demographic-correlation results (base models compress inter-group differences to roughly 70% of human magnitude while post-trained models exaggerate by about 2x) are striking. No free parameters are fitted; evaluation is against external ground truth. The first-token appendix corroborates the direction. The Persona Selection Model is presented as interpretation, not evidence — fine.

Where it wobbles: the central comparison is not controlled for prompt scaffold. Base emulation uses an interviewer-participant dialogue with demographics in a prior turn; post-trained gets a system instruction naming the attribute. Estimation uses a partial string for base and JSON for post-trained. Temperature is unmatched (post-trained 1.5, base unreported). Any of these could drive part of the gap. The paper never tests the crossed conditions. Given that the entire causal story in Section 5 rests on an 'assistant persona bottleneck,' that is a real gap. Also, Table 1 shows the Olmo-7B base beats its instruct variant on estimation TVD (0.278 vs 0.285), so the estimation claim should be softened. The LLM judge's kappa of 0.66 against a single in-team annotator is moderate; a second annotator or a disclosed judge model would help.

None of this kills the paper. The emulation finding is likely to survive matched scaffolds — the self-similarity numbers (8–14x more collapse in post-trained outputs) are independent of scaffold. But as written, the attribution to post-training per se is not established.

Who benefits: anyone picking a model for synthetic respondents or for survey-distribution prediction. It deserves a serious referee, but the revision should add crossed-scaffold robustness checks, report base-model temperature, disclose the judge model, and qualify the estimation exception.

Referee Report

4 major / 4 minor

Summary. The paper distinguishes two tasks for LLM-based opinion simulation: emulation (generating individual responses that aggregate into a population distribution) and estimation (directly predicting the population distribution). Using three matched base/post-trained model pairs on 59 Pew American Trends Panel items, the authors report that base models are stronger emulators on both total variation and Wasserstein distance and better preserve demographic structure, while post-trained models are stronger estimators when asked explicitly for distributions. The authors propose that downstream task form should determine which model type to use, and they interpret the tradeoff through an 'Assistant persona bottleneck' in post-training.

Significance. The emulation–estimation distinction is a useful organizing principle and may help reconcile conflicting results in the LLM opinion-simulation literature. The paper has clear strengths: it evaluates frozen models against external Pew ground truth, fits no parameters, validates the LLM judge against human annotation, includes a first-token extraction corroboration in Appendix A.6, and makes explicit limitations. If the central claims hold, the paper offers actionable model-selection guidance. However, the current evidence is weakened by unmatched prompt scaffolds and sampling procedures across base and post-trained conditions, and by at least one pair-level reversal in the estimation result. The core claims are defensible but need additional robustness experiments before they can be considered established.

major comments (4)

[§3.1, Appendices A.2.2–A.2.3, §4.1] The central emulation comparison is confounded by prompt scaffold: base models are prompted with an interviewer-participant dialogue and demographic conditioning inserted as a prior turn (Figure 5), while post-trained models receive a system instruction that names the attribute (Figure 7). The causal account in §5 attributes the emulation gap to post-training, but any of the scaffold differences could drive the observed base advantage. In addition, post-trained emulation samples at temperature 1.5 while the base-model temperature is not reported, and temperature directly affects the output-diversity measures in Table 4. Please report the base sampling temperature and add matched-scaffold robustness tests, e.g., post-trained models under the base dialogue scaffold and base models under the post-trained system scaffold, with temperatures held fixed.
[§3.2, Appendices A.3.1–A.3.2] The estimation comparison is similarly scaffold- and procedure-dependent: base models complete a partial distribution string (Figure 8) while post-trained models must return a JSON object with integer percentages summing to 100 (Figure 9), and base estimates average multiple completions while post-trained estimates use a single greedy decode. The claim that post-trained models are stronger estimators is therefore not yet isolated from output-format and sampling effects. Please test both model types under a common output format and sampling procedure, for example by prompting post-trained models with the partial-string scaffold and base models with JSON instructions.
[Table 1 and Table 6] The estimation claim is not consistent for one of the three pairs: Olmo-3-7B-Instruct has higher mean TVD (0.285) than its base counterpart Olmo-3-1025-7B (0.278), and Table 6 shows the reversal is concentrated in the Very Liberal and Very Conservative conditions (0.387 vs. 0.365 and 0.370 vs. 0.268). The abstract and §4.3 state that post-trained models are stronger estimators without this qualification. Please either weaken the claim to an average effect and report per-pair consistency, or provide an analysis explaining the reversal.
[Table 2 and §4.2] For Olmo-3-7B-Instruct, the demographic-structure correlation is not statistically significant (ρ = 0.350, p = 0.201), yet the text groups this model with others under 'weaker structural alignment' and uses the aggregate interpretation to support the emulation claim. Because demographic structure preservation is part of the central emulation finding, this non-significant result should be explicitly acknowledged and the strength of the corresponding claim moderated.

minor comments (4)

[Table 1] The post-trained rows for Qwen3-14B and Olmo-3-1125-32B appear to be missing the 'Instruct' label, making the table difficult to parse; please add consistent labels for all rows.
[§4.2 and Appendix A.1] The phrase 'across all six models' is ambiguous because there are three matched pairs of model variants; please say 'across all six model variants (three matched pairs)'.
[Appendix A.2.1] The human annotator is reported as a member of the research team; please state whether the LLM judge and the annotator were blinded to model type and whether any independent annotators were used, since the judge validation underlies the emulation pipeline.
[Appendix A.6] Figure 11 is described as spanning '8B to 14B scale' but only Qwen3-14B appears in the main experiments; please clarify which additional Qwen sizes were evaluated and where the results are available.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper evaluates frozen models against an external Pew benchmark with no fitted parameters whose outputs are renamed as predictions.

full rationale

The paper's central claims are empirical comparisons of frozen base and post-trained models against ground-truth distributions from the Pew American Trends Panel. No parameter is fitted to the target distributions and then reported as a prediction; the emulated and estimated outputs are produced by the models themselves and scored with standard distance metrics (TVD and Wasserstein) defined in Appendix A.4. The LLM judge used to map free-text responses is validated against human annotations (Cohen's κ = 0.66), so the evaluation pipeline is anchored to external human judgment rather than to the models' own outputs. The cited prior work, including the Persona Selection Model, is offered as a theoretical interpretation after the empirical results, not as evidence used to derive them; the claims stand on the measured distances in Tables 1 and 2. The main methodological concern is that base and post-trained models receive different prompt scaffolds and sampling temperatures, which is a potential confound in the experimental design, but this is not circularity: the compared quantities are not defined in terms of one another, and no result is forced by construction. The paper is self-contained against an external benchmark, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces a conceptual distinction (emulation vs estimation) but no new physical or mathematical entities. The evaluation relies on unstated design choices: the judge model, the base-model sampling temperature, and the prompt formats, which are listed as free parameters or assumptions rather than fitted values.

free parameters (2)

Post-trained emulation sampling temperature = 1.5
Set to increase output diversity for post-trained models; base models' temperature is not reported, creating an unmatched sampling protocol.
LLM judge model = not specified
The model that maps free-text responses to categories is not named; its behavior is a hidden component of the emulation results.

assumptions (4)

domain assumption Pew ATP Wave 54 ground-truth distributions are valid human opinion measurements.
All fidelity scores are computed against this 2019 survey reference; if it is not representative, every comparison is affected (Section 4.1).
domain assumption The LLM judge maps open-response text to answer categories as accurately as a human annotator.
The judge achieves Cohen's kappa 0.66 against a single in-team annotator (Appendix A.2.1); moderate agreement means some mapping noise is present.
domain assumption Open-response generation is a valid, positional-bias-free instantiation of emulation.
The paper argues first-token extraction is confounded by positional bias and adopts open-response sampling (Section 3.1), but this is an operational assumption.
ad hoc to paper Observed differences between base and post-trained models are attributable to training status, not to the differing prompt scaffolds and sampling temperatures.
Base models use an interviewer-participant dialogue and an unspecified temperature; post-trained models use system instructions and temperature 1.5 (Figures 4 to 7), so the comparison conflates scaffolding with training status.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation." pith.science (2026). https://pith.science/paper/SB4MG3NM

@misc{pith2026260803044,
  author       = {Pith},
  title        = {Pith review of: Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SB4MG3NM}},
  note         = {Machine review of arXiv:2608.03044}
}

read the original abstract

Large language models are increasingly used to simulate human opinions, but prior work reports conflicting results: some studies find promising alignment with human survey data, while others find persona collapse and weak demographic sensitivity. We show that much of this conflict stems from conflating two distinct tasks. We call the first task emulation, in which models generate individual responses that aggregate into a population distribution. We call the second task estimation, in which models directly predict the population distribution. Evaluating six matched base and post-trained models on the Pew American Trends Panel, we find that base models are stronger emulators: they produce response distributions closer to human ground truth and better preserve demographic structure. Post-trained models are stronger estimators, producing more accurate distributional predictions when asked directly. We propose that model selection for human simulation should be guided by whether the task requires generating text or predicting distributions.

Figures

Figures reproduced from arXiv: 2608.03044 by the authors.

**Figure 3.** Judge prompt used to map free-form model [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 4.** Prompt scaffold for unconditioned base-model [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Figures from the paper (6 more)

**Figure 5.** Figure 5: Prompt scaffold for demographically con [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Prompt scaffold for unconditioned posttrained model emulation. The system instruction frames the model as a survey participant and requests a plaintext response to the survey question. Prompt <system>You are simulating a participant’s answer to a survey question. Gen…

**Figure 8.** Figure 8: Prompt scaffold for base-model estimation. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

**Figure 9.** Figure 9: Prompt scaffold for post-trained model estima [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

**Figure 10.** Figure 10: Per-condition emulation error (TVD). Lower [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]

**Figure 11.** Figure 11: First-token emulation error (Wasserstein [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 15 canonical work pages

[1]

2026 , month = feb, day =

Marks, Sam and Lindsey, Jack and Olah, Christopher , title =. 2026 , month = feb, day =

work page 2026
[2]

Proceedings of the 40th International Conference on Machine Learning , articleno =

Santurkar, Shibani and Durmus, Esin and Ladhak, Faisal and Lee, Cinoo and Liang, Percy and Hashimoto, Tatsunori , title =. Proceedings of the 40th International Conference on Machine Learning , articleno =. 2023 , publisher =

2023
[3]

Benchmarking Distributional Alignment of Large Language Models

Meister, Nicole and Guestrin, Carlos and Hashimoto, Tatsunori. Benchmarking Distributional Alignment of Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.naacl-long.2

doi:10.18653/v1/2025.naacl-long.2 2025
[4]

Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations

Cao, Yong and Liu, Haijiang and Arora, Arnav and Augenstein, Isabelle and R. Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 20...

doi:10.18653/v1/2025.naacl-long.162 2025
[5]

Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions

Suh, Joseph and Jahanparast, Erfan and Moon, Suhong and Kang, Minwoo and Chang, Serina. Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.1028

doi:10.18653/v1/2025.acl-long.1028 2025
[6]

Out of One, Many: Using Language Models to Simulate Human Samples , volume =

Argyle, Lisa and Busby, Ethan and Fulda, Nancy and Gubler, Joshua and Rytting, Christopher and Wingate, David , year =. Out of One, Many: Using Language Models to Simulate Human Samples , volume =. Political Analysis , doi =

work page
[7]

, title =

Park, Joon Sung and O'Brien, Joseph and Cai, Carrie Jun and Morris, Meredith Ringel and Liang, Percy and Bernstein, Michael S. , title =. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , articleno =. 2023 , isbn =. doi:10.1145/3586183.3606763 , abstract =

work page arXiv 2023
[8]

``My Answer is C '': First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

Wang, Xinpeng and Ma, Bolei and Hu, Chengzhi and Weber-Genzel, Leon and R. ``My Answer is C '': First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.441

doi:10.18653/v1/2024.findings-acl.441 2024

Show all 35 references

[9]

Virtual Personas for Language Models via an Anthology of Backstories

Moon, Suhong and Abdulhai, Marwa and Kang, Minwoo and Suh, Joseph and Soedarmadji, Widyadewi and Behar, Eran Kohen and Chan, David M. Virtual Personas for Language Models via an Anthology of Backstories. Proceedings of the 2024 Conference on Empirical Methods in Natural Langua...

2024 doi
[10]

Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLM s

Sun, Huaman and Pei, Jiaxin and Choi, Minje and Jurgens, David. Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLM s. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computat...

2025 doi
[11]

Systematic Biases in LLM Simulations of Debates

Taubenfeld, Amir and Dover, Yaniv and Reichart, Roi and Goldstein, Ariel. Systematic Biases in LLM Simulations of Debates. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.16

2024 doi
[12]

2024 , eprint=

Towards Measuring the Representation of Subjective Global Opinions in Language Models , author=. 2024 , eprint=

2024
[13]

and Rosenthal, Seth A

Lee, Sanguk and Peng, Tai-Quan and Goldberg, Matthew H. and Rosenthal, Seth A. and Kotcher, John E. and Maibach, Edward W. and Leiserowitz, Anthony , title =. PLOS Climate , volume =. 2024 , doi =

2024
[14]

2024 , eprint=

Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information , author=. 2024 , eprint=

2024
[15]

2026 , eprint=

LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals , author=. 2026 , eprint=

2026
[16]

2025 , eprint=

LLM Generated Persona is a Promise with a Catch , author=. 2025 , eprint=

2025
[17]

Do LLM s Exhibit Human-like Response Biases? A Case Study in Survey Design

Tjuatja, Lindia and Chen, Valerie and Wu, Tongshuang and Talwalkwar, Ameet and Neubig, Graham. Do LLM s Exhibit Human-like Response Biases? A Case Study in Survey Design. Transactions of the Association for Computational Linguistics. 2024. doi:10.1162/tacl_a_00685

2024 doi
[18]

Questioning the survey responses of large language models , year =

Dominguez-Olmedo, Ricardo and Hardt, Moritz and Mendler-D\". Questioning the survey responses of large language models , year =. Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =
[19]

2026 , eprint=

The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models , author=. 2026 , eprint=

2026
[20]

2026 , eprint=

Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents , author=. 2026 , eprint=

2026
[21]

The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

Measuring what Matters: Construct Validity in Large Language Model Benchmarks , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=
[22]

2026 , eprint=

This human study did not involve human subjects: Validating LLM simulations as behavioral evidence , author=. 2026 , eprint=

2026
[23]

2026 , url =

Claude Opus 4.6 System Card , institution =. 2026 , url =

2026
[24]

2025 , eprint=

Qwen3 Technical Report , author=. 2025 , eprint=

2025
[25]

2026 , eprint=

Olmo 3 , author=. 2026 , eprint=

2026
[26]

2023 , eprint=

Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia , author=. 2023 , eprint=

2023
[27]

On the Relationship between Truth and Political Bias in Language Models

Fulay, Suyash and Brannon, William and Mohanty, Shrestha and Overney, Cassandra and Poole-Dayan, Elinor and Roy, Deb and Kabbara, Jad. On the Relationship between Truth and Political Bias in Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural La...

2024 doi
[28]

2025 , eprint=

LLM Social Simulations Are a Promising Research Method , author=. 2025 , eprint=

2025
[29]

2026 , eprint=

The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models , author=. 2026 , eprint=

2026
[30]

arXiv preprint arXiv:2502.00561 , year=

Position: Evaluating generative ai systems is a social science measurement challenge , author=. arXiv preprint arXiv:2502.00561 , year=

arXiv
[31]

2020 , month = jan, day =

2020
[32]

2025 , eprint=

Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification , author=. 2025 , eprint=

2025
[33]

2026 , eprint=

Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approach , author=. 2026 , eprint=

2026
[34]

Language Models as Agent Models

Andreas, Jacob. Language Models as Agent Models. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022. doi:10.18653/v1/2022.findings-emnlp.423

2022 doi
[35]

Nature , author =

Role play with large language models , volume =. Nature , author =. 2023 , pages =. doi:10.1038/s41586-023-06647-8 , abstract =

2023 doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

2026 , month = feb, day =

Marks, Sam and Lindsey, Jack and Olah, Christopher , title =. 2026 , month = feb, day =

work page 2026

[2] [2]

Proceedings of the 40th International Conference on Machine Learning , articleno =

Santurkar, Shibani and Durmus, Esin and Ladhak, Faisal and Lee, Cinoo and Liang, Percy and Hashimoto, Tatsunori , title =. Proceedings of the 40th International Conference on Machine Learning , articleno =. 2023 , publisher =

2023

[3] [3]

Benchmarking Distributional Alignment of Large Language Models

Meister, Nicole and Guestrin, Carlos and Hashimoto, Tatsunori. Benchmarking Distributional Alignment of Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.naacl-long.2

doi:10.18653/v1/2025.naacl-long.2 2025

[4] [4]

Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations

Cao, Yong and Liu, Haijiang and Arora, Arnav and Augenstein, Isabelle and R. Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 20...

doi:10.18653/v1/2025.naacl-long.162 2025

[5] [5]

Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions

Suh, Joseph and Jahanparast, Erfan and Moon, Suhong and Kang, Minwoo and Chang, Serina. Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.1028

doi:10.18653/v1/2025.acl-long.1028 2025

[6] [6]

Out of One, Many: Using Language Models to Simulate Human Samples , volume =

Argyle, Lisa and Busby, Ethan and Fulda, Nancy and Gubler, Joshua and Rytting, Christopher and Wingate, David , year =. Out of One, Many: Using Language Models to Simulate Human Samples , volume =. Political Analysis , doi =

work page

[7] [7]

, title =

Park, Joon Sung and O'Brien, Joseph and Cai, Carrie Jun and Morris, Meredith Ringel and Liang, Percy and Bernstein, Michael S. , title =. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , articleno =. 2023 , isbn =. doi:10.1145/3586183.3606763 , abstract =

work page arXiv 2023

[8] [8]

``My Answer is C '': First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

Wang, Xinpeng and Ma, Bolei and Hu, Chengzhi and Weber-Genzel, Leon and R. ``My Answer is C '': First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.441

doi:10.18653/v1/2024.findings-acl.441 2024