LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing

Donghyuk Shin; Jin Soo Han; Jongho Jeong; Sunwoong Kim

REVIEW 5 major objections 7 minor 1 cited by

LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that supplying a language model with a survey respondent's demographics and prior answers yields a generated persona whose responses track the real person's individual choices, and that this makes LLM-generated panels…

desk verdict Useful aggregate-level survey simulation, but the individual-level replication claim is not supported by the evidence. read the letter →

arxiv 2412.03162 v2 pith:3CTHIVIL submitted 2024-12-04 cs.CY

Sunwoong Kim , Jongho Jeong , Jin Soo Han , Donghyuk Shin This is my paper · ORCID

classification cs.CY

keywords LLM-Mirrorsurveypre-testingpersonagenerationindividual-levelresponsereplicationPLS-SEMGPT-4odistributionssimulation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Surveys are expensive to pilot, so the authors ask whether a large language model can stand in for human respondents during pre-testing. Their central claim is that when the model is given each respondent's demographics and prior answers, it produces responses that match the real respondents not just on average but at the level of individual choices, and that the same holds when those details are condensed into a generated 'LLM-Mirror' persona. If true, researchers could test questionnaire items and structural hypotheses before fielding a survey, saving the cost and time of repeated pilots. The evidence is drawn from two published PLS-SEM datasets—an ad-blocker attitude survey and a bank-customer loyalty survey—and compares path coefficients, distribution distances, and per-question agreement.

What carries the argument

The load-bearing object is the LLM-Mirror persona: a short user profile generated by feeding the model a respondent's demographic attributes and their prior answers to survey items for the explanatory latent variables, then using that persona as the prompt for generating answers to the remaining items. The persona condenses respondent-specific information into a reusable prompt, which is what distinguishes the method from generic baseline or demographic-only prompting. The evaluation machinery is PLS-SEM, a partial-least-squares structural equation model that estimates path coefficients between latent variables; the authors compare coefficients estimated from human responses with those from each prompting condition, and they supplement that comparison with Jensen-Shannon divergence, Wasserstein distance, and per-question agreement.

What would settle it

Re-run the same per-question evaluation with a baseline that always picks the most common human response for each item, or samples from the human marginal distribution; if that baseline matches or exceeds the reported 52–73 percent consistency, individual-level mirroring is not demonstrated. A complementary test is to compute per-respondent correlations between LLM-Mirror and human answer vectors; near-zero or weakly positive correlations would falsify the claim that the model reproduces individual decision patterns.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a persona-based prompt—constructed from a respondent's demographics together with their answers to the questions that define the explanatory latent variables—lets GPT-4o generate survey responses that track the actual respondent's answers and reproduce the path coefficients of the original PLS-SEM model. In Study 1 and in Case 1 of Study 2, the Omni prompt (full prior questions and answers) and the LLM-Mirror prompt (persona only) both yield coefficients that match the human estimates in sign, significance, and rough magnitude, whereas prompts without prior responses diverge. The paper also reports Jensen-Shannon divergences and Wasserstein distances that are much smaller for these two conditions, and per-question consistency rates between 52 and 73 percent. In the more complex Case 2 model, the approach still tracks most paths but misses some insignificant relationships, which the authors attribute to model access and suggest fine-tuning could improve.

Load-bearing premise

The load-bearing assumption is that agreement rates of 52 to 73 percent between LLM-Mirror and human answers, with no chance-level or majority-class baseline, demonstrate individual-level replication; if a trivial predictor matches those rates, the central claim does not follow.

Editorial extensions

If this is right

Survey pre-testing could be run on LLM-generated persona panels before human pilots, flagging weak items or non-significant paths early.
Because the Omni and LLM-Mirror prompts reproduce most path coefficients, researchers could use the approach to sanity-check which structural relationships are likely to survive data collection.
Demographics alone are not enough; the comparisons show that prior response information is what moves LLM outputs toward human answers, so persona construction should include substantive prior answers.
Even in the complex Case 2 model, most significant paths are recovered, suggesting the method scales to multi-mediator models with caveats.
The LLM-Mirror persona uses only a compact persona rather than full question-by-question history, making it a practical option when detailed prior responses are unavailable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 52–73 percent consistency rates are not benchmarked against a majority-class or chance predictor; until they are, 'individual-level replication' is an upper-bound claim.
PLS-SEM path coefficients are estimated from aggregate covariance structure, so close coefficient alignment can occur even if the LLM is not tracking each individual; per-respondent correlations across items would be a stricter test.
A natural next experiment is to use LLM-Mirror personas to generate a synthetic pilot, run the actual survey on a small human sample, and compare whether the pre-test would have caught the same design problems.
If the effect holds, the approach could be tested on underrepresented groups by conditioning personas on demographic strata, though representation errors in the LLM's prior would propagate into the mirror.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — read this paper on LLM-generated personas for survey pre-testing. The headline is overstated: the paper claims individual-level replication, but the evidence only supports distributional and structural alignment. Still, there is something real here. The authors extend earlier LLM-survey work from aggregate distribution matching to PLS-SEM path-coefficient replication and introduce a generated-persona condition (LLM-Mirror) that is practical for pre-testing. Study 1 and Case 1 show path coefficients that line up reasonably with human data, and the distributional metrics improve clearly when prior responses are included. The authors also report Case 2 failures honestly, which counts for something.

The soft spot is the central claim. The only individual-level evidence is the three-bin consistency analysis in Appendix B (Tables 13-15), with mean agreement rates of 52-73%. No chance baseline, no majority-class baseline, no per-individual correlation. With Likert data skewed to one end, always choosing the modal category could hit those numbers; the 3-bin grouping makes the task much easier than exact matching. Distributional metrics and path coefficients are aggregate quantities; they cannot show that the model predicted the same person's answer. On top of that, the Omni and LLM-Mirror prompts feed the respondent's own prior answers into the prompt that generates the same person's outcomes, so agreement partly reflects statistical coupling, not faithful mirroring. Case 2 makes the overclaim concrete: generated paths include a wrong sign for LIKE-to-LOY, a significant COMP-to-LOY where humans are insignificant, and an insignificant COMP-to-SAT where humans are significant. The authors acknowledge these, but they sit awkwardly with the abstract's claim that responses 'closely follow human responses at the individual level.'

For a pre-testing tool, the aggregate results may already be useful—if you want to know whether a questionnaire will produce expected structural relationships, this is a cheap screen. But the individual-level claim needs much better evidence: per-individual metrics (correlation or accuracy against a chance baseline), repeated sampling, and ideally a no-prior-answers ablation. This paper deserves a serious referee, not because the claims are proven, but because the question is important and the method is promising. The referee should ask for the missing baselines and a more careful abstract.

Referee Report

5 major / 7 minor

Summary. The paper introduces LLM-Mirror, a generated-persona approach for pre-testing surveys. Using two existing survey datasets (an ad-blocker study and a bank-customer study), the authors compare four prompting strategies: Baseline (survey context only), Demo (adds demographics), Omni (adds prior questions and answers), and LLM-Mirror (uses a generated persona). They evaluate alignment through Jensen-Shannon divergence, Wasserstein distance, PLS-SEM path coefficients, and coarse category agreement rates in Appendix B. The paper claims that LLMs provided with respondent-specific information can reproduce individual human responses and that LLM-Mirror responses closely follow human responses at the individual level, with implications for using LLMs to pre-test surveys and structural models.

Significance. If the individual-level replication claim were established, the paper would make a useful practical contribution to survey pre-testing and to the literature on LLMs as social-science agents. The authors are right to move beyond distributional comparisons, and they deserve credit for reporting Case 2 failures openly and for releasing code and data. However, the current evidence does not support the headline individual-level claim: the main metrics are aggregate/distributional, and the only individual-level analysis is a coarse agreement rate with no chance or majority-class baseline. The paper's value is therefore more modest than claimed, though the underlying idea and datasets may be salvageable with additional validation.

major comments (5)

[Appendix B, Tables 13-15] The central claim that LLM-Mirror 'closely follows human responses at the individual level' rests on coarse agreement rates of 52.56% to 73.01%. These rates are reported as means with no chance baseline, no majority-class baseline, and no per-individual correlation or exact-match rate. Since the 7-point responses are collapsed into three bins (1-3, 4, 5-7), a predictor that always chooses the modal bin could achieve substantial agreement without mirroring any individual. The paper must report exact 7-point agreement, per-individual correlation coefficients (e.g., rank correlation), and comparison against a trivial majority-class predictor. Without these, the individual-level claim is unsupported.
[Evaluation Metrics] Jensen-Shannon divergence, Wasserstein distance, and PLS-SEM path coefficients are aggregate or covariance-level quantities. They compare marginal distributions or structural relations, not whether the LLM's answer matches the same person's answer. Many joint distributions can share marginals and path coefficients while disagreeing at the individual level. The paper should either present individual-level metrics in the main text or explicitly restrict its conclusions to distributional and structural alignment, removing the 'individual-level' phrasing from the abstract.
[Methodology, Table 1 and Experimental Design] In Study 1 and Case 1, the Omni-prompt condition provides the respondent's own prior answers to the explanatory-variable items and then asks the LLM to predict that same respondent's outcome items. High agreement may partly reflect statistical coupling between the provided and predicted variables rather than faithful replication of an individual's decision process. A control condition that pairs prior answers from one respondent with outcomes from another, or a held-out evaluation, is needed to quantify this effect. The LLM-Mirror construction also needs clarification: the introduction says personas are based on demographics and prior responses, but Table 1 lists both as absent; if prior responses are used to generate the persona, the same coupling concern applies.
[Table 6, Case 2] The Case 2 results contain notable mismatches: LLM-Mirror and Omni produce a significant negative LIKE-to-LOY path where the human path is positive and significant, a significant COMP-to-LOY path where the human path is insignificant, and an insignificant COMP-to-SAT path where the human path is significant. These are not minor deviations; they reverse or change the qualitative conclusions of the structural model. The text acknowledges limitations, but the abstract and conclusion still claim that 'LLM-Mirror responses closely follow human responses at the individual level.' The claims need to be tempered to reflect that the approach works for some structures and fails for others, and the failure cases should be treated as boundary conditions.
[Experiment Settings] No information is given about the number of LLM generations, sampling temperature, top-p, or other decoding parameters. The PLS-SEM standard errors appear to come from bootstrapping within a single set of LLM-generated responses, which does not capture variability across LLM samples. The agreement rates in Appendix B could change substantially across random draws. The paper should report repeated sampling statistics (e.g., means and confidence intervals over multiple generations) and the exact decoding settings used.

minor comments (7)

[Table 2] The significance notation 'p < 0.5' is almost certainly a typo for 'p < 0.05'; please correct it.
[Methodology, Table 1] The relationship between the LLM-Mirror persona and the inputs used to construct it is unclear. Please specify precisely what information the generated persona contains and whether any of the outcome-related prior responses are encoded in the persona text.
[Appendix B] The phrase 'Consistent Analysis' is vague; the paper should define whether the reported percentages are averaged over items, respondents, or both, and should report the full distribution of agreement rates rather than only the mean.
[Page near Table 9] There is a typo: 'Jesen-Shannon divergence' should be 'Jensen-Shannon divergence'.
[Figure 3] The caption 'Fraction of Theoretical model' is unclear; it should likely read 'Theoretical model' or 'Fraction of the theoretical model tested'.
[Appendix D] The appendix heading says 'Damberg, Svenja, and Ringle (2023)' while the rest of the paper cites Damberg, Schwaiger, and Ringle (2022); please align the year and author order.
[Experiment Settings] Please report the LLM version, inference hyperparameters (temperature, top-p, max tokens), and number of independent generations per participant so that the results are reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical comparisons test conditional prompting, and no derivation reduces to its own inputs.

full rationale

The paper's central claim is an empirical one: that GPT-4o, when prompted with respondent demographics and/or prior survey answers or a persona derived from them, produces remaining survey responses that are close to the same individuals' actual responses. The evaluation uses distributional divergences, PLS-SEM path coefficients, and coarse category-level agreement rates. None of these quantities is defined in terms of the quantities it is used to establish. In the Omni and LLM-Mirror conditions the prompt does include the respondent's own prior answers to explanatory-variable items, so the generated outcome items are conditioned on correlated information from the same person; but the outcome items themselves are not in the prompt, and agreement is not an identity. The path coefficients are re-estimated from the LLM's outputs, not imported, and no parameter is fitted to the target data before the comparison. The paper cites prior work only for datasets and general LLM-simulation context, and no load-bearing claim rests on the authors' own prior theorems or uniqueness results. The absence of chance or majority-class baselines for Appendix B agreement rates is a statistical-rigor limitation, not a circularity of derivation. Accordingly, the derivation chain is not circular; concerns about baselines and prompt information leakage belong to an evidence-quality assessment rather than to circularity analysis.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The paper does not fit numeric parameters to data, but it relies on hand-chosen evaluation thresholds and undisclosed LLM decoding settings. The central assumptions are that PLS-SEM path coefficients can be read causally, that LLM responses reflect latent attitudes rather than prompt surface patterns, and that the original datasets are valid. The LLM-Mirror persona is an invented conceptual entity with no independent, externally checkable definition beyond the paper's own response-generation procedure.

free parameters (2)

LLM decoding temperature and sampling parameters = not reported
GPT-4o response generation depends on sampling settings, which are not stated; without them, single-response comparisons and the consistency rates are not reproducible.
coarse agreement category thresholds = 1-3, 4, 5-7
Discretization of 7-point Likert responses into disagreement, neutral, and agreement is chosen by hand and fully determines the consistency percentages in Appendix B.

assumptions (3)

domain assumption PLS-SEM is a causal analysis method and path coefficients reveal underlying mechanisms of human behavior.
Invoked in the Abstract and Methodology; PLS-SEM is generally a correlational technique, so the causal interpretation is an unsupported assumption.
domain assumption The LLM's answers to outcome items are generated from the respondent's latent attitudes rather than from surface patterns in the prompt.
The individual-level replication claim depends on the LLM internalizing the respondent's perspective from demographics and prior answers; the paper does not test this mechanism.
domain assumption The original human datasets are accessible, correctly transcribed, and representative of the populations described.
Replication depends on the Redondo and Aznar (2018) and Damberg et al. (2022) datasets, which the paper does not ship and one of which comes from a commercial research firm.

invented entities (1)

LLM-Mirror persona
purpose: A text persona generated from a respondent's demographics and prior answers, used as the prompt for predicting the respondent's remaining survey responses.
The persona generation procedure, prompt template, and model settings are not disclosed, so the entity has no falsifiable handle outside the paper; its success is measured only through the responses it generates.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing." pith.science (2026). https://pith.science/paper/3CTHIVIL

@misc{pith2026241203162,
  author       = {Pith},
  title        = {Pith review of: LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3CTHIVIL}},
  note         = {Machine review of arXiv:2412.03162}
}

read the original abstract

Surveys are widely used in social sciences to understand human behavior, but their implementation often involves iterative adjustments that demand significant effort and resources. To this end, researchers have increasingly turned to large language models (LLMs) to simulate human behavior. While existing studies have focused on distributional similarities, individual-level comparisons remain underexplored. Building upon prior work, we investigate whether providing LLMs with respondents' prior information can replicate both statistical distributions and individual decision-making patterns using Partial Least Squares Structural Equation Modeling (PLS-SEM), a well-established causal analysis method. We also introduce the concept of the LLM-Mirror, user personas generated by supplying respondent-specific information to the LLM. By comparing responses generated by the LLM-Mirror with actual individual survey responses, we assess its effectiveness in replicating individual-level outcomes. Our findings show that: (1) PLS-SEM analysis shows LLM-generated responses align with human responses, (2) LLMs, when provided with respondent-specific information, are capable of reproducing individual human responses, and (3) LLM-Mirror responses closely follow human responses at the individual level. These findings highlight the potential of LLMs as a complementary tool for pre-testing surveys and optimizing research design.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Exploring LLMs for Automated Generation and Adaptation of Questionnaires
cs.HC 2025-01 conditional novelty 5.0 of 10

LLM-generated survey questions were rated as clear and specific, while LLM-based pretesting improved some adapted questions but often made original questions wordier and less clear.

Reference graph

Works this paper leans on

18 extracted references · 6 canonical work pages · cited by 1 Pith paper

[3]

Harvard Business School Marketing Unit Working Paper, No

Using LLMs for Market Re- search. Harvard Business School Marketing Unit Working Paper, No. 23-062. Available at SSRN: doi.org/10.2139/ssrn.4395751. Chaudhuri A., Holbrook M. B

doi:10.2139/ssrn.4395751
[4]

Journal of Marketing , 65(2):81 -93

The Chain of Effects from Brand Trust and Brand Affect to Brand Performance: The Role of Brand Loyalty. Journal of Marketing , 65(2):81 -93. doi.org/10.1509/jmkg.65.2.81.18255. Chio S. Y.; Lee H.; Yoo Y

work page doi:10.1509/jmkg.65.2.81.18255
[8]

Marketing Science, 43(4):709-722

Frontiers: Can Large Language Models Capture Human Preferences?. Marketing Science, 43(4):709-722. doi.org/10.1287/mksc.2023.0306. Groves, R. M

arXiv 2023
[11]

arXiv:2301.07543

Large Language Models as Simulated Eco- nomic Agents: What Can We Learn from Homo Silicus?. arXiv:2301.07543. doi.org/10.48550/arXiv.2301.07543. Hulland, J.; Baumgartner, H.; Smith, K. M

doi:10.48550/arxiv.2301.07543
[14]

Telematics and Informatics , 35(6):1607 -1616

To use or not to use ad blockers? The roles of knowledge of ad blockers and attitude toward online ad- vertising. Telematics and Informatics , 35(6):1607 -1616. doi.org/10.1016/j.tele.2018.04.008. Tjuatja L.; Chen V.; Wu T.; Talwalkwar A.; Neubig G

work page doi:10.1016/j.tele.2018.04.008 2018
[15]

Transactions of the Association for Computational Linguistics, 12:1011-1026

Do LLMs Exhibit Human-like Response Biases? A Case Study in Sur- vey Design. Transactions of the Association for Computational Linguistics, 12:1011-1026. doi.org/10.1162/tacl_a_00685. Venkatesh V.; Thong J.Y.L.; Xu X

doi:10.1162/tacl_a_00685
[17]

The Wharton School Research Paper

How Much Should We Trust Large Language Model-Based Measures for Accounting and Finance Research?. The Wharton School Research Paper . Available at SSRN: doi.org/10.2139/ssrn.4983334. Zhang Y.; Mao S.; Ge T.; Wang X.; A de Winter.; Xia Y.; We W.; Song T.; Lan M.; Wei F

work page doi:10.2139/ssrn.4983334
[18]

arXiv:2404.01230

LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models. arXiv:2404.01230. doi.org/10.48550/arXiv.2404.01230. Appendices Appendix A Baseline prompt Demo prompt LLM- Mirror Omni prompt Q1 0.7748 0.8311 0.2163 0.2667 Q2 0.7985 0.5970 0.2326 0.2741 Q3 1.1748 1.1259 0.3985 0.3719 Q4 0.7881 0.7778 0.4607 0.4770 Q5 0.8711 1.1081 0.5585 0...

arXiv doi:10.48550/arxiv.2404.01230 2018

Show all 18 references

[178]

doi.org/10.2307/41410412. Yoo M

doi
[1991]

IEEE Transactions on Information Theory , 37(1):145 -151

Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory , 37(1):145 -151. doi.org/10.1109/18.61115. Redondo I., Aznar G

doi
[2001]

MIS Quarterly , 25(3):351-370

Understanding Information Systems Con- tinuance: An Expectation -Confirmation Model. MIS Quarterly , 25(3):351-370. doi.org/10.2307/3250921. Brand J.; Israeli A.; Ngwe D

doi
[2006]

Public Opinion Quarterly , 70(5): 646 -675

Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly , 70(5): 646 -675. doi.org/10.1093/poq/nfl033. Hamari J.; Shernoff D. J.; Rowe E.; Coller B.; Asbell -Clarke J.; Edwards T

doi
[2010]

MIS Quar- terly, 34(4):855-870

The Impact of Information Tech- nology and Transactive Memory Systems on Knowledge Sharing, Application, and Team Performance: A Field Study. MIS Quar- terly, 34(4):855-870. doi.org/10.2307/25750708. Damberg S.; Schwaiger M..; Ringle C. M

doi
[2016]

Computers in Human Behavior , 54:170 -179

Challenging games help students learn: An em- pirical study on engagement, flow and immersion in game -based learning. Computers in Human Behavior , 54:170 -179. doi.org/10.1016/j.chb.2015.07.045. Horton, J. J

2015 doi
[2018]

Journal of the Academy of Marketing Sci- ence, 46: 92-108

Marketing sur- vey research best practices: evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Sci- ence, 46: 92-108. doi.org/10.1007/s11747-017-0532-y. Lin J

doi
[2022]

Journal of Marketing Analytics, 10:3-18

What’s important for relationship management? The mediating roles of relational trust and satisfaction for loyalty of cooperative banks’ customers. Journal of Marketing Analytics, 10:3-18. doi.org/10.1057/s41270- 021-00147-2. Deng Y.; Mirrokni V.; Leme R. P.; Zhang H.; Zuo S

doi
[2023]

Political Analysis , 31(3):337 -351

Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis , 31(3):337 -351. doi:10.1017/pan.2023.2. Bhattacherjee A

2023 doi
[2024]

arXiv:2410.19599

Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina. arXiv:2410.19599. doi.org/10.48550/arXiv.2410.19599. Goli A., Singh A

doi arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [3]

Harvard Business School Marketing Unit Working Paper, No

Using LLMs for Market Re- search. Harvard Business School Marketing Unit Working Paper, No. 23-062. Available at SSRN: doi.org/10.2139/ssrn.4395751. Chaudhuri A., Holbrook M. B

doi:10.2139/ssrn.4395751

[2] [4]

Journal of Marketing , 65(2):81 -93

The Chain of Effects from Brand Trust and Brand Affect to Brand Performance: The Role of Brand Loyalty. Journal of Marketing , 65(2):81 -93. doi.org/10.1509/jmkg.65.2.81.18255. Chio S. Y.; Lee H.; Yoo Y

work page doi:10.1509/jmkg.65.2.81.18255

[3] [8]

Marketing Science, 43(4):709-722

Frontiers: Can Large Language Models Capture Human Preferences?. Marketing Science, 43(4):709-722. doi.org/10.1287/mksc.2023.0306. Groves, R. M

arXiv 2023

[4] [11]

arXiv:2301.07543

Large Language Models as Simulated Eco- nomic Agents: What Can We Learn from Homo Silicus?. arXiv:2301.07543. doi.org/10.48550/arXiv.2301.07543. Hulland, J.; Baumgartner, H.; Smith, K. M

doi:10.48550/arxiv.2301.07543

[5] [14]

Telematics and Informatics , 35(6):1607 -1616

To use or not to use ad blockers? The roles of knowledge of ad blockers and attitude toward online ad- vertising. Telematics and Informatics , 35(6):1607 -1616. doi.org/10.1016/j.tele.2018.04.008. Tjuatja L.; Chen V.; Wu T.; Talwalkwar A.; Neubig G

work page doi:10.1016/j.tele.2018.04.008 2018

[6] [15]

Transactions of the Association for Computational Linguistics, 12:1011-1026

Do LLMs Exhibit Human-like Response Biases? A Case Study in Sur- vey Design. Transactions of the Association for Computational Linguistics, 12:1011-1026. doi.org/10.1162/tacl_a_00685. Venkatesh V.; Thong J.Y.L.; Xu X

doi:10.1162/tacl_a_00685

[7] [17]

The Wharton School Research Paper

How Much Should We Trust Large Language Model-Based Measures for Accounting and Finance Research?. The Wharton School Research Paper . Available at SSRN: doi.org/10.2139/ssrn.4983334. Zhang Y.; Mao S.; Ge T.; Wang X.; A de Winter.; Xia Y.; We W.; Song T.; Lan M.; Wei F

work page doi:10.2139/ssrn.4983334

[8] [18]

arXiv:2404.01230

LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models. arXiv:2404.01230. doi.org/10.48550/arXiv.2404.01230. Appendices Appendix A Baseline prompt Demo prompt LLM- Mirror Omni prompt Q1 0.7748 0.8311 0.2163 0.2667 Q2 0.7985 0.5970 0.2326 0.2741 Q3 1.1748 1.1259 0.3985 0.3719 Q4 0.7881 0.7778 0.4607 0.4770 Q5 0.8711 1.1081 0.5585 0...

arXiv doi:10.48550/arxiv.2404.01230 2018