Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events

Chen Gao; Chuan Ai; Kaisheng Lai; Lingnan He; Mengzhu Liu; Quanjun Yin; Xinghong Li; Xin Lu; Yingfeng Chen; Yong Li

REVIEW 3 major objections 5 minor 37 references

Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Psychology-driven LLM agents can predict social-media panic during sudden disasters by simulating users' psychological chains, reaching 86% accuracy and 0.87 AUC and beating baselines by 12.6% to 21.7%.

desk verdict A promising dataset and a psychology-grounded agent design, but the reported performance gain is not supported because the comparison sets differ and the evaluation loop is contaminated by the same annotations used to steer generation. read the letter →

arxiv 2505.16455 v1 pith:PTH4BFIN submitted 2025-05-22 cs.AI cs.CY

Mengzhu Liu , Zhengqiu Zhu , Chuan Ai , Chen Gao , Xinghong Li , Lingnan He , Kaisheng Lai , Yingfeng Chen

show 3 more authors

Xin Lu Yong Li Quanjun Yin

This is my paper · ORCID

classification cs.AIcs.CY

keywords panicemotionpredictionLLMagentsriskperceptionarousaltheorysocialmediadisastereventsexplainableAIchain-of-thoughtpromptinghuman-LLMcollaborativeannotation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that panic on social media during sudden disasters is better predicted by simulating the psychological chain that produces it than by statistically fitting pre-disaster features. The authors build PsychoAgent, an LLM role-playing framework that feeds a user's pre-disaster posts, disaster context, and psychological priors into a chain-of-thought agent, which generates the tweets that user would most likely post after the event. A fine-tuned BERT discriminator labels those generated tweets for panic, and a user is flagged if any generated tweet carries panic. On a new Hurricane Sandy dataset (COPE), the approach reaches 86% accuracy and 0.87 AUC, beating feature-engineered, deep-learning, and direct-LLM baselines by 12.6% to 21.7%. The authors frame the result as a shift from opaque data-driven fitting to transparent role-based simulation with mechanistic interpretation.

What carries the argument

The central machinery is the CoT-driven LLM role-playing agent, grounded in the Psychological Preparedness for Disaster Threat Scale (PPDTS) and emotion arousal theory. The agent is initialized with psychological knowledge, disaster data, and user profiles (Big Five traits, sentiment trends, topic focus, and tone), then walked through four stages: disaster perception, risk perception (scoring PPDTS items 1–4), panic arousal (computing a panic probability from four factors), and posting response (generating predicted tweets). A multi-expert consistency check — psychological, linguistic, factual, and emotional — filters the generated text, and a fine-tuned BERT discriminator converts it into a panic signal. This chain of stages is what carries the argument: it replaces direct classification with a simulated psychological pathway, and the ablation study shows that removing the risk-sensing and emotion-arousal stages collapses accuracy from 0.86 to 0.66.

What would settle it

Have independent human annotators label a random sample of the tweets generated by PsychoAgent and compare their panic labels with the fine-tuned BERT discriminator's outputs; if agreement is low, the reported accuracy measures the discriminator's bias rather than the agent's predictive fidelity. Alternatively, rerun the full pipeline on a second disaster event with human-annotated panic labels and check whether the 12.6–21.7% improvement over baselines persists.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that panic emotion prediction should be treated as a role-based simulation of the psychological chain — disaster perception, risk perception, panic arousal, and posting behavior — rather than a classification of observable features. The evidence is that PsychoAgent, driven by chain-of-thought prompts over a psychological preparedness scale, produces generated texts whose panic labels, assigned by a self-trained BERT model, align with human annotations at 86% accuracy and 0.87 AUC, outperforming all baselines. The authors further claim that this constitutes a paradigm shift: instead of fitting correlations between pre-disaster features and post-disaster emotions, the model reconstructs the cognitive-emotional mechanism, yielding predictions that can be explained in psychological terms.

Load-bearing premise

The load-bearing assumption is that the BERT discriminator, fine-tuned on 1,065 human-LLM labeled entries, labels panic in LLM-generated tweets the same way it would label authentic user posts; if the generated texts shift in style, the reported accuracy may reflect agreement with the annotation style rather than true panic.

Editorial extensions

If this is right

If the central claim holds, panic prediction during emergencies can run without waiting for post-disaster behavior: the model generates that behavior from pre-disaster profiles and disaster context.
The method shows that a 14-billion-parameter LLM scaffolded by psychological reasoning can outperform much larger models used for direct prediction, which matters for deployment under computing constraints.
Every prediction comes with a traceable chain — which risk-perception factors, personality traits, and disaster features drove the panic probability — enabling crisis managers to see why a user was flagged.
The framework is designed to transfer to other sudden disasters: the same psychological chain and prompts apply, with only the disaster-specific data swapped in.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension would be to run PsychoAgent on a second disaster event (for example, an earthquake or flood) with independent human panic labels; if the 12.6–21.7% gains persist, the psychological-chain mechanism is event-general rather than a Hurricane Sandy artifact.
The reported accuracy depends on the BERT discriminator's ability to label LLM-generated text. Having independent human annotators label a sample of the generated tweets and comparing against the discriminator would settle whether the score reflects true panic detection or stylistic consistency with the training distribution.
If role-based simulation is the actual source of the gain, then substituting a different risk-perception instrument for PPDTS should shift predictions in theoretically predictable directions; testing that would separate the psychological mechanism from prompt engineering effects.
The authors call the work a paradigm shift; the evidence shown supports a competitive, interpretable method, but demonstrating superiority over data-driven approaches beyond this one dataset would be needed to justify the stronger framing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: COPE is a real contribution and the PPDTS-driven role-play idea is worth a look, but the paper's central claim—a 12.6–21.7% improvement over baselines—is not supported by the evidence as presented. I'd still send it to review, but the authors need to fix the comparison and the evaluation loop.

What's new: COPE is a dual-phase panic-annotated dataset from Hurricane Sandy tweets, with a documented human-LLM annotation pipeline and a one-veto user-level label. The idea of grounding an LLM role-playing agent in a validated psychological scale (PPDTS) and adding a multi-expert verification step is a reasonable combination I haven't seen in panic-prediction work. The paper is clearly written, the appendices are thorough, and the code is promised publicly.

The soft spots are serious. First, Table 1 evaluates PsychoAgent on 1,711 users after questionnaire screening, while baselines use 1,813. Section 4.1 says all models use the same input features, data split, and evaluation metrics, but the table contradicts that. There's no analysis of the 102 excluded users' panic rate or features. If those users are the hard cases, the gain could be pure selection. The ablation table has the same problem, with four different support sizes.

Second, the final label comes from a BERT discriminator fine-tuned on the same COPE panic annotations that condition the LLM's text generation. The Stage 4 prompt explicitly tells the LLM to write more panic-laden text when the predicted probability is above 51% and calmer text below 49%. So the classifier is likely verifying prompt adherence, not predicting user panic. The Limitations section admits stylistic divergence but doesn't address the circularity.

The 'paradigm shift' framing is overblown; this is an incremental application of generative agents to a crisis domain, not a new scientific law. The scalability study is interesting but inherits the same evaluation flaw.

If I were the editor, I'd send this to peer review, because the dataset and the psychological modeling idea merit serious scrutiny. The authors should be required to rerun all baselines on the exact 1,711-user subset and to validate the BERT discriminator on held-out human-validated generated text. With that revision, the paper could be credible. As it stands, the headline improvement is not.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes PsychoAgent, a psychology-driven LLM agent framework for predicting whether individual Twitter users will express panic after a sudden disaster. The authors build the COPE dataset from Hurricane Sandy tweets, annotate post-disaster texts through a human-LLM collaboration, and fine-tune a BERT discriminator on those annotations. The agent is constructed from pre-disaster user features (Big Five personality, sentiment trends, LDA topics, linguistic tone), disaster metadata, and PPDTS-based risk assessment; it generates a synthetic post-disaster tweet, which is vetted by a multi-expert LLM evaluator and then classified by the BERT discriminator to produce a user-level panic label. The paper reports 86% accuracy and 0.87 AUC, outperforming ten baselines by 12.6–21.7%, and presents ablation, scalability, and case studies.

Significance. The strengths of the paper are the COPE dataset (a public, fine-grained panic-annotation resource), the public implementation at an anonymous repository, and the attempt to operationalize psychological constructs (risk perception, emotion arousal) inside a generative agent. If the evaluation were valid, the framework would be a useful step toward interpretable crisis emotion prediction. However, the significance is currently conditional: the headline performance claims rest on an unequal evaluation set and on a label-generation loop whose final BERT classification is not independent of the panic probability the agent computes. These issues must be resolved before the reported gains can be attributed to the framework's psychological modeling.

major comments (3)

[§4.1, Table 1 and Table 2] The claim in §4.1 that all models use the same input features, data split, and evaluation metrics is contradicted by the support counts in Table 1. PsychoAgent is evaluated on 1,711 users after "Stage 4 questionnaire validity screening," while SVM, LR, RF, XGBoost, Bi-LSTM, Transformer, PredNet, ChatGPT-4o, and THOR-ISA are evaluated on 1,813 users and DeepSeek-v3 on 1,627 users. The ablation rows in Table 2 use supports of 1,813, 1,700, and 1,768. If the 102 users excluded from the PsychoAgent evaluation or the other differing subsets are not random, the reported 12.6–21.7% improvement can be a selection artifact. Please rerun all models on a common evaluation set and report label distributions and feature statistics for the excluded users.
[§3.3, Appendix A.4.4, Appendix A.2.4] The final label is not an independent measurement. The Panic Arousal Stage produces a panic probability P_panic; the Stage 4 generation prompt instructs the agent to write panic-flavored text when P_panic > 51% and calm text when P_panic < 49%; the fine-tuned BERT discriminator, trained on the same COPE panic annotations, then converts that generated text into the reported label. The BERT classifier is therefore largely a transducer of the agent's own probability estimate, and its training labels are the same labels the framework was designed to reproduce. The reported 86% accuracy and 0.87 AUC thus measure consistency with the annotation style encoded in the prompts and the discriminator, not independent predictive accuracy. Please evaluate against held-out human labels on real post-disaster posts, or use an independent labeler for the generated texts.
[§4.1, §4.2, Eq. (2)] Even setting aside the support mismatch, the baseline comparison is not on equal footing. The feature-engineered ML and DL baselines are described as using user features (Big Five, sentiment, linguistic, topic), while PsychoAgent additionally uses post-disaster physical features P_{t+1} from Eq. (2) and the hurricane data from the disaster perception stage. The paper should either include the same physical features in all baselines or explicitly quantify their contribution through an ablation.

minor comments (5)

[§3.1, Figure 2] The heading "Overview of PsycoAgent" misspells the framework name, and the same typo appears in Figure 2; please correct the spelling throughout.
[Table 1, note (b)] The table note should state how many users were removed at each screening step and report the panic/non-panic distribution before and after screening; the current description ("invalid questionnaires containing fewer than 18 responses") is insufficient to assess selection bias.
[Appendix A.4.4, Stage 4 prompt] The generation prompt specifies behavior for panic probability >51% and <49%, but leaves the interval [49%, 51%] undefined; please clarify what the agent should do in that range.
[Appendix A.3.1] The personality model is validated only by consistency on 50% splits, which does not establish validity against an external ground truth; please replace this with a reported benchmark accuracy or a proper reliability measure.
[Figure 1, Figures 5 and 6] Figure 1 mixes full-width and half-width characters (e.g., "Event characteristics（Physical domain）"), and Figures 5 and 6 contain awkward repeated phrasing such as "The detection of Panic Emotion in prediction Text"; proofreading is needed.

Circularity Check

2 steps flagged · score 8.0 of 10

PsychoAgent's headline accuracy is a closed loop: the posting prompt constructs tweets from the model's own panic probability, a MoE check forces that same probability as the target, and the BERT discriminator converts the self-generated text back into the predicted label.

self definitional [Appendix A.6, Stage 4: Posting Response Stage (Predictive Text Generation); Section 3.3, Eq. (11)]
"Consider the user’s panic probability panic probability%; 5. If panic probability >51%, tweets should directly convey more panic, fear and anxiety, and tweets should include more emotional amplifiers, or more EMPHATIC capitalized words, or more repeated punctuation marks, or sensory details; 6. If panic probability <49%, tweets should reflect more calmness and rationality, and tweets should show more composed language."

The tweet that is later classified by the fine-tuned BERT discriminator is not an independent behavioral realization: it is generated from the model's own P_panic under an explicit threshold rule. Section 3.1 then uses the BERT output with a one-vote veto to produce the user-level panic label, and Table 1 reports this as a prediction. Because the text is constructed to encode P_panic, the BERT label is a deterministic function of P_panic up to classifier noise. The 86% accuracy and 0.87 AUC therefore measure the internal consistency of the LLM prompt with the BERT annotator, not an out-of-sample prediction of a user's panic.
self definitional [Appendix A.6, Stage 4: Posting Response Stage (Multi-Expert Evaluation); Section 3.3, Eq. (12)]
"Panic Probability Alignment: Use the user’s panic probability value panic probability% [Response Format]: Expert: YES/NO (reason) LLM answer: Psychological: YES ... Panic: YES (The tweet aligns with the user’s panic probability value of 55%, showing neutral concern without panic, which is appropriate for the given probability range.)"

The MoE system is presented as an independent validation of the generated text, but one of its four experts is explicitly instructed to check that the tweet matches the very P_panic that was used to generate it. Since the generation prompt already hard-codes panic probability above 51% as panic text, the expert approval cannot fail except by chance. Only texts passing this self-consistency filter are passed to the BERT discriminator, so the final prediction is a closed loop: P_panic to panic-flavored text to BERT panic label to user label.

full rationale

PsychoAgent does not rest on self-citation: it cites external psychology and ML works, and its methodological novelty is independent of any author-specific prior result. The circularity is internal to the evaluation loop. In the posting-response stage, the agent is told to generate panic-laden text when its computed P_panic exceeds 51% and calm text below 49%; the same P_panic is then used as a target in the Panic Probability Alignment check by the MoE evaluator, and only texts that pass are fed to the fine-tuned BERT discriminator. The final user-level label is therefore a function of the model's own P_panic, not an independent observation. The reported 86% accuracy and 0.87 AUC (Table 1) quantify the consistency of the prompt-to-text-to-BERT pipeline with the COPE annotation style, rather than a genuine out-of-sample prediction of panic. Separately, the paper's assertion that all models use the same data split is undermined by unequal supports in Tables 1 and 2 (PsychoAgent 1,711 vs. baselines 1,813; ablation supports 1,700 to 1,813); this is a comparison-validity problem, not circularity, and would need to be fixed regardless of the loop.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework contributes a prompt-and-evaluation scaffold rather than a derivation. The key premises are the psychological validity of PPDTS and LLM role-play, plus the validity of hand-chosen thresholds and screening rules. No new physical or fictitious entities are introduced; the latent panic probability is a model output, not a newly postulated entity.

free parameters (5)

Panic probability generation threshold (51%/49%) = 51% panic / 49% calm
Hand-chosen rule in Appendix A.4.4 that directly controls whether generated tweets express panic, and hence the final BERT-based label.
One-vote veto rule = 1 panic text per user
Hand-chosen aggregation rule mapping text-level panic to user-level panic in Sections 2.2 and 3.1.
PPDTS factor weights = 25% each for awareness, coping, uncertainty, novelty
Stage 3 prompt in Appendix A.6 assigns equal weights without empirical justification.
Stage 4 validity screening criterion = exclude users with fewer than 18 returned questionnaire items
Post-hoc screening in Table 1 note changes the evaluation set from 1,813 to 1,711 users.
LDA topic count and keyword count = 25 topics, 10 keywords per topic
Hand-configured in Appendix A.3.2; part of feature extraction but not central to the psychological claim.

assumptions (5)

domain assumption Panic emotion formation follows the physical-to-information-to-cognitive-to-social pathway shown in Figure 1, and PPDTS scores reliably measure risk perception.
The entire agent design rests on this psychological pathway; PPDTS is adopted from McLennan et al. without independent validation in this social media context.
domain assumption Big Five traits from bert-base-personality and sentiment/topic/tone features extracted from pre-disaster posts are stable predictors of post-disaster panic.
Section 3.2 relies on this; the consistency check in Table 8 reports intra-user agreement, not accuracy or predictive validity.
domain assumption LLM role-play faithfully simulates an individual user's psychological chain from pre-disaster data.
Equations (8)-(11) depend on this; Section 7 Limitations acknowledges hallucinations and stylistic divergence.
domain assumption User-level panic can be inferred from at least one panic-labeled text (one-veto rule).
Sections 2.2 and 3.1 treat text-level annotations as sufficient for user-level ground truth.
domain assumption The human-LLM collaborative labels in COPE are correct enough to serve as ground truth for both training and evaluation.
Section 2.2 describes three rounds of cross-validation but reports no inter-annotator agreement statistics.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events." pith.science (2026). https://pith.science/paper/PTH4BFIN

@misc{pith2026250516455,
  author       = {Pith},
  title        = {Pith review of: Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTH4BFIN}},
  note         = {Machine review of arXiv:2505.16455}
}

read the original abstract

During sudden disaster events, accurately predicting public panic sentiment on social media is crucial for proactive governance and crisis management. Current efforts on this problem face three main challenges: lack of finely annotated data hinders emotion prediction studies, unmodeled risk perception causes prediction inaccuracies, and insufficient interpretability of panic formation mechanisms. We address these issues by proposing a Psychology-driven generative Agent framework (PsychoAgent) for explainable panic prediction based on emotion arousal theory. Specifically, we first construct a fine-grained open panic emotion dataset (namely COPE) via human-large language models (LLMs) collaboration to mitigate semantic bias. Then, we develop a framework integrating cross-domain heterogeneous data grounded in psychological mechanisms to model risk perception and cognitive differences in emotion generation. To enhance interpretability, we design an LLM-based role-playing agent that simulates individual psychological chains through dedicatedly designed prompts. Experimental results on our annotated dataset show that PsychoAgent improves panic emotion prediction performance by 12.6% to 21.7% compared to baseline models. Furthermore, the explainability and generalization of our approach is validated. Crucially, this represents a paradigm shift from opaque "data-driven fitting" to transparent "role-based simulation with mechanistic interpretation" for panic emotion prediction during emergencies. Our implementation is publicly available at: https://anonymous.4open.science/r/PsychoAgent-19DD.

Figures

Figures reproduced from arXiv: 2505.16455 by the authors.

**Figure 2.** Overview of the proposed panic emotion prediction framework——PsychoAgent. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Detailed stages for building LLM-based agent driven by CoT in the cognitive domain module. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Crowdsourced validation protocol for panic [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: Non-panic User Prediction Case. User Personalized Profile User id 781886935 Panic Emotion Label Panic Users Event Attitude Emotional Trend Topics of Interest Technology and Innovation, Sports and Entertainment, Society and News [0, 0, -1, 0, 0, 0, 1, 1, 1, 0, 0] indiff…

**Figure 6.** Figure 6: Panic User Prediction Case. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 33 canonical work pages

[1]

Data Filtering: Removed malformed, highly re- dundant (cosine similarity >0.85), or invalid entries (posts with <5 meaningful tokens)

work page
[2]

Irrelevant Content Removal: Leveraged human- LLM collaboration to exclude non-Sandy- related texts from the keyword-crawled public dataset(similar to Section 3.2)

work page
[3]

User Selection: Retained users who posted in both phases, with ≥10 pre-disaster tweets

work page
[4]

arXiv preprint arXiv:2005.10200

Bertweet: A pre-trained language model for english tweets. arXiv preprint arXiv:2005.10200. World Health Organization and 1 others. 2020. Man- aging the covid-19 infodemic: promoting healthy behaviours and mitigating the harm from misinforma- tion and disinformation. joint statement by who, un, unicef, undp. UNESCO, UNAIDS, ITU, UN Global Pulse, and IFRC....

arXiv 2005
[5]

arXiv preprint arXiv:2411.10109

Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109. D Pierson. 2011. Japan radiation fears spark panic salt- buying in china. Los Angeles Times March, 18:2011. Yiting Ran, Xintao Wang, Rui Xu, Xinfeng Yuan, Ji- aqing Liang, Deqing Yang, and Yanghua Xiao. 2024. Capturing minds, not just words: Enhancing role- playing language mod...

arXiv 2011
[6]

LLM answer: Stay safe everyone! The hurricane is getting stronger—winds up to 155 km/h now

If panic probability <49%, tweets should reflect more calmness and rationality, and tweets should show more composed language. LLM answer: Stay safe everyone! The hurricane is getting stronger—winds up to 155 km/h now. I’m prepping supplies and staying informed. Let’s all follow safety guidelines and look out for each other. #HurricaneAlert #StayPrepared ...

work page
[9]

RT @xxx:

Text Sanitization: Stripped URLs, retweets ("RT @xxx:"), and non-alphanumeric characters; Prompt for LLM-assisted annotation (Hurricane Related) Question prompt: You are a helpful assistant that identifies text rele- vance to Hurricane Sandy using specific keywords and related terms. Evaluate the following text for relevance to Hurricane Sandy. Focus only...

work page
[10]

A.2.2 LLM-Assisted Annotation Prompts The ChatGPT-4o assisted annotation prompts used in our dataset labeling process are shown as fol- lows

Temporal Stratification: Split the dataset into pre-disaster and peri-disaster subsets based on timestamps. A.2.2 LLM-Assisted Annotation Prompts The ChatGPT-4o assisted annotation prompts used in our dataset labeling process are shown as fol- lows. Since LLMs are trained on extensive cor- pus data, they inherently contain general knowl- edge about Hurric...

work page 2023

Show all 37 references

[11]

EmotionBench Multi- scenario stories Emotion assessment Contextual GPT-3.5-Turbo Partial (Context) Binary (Mou et al.,
[12]

/ News / Cross-doc / / Ekman-6 (Li et al., 2020) / Text dialogues / / / / Ekman-6 /4-class (Gao et al., 2024) / News / / RoBERTa+ChatGPT Partial (Explanation- Assisted) 8-class (Houlihan et al.,

2020
[13]

Split or Steal

Game data "Split or Steal" Intuitive psychology Preferences / Full-chain 20-class (Alsaedi et al.,
[15]

Psychological Principles: psychology
[16]

Hurricane monitoring data (Markdown): hurricane table
[17]

Directly output the final answer

User Profile (JSON): user info) Please always: 1. Directly output the final answer
[18]

Disable any thought process
[19]

LLM answer: Data understood

Use plain text format. LLM answer: Data understood. Stage 2: Risk Perception Stage Question prompt: Answer the following questions. For each question below, follow this reasoning chain:
[20]

Question Comprehension
[21]

Synthesis Questions:questions Response Requirements:
[22]

Q[number]: [score](reason)

Response Format: "Q[number]: [score](reason)" 2. Use 1-4 rating scale LLM answer:
[23]

Q1: 3 (User has interests in natural disasters and weather, indicating familiarity with preparedness materials, but emotional stability is not stable, suggesting some uncertainty.)
[24]

Q2: 2 (User’s conscientiousness is above baseline, suggesting some preparedness knowledge, but agreeableness is below mean, indicating potential skepticism about adequacy.)
[25]

Stage 3: Panic Arousal Stage Question prompt: Based on the previous stages, silently analyze these panic factors and assign a score (1-5) to each:

Q3: x . . . . . . Stage 3: Panic Arousal Stage Question prompt: Based on the previous stages, silently analyze these panic factors and assign a score (1-5) to each:
[26]

Coping Efficacy and Sense of Control
[27]

Calculate panic probability

Novelty of Risk Each factor contributes 25% weight. Calculate panic probability. Response Format: factors: [1-5]/5 (reason) [panic probability (e.g., 50%)] LLM answer: Awareness: 4/5 (User has interests in natural disasters and weather, indicating deep awareness of hurricane d...
[29]

[Tweet text with #hashtags]

[Response Format]: "[Tweet text with #hashtags]"
[30]

Consider the user’s panic probability panic probability%
[31]

If panic probability >51%, tweets should directly convey more panic, fear and anxiety, and tweets should include more emotional amplifiers, or more EMPHATIC capitalized words, or more repeated punctuation marks, or sensory details
[33]

Generate EXACTLY 1 possible tweets
[34]

[Tweet text with #hashtags]

Use exactly this format: "[Tweet text with #hashtags]"
[35]

Psychological Validation: Check if the tweet aligns with the user’s psychological profile
[36]

Linguistic Validation: Verify if the tweet’s language style is consistent with the user’s historical style
[37]

Factual Validation: Confirm if the tweet is relevant to Hurricane Sandy and factually accurate
[38]

Staying cautious but hopeful

Panic Probability Alignment: Use the user’s panic probability value panic probability% [Response Format]: Expert: YES/NO (reason) LLM answer: Psychological: YES (The tweet aligns with the user’s psychological profile, as the user has an interest in natural disasters and a cons...
[2020]

Natural Hazards, 101(1):297–307

Conceptualising and measuring psychological preparedness for disaster: The psychological pre- paredness for disaster threat scale. Natural Hazards, 101(1):297–307. Sandra Mitrovic and Vani Kanjirangat. 2022. Enhanc- ing bert performance with contextual valence shifters for pan...

2022
[2022]

text"]. Describe this user’s overall tone of voice on the social media with three words. Only output in the exact format: xxx, xxx, xxx

/ Social media / / / / 5-class Ours Disaster Emergencies Risk perception + Emotional arousal Multi- domain fusion LLM+MoE Full-chain Panic- specific Table 5: Comparative summary of emotion prediction research status. Dedicated dataset (whether creating task- specific data), Sc...
[2023]

arXiv preprint arXiv:2308.09597

Chatharuhi: Reviving anime character in reality via large language model. arXiv preprint arXiv:2308.09597. Dayu Li, Yang Li, and Suge Wang. 2020. Interactive double states emotion cell model for textual dia- logue emotion prediction. Knowledge-Based Sys- tems, 189:105084. Aixi...

2020 arXiv
[2024]

arXiv preprint arXiv:2412.19437

Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. William Lotter, Gabriel Kreiman, and David Cox. 2016. Deep predictive coding networks for video predic- tion and unsupervised learning. arXiv preprint arXiv:1605.08104. Jim McLennan, Mathew D Marques, and Danielle Every

2016 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Data Filtering: Removed malformed, highly re- dundant (cosine similarity >0.85), or invalid entries (posts with <5 meaningful tokens)

work page

[2] [2]

Irrelevant Content Removal: Leveraged human- LLM collaboration to exclude non-Sandy- related texts from the keyword-crawled public dataset(similar to Section 3.2)

work page

[3] [3]

User Selection: Retained users who posted in both phases, with ≥10 pre-disaster tweets

work page

[4] [4]

arXiv preprint arXiv:2005.10200

Bertweet: A pre-trained language model for english tweets. arXiv preprint arXiv:2005.10200. World Health Organization and 1 others. 2020. Man- aging the covid-19 infodemic: promoting healthy behaviours and mitigating the harm from misinforma- tion and disinformation. joint statement by who, un, unicef, undp. UNESCO, UNAIDS, ITU, UN Global Pulse, and IFRC....

arXiv 2005

[5] [5]

arXiv preprint arXiv:2411.10109

Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109. D Pierson. 2011. Japan radiation fears spark panic salt- buying in china. Los Angeles Times March, 18:2011. Yiting Ran, Xintao Wang, Rui Xu, Xinfeng Yuan, Ji- aqing Liang, Deqing Yang, and Yanghua Xiao. 2024. Capturing minds, not just words: Enhancing role- playing language mod...

arXiv 2011

[6] [6]

LLM answer: Stay safe everyone! The hurricane is getting stronger—winds up to 155 km/h now

If panic probability <49%, tweets should reflect more calmness and rationality, and tweets should show more composed language. LLM answer: Stay safe everyone! The hurricane is getting stronger—winds up to 155 km/h now. I’m prepping supplies and staying informed. Let’s all follow safety guidelines and look out for each other. #HurricaneAlert #StayPrepared ...

work page

[7] [9]

RT @xxx:

Text Sanitization: Stripped URLs, retweets ("RT @xxx:"), and non-alphanumeric characters; Prompt for LLM-assisted annotation (Hurricane Related) Question prompt: You are a helpful assistant that identifies text rele- vance to Hurricane Sandy using specific keywords and related terms. Evaluate the following text for relevance to Hurricane Sandy. Focus only...

work page

[8] [10]

A.2.2 LLM-Assisted Annotation Prompts The ChatGPT-4o assisted annotation prompts used in our dataset labeling process are shown as fol- lows

Temporal Stratification: Split the dataset into pre-disaster and peri-disaster subsets based on timestamps. A.2.2 LLM-Assisted Annotation Prompts The ChatGPT-4o assisted annotation prompts used in our dataset labeling process are shown as fol- lows. Since LLMs are trained on extensive cor- pus data, they inherently contain general knowl- edge about Hurric...

work page 2023