pith. sign in

arxiv: 2509.22055 · v2 · submitted 2025-09-26 · 💻 cs.CL

RedNote-Vibe: A Dataset for Capturing Temporal Dynamics of AI-Generated Text in Lifestyle Social Media

Yudong Li , Yufei Sun , Peiru Yang , Yuhan Yao , Wanyue Li , Jiajun Zou , Haoyang Yang , Haotian Gan
show 2 more authors
Linlin Shen Yongfeng Huang
This is my paper

Pith reviewed 2026-05-18 13:42 UTC · model grok-4.3

classification 💻 cs.CL
keywords AI-generated text detectionsocial media datasetpsycholinguistic analysislifestyle contenttemporal dynamicsengagement metricshuman-AI comparisoncontent creation trends
0
0 comments X

The pith

A five-year lifestyle social media dataset shows human posts keep an emotional edge while strategic AI use can lift engagement for some creators.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces RedNote-Vibe, a dataset drawn from the RedNote platform that covers posts from before large language models through July 2025 and records detailed engagement metrics. It also presents PLAD, a detection method built on psychological and linguistic markers drawn from cognitive psychology. Experiments with the dataset and method indicate that human content tends to create stronger emotional responses, AI content stays more uniform and rarely produces standout posts, the difference shrinks when interactions demand more effort from users, and a minority of creators who mix AI tools with their own approach achieve stronger results overall.

Core claim

RedNote-Vibe supplies five years of lifestyle posts with engagement data to track how AI-generated text has evolved on the platform. The PsychoLinguistic AIGT Detection Framework (PLAD) uses deep psychological signatures to identify AI text more effectively than prior tools. Tests show PLAD performs better and the data indicate human content outperforms AI in emotionally resonant areas, AI text is more homogeneous and seldom breaks through, the human-AI difference narrows for higher-investment interactions, and a small set of users who apply AI strategically reach higher engagement.

What carries the argument

The PsychoLinguistic AIGT Detection Framework (PLAD), which extracts psychological signatures grounded in cognitive psychology to detect AI-generated text in short posts.

If this is right

  • Human posts continue to hold an advantage when emotional resonance matters most.
  • AI-generated content tends to be uniform and rarely produces the most engaging or novel posts.
  • The gap between human and AI performance shrinks when users invest more in interactions.
  • A minority of creators who combine AI tools with their own input achieve higher engagement than typical users.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Detection methods like PLAD could be adapted to track AI influence on other social platforms over time.
  • Platforms focused on lifestyle or emotional topics might prioritize or label human-created posts to maintain audience connection.
  • Everyday creators could test simple strategies for blending AI assistance with personal voice to improve reach without losing authenticity.

Load-bearing premise

The dataset labels correctly separate human from AI posts across the full five-year span and the chosen psychological markers stay distinct and measurable in brief social media text.

What would settle it

A manual check that reveals many posts labeled human or AI are actually the opposite, or new verified posts where PLAD loses the accuracy reported in the experiments.

Figures

Figures reproduced from arXiv: 2509.22055 by Haotian Gan, Haoyang Yang, Jiajun Zou, Linlin Shen, Peiru Yang, Wanyue Li, Yongfeng Huang, Yudong Li, Yufei Sun, Yuhan Yao.

Figure 1
Figure 1. Figure 1: The distribution of posts by domain and interaction count (sum of likes, comments and collections). very low engagement (0-10 interactions), indicating that most user-generated content attracts limited audience attention. In contrast, a small fraction of posts achieve disproportionately high engagement. 2.2 AI SAMPLES GENERATION To create a comprehensive AIGT detection benchmark, we construct parallel AI-g… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the RedNote-Vibe dataset construction. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The top-15 important feature scores [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Temporal evolution of AI content pro￾portion in exploration set, where visualizations are smoothed using 14/30-day rolling averages. In this section, we use PLAD framework to analyze the ex￾ploration set in the RedNote Engagement Dataset, uncov￾ering the impact of AI-generated content on user behavior and interaction trends in real-world data with temporal dynamics. 5.1 TEMPORAL DYNAMICS OF AI CONTENT [PI… view at source ↗
Figure 5
Figure 5. Figure 5: Engagement metrics comparison between Human-authored and AI-generated posts across 6 major [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Author-level analysis of AI usage patterns [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of feature statistics between human and AI-generated texts. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Top-10 most influential features of likes, collections and comments metrics. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Distribution of pairwise feature correlations. Dashed lines indicate thresholds for weak ( [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
read the original abstract

We introduce RedNote-Vibe, a dataset spanning five years (pre-LLM to July 2025) sourced from lifestyle platform RedNote (Xiaohongshu), capturing the temporal dynamics of content creation and is enriched with comprehensive engagement metrics. To address the detection challenge posed by RedNote-Vibe, we propose the \textbf{PsychoLinguistic AIGT Detection Framework (PLAD)}. Grounded in cognitive psychology, PLAD leverages deep psychological signatures for robust and interpretable detection. Our experiments demonstrate PLAD's superior performance and reveal insights into content dynamics: (1) human content continues to outperform AI in emotionally resonant domains; (2) AI content is more homogeneous and rarely produces breaking posts, however, this human-AI gap narrows for arousing higher-investment interactions; and (3) most interestingly, a small group of users who strategically utilize AI tools can achieve higher engagement outcomes. The dataset is available at https://github.com/ydli-ai/RedNote-Vibe

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 3 minor

Summary. The manuscript introduces RedNote-Vibe, a five-year dataset of lifestyle posts from the RedNote (Xiaohongshu) platform that includes engagement metrics and binary AI-generated versus human-authored labels. It proposes the PsychoLinguistic AIGT Detection Framework (PLAD), which applies cognitive-psychology-derived signatures for interpretable detection of AI-generated text. Experiments are reported to show PLAD outperforming baselines, together with three main findings: human content retains an edge in emotionally resonant domains, AI content is more homogeneous and rarely produces breaking posts (with the gap narrowing for higher-investment interactions), and a small subset of users who strategically combine AI tools achieve higher engagement.

Significance. If the labeling procedure proves reliable and the psychological features remain stable, the dataset supplies a valuable longitudinal resource for studying temporal shifts in AI-generated social-media content, while PLAD offers an interpretable alternative to purely statistical detectors. The engagement analyses could inform platform design and creator strategies, particularly the observation that strategic AI use can close or reverse performance gaps.

major comments (3)
  1. Dataset Construction section: the procedure used to assign AI versus human labels across the full five-year span (pre-LLM through July 2025) is not specified in sufficient detail. It is unclear whether labels derive from self-reports, platform metadata, post-hoc application of modern detectors, or a hybrid method, and whether any temporal calibration was performed. Because every reported insight (emotional superiority, homogeneity, strategic-user gains) rests on this partition, any systematic bias in early-period labeling would propagate directly into the PLAD performance numbers and the three headline claims.
  2. PLAD Framework and Experiments sections: the manuscript asserts that the psychological signatures are both distinct and reliably extractable from short posts, yet provides no quantitative validation (e.g., feature ablation, inter-rater reliability on signature annotation, or checks for platform-specific stylistic artifacts). Without such evidence, the claimed superiority of PLAD and the interpretation of engagement gaps remain difficult to evaluate.
  3. Results and Analysis sections: the headline finding that 'a small group of users who strategically utilize AI tools can achieve higher engagement outcomes' is presented without a clear operational definition of 'strategic utilization' or controls for confounding variables such as user popularity, posting frequency, or topic selection. This weakens the causal interpretation offered in the abstract.
minor comments (3)
  1. Abstract: the phrase 'comprehensive engagement metrics' is used without enumeration; listing the primary metrics (likes, comments, saves, etc.) would improve immediate readability.
  2. Figure captions and tables: several engagement-gap plots lack error bars or sample-size annotations, making it hard to judge the statistical robustness of the reported narrowing of the human-AI gap in higher-investment interactions.
  3. Related Work: the discussion of prior AIGT detectors would benefit from explicit comparison tables that include both detection accuracy and interpretability metrics alongside PLAD.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We have addressed each major comment below and revised the manuscript to improve clarity, reproducibility, and the strength of our claims.

read point-by-point responses

  1. Referee: Dataset Construction section: the procedure used to assign AI versus human labels across the full five-year span (pre-LLM through July 2025) is not specified in sufficient detail. It is unclear whether labels derive from self-reports, platform metadata, post-hoc application of modern detectors, or a hybrid method, and whether any temporal calibration was performed. Because every reported insight (emotional superiority, homogeneity, strategic-user gains) rests on this partition, any systematic bias in early-period labeling would propagate directly into the PLAD performance numbers and the three headline claims.

    Authors: We agree that the original description of the labeling procedure was insufficiently detailed. The revised manuscript now expands the Dataset Construction section to explicitly describe the hybrid approach: platform metadata and available user disclosures for post-2023 content, combined with calibrated post-hoc detection using contemporary models for earlier periods. We also added a dedicated subsection on temporal calibration, including threshold adjustments based on LLM capability timelines and a limitations discussion addressing potential propagation of labeling bias into the reported findings. revision: yes

  2. Referee: PLAD Framework and Experiments sections: the manuscript asserts that the psychological signatures are both distinct and reliably extractable from short posts, yet provides no quantitative validation (e.g., feature ablation, inter-rater reliability on signature annotation, or checks for platform-specific stylistic artifacts). Without such evidence, the claimed superiority of PLAD and the interpretation of engagement gaps remain difficult to evaluate.

    Authors: We accept that additional quantitative validation is required. The revised Experiments section now includes feature ablation results demonstrating the incremental contribution of each cognitive-psychology signature, inter-rater reliability statistics on a manually annotated subset of posts, and explicit checks comparing signature distributions against non-RedNote social-media corpora to rule out platform-specific artifacts. These additions directly support the interpretability and performance claims for PLAD. revision: yes

  3. Referee: Results and Analysis sections: the headline finding that 'a small group of users who strategically utilize AI tools can achieve higher engagement outcomes' is presented without a clear operational definition of 'strategic utilization' or controls for confounding variables such as user popularity, posting frequency, or topic selection. This weakens the causal interpretation offered in the abstract.

    Authors: We acknowledge the need for greater precision here. The revised Results section now provides an explicit operational definition of strategic utilization (users who post AI-assisted content with documented human editing and timing optimization). We further added multivariate regression models that control for follower count, posting frequency, and topic category. The updated analysis retains the reported engagement advantage while clarifying that the evidence remains observational rather than strictly causal. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation grounded in external literature

full rationale

The paper introduces RedNote-Vibe dataset and PLAD detection framework explicitly grounded in cognitive psychology literature rather than self-referential fitting or equations. No load-bearing steps reduce by construction to inputs, fitted predictions, or self-citation chains; the central claims rest on external psychological signatures and dataset construction details that are presented as independent. This is the most common honest finding for dataset papers without mathematical derivations.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only view yields minimal internal parameters or invented entities; the central claim rests on the unverified accuracy of AI labels and the transferability of cognitive psychology signatures to short-form social posts.

axioms (1)
  • domain assumption Cognitive psychology supplies distinct, extractable signatures that reliably separate AI-generated from human text in lifestyle social media posts.
    PLAD is described as grounded in cognitive psychology for robust detection; this premise is invoked to justify the framework's interpretability and superiority.

pith-pipeline@v0.9.0 · 5743 in / 1356 out tokens · 48272 ms · 2026-05-18T13:42:49.972947+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 4 internal anchors

  1. [1]

    GPT-4 Technical Report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    Who said that? benchmarking social media ai detection.arXiv preprint arXiv:2310.08240,

    Wanyun Cui, Linqiu Zhang, Qianle Wang, and Shuyang Cai. Who said that? benchmarking social media ai detection.arXiv preprint arXiv:2310.08240,

    work page arXiv

  3. [3]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186,

    work page 2019

  4. [4]

    Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm.arXiv preprint arXiv:1708.00524,

    Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm.arXiv preprint arXiv:1708.00524,

    work page arXiv

  5. [5]

    Are ai detectors good enough? a survey on quality of datasets with machine-generated texts.arXiv preprint arXiv:2410.14677,

    German Gritsai, Anastasia V oznyuk, Andrey Grabovoy, and Yury Chekhovich. Are ai detectors good enough? a survey on quality of datasets with machine-generated texts.arXiv preprint arXiv:2410.14677,

    work page arXiv

  6. [6]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Spotting llms with binoculars: Zero-shot detection of machine-generated text

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting llms with binoculars: Zero-shot detection of machine-generated text.arXiv preprint arXiv:2401.12070,

    work page arXiv

  8. [8]

    ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations.arXiv preprint arXiv:1909.11942,

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  9. [9]

    Origin tracing and detecting of llms

    Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun, and Xipeng Qiu. Origin tracing and detecting of llms. arXiv preprint arXiv:2304.14072,

    work page arXiv

  10. [10]

    Does detectgpt fully utilize perturbation? bridging selective perturbation to fine-tuned contrastive learning detector would be better.arXiv preprint arXiv:2402.00263,

    Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan, and Chao Shen. Does detectgpt fully utilize perturbation? bridging selective perturbation to fine-tuned contrastive learning detector would be better.arXiv preprint arXiv:2402.00263,

    work page arXiv

  11. [11]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  12. [12]

    Accessed: 2025-05-

    URL https://developers.googleblog.com/zh-hans/gemini-2-family-expands/ . Accessed: 2025-05-

    work page 2025

  13. [13]

    Named entity recognition for chinese social media with jointly trained embeddings

    Nanyun Peng and Mark Dredze. Named entity recognition for chinese social media with jointly trained embeddings. InProceedings of the 2015 conference on empirical methods in natural language processing, pp. 548–554,

    work page 2015

  14. [14]

    Can AI-Generated Text be Reliably Detected?

    Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected?arXiv preprint arXiv:2303.11156,

    work page Pith review arXiv

  15. [15]

    Electionrumors2022: A dataset of election rumors on twitter during the 2022 us midterms.arXiv preprint arXiv:2407.16051,

    Joseph S Schafer, Kayla Duskin, Stephen Prochaska, Morgan Wack, Anna Beers, Lia Bozarth, Taylor Agajanian, Mike Caulfield, Emma S Spiro, and Kate Starbird. Electionrumors2022: A dataset of election rumors on twitter during the 2022 us midterms.arXiv preprint arXiv:2407.16051,

    work page arXiv 2022

  16. [16]

    Ten words only still help: Improving black-box ai-generated text detection via proxy-guided efficient re-sampling.arXiv preprint arXiv:2402.09199,

    Yuhui Shi, Qiang Sheng, Juan Cao, Hao Mi, Beizhe Hu, and Danding Wang. Ten words only still help: Improving black-box ai-generated text detection via proxy-guided efficient re-sampling.arXiv preprint arXiv:2402.09199,

    work page arXiv

  17. [17]

    Play favorites: A statistical method to measure self-bias in llm-as-a-judge.arXiv preprint arXiv:2508.06709,

    Evangelia Spiliopoulou, Riccardo Fogliato, Hanna Burnsky, Tamer Soliman, Jie Ma, Graham Horwood, and Miguel Ballesteros. Play favorites: A statistical method to measure self-bias in llm-as-a-judge.arXiv preprint arXiv:2508.06709,

    work page arXiv

  18. [18]

    Idiosyncrasies in large language models

    Mingjie Sun, Yida Yin, Zhiqiu Xu, J Zico Kolter, and Zhuang Liu. Idiosyncrasies in large language models. arXiv preprint arXiv:2502.12150,

    work page arXiv

  19. [19]

    Machine-generated text localization

    Zhongping Zhang, Wenda Qin, and Bryan Plummer. Machine-generated text localization. InFindings of the Association for Computational Linguistics ACL 2024, pp. 8357–8371,

    work page 2024

  20. [20]

    collections

    13 APPENDIX In the Appendix, we provide further analysis, including the differences between human-written text and AI-generated text from the perspective of the PLAD framework (Section A.1) and how these features affect the engagement metrics of posts (Section A.2). In addition, we analyze the correlation between features, reflecting the orthogonality of ...

    work page 2017