REVIEW 5 major objections 6 minor 41 references
Large language model as user daily behavior data generator: balancing population diversity and individual personality
T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that a properly prompted large language model can generate synthetic user behavior data that balances population diversity with individual personality, improving downstream behavior prediction by up to 18.9%.
desk verdict BehaviorGen is a practical LLM-based synthetic behavior data pipeline with a clear three-scenario evaluation, but the absence of a knowledge-free baseline and significance testing leaves the core claim about LLM behavioral knowledge only partially supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the prompt design: a role-setting system prompt, strict output formatting with constrained value ranges, and segmented generation in weekly blocks to prevent context drift. A five-field user profile and a small real record sample are injected as conditioning context. Weekly segmentation is the component that balances diversity against faithfulness: in the ablations, removing it drops Pass@1 from 100% to 22.5%, while removing format restrictions makes generated output entirely unusable.
What would settle it
Run BehaviorGen's fine-tuning augmentation on a behavior domain that is systematically absent from common web text, such as specialized occupational routines, and compare against training on the limited real data alone; if the synthetic-data gain disappears, the pretrained-knowledge assumption fails.
Extended reading notes
Core claim
BehaviorGen prompts GPT-4o with a five-attribute profile (age, education, gender, consumption, occupation) plus a few real behavior events, and requests output in a strict '[weekday, timestamp, loc, intent]' format with weekly segmentation. The paper's discovery is that this simple recipe makes the LLM generate behavior sequences that simultaneously carry population-level diversity and individual-level personality. In the Tencent mobility dataset and a smartphone usage dataset, models trained with BehaviorGen data beat SeqGAN, DiffuSeq, and UPC_SDG in every usage scenario tested. Synthetic data replaces real data in fine-tuning with roughly 62-88% of the real-data performance gain, and augmenting around 100 real records with generated data improves prediction by up to 18.9%.
Load-bearing premise
The load-bearing premise is that GPT-4o's pretraining corpus contains enough reliable, generalizable knowledge about ordinary human daily routines that a profile plus a handful of real records suffices to generate behavior distributions that improve discriminative prediction models.
Editorial extensions
If this is right
- Fine-tuning replacement with BehaviorGen data retains about 62-88% of real-data gains, so individual-level personalization may not require direct access to real user records.
- Pretraining augmentation lifts population-level prediction by up to 2.6% (mobility) and 6.9% (smartphone usage), suggesting synthetic data can expand limited population corpora.
- Augmenting roughly 100 real records per user yields up to 18.9% improvement, making cold-start and long-tail behavior prediction more tractable.
- The ablation results imply that format rigidity is indispensable and that weekly segmentation is the main lever controlling the diversity-fidelity trade-off.
Reading between the lines
- The same profile-plus-events conditioning should transfer to other structured behavioral log domains, such as app installs, purchase histories, or wearable time series, because nothing in the prompt design is specific to mobility or phone usage.
- The reported privacy guarantees (uniqueness, membership-inference resistance, epsilon below 4) are measured for the specific prompt lengths and profile fields used; longer real-behavior excerpts or richer profiles would raise re-identification risk, so the privacy claim scales inversely with conditioning detail.
- A direct test of whether LLM knowledge, rather than mere data-volume regularization, drives the gains would compare BehaviorGen against duplication or noise-injection baselines at equal data size; if those close the gap, the contribution is augmentation, not pretrained behavioral knowledge.
- The paper's no_profile ablation suggests profile fields carry different weight; an attribute-level contribution analysis could identify a minimal privacy-preserving profile that exposes fewer personal attributes while keeping most of the gain.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. BehaviorGen is a framework that prompts GPT-4o to generate synthetic daily-behavior sequences from a user profile and a small set of real records, with weekly segmentation and strict output formatting. The authors evaluate the generated data in three usage scenarios (pretraining augmentation, fine-tuning replacement, and fine-tuning augmentation) on two real datasets (Tencent and Smartphone) and two prediction backbones (Bert4Rec and PITuning), reporting gains up to 18.9% in fine-tuning augmentation and replacement rates around 62-88% of real-data fine-tuning gains. They also include ablation and privacy analyses. The core claim is that LLMs have already captured enough behavioral knowledge through pretraining to generate data that balances population diversity and individual personality, and that this synthetic data improves downstream behavior prediction.
Significance. If the empirical claims held, BehaviorGen would be a practically useful contribution: it would show that a single LLM-based generator can serve multiple downstream behavior-modeling scenarios, potentially reducing the need for sensitive real user data. The manuscript has several strengths: it uses two real datasets and two backbone models, covers three distinct usage scenarios, provides a detailed prompt in Appendix A.3, and includes privacy analyses (uniqueness testing, membership inference attacks, and an empirical differential-privacy estimate). These go beyond many single-domain synthetic-data papers. However, the current evidence is not yet sufficient to support the central claims: there are no error bars or repeated runs, the comparison lacks a knowledge-free resampling baseline, the ablation interpretation is internally inconsistent, and several reported cells contradict the prose. The privacy analysis also overstates the formal guarantee.
major comments (5)
- [§5.2, Tables 1-3] All reported numbers are single-run point estimates with no standard deviations, confidence intervals, or significance tests. Since the central claim is that BehaviorGen 'consistently outperforms' baselines and achieves gains up to 18.9%, the absence of repeated runs across random seeds and model initializations makes the magnitude and reliability of these gains impossible to assess. This is especially important for the small differences in Table 3, such as Smartphone Bert4Rec N@3 (0.612 for Ours vs 0.600 for UPC_SDG), where noise could easily change the conclusion.
- [§1 and §5.1.3, Tables 1-3] The paper attributes the success of BehaviorGen to behavioral knowledge acquired by LLMs during pretraining, but no baseline isolates that contribution. SeqGAN, DiffuSeq, and UPC_SDG all generate from the real training data, and none is a knowledge-free control that simply adds more real-like sequences (e.g., bootstrapped subsequences or a Markov model over (time, location, intent) transitions). Without such a control, the gains could be explained by data augmentation alone, and the central premise that GPT-4o's pretrained behavioral knowledge is the driving factor remains untested.
- [§5.3, Table 4] The ablation results are internally inconsistent with the stated metric directions. The paper defines KS_P so that higher values indicate better alignment and BD so that lower values indicate greater similarity; under those definitions the 'no_segment' variant achieves KS_P=0.489 (vs 0.327 for 'our') and BD=0.035 (vs 0.050), dominating the proposed configuration on two of the four distributional metrics. The prose nevertheless concludes that segmented generation 'enhances diversity and textual coherence' and produces outputs 'closely aligned with the intended data distribution.' That conclusion is not supported by the table, and the ablation does not establish that weekly segmentation improves the diversity-faithfulness trade-off. Moreover, the relation between these distributional metrics and the downstream prediction gains in Tables 1-3 is never shown.
- [§5.2, Tables 1 and 3] Several 'improvement' values are negative relative to the best real-data or baseline result, e.g., Table 1 PITuning/Tencent N@3 -1.8% and N@5 -0.3%, and Table 3 Bert4Rec/Tencent N@3 -0.1% and N@5 -0.8%; Table 2 additionally reports a negative replacement rate for Smartphone Bert4Rec recall (-23.1%). These cells are not mentioned in the text, which states that models trained with BehaviorGen data 'consistently outperform' baselines and that synthetic fine-tuning 'significantly outperformed those fine-tuned solely on real data.' The paper should report per-cell comparisons, state exactly which baseline is used in the improvement formula, and correct the prose so that it does not contradict the tables.
- [Appendix A.5] The differential privacy discussion is not a formal privacy guarantee for the generation pipeline. The reported epsilon is obtained by fitting two Gaussian distributions to overlapping-ratio values and using TensorFlow Privacy, which gives an empirical estimate for a proxy statistic, not a proof that the LLM output distribution satisfies differential privacy. Since the prompt contains real user records, the output depends on them in a complex way, and the statement that 'our model achieves a maximum privacy budget of epsilon < 4' overstates the protection. If privacy is a central motivation, this section should be reframed as an empirical similarity/uniqueness analysis, or the pipeline should be modified to provide an actual DP guarantee.
minor comments (6)
- [Abstract and §5.1] The abstract contains the typo 'pertaining augmentation'; this should be 'pretraining augmentation'.
- [§3.1] The symbol x_i is used both for an individual behavior tuple and for the full sequence, which is confusing; please use different notation (e.g., s for a sequence).
- [References] The reference entry for DeMontjoye (2013) is mislabeled as 'Seqgan: Sequence generative adversarial nets with policy gradient'; this citation error should be corrected.
- [§5.3] KS_P is not a standard measure of 'alignment' with higher-is-better; if it is the Kolmogorov-Smirnov statistic, lower values indicate closeness, so the definition and the metric need clarification.
- [Appendix A.4 and A.5] The text refers to Figures 4-7, but these figures are not included in the main text and some appear missing from the appendix; please ensure all figures are present and properly referenced.
- [Throughout] The manuscript does not state whether code and data will be released; for reproducibility, a public or anonymous release would be helpful.
Circularity Check
Fine-tuning replacement is evaluated on synthetic test data generated by the same LLM, so the 62-88% replacement rates reduce to a self-consistency check rather than real-data transfer.
-
fitted input called prediction
[Section 4.2.2 (Finetuning Replacement), Eq. (4); results in Table 2/Section 5.2]
"Specifically, we generate behavioral data for users who included in the finetuning phase. The synthesized data is then partitioned into training, testing, and validation sets, facilitating the fine-tuning of the population-level model."
Eq. (1) defines the generated sequence as a function G([x1,...,xI]) of the user's real behavior events. The replacement experiment then splits the output of that same G into train and test partitions and reports replacement rates of 62.0%/87.8% relative to real-data fine-tuning. The test labels are therefore produced by the same LLM, prompted with the same user's real events, that generated the training data. A model trained and tested on two slices of G's output is measuring its ability to fit the generator's conditional distribution, not its ability to substitute for real held-out behavior.
full rationale
The paper is an empirical pipeline, not a mathematical derivation, so most of the derivation chain is not circular. Pretraining augmentation and fine-tuning augmentation are evaluated against real held-out data and against non-LLM generative baselines (SeqGAN, DiffuSeq, UPC_SDG); neither reduces to its inputs. The choice of weekly segmentation is tuned on a 20-user batch (Appendix A.4), but that is ordinary hyperparameter selection rather than a prediction. The self-citations to the authors' PITuning and to earlier LLM-mobility work are supportive background, not the load-bearing argument: Bert4Rec is used as an independent backbone and the downstream gains are measured on real prediction tasks. The one construction-level circularity is the finetuning-replacement scenario: as written, the synthetic data is split into training and testing, so the replacement rate is computed on data generated by the same LLM that was prompted with the users' real events. That makes the 62-88% replacement claim a measure of fit to the generator's own output rather than a transfer result to real data. Because this affects one of the paper's three headline scenarios, the overall score is 6 rather than 0.
Assumptions & free parameters
free parameters (3)
- weekly segment length =
7 days
- number of real records used as context =
about 100 logs (fine-tuning augmentation)
- generated sequence length =
at least 90 or 100 lines per month
assumptions (3)
- domain assumption LLM pretraining has captured reliable knowledge of human daily behavior patterns.
- domain assumption Behavior prediction accuracy, measured by Precision, Recall, and NDCG, is a valid proxy for synthetic data quality.
- domain assumption The two datasets are representative of daily human behavior and are properly anonymized.
Cite this review
Pith. "Pith review of Large language model as user daily behavior data generator: balancing population diversity and individual personality." pith.science (2026). https://pith.science/paper/VBQVEHRY
@misc{pith2026250517615,
author = {Pith},
title = {Pith review of: Large language model as user daily behavior data generator: balancing population diversity and individual personality},
year = {2026},
howpublished = {\url{https://pith.science/paper/VBQVEHRY}},
note = {Machine review of arXiv:2505.17615}
}
read the original abstract
Predicting human daily behavior is challenging due to the complexity of routine patterns and short-term fluctuations. While data-driven models have improved behavior prediction by leveraging empirical data from various platforms and devices, the reliance on sensitive, large-scale user data raises privacy concerns and limits data availability. Synthetic data generation has emerged as a promising solution, though existing methods are often limited to specific applications. In this work, we introduce BehaviorGen, a framework that uses large language models (LLMs) to generate high-quality synthetic behavior data. By simulating user behavior based on profiles and real events, BehaviorGen supports data augmentation and replacement in behavior prediction models. We evaluate its performance in scenarios such as pertaining augmentation, fine-tuning replacement, and fine-tuning augmentation, achieving significant improvements in human mobility and smartphone usage predictions, with gains of up to 18.9%. Our results demonstrate the potential of BehaviorGen to enhance user behavior modeling through flexible and privacy-preserving synthetic data generation.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. https://doi.org/10.1145/2976749.2978318 Deep learning with differential privacy . In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS '16, page 308–318, New York, NY, USA. Association for Computing Machinery
arXiv 2016
-
[2]
Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165
arXiv 2020
-
[3]
Derek Chen, Celine Lee, Yunan Lu, Domenic Rosati, and Zhou Yu. 2023. Mixture of soft prompts for controllable data generation. arXiv preprint arXiv:2303.01580
work page Pith review arXiv 2023
-
[4]
Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021. Autodebias: Learning to debias for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 21--30
work page 2021
-
[5]
Hyunji Chung and Sangjin Lee. 2018. Intelligent virtual assistant knows your life. arXiv preprint arXiv:1803.00466
arXiv 2018
-
[6]
YA DeMontjoye. 2013. https://doi.org/10.1038/srep01376 Seqgan: Sequence generative adversarial nets with policy gradient . 31(1)
-
[7]
Jingtao Ding, Fuli Feng, Xiangnan He, Guanghui Yu, Yong Li, and Depeng Jin. 2018. An improved sampler for bayesian personalized ranking by leveraging view data. In Companion proceedings of the the web conference 2018, pages 13--14
work page 2018
-
[8]
Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. 2020. Simplify and robustify negative sampling for implicit collaborative filtering. Advances in Neural Information Processing Systems, 33:1094--1105
work page 2020
Show all 41 references
-
[9]
Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, et al. 2024. Understanding world or predicting future? a comprehensive survey of world models. arXiv preprint arXiv:2411.14499
2024
-
[10]
Jiahui Gong, Jingtao Ding, Fanjin Meng, Guilong Chen, Hong Chen, Shen Zhao, Haisheng Lu, and Yong Li. 2024. https://doi.org/10.1145/3637528.3671984 A population-to-individual tuning framework for adapting pretrained lm to on-device user intent prediction . In Proceedings of th...
2024
-
[11]
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. 2022. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933
2022 arXiv
-
[12]
Xu Guo and Yiqiang Chen. 2024. Generative ai for synthetic data generation: Methods, challenges and the future. arXiv preprint arXiv:2403.04190
2024 arXiv
-
[13]
Xu Guo, Boyang Li, and Han Yu. 2022. Improving the sample efficiency of prompt tuning with domain adaptation. arXiv preprint arXiv:2210.02952
2022 arXiv
-
[14]
Xingwei He, Zhenghao Lin, Yeyun Gong, Alex Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, Weizhu Chen, et al. 2023. Annollm: Making large language models to be better crowdsourced annotators. arXiv preprint arXiv:2303.16854
2023 arXiv
-
[15]
Tong Li, Tong Xia, Huandong Wang, Zhen Tu, Sasu Tarkoma, Zhu Han, and Pan Hui. 2022. Smartphone app usage analysis: datasets, methods, and applications. IEEE Communications Surveys & Tutorials, 24(2):937--966
2022
-
[16]
Xuchuan Li, Fei Huang, Jianrong Lv, Zhixiong Xiao, Guolong Li, and Yang Yue. 2024. Be more real: Travel diary generation using llm agents and individual profiles. arXiv preprint arXiv:2407.18932
2024 arXiv
-
[17]
Fan Liu, Zhiyong Cheng, Huilin Chen, Yinwei Wei, Liqiang Nie, and Mohan Kankanhalli. 2022. Privacy-preserving synthetic data generation for recommendation systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieva...
2022
-
[18]
Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On llms-driven synthetic data generation, curation, and evaluation: A survey. arXiv preprint arXiv:2406.15126
2024 arXiv
-
[19]
Xufang Luo, Zheng Liu, Shitao Xiao, Xing Xie, and Dongsheng Li. 2022. Mindsim: user simulator for news recommenders. In WWW, pages 2067--2077
2022
-
[20]
Devaki Nadkarni. 2016. Superforecasting: The art & science of forecasting. Anvesha, 9(1):61
2016
-
[21]
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1--22
2023
-
[22]
Vladimir Provalov, Elizaveta Stavinova, and Petr Chunaev. 2021. https://doi.org/10.1109/ICDMW53433.2021.00014 Synevarec: A framework for evaluating recommender systems on synthetic data classes . In 2021 International Conference on Data Mining Workshops (ICDMW), pages 55--64
2021
-
[23]
Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended abstracts of the 2021 CHI conference on human factors in computing systems, pages 1--7
2021
-
[24]
Germans Savcisens, Tina Eliassi-Rad, Lars Kai Hansen, Laust Hvas Mortensen, Lau Lilleholt, Anna Rogers, Ingo Zettler, and Sune Lehmann. 2023. Using sequences of life-events to predict human lives. Nature Computational Science, pages 1--14
2023
-
[25]
Chenyang Shao, Fengli Xu, Bingbing Fan, Jingtao Ding, Yuan Yuan, Meng Wang, and Yong Li. 2024. Beyond imitation: Generating human mobility from context-aware reasoning with large language models. arXiv preprint arXiv:2402.09836
2024 arXiv
-
[26]
Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, and An-Xiang Zeng. 2019. Virtual-taobao: Virtualizing real-world online retail environment for reinforcement learning. In AAAI, volume 33, pages 4902--4909
2019
-
[27]
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. https://doi.org/10.1109/SP.2017.41 Membership inference attacks against machine learning models . In 2017 IEEE Symposium on Security and Privacy (SP), pages 3--18
2017 doi
-
[28]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, ...
2019
-
[29]
Amrita S Tulshan and Sudhir Namdeorao Dhage. 2019. Survey on virtual assistant: Google assistant, siri, cortana, alexa. In Advances in Signal Processing and Intelligent Recognition Systems: 4th International Symposium SIRS 2018, Bangalore, India, September 19--22, 2018, Revise...
2019
-
[30]
Jiawei Wang, Renhe Jiang, Chuang Yang, Zengqing Wu, Makoto Onizuka, Ryosuke Shibasaki, and Chuan Xiao. 2024. Large language models as urban residents: An llm agent framework for personal mobility generation. arXiv preprint arXiv:2402.14744
2024 arXiv
-
[31]
Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. https://doi.org/10.1609/aaai.v31i1.10804 Seqgan: Sequence generative adversarial nets with policy gradient . Proceedings of the AAAI Conference on Artificial Intelligence, 31(1)
2017 doi
-
[32]
Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. 2023. Large language model as attributed training data generator: A tale of diversity and bias. Advances in neural information processing systems
2023
-
[33]
Yuan Yuan, Jingtao Ding, Depeng Jin, and Yong Li. 2025. Learning the complexity of urban mobility with deep generative network. PNAS nexus, 4(5):pgaf081
2025
-
[34]
Yuan Yuan, Huandong Wang, Jingtao Ding, Depeng Jin, and Yong Li. 2023. Learning to simulate daily activities via modeling dynamic human needs. pages 906--916
2023
-
[35]
Hongtao Zhang and Lingcheng Dai. 2018. Mobility prediction: A survey on state-of-the-art schemes and future applications. IEEE access, 7:802--822
2018
-
[36]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM computing surveys (CSUR), 52(1):1--38
2019
-
[37]
Yu Zhang, Shutong Qiao, Jiaqi Zhang, Tzu-Heng Lin, Chen Gao, and Yong Li. 2025. A survey of large language model empowered agents for recommendation and search: Towards next-generation information retrieval. arXiv preprint arXiv:2503.05659
2025 arXiv
-
[38]
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223
2023 arXiv
-
[39]
Larisa Zherdeva, Denis Zherdev, and Artem Nikonorov. 2021. https://doi.org/10.1109/ITNT52450.2021.9649073 Prediction of human behavior with synthetic data . In 2021 International Conference on Information Technology and Nanotechnology (ITNT), pages 1--6
2021
-
[40]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[41]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.