REVIEW 4 major objections 5 minor 1 cited by
A Large-scale Dataset with Behavior, Attributes, and Content of Mobile Short-video Platform
T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read A released short-video dataset combines 10,000 users' behavior, attributes, and 153,561 raw videos in one package.
desk verdict A useful dataset paper whose central content claim is currently ambiguous — the released artifact needs to be pinned down before the community can trust what is actually delivered. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the dataset itself, structured as three coupled layers: behavior records captured through a proxy agent, user and video attributes, and preprocessed video content. The content layer carries the novelty: each video is divided into 8 equal-length clips, a 256-dimensional visual feature is extracted per clip with pretrained convolutional and transformer visual encoders, and bilingual ASR text is generated from the audio. The four-fold validation uses this same machinery: embedding visualization checks the semantic quality of the visual features, the recommendation benchmark uses the interaction and feature layers, and the filter-bubble study relies on the hierarchical video category system.
What would settle it
Download the released package and check whether at least one of the 153,561 raw video files described in Section 2.3 is actually present and playable; if only URLs or precomputed feature vectors are distributed, the content-layer claim fails.
Extended reading notes
Core claim
The paper's own claim is that the dataset's distinctive contribution is combining three layers on a real mobile short-video platform: behavior logs (implicit watching time plus explicit like, follow, collect, forward, comment, and hate signals), attribute data (6 user-side and 9 video-side attributes, including a three-level hierarchical video category system), and content data (raw video files, per-clip visual features, and bilingual speech-to-text text). On this basis, four validations show the data is usable: the interaction and attribute distributions are diverse, the visual features cluster by video category, recommendation algorithms produce informative rankings with a multimodal method performing best, and filter-bubble ratios rise for inactive users while staying stable for active users. The paper presents these results as evidence that the dataset can support user modeling, social science, and human behavior research.
Load-bearing premise
The claim stands on the release actually containing the raw video files described in Section 2.3 rather than only public URLs and precomputed features.
Editorial extensions
If this is right
- Researchers can benchmark both general and multimodal recommendation algorithms on one dataset, with the paper reporting the best result for a multimodal method that leverages video content.
- The dataset supports filter-bubble research: the reported analysis distinguishes active from inactive users and finds that their content-diversity trajectories diverge over the first week.
- Because hierarchical video categories and bilingual ASR text are included, the dataset can support multi-level content understanding rather than only interaction prediction.
- The future extensions named by the paper, such as finer-grained video semantics and longer observation windows, would build directly on the same data structure.
Reading between the lines
- Beyond the paper: if the released archive actually contains the 3.2 TB of raw videos, the dataset becomes a possible training ground for large multimodal models on short-video content; if it contains only URLs and precomputed features, it is materially a feature benchmark and the raw-content claim would be false.
- Beyond the paper: the city-tier and device-price attributes invite studies of geographic and economic differences in short-video consumption, but the paper itself only reports distributions and does not test such relationships.
- Beyond the paper: pairing the per-clip visual features with the bilingual ASR text could support cross-modal retrieval or captioning tasks, which the paper does not explicitly evaluate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ShortVideo_dataset, a large-scale dataset from a real mobile short-video platform, containing 10,000 voluntary users, 153,561 videos, and 1,019,568 interactions. The authors describe user behavior data (implicit and explicit feedback), user and video attributes (demographics, geography, categories, tags), and video content data (raw videos plus precomputed visual features and ASR text). They provide four technical validations: interaction and attribute richness analysis, t-SNE-based video feature quality assessment, benchmarking of eight recommendation algorithms, and a filter bubble study over seven days. The stated contribution is a dataset that combines behavior, attributes, and content, with public code and data at a GitHub repository.
Significance. If the dataset is released as described, it would be a valuable resource for short-video recommendation, user modeling, and computational social science, because it combines interaction logs, user attributes, and multimodal video content in one resource. The paper explicitly reports voluntary informed consent, anonymization, and ethical review, and it ships code for data analysis. The recommendation benchmark covers both general and multimodal methods, and the filter bubble analysis targets a socially relevant phenomenon. These strengths are real, but the significance is conditional: the most distinctive part of the claimed contribution, the raw video content, is contradicted by the distribution statement in Section 4, and several validations are qualitative or lack statistical grounding.
major comments (4)
- [Section 2.3 and Section 4] The paper states in Section 2.3 that the dataset includes "the raw files of 153,561 videos, totaling 3.2 TB," but Section 4 states "to avoid copyright issues, we will provide the original video URLs which can be accessed publicly." These are materially different artifacts: raw MP4 files are not the same as URL references. This contradiction bears directly on the central claim that the dataset provides video content, so the manuscript must specify the exact downloadable items (raw videos, feature vectors, ASR text, URL manifest) and reconcile the two statements before the contribution can be evaluated.
- [Section 2 and Section 2.2] There is an internal inconsistency about the age composition of the sample. Section 2 says the volunteers were recruited "excluding users under 20," while Section 2.2 says "we do not exclude the minor users but their data has been removed in the actual dataset." These statements cannot both be true, and the issue matters for the privacy and ethical claims. Please clarify whether the released data contains any users under 20 and whether the reported statistics in Section 3 include or exclude such users.
- [Section 3.2] The validation of the video content features relies solely on visual inspection of t-SNE plots, with no quantitative measure of cluster quality, category separability, or downstream effectiveness. The claim that the features "effectively capture the semantic information of the videos" is therefore not established. Please add a numerical evaluation, such as category classification accuracy with a held-out split, retrieval precision, or a clustering metric with appropriate baselines.
- [Section 3.4] The filter bubble metric is ambiguous. Coverage is defined as N_seen(u,c)/N_all(c), but N_seen(u,c) is described as "the number of categories a user u has interacted with," while N_all(c) is "the total available categories at level c." It is unclear whether the numerator is a count of categories or a count of interactions, and whether the metric is a fraction, a proportion, or an average over categories. In addition, the active-user threshold of at least 3 videos per day is ad hoc; please justify it or report sensitivity to the threshold.
minor comments (5)
- [Section 3.3, Table 1] The benchmark results are presented as point estimates without standard deviations or significance tests. Please state the number of random seeds and describe the hyperparameter settings or state that the pipeline of [16] was followed exactly.
- [Section 3.4, Figure 5] The two panels of Figure 5 appear nearly identical at the resolution provided, which makes it hard to see the claimed difference between active and inactive users. Please use clearer legends, distinct y-axis ranges, or difference curves.
- [Section 2.3] The extraction of visual features is said to use "pre-trained ResNet and ViT," but no references or version details are given for these models. Please provide citations and the exact feature extraction layer or pooling method.
- [Section 4] The phrase "the ImageNet license" is unclear, since ImageNet does not have a single standard license and Kuaishou videos are third-party content. Please clarify the exact license terms under which the videos or URLs are distributed.
- [Throughout] The paper would benefit from a table listing the released files, their formats, and sizes, as well as a data card or schema description for the user, video, and interaction files.
Circularity Check
No circular derivation; the paper is a descriptive dataset paper whose validations are self-contained, though Section 2.3 and Section 4 contain a release-spec inconsistency that is not a circularity.
full rationale
This is a dataset description paper, not a derivation paper. There is no claimed chain of equations from first principles to a predicted quantity. The four validations are descriptive: interaction and attribute distributions, t-SNE visualization of pre-extracted visual features, benchmark runs of eight external recommender algorithms, and a filter-bubble ratio computed from an explicitly stated definition. None of these fits a parameter and then predicts a closely related quantity; none invokes a uniqueness theorem; and none relies on a load-bearing self-citation (the reference list contains no self-citations). The content-feature t-SNE check is not circular because the category labels come from video titles via K-means plus manual labeling, while the visual features come from pre-trained ResNet and ViT applied to video frames, i.e., separate modalities. The benchmark uses standard algorithms from external groups and compares with an external survey's finding that multimodal methods such as BM3 are strong; this is external evidence, not circularity. One inconsistency should be flagged as a data-release specification problem, not circularity: Section 2.3 states 'Our dataset includes the raw files of 153,561 videos watched by users, totaling 3,998 hours in duration and 3.2 TB in size,' while Section 4 states 'To avoid copyright issues, we will provide the original video URLs which can be accessed publicly.' These describe different release artifacts and must be reconciled before the central content-data claim can be evaluated, but the contradiction does not make any reasoning circular. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- active user threshold =
3 videos/day
assumptions (4)
- domain assumption Recorded interactions from proxy agent reflect natural user behavior
- domain assumption Volunteer sample is representative enough for population-level claims
- domain assumption Video categories from K-means plus manual labeling are valid ground truth
- domain assumption t-SNE visual separation implies semantic quality of visual features
Cite this review
Pith. "Pith review of A Large-scale Dataset with Behavior, Attributes, and Content of Mobile Short-video Platform." pith.science (2026). https://pith.science/paper/B5ESMMPN
@misc{pith2026250205922,
author = {Pith},
title = {Pith review of: A Large-scale Dataset with Behavior, Attributes, and Content of Mobile Short-video Platform},
year = {2026},
howpublished = {\url{https://pith.science/paper/B5ESMMPN}},
note = {Machine review of arXiv:2502.05922}
}
read the original abstract
Short-video platforms show an increasing impact on people's daily lives nowadays, with billions of active users spending plenty of time each day. The interactions between users and online platforms give rise to many scientific problems across computational social science and artificial intelligence. However, despite the rapid development of short-video platforms, currently there are serious shortcomings in existing relevant datasets on three aspects: inadequate user-video feedback, limited user attributes and lack of video content. To address these problems, we provide a large-scale dataset with rich user behavior, attributes and video content from a real mobile short-video platform. This dataset covers 10,000 voluntary users and 153,561 videos, and we conduct four-fold technical validations of the dataset. First, we verify the richness of the behavior and attribute data. Second, we confirm the representing ability of the content features. Third, we provide benchmarking results on recommendation algorithms with our dataset. Finally, we explore the filter bubble phenomenon on the platform using the dataset. We believe the dataset could support the broad research community, including but not limited to user modeling, social science, human behavior understanding, etc. The dataset and code is available at https://github.com/tsinghua-fib-lab/ShortVideo_dataset.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems
AgentRecBench is a public text-based benchmark for LLM recommendation agents, but its headline claim of agent superiority is undercut by its own tables.
Reference graph
Works this paper leans on
-
[1]
Xu Chen, Jingsen Zhang, Lei Wang, Quanyu Dai, Zhenhua Dong, Ruiming Tang, Rui Zhang, Li Chen, and Ji-Rong Wen. 2023. REASONER: An Explainable Recom- mendation Dataset with Multi-aspect Real User Labeled Ground Truths Towards more Measurable Explainable Recommendation. arXiv preprint arXiv:2303.00168 (2023). 1 2 3 4 5 6 7 Day 0.300 0.325 0.350 0.375 0.400 ...
work page Pith review arXiv 2023
-
[2]
Chenbo Fu, Qiushun Che, Zhanghao Li, Fengyan Yuan, and Yong Min. 2024. Heavy users fail to fall into filter bubbles: evidence from a Chinese online video platform. Frontiers in Physics 12 (2024), 1423851
work page 2024
-
[3]
Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 540–550
work page 2022
-
[4]
Yingqiang Ge, Shuya Zhao, Honglu Zhou, Changhua Pei, Fei Sun, Wenwu Ou, and Yongfeng Zhang. 2020. Understanding echo chambers in e-commerce recom- mender systems. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval . 2261–2270
work page 2020
-
[5]
Zhiqiang Guo, Jianjun Li, Guohui Li, Chaoyang Wang, Si Shi, and Bin Ruan. 2024. LGMRec: Local and Global Graph Learning for Multimodal Recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 8454–8462
2024
-
[6]
Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI conference on artificial intelligence, Vol. 30
2016
-
[7]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval . 639–648
2020
-
[8]
Christian Montag, Zhiying Zhao, Cornelia Sindermann, Lei Xu, Meina Fu, Jialin Li, Xiaoxiao Zheng, Keshuang Li, Keith M Kendrick, Jing Dai, et al. 2018. Internet communication disorder and the structure of the human brain: Initial insights on WeChat addiction. Scientific reports 8, 1 (2018), 2155
work page 2018
Show all 19 references
-
[9]
Tien T Nguyen, Pik-Mai Hui, F Maxwell Harper, Loren Terveen, and Joseph A Konstan. 2014. Exploring the filter bubble: the effect of using recommender systems on content diversity. In Proceedings of the 23rd international conference on World wide web. 677–686
2014
-
[10]
Yongxin Ni, Yu Cheng, Xiangyan Liu, Junchen Fu, Youhua Li, Xiangnan He, Yongfeng Zhang, and Fajie Yuan. 2023. A Content-Driven Micro-Video Recom- mendation Dataset at Scale. arXiv preprint arXiv:2309.15379 (2023)
2023 arXiv
-
[11]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme
-
[12]
Tongyi SpeechTeam. 2024. FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs. arXiv preprint arXiv:2407.04051 (2024)
2024 arXiv
-
[13]
Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, and Tat-Seng Chua. 2020. Graph-refined convolutional network for multimedia recommendation with implicit feedback. In Proceedings of the 28th ACM international conference on multimedia. 3541–3549
2020
-
[14]
Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, and Tat-Seng Chua. 2019. MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. In Proceedings of the 27th ACM international conference on multimedia. 1437–1445
2019
-
[15]
Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, YU Chenyun, Bo Hu, Zang Li, et al. 2022. Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems. In Thirty-sixth Conference on Neural Information Processing Systems Datasets ...
2022
-
[16]
Hongyu Zhou, Xin Zhou, Zhiwei Zeng, Lingzi Zhang, and Zhiqi Shen. 2023. A comprehensive survey on multimodal recommender systems: Taxonomy, evalua- tion, and future directions. arXiv preprint arXiv:2302.04473 (2023)
2023 arXiv
-
[17]
Xin Zhou, Donghui Lin, Yong Liu, and Chunyan Miao. 2023. Layer-refined graph convolutional networks for recommendation. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) . IEEE, 1247–1259
2023
-
[18]
Xin Zhou, Hongyu Zhou, Yong Liu, Zhiwei Zeng, Chunyan Miao, Pengwei Wang, Yuan You, and Feijun Jiang. 2023. Bootstrap latent representations for multi-modal recommendation. In Proceedings of the ACM Web Conference 2023 . 845–854
2023
-
[2009]
In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence . 452–461
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.