YTCommentVerse: A Multi-Category Multi-Lingual YouTube Comment Corpus

Biswadeep Khan; Hridoy Sankar Dutta

REVIEW 3 major objections 5 minor 20 references

YTCommentVerse: A Multi-Category Multi-Lingual YouTube Comment Corpus

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces YTCommentVerse, a dataset of more than 32 million YouTube comments from 178,000 videos, spanning 15 content categories and over 50 languages.

desk verdict A potentially valuable 32M-comment YouTube corpus whose central statistics don't add up in Table 2; worth a serious referee but not a citation yet. read the letter →

arxiv 2509.11057 v1 pith:NSZ6UGXU submitted 2025-09-14 cs.SI

Hridoy Sankar Dutta , Biswadeep Khan This is my paper

classification cs.SI

keywords YouTubecommentdatasetmultilingualmulti-categorysentimentanalysistoxicitysocialnetworksupvotes

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents YTCommentVerse, a large public corpus of YouTube comments intended to fill a gap in social media datasets for video-sharing platforms. The authors claim it is the first dataset of its kind to combine more than 32 million comments, 15 content categories, and over 50 languages with rich per-comment metadata. The dataset is built by scraping videos and comments from top-ranked channels, with personally identifiable information removed. The authors argue this resource enables research on sentiment, toxicity, engagement, cross-cultural discourse, and fairness audits that were previously difficult because existing YouTube datasets were small or lacked metadata.

What carries the argument

The key object is the dataset itself and the construction pipeline behind it. The authors select top channels by subscriber count from a public channel-ranking service, group them into 15 categories, and use custom web scrapers to enumerate videos and collect comments along with metadata. The dataset's eight-field record structure (video ID, comment ID, anonymized commenter name and channel ID, comment text, upvotes, original channel ID, category) is what carries the argument, since it combines linguistic content with engagement signals and domain labels, making possible the paper's benchmark studies of sentiment, embeddings, and toxicity.

What would settle it

Recalculate the sum of the counts in Table 2 and compare it with the stated total of 32,236,173. If the sum is substantially larger, the language distribution percentages and the total count cannot both be correct; similarly, recomputing English's share from its raw count and the stated total would show whether the 22% figure is consistent.

Watch

Extended reading notes

Core claim

The central claim is that YTCommentVerse provides, to the best of the authors' knowledge, the first large-scale multilingual and multi-category YouTube comment corpus. It contains over 32 million comments collected from 178,000 videos, contributed by more than 20 million unique users across 15 content categories such as Music, News, Education, and Entertainment. Each comment is annotated with video ID, comment ID, anonymized commenter name and channel ID, the comment text, upvote count, original channel ID, and a category label. The paper also reports benchmark analyses: sentiment analysis shows positive engagement overall with Music the most positive category; embedding visualization reveal

Load-bearing premise

The central claim depends on the accuracy and internal consistency of the reported dataset statistics; in particular, Table 2's per-language counts appear to sum to about 40 million while the paper reports 32.2 million total comments, and English is listed as 22% even though 8.9 million is roughly 27.6% of 32.2 million.

Editorial extensions

If this is right

Researchers can use YTCommentVerse to train and evaluate cross-lingual sentiment and toxicity classifiers on naturally occurring video comments, covering high- and low-resource languages.
The combination of category labels, upvotes, and comment length enables studies of engagement patterns and how they vary across content domains.
The large multilingual collection supports construction of benchmarks for language models fine-tuned on video-sharing platform discourse, potentially improving moderation tools.
The dataset can be used for fairness audits, examining whether sentiment or toxicity predictions differ across languages or regions.
It provides a baseline for comparing YouTube comment behavior with other social media platforms that have existing public datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If Table 2's language counts are taken at face value, they sum to roughly 40 million rather than the reported 32.2 million; reconciling this discrepancy could change the reported language shares and the claimed corpus size.
Since channels are selected by subscriber rank, the corpus skews toward popular, top-tier creators; findings about 'typical' YouTube comments may not generalize to the long tail of small channels or to non-English-speaking creator communities.
The upvote variable is right-skewed (most comments receive few votes), so engagement-driven analyses may need to treat upvotes as a sparse or ordinal signal rather than a continuous outcome.
Combining the category labels with language labels could let researchers study how the same topic (e.g., music) is discussed differently across languages, a direction the paper mentions but does not fully explore.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Yours is a dataset paper that could matter if the numbers are right, and the numbers are exactly what I don't trust yet. The headline claim — 32M comments, 50 languages, 15 categories — is plausible and genuinely new relative to the prior datasets they list (the largest looks like 180K). That scale with category labels and upvote metadata would be a real resource for content moderation and cross-cultural work. Credit where due: the paper ships a sample record, compares against prior corpora, and its own Limitations section is candid about the dataset being non-temporal, text-only, and biased toward public comments. Those are honest, useful admissions.

The soft spot is load-bearing. Table 2, the only evidence for the "50+ languages" claim, is internally inconsistent: the per-language counts sum to roughly 40M, not the stated 32.2M; the percentages sum to about 96%, not 100%; and English is reported at 22% although 8.9M/32.2M is 27.6%. That is not a rounding error. Either the total is understated or the language counts are inflated, and because the paper never says which language detector it used or how accurate it was, a reader can't resolve the discrepancy. The stress-test note is right that this directly weakens the first-large-scale-multilingual claim.

Two other things bothered me. First, Section 6 says the full dataset is only available by emailing the authors, while Section 7 says it's on Zenodo as an SQLite file. Those should not both be true. Second, the benchmark analyses are too thin to reproduce: the sentiment model is not named, and the UMAP/DBSCAN parameters are missing. The toxicity analysis at least names Perspective API.

On the whole, the core idea is sound and the resource would be valuable, but the paper as written does not support its own statistics. I would not cite it until the authors release a corrected Table 2, specify the language identification method, and reconcile the access statements. That said, I'd still send it to peer review: the dataset is potentially important, and the problems are fixable with an honest revision. A good referee would ask for exactly those changes.

For your reading group: maybe, if you want to talk about what counts as evidence in dataset papers.

Referee Report

3 major / 5 minor

Summary. The paper introduces YTCommentVerse, a claimed large-scale multilingual and multi-category YouTube comment dataset. The central claims are that the corpus contains over 32 million comments from 178,000 videos, contributed by more than 20 million unique users, spanning 15 YouTube content categories and over 50 languages. The paper describes the data collection process (via SocialBlade-ranked channels), the schema of the released data, basic descriptive statistics (language distribution, upvotes, comment length), and several exploratory benchmark analyses (sentiment, text embeddings, toxicity). It also discusses FAIR principles, ethical considerations, and release mechanisms. The main contribution is the dataset itself, positioned as a more comprehensive public resource than prior YouTube comment datasets.

Significance. If the dataset is as described and made publicly accessible, it would be a useful resource for multilingual social media analysis, content moderation research, and cross-cultural NLP. The paper explicitly targets a gap: most existing YouTube comment datasets are small, single-language, or metadata-poor. The inclusion of comments across many languages and categories, with per-comment metadata, could support benchmark tasks and studies of engagement and toxicity. Credit is due for planning PII redaction, providing a Hugging Face sample, and discussing FAIR principles. However, the dataset's credibility is currently undermined by an apparent internal inconsistency in the language distribution table, which is the primary quantitative evidence for the 'over 50 languages' claim. The exploratory analyses are not presented with enough methodological detail to be independently reproduced, but the dataset, once corrected, could still be a valuable contribution.

major comments (3)

[Section 3, Table 2] The language distribution in Table 2 is internally inconsistent with the stated dataset scale. Summing the counts in the '# ' columns yields approximately 40.68 million comments, not the 32,236,173 reported in Figure 1. The percentages sum to approximately 100.4%, not 100%. Furthermore, English is listed as 22.0% with 8.9M comments, but 8.9M / 32.236M ≈ 27.6%, while 8.9M / 40.68M ≈ 21.9%, suggesting the percentages were computed against a total of about 40.68M. The authors must reconcile the total count, the per-language counts, and the percentages, and explain the discrepancy (e.g., deduplication, language detection overlap, or data collection timing). Without correction, the 'over 32 million' and 'over 50 languages' claims are not trustworthy.
[Section 3, language distribution method] No language identification method is described beyond 'identified by their ISO 639-1 codes.' The paper does not state which detector or model was used, its version, its confidence thresholds, or how code-switched/short comments were handled. This matters because Table 2 includes both a large 'und' category (2.3M) and very low counts for languages such as Hebrew (1.8K) and Kannada (1.2K). Without this information, the reader cannot diagnose the Table 2 inconsistency or assess the reliability of the multilingual composition claim. Add the detection tool, parameters, and a brief validation (e.g., manual sample accuracy).
[Section 6.1 and Section 7] There is a direct contradiction about dataset availability. Section 6.1 states that due to scale, YTCommentVerse 'could only be obtained by directly emailing the authors,' while Section 7 states that it 'is currently uploaded to Zenodo in the form of an SQLite database.' These are incompatible. Since the paper's value depends on the dataset being publicly accessible and reproducible, clarify the current release mechanism and provide a stable DOI/URL for the full dataset, not only a 100K subset.

minor comments (5)

[Abstract / Section 1] The abstract claims 'over 32 million comments' while Table 1 says '30M+'. Use a consistent figure throughout, or explicitly note that the 30M+ is a rounded lower bound.
[Section 2.2 / Figure 1] The sample JSON in Figure 1 contains typos and formatting issues, e.g., 'c om m en t o rC h a nn e l ID' and 'o ri gi na lC han ne lI D'. Also, 'commentor' is misspelled repeatedly; the standard spelling is 'commenter'. These should be corrected in the dataset schema and documentation.
[Section 4] The benchmark experiments (sentiment, embeddings, toxicity) lack reproducibility details: which sentiment model, which embedding model, UMAP parameters, DBSCAN eps/min_samples, and Perspective API version. Since these are presented as 'benchmark results,' at least a brief methodological note is needed. Otherwise, the figures are only illustrative.
[Figure 2/3/4/5] The figures have minimal captions and are not referenced with enough specificity in the text. For instance, Figure 4's UMAP plot omits the number of points shown, perplexity, and whether the 100k sample was balanced by category. Add such details in captions or the text.
[References] Several references are incomplete or inconsistently formatted, e.g., entries with only first author and 'et al.' without full author lists. Also check reference [11] (NBC News) and [13] for proper pagination and access dates.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning: descriptive dataset paper with no derived predictions or fitted inputs.

full rationale

YTCommentVerse is a dataset introduction and descriptive analysis paper. There is no derivation chain in which an output is defined in terms of an input or a fitted parameter is relabeled as a prediction. The central claims are corpus existence, scale, and composition, supported directly by the data collection procedure described in Section 2. The benchmark analyses in Section 4 (sentiment, UMAP embeddings, toxicity) are exploratory summaries, not predictions derived from the data by construction. Table 2 contains arithmetic inconsistencies, but those are internal consistency/accuracy concerns, not circularity: the reported language counts are not defined in terms of the paper's conclusions, and no conclusion is forced by a self-citation. The paper does not invoke any uniqueness theorem or prior work by the same authors to justify its choices. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The main assumptions are domain-level: the accuracy of third-party category labels, the language detection pipeline, and the validity of sentiment/toxicity models. No free parameters are fitted in this dataset paper; all analysis is descriptive.

assumptions (4)

domain assumption SocialBlade category labels are accurate and correspond to YouTube's own video categories.
The dataset's multi-category structure rests entirely on these labels; no validation against YouTube's native categories is described (Section 2.1).
domain assumption The language identification tool used to produce Table 2 is accurate and applied uniformly.
The language distribution is a headline characteristic, but the paper never specifies which tool was used or its error rates.
domain assumption Sentiment and toxicity measurements from the chosen models (e.g., Perspective API) are valid cross-culturally.
The benchmark analysis interprets cross-category sentiment and toxicity differences as meaningful, which presupposes model validity across the 50 languages in the dataset.
domain assumption Public YouTube comments are representative of viewer sentiment and engagement.
The paper uses comments as a lens onto user engagement, though it later acknowledges the bias in limitations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of YTCommentVerse: A Multi-Category Multi-Lingual YouTube Comment Corpus." pith.science (2026). https://pith.science/paper/NSZ6UGXU

@misc{pith2026250911057,
  author       = {Pith},
  title        = {Pith review of: YTCommentVerse: A Multi-Category Multi-Lingual YouTube Comment Corpus},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NSZ6UGXU}},
  note         = {Machine review of arXiv:2509.11057}
}

read the original abstract

In this paper, we introduce YTCommentVerse, a large-scale multilingual and multi-category dataset of YouTube comments. It contains over 32 million comments from 178,000 videos contributed by more than 20 million unique users spanning 15 distinct YouTube content categories such as Music, News, Education and Entertainment. Each comment in the dataset includes video and comment IDs, user channel details, upvotes and category labels. With comments in over 50 languages, YTCommentVerse provides a rich resource for exploring sentiment, toxicity and engagement patterns across diverse cultural and topical contexts. This dataset helps fill a major gap in publicly available social media datasets particularly for analyzing video sharing platforms by combining multiple languages, detailed categories and other metadata.

Figures

Figures reproduced from arXiv: 2509.11057 by the authors.

**Figure 2.** Upvotes distribution for YouTube categories. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 4.** A high-dimensional embedding of YouTube video [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 3.** Figure 3: Sentiment vs. Upvotes distribution for YouTube [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]

**Figure 5.** Figure 5: Toxicity Score for YouTube categories. • Content Moderation: The combination of comment text and engagement metrics such as upvotes offers a valuable resource for developing and validating models to detect toxic, obscene or other harmful content in online discussions. …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 1 canonical work pages

[1]

2022.YouTube Comments Dataset

Atif Aliak. 2022.YouTube Comments Dataset. https://www.kaggle.com/datasets/ atifaliak/youtube-comments-dataset Accessed: 2025-06-01

2022
[2]

Malak Aljabri, Rachid Zagrouba, Afrah Shaahid, Fatima Alnasser, Asalah Saleh, and Dorieh M Alomari. 2023. Machine learning-based social media bot detection: a comprehensive literature review.Social Network Analysis and Mining13, 1 (2023), 20

2023
[3]

Mansour Alsaleh, Abdulrahman Alarifi, Abdul Malik Al-Salman, Mohammed Alfayez, and Abdulmajeed Almuhaysin. 2014. Tsd: Detecting sybil accounts in twitter. In2014 13th International Conference on Machine Learning and Applications. IEEE, 463–469. 5https://huggingface.co/datasets/hridaydutta123/YT-100K 6https://zenodo.org/records/15678816 YTCommentVerse: A M...

arXiv 2014
[4]

breadlicker45. 2023. YouTube Comments 180k. https://huggingface.co/datasets/ breadlicker45/youtube-comments-180k. doi:10.57967/hf/4742 Accessed: 2025-06- 16

work page doi:10.57967/hf/4742 2023
[5]

Nikan Chavoshi, Hossein Hamooni, and Abdullah Mueen. 2016. Debot: Twitter bot detection via warped correlation.. InIcdm, Vol. 18. 28–65

2016
[6]

Giulio Corsi, Elizabeth Seger, et al . 2024. Crowdsourcing the Mitigation of disinformation and misinformation: The case of spontaneous community-based moderation on Reddit.Online Social Networks and Media43 (2024), 100291

2024
[7]

Sindhu Kiranmai Ernala, Moira Burke, Alex Leavitt, and Nicole B Ellison. 2020. How well do people report time spent on Facebook? An evaluation of established survey questions with recommendations. InProceedings of the 2020 CHI conference on human factors in computing systems. 1–14

2020
[8]

Maria Giatsoglou, Despoina Chatzakou, Neil Shah, Christos Faloutsos, and Athena Vakali. 2015. Retweeting activity on twitter: Signs of deception. In Advances in Knowledge Discovery and Data Mining: 19th Pacific-Asia Conference, PAKDD 2015, Ho Chi Minh City, Vietnam, May 19-22, 2015, Proceedings, Part I 19. Springer, 122–134

2015

Show all 20 references

[9]

Aditi Gupta, Ponnurangam Kumaraguru, Carlos Castillo, and Patrick Meier
[10]

Kai Nakamura, Sharon Levy, and William Yang Wang. 2019. r/fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection.arXiv preprint arXiv:1911.03854(2019)

2019 arXiv
[11]

2023.YouTube creators can now dub videos in multiple lan- guages

NBC News. 2023.YouTube creators can now dub videos in multiple lan- guages. https://www.nbcnews.com/tech/tech-news/youtube-creators-can-now- dub-videos-multiple-languages-rcna72047 Accessed: 2025-06-17

2023
[12]

Daniela Perrotta, André Grow, Francesco Rampazzo, Jorge Cimentada, Emanuele Del Fava, Sofia Gil-Clavel, and Emilio Zagheni. 2021. Behaviours and attitudes in response to the COVID-19 pandemic: insights from a cross-national Facebook survey.EPJ data science10, 1 (2021), 17

2021
[13]

Bernhard Rieder, Erik Borra, Òscar Coromina, and Ariadna Matamoros-Fernández
[14]

Flora Sakketou, Joan Plepi, Riccardo Cervero, Henri-Jacques Geiss, Paolo Rosso, and Lucie Flek. 2022. Factoid: A new dataset for identifying misinformation spreaders and political bias.arXiv preprint arXiv:2205.06181(2022)

2022 arXiv
[15]

2021.COVID-19 YouTube Comments Dataset

Seungguini. 2021.COVID-19 YouTube Comments Dataset. https://www.kaggle. com/datasets/seungguini/youtube-comments-for-covid19-related-videos Ac- cessed: 2025-06-01

2021
[16]

Neil Shah, Alex Beutel, Brian Gallagher, and Christos Faloutsos. 2014. Spotting suspicious link behavior with fbox: An adversarial perspective. In2014 IEEE International conference on data mining. IEEE, 959–964

2014
[17]

2025.23 Essential YouTube Statistics You Need to Know in 2025

Jack Shepherd. 2025.23 Essential YouTube Statistics You Need to Know in 2025. https://thesocialshepherd.com/blog/youtube-statistics Accessed: 2025-06-17

2025
[18]

Ahsen Waheed. 2023. YouTube Comments Spam Dataset. https://www.kaggle. com/datasets/ahsenwaheed/youtube-comments-spam-dataset. Accessed: 2025- 06-16

2023
[2014]

InSocial Informatics: 6th International Conference, SocInfo 2014, Barcelona, Spain, November 11-13, 2014

Tweetcred: Real-time credibility assessment of content on twitter. InSocial Informatics: 6th International Conference, SocInfo 2014, Barcelona, Spain, November 11-13, 2014. Proceedings 6. Springer, 228–243

2014
[2023]

Making a living in the creator economy: A large-scale study of linking on YouTube.Social Media+ Society9, 2 (2023), 20563051231180628

2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

2022.YouTube Comments Dataset

Atif Aliak. 2022.YouTube Comments Dataset. https://www.kaggle.com/datasets/ atifaliak/youtube-comments-dataset Accessed: 2025-06-01

2022

[2] [2]

Malak Aljabri, Rachid Zagrouba, Afrah Shaahid, Fatima Alnasser, Asalah Saleh, and Dorieh M Alomari. 2023. Machine learning-based social media bot detection: a comprehensive literature review.Social Network Analysis and Mining13, 1 (2023), 20

2023

[3] [3]

Mansour Alsaleh, Abdulrahman Alarifi, Abdul Malik Al-Salman, Mohammed Alfayez, and Abdulmajeed Almuhaysin. 2014. Tsd: Detecting sybil accounts in twitter. In2014 13th International Conference on Machine Learning and Applications. IEEE, 463–469. 5https://huggingface.co/datasets/hridaydutta123/YT-100K 6https://zenodo.org/records/15678816 YTCommentVerse: A M...

arXiv 2014

[4] [4]

breadlicker45. 2023. YouTube Comments 180k. https://huggingface.co/datasets/ breadlicker45/youtube-comments-180k. doi:10.57967/hf/4742 Accessed: 2025-06- 16

work page doi:10.57967/hf/4742 2023

[5] [5]

Nikan Chavoshi, Hossein Hamooni, and Abdullah Mueen. 2016. Debot: Twitter bot detection via warped correlation.. InIcdm, Vol. 18. 28–65

2016

[6] [6]

Giulio Corsi, Elizabeth Seger, et al . 2024. Crowdsourcing the Mitigation of disinformation and misinformation: The case of spontaneous community-based moderation on Reddit.Online Social Networks and Media43 (2024), 100291

2024

[7] [7]

Sindhu Kiranmai Ernala, Moira Burke, Alex Leavitt, and Nicole B Ellison. 2020. How well do people report time spent on Facebook? An evaluation of established survey questions with recommendations. InProceedings of the 2020 CHI conference on human factors in computing systems. 1–14

2020

[8] [8]

Maria Giatsoglou, Despoina Chatzakou, Neil Shah, Christos Faloutsos, and Athena Vakali. 2015. Retweeting activity on twitter: Signs of deception. In Advances in Knowledge Discovery and Data Mining: 19th Pacific-Asia Conference, PAKDD 2015, Ho Chi Minh City, Vietnam, May 19-22, 2015, Proceedings, Part I 19. Springer, 122–134

2015