Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

A Chinese Multi-label Affective Computing Dataset Based on Social Media Network Users

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The authors introduce CMACD, the first Chinese affective computing dataset that pairs each user's MBTI personality type with multi-label emotion intensity scores on 566,900 Weibo posts, and they validate it with benchmark classifiers.

desk verdict A plausible gap-filling Chinese MBTI-emotion resource, but the emotion labels need human validation before the utility claim can be trusted. read the letter →

arxiv 2411.08347 v1 pith:OOYFOX7O submitted 2024-11-13 cs.CV cs.AIcs.CLcs.CY

Jingyi Zhou , Senlin Luo , Haofan Chen This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.CY
keywords affectivecomputingmulti-labelemotiondatasetintensityMBTIpersonalityChinesesocialmediaWeiboEQNframeworkBERT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CMACD, a large Chinese affective computing dataset built from 566,900 Weibo posts by 11,338 users who publicly self-identify with one of the 16 MBTI personality types. The paper's claim is that CMACD is the first dataset to combine personality labels with multi-label emotion annotations: each post carries intensity scores for six emotions (anger, fear, happiness, neutral, sadness, surprise) generated by the authors' EQN framework. The authors argue this fills a gap left by scarce Chinese emotion datasets and by English resources such as GoEmotions, which lack personality labels and intensity values. Validation with several classifiers is offered as evidence of the dataset's usability, with BERT reaching 0.9284 top-emotion accuracy and 0.74 to 0.80 accuracy on the four MBTI axes. If the claim holds, the field gains a reusable benchmark for studying how stable personality traits and fine-grained emotional expression relate in Chinese social media text.

What carries the argument

The load-bearing machinery is the EQN (Extended Quantization Network) annotation framework, a BERT-based model trained on a manually annotated single-label Weibo emotion dataset, then adjusted by regression so that originally labeled emotions are set to 1 and EQN-generated values are retained for previously unlabeled emotions, then retrained to predict six emotion intensities per post. This framework is what turns raw posts into multi-label intensity annotations. The dataset itself is the other half of the machinery: user posts are organized into 16 MBTI folders with per-user CSV files, giving future researchers a ready-made structure for personality-aware emotion modeling.

What would settle it

Randomly sample posts from CMACD and have trained annotators assign independent intensity scores for all six emotions, or at least label all present emotions, then compare their judgments to the EQN scores. If agreement on secondary emotions falls near chance, the multi-label intensity portion of the dataset is not validated.

Watch

Extended reading notes

Core claim

The central discovery is a first-of-its-kind resource: a Chinese multi-label affective computing dataset in which the same user's MBTI personality type and per-post emotion intensities are recorded together. Each of the 566,900 posts is labeled with scores in the range 0 to 1 for anger, fear, happiness, neutral, sadness, and surprise; scores below 0.05 are set to zero, so a post can carry several valid emotions at once. The personality labels come from Weibo users' self-identification, screened by manual review of phrases for each MBTI type, and the emotion intensities come from machine annotation with the EQN framework trained on the SMP2020 single-label Weibo emotion dataset. The authors demonstrate utility by reporting multi-label emotion classification (top-label accuracies from 0.6577 for SVM to 0.9284 for BERT), MBTI four-axis binary classification (BERT accuracies 0.7388 to 0.7963), a 1,000-post manual spot check with 83.1 percent agreement on the top emotion label, and Pearson correlation patterns among emotions that match everyday expectations.

Load-bearing premise

The dataset's value depends on EQN's machine-generated intensity scores for all six emotions being accurate enough to serve as ground truth, even though the model was trained on single labels and the manual check only tested the top emotion.

Editorial extensions

If this is right

  • CMACD gives Chinese-language affective computing a single benchmark that joins personality and emotion, so models can be trained and compared on tasks that require both at once.
  • Researchers can use the paired MBTI and emotion-intensity labels to test whether stable personality types show measurable differences in emotional expression, as the paper's statistics suggest.
  • The BERT results on both emotion and MBTI classification provide reference numbers that future systems can be measured against.
  • The dataset supports downstream work in psychology, education, marketing, finance, and politics by supplying Chinese social-media text with quantified emotional states and creator personality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reliability of the dataset's micro-emotion labels is inherited from EQN, so a direct human multi-label agreement study would tell whether the non-top intensity scores are trustworthy or primarily noise; the paper's spot check only verifies the top label.
  • Because EQN was trained on a single-label dataset, the multi-label intensity annotations are likely shaped by the model's learned label correlations; users of CMACD should treat the intensity vectors as model predictions rather than human ground truth.
  • The skewed distribution across MBTI types, with INFP users outnumbering ESTP users by about 12 to 1, means personality classification benchmarks should be read with the class imbalance in mind; a balanced subsample would give a cleaner estimate of personality signal.
  • A natural extension would be to test whether BERT's high emotion accuracy on CMACD transfers to fully human-annotated multi-label Chinese emotion data, which would indicate how much of the reported performance comes from label patterns rather than text content.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces CMACD, a Chinese multi-label affective computing dataset built from Weibo posts. The dataset consists of 566,900 posts from 11,338 users who publicly self-identify with one of 16 MBTI personality types; each post is annotated by the authors' EQN framework with intensity scores for six emotions (anger, fear, happiness, neutrality, sadness, surprise). The authors report MBTI text-classification benchmarks on four binary axes, multi-label emotion classification results, manual spot-check accuracy for top emotion labels, and correlation analyses between MBTI axes and between emotions. The central claim is that CMACD is the first Chinese dataset integrating personality traits with multi-label emotion intensity labels and that it demonstrates strong utility for affective computing.

Significance. If the emotion labels were shown to be reliable, CMACD would fill a genuine gap: a large Chinese-language resource linking personality (MBTI) with multi-label, intensity-valued emotion annotations would be valuable for psychology, NLP, and computational social science. The scale (566,900 posts, 11,338 users) and the inclusion of public user personality tags are notable strengths, as is the authors' attention to privacy (removing URLs, names, and identifiers). The paper also makes a small public sample and the annotation code available, and it provides baseline benchmarks across several models. However, the central value of the dataset rests on the validity of machine-generated emotion intensity labels, and the current validation evidence is insufficient; this is the main factor limiting the paper's contribution.

major comments (5)
  1. [Methods, 'Labeling Emotions with Intensity Scores Using the EQN Framework'; Technical Validation, 'Manual Spot Check…] The six-emotion intensity labels are generated by the authors' own EQN framework, which is trained on the single-label SMP2020-EWECT corpus using a self-training/regression procedure in which the model's own outputs serve as pseudo-labels for previously unlabeled emotions. The only human validation, reported in the Manual Spot Check, compares the top one or top two EQN energy labels against a single human label on 1,000 posts (83.1% top-1, 92.3% top-2). This check does not evaluate the correctness of the remaining four labels, does not compare intensity values, does not test the t=0.05 threshold, and reports no inter-annotator agreement. Since the paper's central contribution is precisely the multi-label intensity annotation, this validation gap is load-bearing and must be addressed with human evaluation of the full emotion vectors (e.g., exact-match accuracy, per-label precision/recall, and intensity agreement).
  2. [Technical Validation, 'Multi-label Classification Experiment to Assess Dataset Usability' and Table 4] The multi-label classification experiments train and evaluate models on the same EQN-generated labels, so the high E1_Acc and Ex_Acc values (e.g., BERT E1_Acc 0.9284) demonstrate that the labels are learnable and internally consistent, but they do not show that the labels correspond to human judgments. This is a circularity concern: the benchmark only proves that a model can reproduce the annotation tool's outputs. The paper needs an external criterion—such as comparison with independent human multi-label annotations, exact-match agreement, or threshold-sensitivity analysis—before claiming the dataset has 'strong utility.'
  3. [Usage Notes and Data Records] The full CMACD dataset is not actually released: Usage Notes state that access requires an email request, and Code Availability states that only a small sample is publicly available on GitHub. Because the emotion labels are machine-generated and cannot be independently audited without the full label vectors, the absence of public release (or an access protocol with clear terms) prevents independent verification of the central claims and limits the reproducibility of Table 4 and of all downstream analyses.
  4. [Data Records and 'Quantitative Emotional Data Analysis'] There are inconsistent post counts in the manuscript: the Abstract and Data Records state 566,900 posts, while Section 5 states '566,950 posts from 11,338 users'; note that 11,338 users × 50 posts = 566,900. In addition, Table 2's axis sums give 7,058 + 4,281 = 11,339, and the same inconsistency appears for all four axes. These arithmetic inconsistencies undermine confidence in the exact statistics and need to be corrected and reconciled.
  5. [Technical Validation, 'MBTI Axis Correlation Test' and 'Evaluation of the Reasonableness of the Overall Distribution…] The Pearson correlation analyses in Figs. 15 and 16 are internal sanity checks: a weak MBTI axis correlation or a 'reasonable' emotion correlation structure is consistent with many possible label-generation schemes, including biased or artifact-prone ones. These analyses do not provide quantitative evidence of annotation accuracy. The paper should either be more cautious in framing these results or replace them with direct human-validation metrics; in particular, the emotion correlation heatmap should not be presented as evidence that the annotation accuracy is 'relatively high.'
minor comments (6)
  1. [Abstract and Background & Summary] The claims that CMACD is 'the first dataset to unify personality and emotion' and 'the first large-scale dataset of Chinese personality traits' should be substantially justified with a more systematic comparison to prior resources; otherwise these claims read as overstatements.
  2. [Methods, 'Labeling Emotions with Intensity Scores Using the EQN Framework' and Fig. 3] The description of the EQN self-training procedure is difficult to follow: the text says the model's outputs are used as pseudo-labels for unlabeled emotions and then a 'regression adjustment' sets original labels to 1, but it is not explained how the regression targets are formed or why this avoids circularity. More detail is needed, especially since reference 20 is an unpublished preprint.
  3. [Quantitative Emotional Data Analysis, Eq. (1)-(10)] There are notation and clarity issues in the formulas: Eq. (2) defines Pnum as a sum over F(i), which is not obviously the number of posts; the subscripts and summation ranges in Eqs. (3)-(10) are garbled; and the threshold t is written inconsistently. These formulas should be rewritten with clear definitions.
  4. [Table 3] There are several typos in Table 3: KNN reports '0.62.90' and AdaBoost J/P reports '0,5701'. These should be corrected, and the table would benefit from a consistent number of decimal places.
  5. [Technical Validation, 'Multi-label Classification Experiment'] The definition of Ex_Acc is unclear: the text says it is 'Accuracy of labels with non-maximum energy scores,' but it likely means something like exact-match or partial-match accuracy over the non-top labels. Please clarify the metric and state how the label threshold is applied for evaluation.
  6. [Figures 15 and 16] The correlation heatmaps are described in the text but the figures themselves are not shown in the provided manuscript; if they are included in the final version, they need color bars and value labels so that the reported correlation strengths can be verified.

Circularity Check

3 steps flagged · score 6.0 of 10

CMACD's novel multi-label intensity annotations are generated by the authors' own EQN model via self-training, and the main quantitative validation re-learns those same machine labels; the independent manual check covers only top-1/top-2 single emotions, so the central multi-label emotion claim partially reduces to the model's own outputs.

  1. self definitional [Methods, 'Labeling Emotions with Intensity Scores Using the EQN Framework' (Fig. 3)]
    "A regression adjustment was then performed on the annotated dataset, setting values of originally labeled emotions to 1 and retaining EQN-generated values for previously unlabeled data. The model within the EQN framework was retrained on this regressed dataset, which significantly improved its detection performance (in our BERT-based EQN model, accuracy rose from 78.3% to 81.2% after the second training). Finally, this enhanced model was used to label each post in the MBTI dataset with multi-label emotions and intensity scores, forming the CMACD dataset."

    For every emotion that is not the single human label supplied by SMP2020-EWECT, the training target is the EQN model's own generated intensity value; those same outputs are then used to annotate CMACD. The novel six-emotion intensity vectors are therefore defined as the model's own predictions, by construction. Retraining on these pseudo-labels and applying the retrained model to CMACD does not add independent evidence that the multi-label intensities correspond to human judgments.

  2. fitted input called prediction [Technical Validation, 'Validation of Six Emotion Labels', '(1) Multi-label Classification Experiment to Assess Dataset Usability', Table 4]
    "We trained and evaluated our emotion dataset using Support Vector Machine (SVM), FastText, TextCNN, TextRNN, and BERT models. To validate the usability and effectiveness of the dataset, the experiment was conducted using the standard or baseline parameter settings for each algorithm or model, without any special optimization. ..."

    The labels used as training and test targets in this experiment are the EQN-generated labels that the paper is trying to validate. A model trained and tested on the same machine-generated labels can achieve high E1_Acc and Ex_Acc (BERT: 0.9284 and 0.8865) simply because the labels are a learnable function of the text, not because they are human-validated emotion states. Thus the quantitative 'validation' is a self-consistency check: the fitted classifiers reproduce the annotator model's own labels, which is statistically forced once the labels are generated deterministically from the text.

1 more flagged steps
  1. self citation load bearing [Background & Summary and Methods; reference 20]
    "Our previous research20 introduced a micro-emotion annotation framework with continuous intensity scores, and this paper applies that framework to annotate micro-emotion data, establishing the Chinese Multi-label Affective Computing Dataset (CMACD) ... For further details, refer to our paper, 'Extended Quantification Network: An Efficient Micro-Emotion Detection and Annotation Framework'20."

    Reference 20 is an unpublished preprint by the same three authors, and it is the sole source for the EQN framework that produces every emotion intensity label in CMACD. The paper's contribution claim—'CMACD, utilizing our published automatic emotion detection annotation tool, is the first to achieve machine-labeled macro-emotions and micro-emotions'—therefore rests on a self-citation chain. The only independent check offered is a 1,000-post manual spot check that tests only the top-1 and top-2 single emotion labels (83.1% and 92.3%), not the multi-label intensity vectors that constitute the dataset's novelty.

full rationale

The MBTI side of the dataset is not circular: users were selected by self-identification and manual review, and the personality classification benchmarks (e.g., BERT axis accuracies 0.74–0.80) are a legitimate test of learnability of those labels. The emotion side is where the circularity lies. The six-emotion intensity scores, including the micro-emotion multi-label structure, are generated by EQN, the authors' own framework, using a self-training procedure in which EQN's own outputs become the regression targets for previously unlabeled emotions. The main validation of these labels is a multi-label classification experiment trained and tested on the same machine-generated labels, which demonstrates only self-consistency. The manual spot check is genuinely independent for the dominant emotion (top-1/top-2 single-label agreement), and that partially breaks the circle, but it does not validate the remaining four intensity values, the 0.05 threshold, or the multi-label combinations. Because the full CMACD is not released (only a small sample), the EQN-generated labels cannot currently be externally audited. In sum, the paper's headline claim of 'strong utility' for the multi-label intensity annotations is partially forced by the annotation pipeline itself: the novel emotional content is, for the non-primary emotions, the model's own output re-presented as ground truth. I find no evidence of intent to deceive; the self-training step is disclosed. The circularity is real but partial, because a meaningful independent anchor exists in the human single-label SMP2020 data and in the limited manual top-emotion check. Score 6 reflects that one or more validation quantities reduce by construction rather than the whole derivation being vacuous.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The dataset has no new physical or mathematical entities. It rests on hand-set thresholds and domain assumptions about self-reported MBTI labels and model-generated emotion labels. The most consequential free parameter is t=0.05, which defines what counts as a valid emotion label and shapes all reported statistics.

free parameters (4)
  • Valid emotion label threshold t = 0.05
    Hand-set threshold that converts continuous EQN intensity scores into valid versus zero labels. It directly affects every label set and all dataset statistics.
  • Post length bounds = 30 to 150 characters
    Hand-selected inclusion criteria that delete posts outside this range, changing the corpus composition.
  • Minimum posts per user = 50
    Users with fewer than 50 posts after preprocessing are excluded; this affects user selection and sample size.
  • Random posts sampled per user = 50
    Exactly 50 posts are randomly sampled per user with no stated seed, which affects reproducibility and representativeness.
assumptions (4)
  • domain assumption SMP2020-EWECT single-label emotion annotations are reliable ground truth.
    The EQN model is trained on this dataset; any labeling error propagates into CMACD.
  • domain assumption Weibo users' self-declared MBTI labels are accurate personality ground truth.
    Selection uses phrases like 'I am an ISTJ'; no psychological test or independent verification is applied.
  • ad hoc to paper EQN's full-label mapping and regression adjustment create valid multi-label intensity distributions rather than artifacts.
    The retraining loop sets originally labeled emotions to 1 and keeps model-generated values for unlabeled emotions, assuming those generated values are meaningful.
  • domain assumption Emotion correlations consistent with everyday intuition are evidence of annotation accuracy.
    The validation concludes correctness from observing negative correlations between angry/happy and happy/sad.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Chinese Multi-label Affective Computing Dataset Based on Social Media Network Users." pith.science (2026). https://pith.science/paper/OOYFOX7O

@misc{pith2026241108347,
  author       = {Pith},
  title        = {Pith review of: A Chinese Multi-label Affective Computing Dataset Based on Social Media Network Users},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OOYFOX7O}},
  note         = {Machine review of arXiv:2411.08347}
}
read the original abstract

Emotion and personality are central elements in understanding human psychological states. Emotions reflect an individual subjective experiences, while personality reveals relatively stable behavioral and cognitive patterns. Existing affective computing datasets often annotate emotion and personality traits separately, lacking fine-grained labeling of micro-emotions and emotion intensity in both single-label and multi-label classifications. Chinese emotion datasets are extremely scarce, and datasets capturing Chinese user personality traits are even more limited. To address these gaps, this study collected data from the major social media platform Weibo, screening 11,338 valid users from over 50,000 individuals with diverse MBTI personality labels and acquiring 566,900 posts along with the user MBTI personality tags. Using the EQN method, we compiled a multi-label Chinese affective computing dataset that integrates the same user's personality traits with six emotions and micro-emotions, each annotated with intensity levels. Validation results across multiple NLP classification models demonstrate the dataset strong utility. This dataset is designed to advance machine recognition of complex human emotions and provide data support for research in psychology, education, marketing, finance, and politics.

Figures

Figures reproduced from arXiv: 2411.08347 by the authors.

Figure 1
Figure 1. Miniature of a user in CMACD. it is a snapshot of a user in CMACD, where the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Workflow for Creating the CMACD Dataset. creating the CMACD dataset involves: manually collecting users with self personality types and their posts, followed by data filtering and cleaning (including user privacy protection) to create the MBTI dataset. Then framework on a high-quality manually annotated emotional Weibo dataset, is applied to label each Weibo post in the MBTI dataset with multiple emotion intensities… view at source ↗
Figure 4
Figure 4. Emotion Annotation Structure in CMACD. First, the trained model is loaded, and its [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (5 more)
Figure 7
Figure 7. Figure 7: Four-axis octupole user distribution. 5. Quantitative Emotional Data Analysis Using the previously developed EQN framework on a manually annotated dataset consisting of six emotion categories to train the annotation model42. This trained model provides multi for each p…
Figure 9
Figure 9. Figure 9: Overall distribution of the number of people with six emotions in the 16 personality [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Distribution of average energy values for all users across the 16 personality types. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Probability Distribution of Each Emotion Across Personality Types. Based on the [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 15
Figure 15. Figure 15: MBTI four-axis correlation heat map. The absolute value of the Pearson correlation [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BlossomPsy: A User-Centric AI System for Adaptive and Engaging MBTI Personality Assessments

    cs.HC 2026-07 conditional novelty 5.0 of 10

    BlossomPsy combines multi-turn LLM dialogue, photo-based questions, a multi-head classifier, and a modified UCB bandit algorithm to deliver MBTI assessments with higher user engagement and preliminary consistency with...

Reference graph

Works this paper leans on

2 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Chinese EmoBank: Building Valence-Arousal Resources for Dimensional Sentiment Analysis,

    1.Malhotra, A. & Jindal, R. XAI Transformer based Approach for Interpreting Depressed and Suicidal User Behavior on Online Social Networks. Cognitive Systems Research.84, 101186 (2024). 2.Munoz, S. & Iglesias, C. a. Detection of the Severity Level of Depression Signs in Text Combining a Feature-Based Framework with Distributional Representations.APPLIED S...

    arXiv 2024

  2. [20]

    International Mother Language Day

    Jingyi Zhou, Senlin Luo, Haofan Chen. Expansion Quantization Network: An Efficient Micro-emotion Annotation and Detection Framework. Preprint at https://doi.org/10.48550/arXiv.2411.06160 (2024). 21.PENGDanling. General Psychology (Revised Edition).Beijing Normal University Press, (2001). 22.Kashani, A., Pfau, J., El-Nasr, M., & IEEE. Assessing the Impact ...

    work page doi:10.48550/arxiv.2411.06160 2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.