REVIEW 4 major objections 5 minor 14 references
Is Micro-expression Ethnic Leaning?
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper argues that ethnicity affects micro-expression recognition and that adding an ethnicity branch to a motion classifier improves emotion prediction.
desk verdict An honest, under-powered preliminary study; the dataset is worth having, but the experiments do not isolate ethnicity as the driver. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is a dual-branch late-fusion architecture fed by an optical flow image. The optical flow image stacks horizontal and vertical motion displacement ($f_x$, $f_y$) with optical strain ($\epsilon$), computed between onset and apex frames, so the same motion input drives two separate backbones: one fine-tuned with emotion labels (Positive/Negative/Surprise) and one with ethnicity labels (Asian/Non-Asian). Their features are concatenated and passed through a fully connected layer so the final emotion prediction can use ethnic context without weight sharing. Ethnic labels come from a semi-automatic pipeline (FaceXFormer with heuristic human-eye correction), and evaluation is leave-one-subject-out with composite database evaluation, reported as macro-F1.
What would settle it
Annotate the same CASME II/SAMM subjects with multiple independent human raters; if the mono-ethnic versus mixed-ethnic performance gap disappears or reverses when only consensus labels are used, the observed ethnic influence is annotation noise rather than ethnicity.
Extended reading notes
Core claim
The central claim is that ethnicity is not a nuisance variable but a source of signal in micro-expression emotion classification. The paper establishes this in two steps. First, in a controlled prima facie design with 16 subjects per group and balanced emotion classes, the Asian-only and non-Asian-only setups score macro-F1 0.4546 and 0.4785 while the mixed-ethnicity setup scores 0.4375, with the mixed setup effectively losing the smaller positive and surprise classes. Second, in the main benchmark on the joint dataset, the plain motion-based ResNet-18 reaches 0.6210 average macro-F1, while the same motion backbone with an added ethnicity branch reaches 0.6639, and fusing a patch-based TinyViT trained on RGB face texture reaches 0.6826. Qualitative activation maps show different action-unit emphasis between Asian and non-Asian samples for happiness and surprise, which the paper presents as evidence against the emotion universality hypothesis.
Load-bearing premise
The load-bearing premise is that the automatically assigned Asian/non-Asian labels are correct, because the mono-ethnic versus mixed-ethnic performance gap is measured on those labels and no inter-annotator agreement or ground-truth validation is reported.
Editorial extensions
If this is right
- A micro-expression model trained on one ethnic group may not transfer to another, so benchmarks should report or control the ethnic mix of training and test subjects.
- Adding an ethnicity-aware branch is a way to recover part of the lost accuracy on this joint dataset, raising macro-F1 from 0.6210 to 0.6639 with motion features alone.
- Ethnicity is learnable from optical flow motion, not only from static RGB face texture, since the motion-only dual-branch model already improves over the baseline.
- The qualitative activation maps suggest that cross-cultural micro-expression analysis should attend to which facial regions are activated, not just which emotion label is produced.
- The combined annotated dataset, while small and imbalanced, is a reusable testbed for further cross-cultural micro-expression studies.
Reading between the lines
- Permuting the ethnicity labels during training would test whether the accuracy gain comes from ethnicity information or simply from adding a second task; if shuffled labels give the same gain, the extra capacity, not ethnicity, is responsible.
- The Asian/non-Asian dichotomy pools small groups such as Indian and Others into the Asian side, so the measured gap likely underestimates the cultural variation present in the data.
- The reported asymmetric AU activation for happiness and surprise implies a testable prediction: an expression classifier trained mostly on non-Asian faces should be systematically less accurate at detecting surprise and positive emotion on Asian faces.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies whether ethnicity influences micro-expression recognition. The authors construct an ethnically annotated joint CASME2-SAMM dataset using FaceXFormer plus heuristic correction, run a "prima facie" experiment comparing mono-ethnic and mixed-ethnic groups under a leave-one-subject-out protocol, and propose a dual-branch framework that adds an ethnicity-classification branch to a motion-based emotion classifier. They report that the mixed-ethnicity condition yields lower macro-F1 than mono-ethnic conditions, and that adding an ethnicity branch improves average macro-F1 from 0.6210 to 0.6639, with a further improvement to 0.6826 when a transformer is used for RGB texture. Qualitative activation maps are presented as evidence for distinct expression patterns across ethnic groups. The paper argues that these results challenge the universality hypothesis and motivate ethnicity-aware micro-expression recognition.
Significance. The question of whether ethnicity affects micro-expression recognition is timely and underexplored; if established convincingly, it would have implications for both affective computing and cross-cultural psychology. The paper also contributes a public code release and an ethnicity-annotated subset of two standard datasets. However, the evidence presented is currently too confounded to support the central claims. The main value of the submission at this stage is as a preliminary investigation that could become publishable after substantial re-analysis, including controlled ablations, significance testing, and label-quality validation.
major comments (4)
- [Section IV-B, Table V, Equations (1)-(4)] The central benchmarking claim that "ethnic context" improves micro-expression recognition is confounded by architecture and loss-function changes. The baseline is a single ResNet-18 trained with one categorical cross-entropy loss, while the proposed method uses two ResNet-18 backbones with late fusion and an additional ethnicity loss Lethnic. The observed improvement from 0.6210 to 0.6639 could therefore be due to extra model capacity, multi-task regularization, or any auxiliary supervision, rather than the semantic content of the ethnicity labels. The same confound affects the RGB-texture comparison: the architecture changes from ResNet-18 alone to ResNet-18 plus TinyViT at the same time as the input modality changes, so the failure/success cannot be attributed to the ethnicity representation. To support the claim, the authors should add ablations that keep the architecture fixed and replace the ethnicity labels with shuffled labels, gender labels, or a second emotion task.
- [Section III-B and Table IV] The prima facie study is confounded by dataset identity. Given the subject distributions in Table I, the Asian-only group is drawn predominantly from CASME2 and the non-Asian-only group from SAMM, so the mixed-ethnicity condition is also a mixed-dataset condition. The observed average macro-F1 drop (0.4546/0.4785 to 0.4375) is small, and no variance or significance testing is reported for this 16-subject-per-condition experiment. This design cannot distinguish the effect of ethnicity from domain shift between datasets. The authors should report per-dataset performance, add error bars over multiple runs or bootstraps, and ideally include a condition that mixes subjects from both datasets while keeping the ethnic composition constant.
- [Section III-A] The reliability of the ethnicity labels is not established. The labels come from FaceXFormer with "heuristic-driven screening to correct some annotations" by human eyes, but no inter-annotator agreement, validation against known subject metadata, or per-sample confidence measures are reported. The mapping of Indian and "Others" subjects to the Asian group is ad hoc and may introduce label noise. If FaceXFormer's errors are correlated with the source database (e.g., it performs differently on CASME2 versus SAMM images), then both the prima facie study and the benchmarking results could reflect label noise rather than ethnicity. The authors should measure annotation accuracy, report confusion on a held-out subset, or at least conduct a sensitivity analysis with alternative label groupings.
- [Section IV-C and Figures 3-4] The qualitative activation-map analysis is based on a single illustrative example per emotion class and per ethnicity group. The text draws strong conclusions, such as "Asians demonstrate activated maps on both sides of the face instead of one" and that different cultures express surprise with different facial muscle regions, but no quantification, subject-level aggregation, or statistical testing is provided. These observations are anecdotal and do not provide reliable evidence for the paper's challenge to the universality hypothesis. At minimum, the authors should show activation maps for multiple subjects and report averaged activation statistics with error bars.
minor comments (5)
- [Abstract] The abstract uses "stereo-ethnicity" while the body consistently uses "mixed ethnicity"; please unify the terminology.
- [Tables I and II] The mapping of "Others" to Asian is described as "manually heuristic" in Table II and as part of "mapping is done for data distribution balancing" in Table I; the two descriptions are confusingly redundant and should be reconciled.
- [Related Work and References] The text refers to "MEGC 2023 [6]" but the cited reference is titled "Megc2024"; please correct the year or the citation.
- [Section IV-A] The prima facie experiments are said to use "ResNet-18 off-the-shelf features" with a random forest classifier, but this detail appears only in the prose and not in Table IV or its caption; please state the exact feature-extraction and classification protocol in the table caption for reproducibility.
- [Section III-B] The abbreviation CDE is used before it is defined; the full phrase "composite database evaluation" appears only later in Section IV. Please define it at first use.
Circularity Check
No circularity: ethnicity labels are empirical inputs, not fitted outputs of the emotion predictor; reported gains are confounded but not constructively circular.
full rationale
The paper does not derive any quantity from a fitted parameter or from its own definitions. Ethnicity labels are produced once by FaceXFormer with heuristic correction (Section III-A) and then used in two distinct ways: (i) to define comparison groups in the prima facie study (Table IV) and (ii) as supervision for an auxiliary branch Lethnic in Equations 1-4. Emotion classification is never computed from the ethnicity label at test time; the final emotion prediction uses merged features Femotion = FC(concat(femotion, fethnicity)) (Eq. 8), where both feature vectors are functions of the input image/optical flow, not of the ground-truth ethnicity label. The improvements in Table V (0.6210 to 0.6639) are therefore empirical findings, not quantities forced by construction. The primary limitations are experimental confounds: ethnicity is correlated with database identity (CASME2 vs SAMM), the dual-backbone model adds capacity and an extra loss, and no shuffled-label or gender auxiliary-task ablation isolates the semantic content of ethnicity. These are validity threats, not circularity under the hard rules requiring an exhibited equation-level reduction. Reference [4] is a self-citation but is used only as background context ('conventional research had emphasised the spotting and recognition tasks [4]') and is not load-bearing for the central claim. Hence no circular step can be exhibited.
Assumptions & free parameters
free parameters (2)
- Ethnicity label correction threshold =
not quantified
- Mapping of 'Others' to Asian =
11 samples
assumptions (3)
- domain assumption The ethnicity labels assigned by FaceXFormer and subsequently corrected by human inspection are accurate enough for the study.
- ad hoc to paper The Asian/non-Asian dichotomy is a meaningful grouping for studying ethnic influence on micro-expressions.
- domain assumption The remapping of fine-grained emotion labels into Positive, Negative, and Surprise preserves the relevant emotional information.
Cite this review
Pith. "Pith review of Is Micro-expression Ethnic Leaning?." pith.science (2026). https://pith.science/paper/ANU535CT
@misc{pith2026250710209,
author = {Pith},
title = {Pith review of: Is Micro-expression Ethnic Leaning?},
year = {2026},
howpublished = {\url{https://pith.science/paper/ANU535CT}},
note = {Machine review of arXiv:2507.10209}
}
read the original abstract
How much does ethnicity play its part in emotional expression? Emotional expression and micro-expression research probe into understanding human psychological responses to emotional stimuli, thereby revealing substantial hidden yet authentic emotions that can be useful in the event of diagnosis and interviews. While increased attention had been provided to micro-expression analysis, the studies were done under Ekman's assumption of emotion universality, where emotional expressions are identical across cultures and social contexts. Our computational study uncovers some of the influences of ethnic background in expression analysis, leading to an argument that the emotional universality hypothesis is an overgeneralization from the perspective of manual psychological analysis. In this research, we propose to investigate the level of influence of ethnicity in a simulated micro-expression scenario. We construct a cross-cultural micro-expression database and algorithmically annotate the ethnic labels to facilitate the investigation. With the ethnically annotated dataset, we perform a prima facie study to compare mono-ethnicity and stereo-ethnicity in a controlled environment, which uncovers a certain influence of ethnic bias via an experimental way. Building on this finding, we propose a framework that integrates ethnic context into the emotional feature learning process, yielding an ethnically aware framework that recognises ethnicity differences in micro-expression recognition. For improved understanding, qualitative analyses have been done to solidify the preliminary investigation into this new realm of research. Code is publicly available at https://github.com/IcedDoggie/ICMEW2025_EthnicMER
Figures
Reference graph
Works this paper leans on
-
[1]
A survey of automatic facial micro-expression analysis: databases, methods, and challenges,
Yee-Hui Oh, John See, Anh Cat Le Ngo, Raphael C-W Phan, and Vishnu M Baskaran, “A survey of automatic facial micro-expression analysis: databases, methods, and challenges,” Frontiers in psychology , vol. 9, pp. 1128, 2018
work page 2018
-
[2]
Lie catching and microexpressions,
Paul Ekman, “Lie catching and microexpressions,” The philosophy of deception, vol. 1, no. 2, pp. 5, 2009
work page 2009
-
[3]
Constants across cultures in the face and emotion.,
Paul Ekman and Wallace V Friesen, “Constants across cultures in the face and emotion.,” Journal of personality and social psychology , vol. 17, no. 2, pp. 124, 1971
work page 1971
-
[4]
Deep learning for micro-expression recognition: A survey,
Yante Li, Jinsheng Wei, Yang Liu, Janne Kauttonen, and Guoying Zhao, “Deep learning for micro-expression recognition: A survey,” IEEE Transactions on Affective Computing , vol. 13, no. 4, pp. 2028–2046, 2022
work page 2022
-
[5]
Universals and cultural differences in the judgments of facial expressions of emotion.,
Paul Ekman, Wallace V Friesen, Maureen O’sullivan, Anthony Chan, Irene Diacoyanni-Tarlatzis, Karl Heider, Rainer Krause, William Ayhan LeCompte, Tom Pitcairn, Pio E Ricci-Bitti, et al., “Universals and cultural differences in the judgments of facial expressions of emotion.,” Journal of personality and social psychology , vol. 53, no. 4, pp. 712, 1987
work page 1987
-
[6]
Megc2024: Acm multimedia 2024 facial micro-expression grand challenge,
John See, Jingting Li, Adrian K Davison, Gen Bing Liong, Moi Hoon Yap, Wen-Huang Cheng, Xiaobai Li, Xiaopeng Hong, and Su-Jing Wang, “Megc2024: Acm multimedia 2024 facial micro-expression grand challenge,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 11482–11483
work page 2024
-
[7]
Cultural facial expressions dynamically convey emotion category and intensity information,
Chaona Chen, Daniel S Messinger, Cheng Chen, Hongmei Yan, Yaocong Duan, Robin AA Ince, Oliver GB Garrod, Philippe G Schyns, and Rachael E Jack, “Cultural facial expressions dynamically convey emotion category and intensity information,” Current biology , vol. 34, no. 1, pp. 213–223, 2024
work page 2024
-
[8]
Sixteen facial expressions occur in similar contexts worldwide,
Alan S Cowen, Dacher Keltner, Florian Schroff, Brendan Jou, Hartwig Adam, and Gautam Prasad, “Sixteen facial expressions occur in similar contexts worldwide,” Nature, vol. 589, no. 7841, pp. 251–257, 2021
work page 2021
Show all 14 references
-
[9]
Emotion in the wilds of nature: The coherence and contagion of fear during threatening group-based outdoors experiences.,
Craig L Anderson, Maria Monroy, and Dacher Keltner, “Emotion in the wilds of nature: The coherence and contagion of fear during threatening group-based outdoors experiences.,” Emotion, vol. 18, no. 3, pp. 355, 2018
2018
-
[10]
The spontaneous expression of pride and shame: Evidence for biologically innate nonverbal displays,
Jessica L Tracy and David Matsumoto, “The spontaneous expression of pride and shame: Evidence for biologically innate nonverbal displays,” Proceedings of the National Academy of Sciences , vol. 105, no. 33, pp. 11655–11660, 2008
2008
-
[11]
Humor detection system for muse 2023: contextual modeling, pesudo labelling, and post-smoothing,
Mingyu Xu, Shun Chen, Zheng Lian, and Bin Liu, “Humor detection system for muse 2023: contextual modeling, pesudo labelling, and post-smoothing,” in Proceedings of the 4th on Multimodal Sentiment Analysis Challenge and Workshop: Mimicked Emotions, Humour and Personalisation, 2...
2023
-
[12]
Facexformer: A unified transformer for facial analysis,
Kartik Narayan, Vibashan VS, Rama Chellappa, and Vishal M Patel, “Facexformer: A unified transformer for facial analysis,” arXiv preprint arXiv:2403.12960, 2024
2024 arXiv
-
[13]
Towards macro-and micro-expression spotting in video using strain patterns,
Matthew Shreve, Sridhar Godavarthy, Vasant Manohar, Dmitry Goldgof, and Sudeep Sarkar, “Towards macro-and micro-expression spotting in video using strain patterns,” in 2009 Workshop on Applications of Computer Vision (WACV) . IEEE, 2009, pp. 1–6
2009
-
[14]
Secrets of optical flow estimation and their principles,
Deqing Sun, Stefan Roth, and Michael J Black, “Secrets of optical flow estimation and their principles,” in 2010 IEEE computer society conference on computer vision and pattern recognition . IEEE, 2010, pp. 2432–2439
2010
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.