Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A data-driven facial coding system outperforms the standard FACS tool at predicting autism from conversations.

desk verdict Promising unsupervised facial coding system, but the clinical comparison leaks test data through component ordering and post hoc k selection. read the letter →

arxiv 2505.24679 v1 pith:AYRCAGLS submitted 2025-05-30 cs.CV

Evangelos Sariyanidi , Lisa Yankowitz , Robert T. Schultz , John D. Herrington , Birkan Tunc , Jeffrey Cohn This is my paper
classification cs.CV
keywords facialexpressioncodingactionunitsdictionarylearningsparse3Dmorphablemodelautismclassificationbehavioranalysisunsupervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes replacing automated FACS coding with a fully unsupervised, data-driven coding system called Facial Basis, whose units are localized 3D facial movements. It argues that automated FACS has structural limits: many Action Units are excluded, detection accuracy is often too low for behavioral research, and AUs are not additive when they combine. The central claim is that a dictionary learned directly from 3DMM expression coefficients can reconstruct all observable facial movement and still produce interpretable units. When these units are used to classify autistic versus neurotypical adolescents from in-person and remote conversations, the system outperforms OpenFace, the most widely used AU detector. If correct, this points toward a behavioral research paradigm that encodes expressions comprehensively without costly manual annotation.

What carries the argument

The central object is the Facial Basis dictionary $W$, learned by sparse dictionary learning from expression vectors $E\varepsilon$ obtained by 3DMM fitting. The optimization minimizes the reconstruction error $\sum_n \|E\varepsilon_n - W z_n\|^2 + \lambda \sum_n \|z_n\|_1$, with the dictionary atoms $w_i$ a priori allocated to one of six facial features so that every unit is localized. Because the units are additive, non-additive AU combinations such as AU 1+4 can receive a dedicated Basis Unit and be reconstructed as a linear sum.

What would settle it

Fit the model to videos in which head pose is artificially varied, such as steady yaw or pitch rotations with a fixed expression, and check whether the learned Basis Unit coefficients track the pose changes; if they do, the expression/pose separation fails and the autism-classification advantage may be partly an artifact of pose leakage.

Watch

Extended reading notes

Core claim

The proposed coding system, Facial Basis, learns a sparse dictionary of localized expression components called Basis Units from 3DMM expression coefficients, without any manual AU labels. Each Basis Unit is constrained to move only landmarks of one facial feature, so the units are interpretable and physically plausible, unlike the global PCA components of a standard 3DMM expression model. The paper reports that this system reconstructs facial expressions as linear sums of units, handles asymmetric expressions that automated FACS omits, and predicts autism diagnosis more accurately than OpenFace in both face-to-face and remote conversational settings. It also finds that remote conversations require more expression components than in-person ones, suggesting that the two contexts differ in behavioral dynamics.

Load-bearing premise

The load-bearing premise is that the face-model fitting step separates facial expression from head pose and personal face shape cleanly; if pose or identity variation leaks into the expression measurements, the learned units and the clinical results built on them are contaminated.

Editorial extensions

If this is right

  • Behavioral researchers could quantify facial behavior comprehensively without manual FACS coding, training the same unsupervised pipeline on any face-model topology.
  • Expression analyses can include asymmetric movements, since Facial Basis provides separate left- and right-side units that automated FACS toolkits typically omit.
  • If the classification results replicate, telehealth-based autism assessment can use facial expression features comparable to in-person assessment, but should expect that the discriminative behaviors differ between settings.
  • The finding that data-driven coding beats OpenFace supports the broader claim that encoding all observable movement, not just a fixed AU repertoire, improves clinical prediction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same dictionary-learning procedure could be retrained on large spontaneous or clinical populations, producing a task-specific coding system that retains interpretability while covering movement types absent from posed-expression training sets.
  • Beyond the paper: because the method is unsupervised and operates from 3DMM coefficients, it could be applied to historical video archives lacking manual labels, enabling retrospective behavioral studies with a consistent coding system.
  • Beyond the paper: the differing component counts needed for in-person versus remote classification suggest a testable hypothesis: the statistical structure of facial behavior changes with communication medium, so telehealth feature-selection should be tuned on remote data rather than transferred from in-person studies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes a data-driven coding system, 'Facial Basis,' as an alternative to automated FACS for facial expression analysis. A sparse dictionary with a localization constraint is learned on 3DMM expression coefficients from CK+ and MMI, producing 50 localized, additively combinable 'Basis Units.' The authors compare this representation with OpenFace AUs, PCA, and local PCA for classifying autism versus neurotypical participants from in-person (n=42) and remote (n=97) conversational videos, reporting classification accuracy as a function of the number of components used. They report that Facial Basis achieves the highest accuracy on both datasets and that the most predictive behaviors differ between the two recording contexts. The paper includes an open-source implementation and a discussion of limitations, including physically implausible units and the reliance on posed training data.

Significance. If the evaluation were properly controlled, this would be a useful contribution: the idea of learning a localized, additive, unsupervised expression dictionary in 3D space is a sensible response to the structural limitations of automated FACS, and the open-source release makes the pipeline usable by others. The dictionary learning objective in Eq. (4) is clearly specified, and the localization constraint is a simple way to enforce interpretability. The comparison against OpenFace and PCA baselines is appropriate in principle, and the clinical application to telehealth autism assessment is timely. The main weaknesses are in the experimental protocol for the clinical claim: the reported 'outperforms' conclusion is not yet supported because of test-data leakage in component ordering and k selection, and the accuracy differences are presented without confidence intervals or significance tests. These issues are fixable within the manuscript's scope, so the paper merits a major revision rather than rejection.

major comments (3)
  1. [Section IV-B, Fig. 4] The reported accuracy curves are computed with selection on the full evaluation datasets. The text states that 'the expression components are ordered according to their magnitude of activation on the datasets used for the experiments,' meaning the k-th component is chosen by the k-th highest magnitude across the entire F2F or R2R sample, including the participant held out in each leave-one-out fold. The final accuracy for each coding system is then the peak of the resulting curve, so k is also selected post hoc from the test set. This violates the nested-CV principle that the authors apply to the SVM C parameter. It is especially damaging for the headline comparison: Facial Basis, PCA, and local PCA can be evaluated at k=5,...,50, while OpenFace stops at 17, so the number of models tried differs and the chance of an inflated peak for the data-driven systems increases. Please re-run the evaluation with all selection steps (component ordering and k) performed inside each training fold, with the held-out participant used only for final evaluation, and report the CV accuracy of the chosen configuration rather than the envelope of several curves.
  2. [Section IV-C.2, Fig. 4] No uncertainty quantification or statistical test accompanies the accuracy differences. With n=42 and n=97 and curves that fluctuate by several percentage points, the observed gaps (e.g., Facial Basis vs OpenFace) may be within sampling variability. Please report confidence intervals and a valid paired test on the leave-one-out predictions (e.g., McNemar's test or a permutation test over participants), separately for F2F and R2R. The claim that Facial Basis 'outperforms the most frequently used AU detector' cannot be evaluated from point estimates alone.
  3. [Section III-A, Eq. (1)] The entire Facial Basis is learned from 3DMM expression coefficients Eε, so any leakage of pose or identity into ε contaminates the Basis Units and the clinical features derived from them. The authors correctly state that pose/identity/expression decoupling is an active research problem, but the paper does not validate its own fitting pipeline. Please add a validation experiment (e.g., identity invariance on neutral images of different people, pose invariance under yaw/pitch variation, or comparison against ground-truth 3D scans) or, if that is outside scope, state explicitly that the clinical results are conditional on the quality of the 3DMM fitting. This is a validity concern distinct from the cross-validation issue.
minor comments (6)
  1. [Section IV-B] The window length Tw used for windowed cross-correlation is never specified; please provide the exact value and the feature-averaging procedure so that the experiments are reproducible.
  2. [Section IV-B, Implementation details] The choice of K=50 and λ=0.2 is described as 'qualitative inspection'; please provide a quantitative criterion or a sensitivity analysis to show that the clinical results are not sensitive to these choices.
  3. [Section IV-C.2] The head-movement-only accuracy of 32% on R2R is below the chance level expected for a balanced two-class problem; please verify the label and feature preprocessing and discuss this result, as it may indicate a methodological artifact.
  4. [Section III-C, Eq. (4)] The ℓ2 norm constraint on the dictionary atoms is stated in prose but not included in the displayed optimization; please include it explicitly.
  5. [Fig. 4] Because OpenFace has at most 17 components, its curve stops at k=17; please make this explicit in the figure or caption and avoid comparing peak accuracies across different k ranges without noting this asymmetry.
  6. [Abstract and Section I] The statement that the dictionary 'reconstructs all observable facial movements' conflicts with Section V's acknowledgment that the training set is limited to posed expressions and cannot capture all spontaneous action; please temper the wording.

Circularity Check

1 steps flagged · score 4.0 of 10

Dictionary learning is genuinely transferable, but the headline Autism-vs-NT accuracy comparison is partly constructed by test-set model selection: component ordering is computed on the full F2F/R2R data and the reported peak k is read from the test accuracy curves.

  1. fitted input called prediction [Section IV-B (Compared coding systems; Classification pipeline) and Section IV-C.2 (Clinical Classification Results, Fig. 4)]
    "For these experiments, the expression components are ordered according to their magnitude of activation on the datasets used for the experiments. That is, the kth expression component of a coding system is the one that had the kth highest magnitude across the F2F and R2R datasets. ... On the F2F sample, the highest classification accuracy is achieved by the Facial Basis ... All methods reach their peak performance with 10-15 expression components on the F2F sample ... All methods other than automated FACS reach their peak performance around 40-50 units."

    The component ordering is computed once on the full F2F and R2R datasets, before the leave-one-out split, so every held-out participant contributes to deciding which components are the first k. The reported best accuracy is then the maximum of the Fig. 4 curves, with k chosen post hoc (10-15 for F2F, 40-50 for R2R). Thus the operating point of the reported result is selected using test-set labels, making the headline accuracy comparison partially self-confirming rather than a prediction for a prespecified feature set. The asymmetry in k ranges (Facial Basis up to 50 components, OpenFace capped at 17) gives the two systems unequal chances to hit a favorable peak.

full rationale

The construction of the Facial Basis itself is not circular: the dictionary W is learned unsupervised on CK+ and MMI (Section IV-A) and then applied to the separate F2F and R2R clinical samples, so the coding system is not fit to the autism labels. The 3DMM expression/pose/identity decoupling is flagged by the authors as an active research problem (Section III-A), and the chosen λ=0.2 and K=50 are stated as qualitative modeling choices, not as predictions. Self-citations to prior work (e.g., [22], [24], [31], [49]) are used for motivation and components, not as a uniqueness theorem that forces the result. The main circularity-adjacent problem is the evaluation protocol: component ordering and k selection use the full F2F/R2R datasets and the test-set accuracy curves, so the reported peak accuracy of Facial Basis versus OpenFace is partly selected on the test data rather than being a fully out-of-sample prediction. This does not invalidate the transferable dictionary-learning idea, but it means the headline clinical claim should be re-verified with a nested selection of k and component ordering before it is taken as an unbiased comparison.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim does not depend on fitted numerical constants beyond the standard sparse-coding hyperparameters, but the choice of lambda, K, and the test-set-selected k all influence the reported accuracy. The main assumptions are about the validity of 3DMM expression fitting and the transfer from posed to spontaneous data.

free parameters (4)
  • sparsity penalty lambda = 0.2
    Set in Section IV-B 'Implementation details' by qualitative inspection of learned components; controls the trade-off between reconstruction and sparsity in Eq (4).
  • dictionary size K = 50
    Set in Section IV-B; authors state increasing K further led to highly similar components, a qualitative criterion, not a validated selection.
  • number of expression components k for classifier = varies 5-50, peak per method/dataset
    In Section IV-B/IV-C.2, classification accuracy is reported for the first k components and the peak k is used for the 'outperforms' claim; this is a post hoc selection on the test data.
  • WCC window size Tw = not reported
    Mentioned in Section IV-B but the numeric value is not given; affects the feature representation.
assumptions (4)
  • domain assumption The 3DMM expression model E spans all facial expressions of interest and is an adequate substrate for a universal coding system.
    Eq (1)-(3) use Eε as the ground-truth expression representation; the authors acknowledge in Section V that BFM's low resolution misses wrinkles and furrows, so the span is incomplete.
  • domain assumption The 3DMM fitting procedure separates expression from pose and identity.
    Section III-A states that this decoupling is an active research problem and depends on the fitting technique; the ResNet distilled from 3DI is assumed accurate enough.
  • domain assumption CK+ and MMI posed-expression datasets are representative of expression variation in spontaneous conversations.
    Section V acknowledges the dictionary is trained only on posed expressions; transfer to spontaneous clinical data is assumed, and the authors list this as a limitation.
  • ad hoc to paper The six facial feature groups (brows, eyes, nose, mouth) are sufficient for the localization constraint, and every meaningful expression unit can be assigned to exactly one of them.
    Section III-C hard-codes this partition a priori; it is not derived from data and restricts the dictionary to feature-localized units.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism." pith.science (2026). https://pith.science/paper/AYRCAGLS

@misc{pith2026250524679,
  author       = {Pith},
  title        = {Pith review of: Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYRCAGLS}},
  note         = {Machine review of arXiv:2505.24679}
}
read the original abstract

The Facial Action Coding System (FACS) has been used by numerous studies to investigate the links between facial behavior and mental health. The laborious and costly process of FACS coding has motivated the development of machine learning frameworks for Action Unit (AU) detection. Despite intense efforts spanning three decades, the detection accuracy for many AUs is considered to be below the threshold needed for behavioral research. Also, many AUs are excluded altogether, making it impossible to fulfill the ultimate goal of FACS-the representation of any facial expression in its entirety. This paper considers an alternative approach. Instead of creating automated tools that mimic FACS experts, we propose to use a new coding system that mimics the key properties of FACS. Specifically, we construct a data-driven coding system called the Facial Basis, which contains units that correspond to localized and interpretable 3D facial movements, and overcomes three structural limitations of automated FACS coding. First, the proposed method is completely unsupervised, bypassing costly, laborious and variable manual annotation. Second, Facial Basis reconstructs all observable movement, rather than relying on a limited repertoire of recognizable movements (as in automated FACS). Finally, the Facial Basis units are additive, whereas AUs may fail detection when they appear in a non-additive combination. The proposed method outperforms the most frequently used AU detector in predicting autism diagnosis from in-person and remote conversations, highlighting the importance of encoding facial behavior comprehensively. To our knowledge, Facial Basis is the first alternative to FACS for deconstructing facial expressions in videos into localized movements. We provide an open source implementation of the method at github.com/sariyanidi/FacialBasis.

Figures

Figures reproduced from arXiv: 2505.24679 by the authors.

Figure 1
Figure 1. The first five components of the expression model used for 3DMM [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The expression encoded by some Facial Basis Units (BUs). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The AU labels (ground truth) from videos of the MMI dataset and the BU coefficients of the corresponding expression units, plotted over time. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Average feature weights of behavioral components that include [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 66 canonical work pages

  1. [1]

    Facial action coding system,

    P. Ekman and W. V . Friesen, “Facial action coding system,” Environ- mental Psychology & Nonverbal Behavior , 1978

    1978

  2. [2]

    Ekman and E

    P. Ekman and E. L. Rosenberg, What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) . Oxford University Press, 04 2005. [Online]. Available: https://doi.org/10.1093/acprof: oso/9780195179644.001.0001

    arXiv 2005

  3. [3]

    Facial Action Coding System - Paul Ekman Group,

    “Facial Action Coding System - Paul Ekman Group,” https://www. paulekman.com/facial-action-coding-system/, accessed: 2024-12-19

    work page 2024

  4. [4]

    Crossing domains for au coding: Perspectives, approaches, and measures,

    I. O. Ertugrul, J. F. Cohn, L. A. Jeni, Z. Zhang, L. Yin, and Q. Ji, “Crossing domains for au coding: Perspectives, approaches, and measures,” IEEE transactions on biometrics, behavior, and identity science, vol. 2, no. 2, pp. 158–171, 2020

    work page 2020

  5. [5]

    Online learning for matrix factorization and sparse coding

    J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding.” Journal of Machine Learning Research, vol. 11, no. 1, 2010

    work page 2010

  6. [6]

    Using telehealth to provide interventions for children with asd: A systematic review,

    Y . L. de Nocker and C. K. Toolan, “Using telehealth to provide interventions for children with asd: A systematic review,” Review Journal of Autism and Developmental Disorders , vol. 10, no. 1, pp. 82–112, 2023

    work page 2023

  7. [7]

    OpenFace: An open source facial behavior analysis toolkit,

    T. Baltru ˇsaitis, P. Robinson, and L.-P. Morency, “OpenFace: An open source facial behavior analysis toolkit,” in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) , Mar. 2016, pp. 1–10. [Online]. Available: https://ieeexplore.ieee.org/document/ 7477553

    work page 2016

  8. [8]

    Automated facial expression recognition based on facs action units,

    J. J. Lien, T. Kanade, J. F. Cohn, and C.-C. Li, “Automated facial expression recognition based on facs action units,” in Proceedings third IEEE international conference on automatic face and gesture recognition. IEEE, 1998, pp. 390–395

    work page 1998

Show all 69 references
  1. [9]

    Coding, analysis, interpretation, and recognition of facial expressions,

    I. A. Essa and A. P. Pentland, “Coding, analysis, interpretation, and recognition of facial expressions,” IEEE transactions on pattern analysis and machine intelligence , vol. 19, no. 7, pp. 757–763, 1997

    1997

  2. [10]

    Measuring facial expressions by computer image analysis,

    M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, “Measuring facial expressions by computer image analysis,” Psychophysiology, vol. 36, no. 2, pp. 253–263, 1999

    1999

  3. [11]

    Automatic analysis of facial affect: A survey of registration, representation, and recognition,

    E. Sariyanidi, H. Gunes, and A. Cavallaro, “Automatic analysis of facial affect: A survey of registration, representation, and recognition,” IEEE transactions on pattern analysis and machine intelligence , vol. 37, no. 6, pp. 1113–1133, 2014

    2014

  4. [12]

    Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition,

    Y . Chang and S. Wang, “Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, pp. 20 385–20 394, iSSN: 2575-7075. [Online]. Available: https://ieee...

    2022

  5. [13]

    FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features,

    Y . Yin, D. Chang, G. Song, S. Sang, T. Zhi, J. Liu, L. Luo, and M. Soleymani, “FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Waikoloa, HI, USA: IEEE, Jan. 2024, pp. 6...

    2024

  6. [14]

    Facial Action Unit Detection and Intensity Estimation From Self-Supervised Representation,

    B. Ma, R. An, W. Zhang, Y . Ding, Z. Zhao, R. Zhang, T. Lv, C. Fan, and Z. Hu, “Facial Action Unit Detection and Intensity Estimation From Self-Supervised Representation,” IEEE Transactions on Affective Computing, vol. 15, no. 3, pp. 1669–1683, Jul. 2024, conference Name: IEEE...

    2024

  7. [15]

    Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection,

    X. Liu, K. Yuan, X. Niu, J. Shi, Z. Yu, H. Yue, and J. Yang, “Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection,” IEEE Transactions on Affective Computing , pp. 1–15, 2024, conference Name: IEEE Transactions on Affective Computing. [Onli...

    2024

  8. [16]

    PyAFAR: Python-based Automated Facial Action Recognition library for use in Infants and Adults

    S. Hinduja, I. O. Ertugrul, M. Bilalpur, D. S. Messinger, and J. F. Cohn, “PyAFAR: Python-based Automated Facial Action Recognition library for use in Infants and Adults.” IEEE Computer Society, Sep. 2023, pp. 1–3. [Online]. Available: https://www.computer.org/ csdl/proceeding...

    2023

  9. [17]

    Multimodal spontaneous emotion corpus for human behavior analysis,

    Z. Zhang, J. M. Girard, Y . Wu, X. Zhang, P. Liu, U. Ciftci, S. Canavan, M. Reale, A. Horowitz, H. Yang et al. , “Multimodal spontaneous emotion corpus for human behavior analysis,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 3438–3446

    2016

  10. [18]

    Affectiva-mit facial expression dataset (am-fed): Naturalistic and spontaneous facial expressions collected,

    D. McDuff, R. Kaliouby, T. Senechal, M. Amr, J. Cohn, and R. Picard, “Affectiva-mit facial expression dataset (am-fed): Naturalistic and spontaneous facial expressions collected,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2013...

    2013

  11. [19]

    Py-feat: Python facial expression analysis toolbox,

    J. H. Cheong, E. Jolly, T. Xie, S. Byrne, M. Kenney, and L. J. Chang, “Py-feat: Python facial expression analysis toolbox,” Affective Science, vol. 4, no. 4, pp. 781–796, 2023

    2023

  12. [20]

    Facial action coding system, a human face,

    P. Ekman, W. V . Friesen, and J. C. Hager, “Facial action coding system, a human face,” ETC What is the ETC , 2002

    2002

  13. [21]

    Foundations of human computing: Facial expression and emotion,

    J. F. Cohn, “Foundations of human computing: Facial expression and emotion,” in Proceedings of the 8th international conference on Multimodal interfaces, 2006, pp. 233–238

    2006

  14. [22]

    Learning bases of ac- tivity for facial expression recognition,

    E. Sariyanidi, H. Gunes, and A. Cavallaro, “Learning bases of ac- tivity for facial expression recognition,” IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1965–1978, 2017

    1965

  15. [23]

    3d morphable face models—past, present, and future,

    B. Egger, W. A. Smith, A. Tewari, S. Wuhrer, M. Zollhoefer, T. Beeler, F. Bernard, T. Bolkart, A. Kortylewski, S. Romdhani et al. , “3d morphable face models—past, present, and future,” vol. 39, no. 5, pp. 1–38, 2020

    2020

  16. [24]

    Inequality- constrained 3d morphable face model fitting,

    E. Sariyanidi, C. J. Zampella, R. T. Schultz, and B. Tunc ¸, “Inequality- constrained 3d morphable face model fitting,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  17. [25]

    Facewarehouse: A 3d facial expression database for visual computing,

    C. Cao, Y . Weng, S. Zhou, Y . Tong, and K. Zhou, “Facewarehouse: A 3d facial expression database for visual computing,” vol. 20, no. 3, pp. 413–425, 2013

    2013

  18. [26]

    Anjyo, Blendshape Facial Animation

    K. Anjyo, Blendshape Facial Animation . Cham: Springer International Publishing, 2018, pp. 2145–2155. [Online]. Available: https://doi.org/10.1007/978-3-319-14418-4 2

    2018 doi

  19. [27]

    Multiple comparisons: philosophies and illustra- tions,

    D. Curran-Everett, “Multiple comparisons: philosophies and illustra- tions,” American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, vol. 279, no. 1, pp. R1–R8, 2000

    2000

  20. [28]

    C. Lord, M. Rutter, P. DiLavore, S. Risi, K. Gotham, and S. Bishop, Autism Diagnostic Observation Schedule, Second Edition: ADOS-2 , 2012

    2012

  21. [29]

    Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions,

    J. C. Koehler, M. S. Dong, A. M. Bierlich, S. Fischer, J. Sp ¨ath, I. S. Plank, N. Koutsouleris, and C. M. Falter-Wagner, “Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions,” Translational Psychiatry, vol. 14, ...

    2024

  22. [30]

    Discriminative few shot learning of facial dynamics in interview videos for autism trait classification,

    N. Zhang, M. Ruan, S. Wang, L. Paul, and X. Li, “Discriminative few shot learning of facial dynamics in interview videos for autism trait classification,” IEEE Transactions on Affective Computing , vol. 14, no. 2, pp. 1110–1124, 2022

    2022

  23. [31]

    Comparison of human experts and ai in pre- dicting autism from facial behavior,

    E. Sariyanidi, C. J. Zampella, E. DeJardin, J. D. Herrington, R. T. Schultz, and B. Tunc, “Comparison of human experts and ai in pre- dicting autism from facial behavior,” in CEUR workshop proceedings, vol. 3359, no. ITAH. NIH Public Access, 2023, p. 48

    2023

  24. [32]

    Computer vision analysis of reduced interpersonal affect coordination in youth with autism spectrum disorder,

    C. J. Zampella, L. Bennetto, and J. D. Herrington, “Computer vision analysis of reduced interpersonal affect coordination in youth with autism spectrum disorder,” Autism Research, vol. 13, no. 12, pp. 2133– 2142, 2020

    2020

  25. [33]

    Atypical head movement during face-to-face interaction in children with autism spectrum disorder,

    Z. Zhao, Z. Zhu, X. Zhang, H. Tang, J. Xing, X. Hu, J. Lu, Q. Peng, and X. Qu, “Atypical head movement during face-to-face interaction in children with autism spectrum disorder,” Autism Research, vol. 14, no. 6, pp. 1197–1208, 2021

    2021

  26. [34]

    Detecting autism by analyzing a simu- lated social interaction,

    H. Drimalla, N. Landwehr, I. Baskow, B. Behnia, S. Roepke, I. Dziobek, and T. Scheffer, “Detecting autism by analyzing a simu- lated social interaction,” in Machine Learning and Knowledge Discov- ery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September...

    2018

  27. [35]

    Toward the detection of reduced emotion expression inten- sity: an autism sibling study,

    B. Turan, P. Algedik Demirayak, E. Yildirim Demirdogen, M. Gulsen, H. C. Cubukcu, M. Guler, H. Alarslan, A. E. Yilmaz, and O. B. Dursun, “Toward the detection of reduced emotion expression inten- sity: an autism sibling study,” Journal of Clinical and Experimental Neuropsychol...

    2023

  28. [36]

    Facial expression analysis in children with autism spectrum disorder using a refined human- robot-game platform for active learning,

    N. I. Sol ´orzano Alc´ıvar, D. F. Paillacho Chiluiza, M. X. Arce Sierra, A. J. Pincay Lino, and E. A. Eras Zamora, “Facial expression analysis in children with autism spectrum disorder using a refined human- robot-game platform for active learning,” Behaviour & Information Tec...

    2024

  29. [37]

    Capturing fear through eyes to aid in restoring social functioning for neuropsychiatric disorders: Machine learning research evidence from the emotion face database,

    Y . Li, H. Liu, H. Feng, X. Shen, Z. Chen, W. Luo, N. Li, and S. Tan, “Capturing fear through eyes to aid in restoring social functioning for neuropsychiatric disorders: Machine learning research evidence from the emotion face database,” 2024

    2024

  30. [38]

    On scalable and interpretable autism detection from social interaction behavior,

    W. Saakyan, M. Norden, L. Herrmann, S. Kirsch, M. Lin, S. Guen- delman, I. Dziobek, and H. Drimalla, “On scalable and interpretable autism detection from social interaction behavior,” in 2023 11th International Conference on Affective Computing and Intelligent In- teraction (A...

    2023

  31. [39]

    Is smiling the key? machine learning analytics detect subtle patterns in micro-expressions of infants with asd,

    G. Alvari, C. Furlanello, and P. Venuti, “Is smiling the key? machine learning analytics detect subtle patterns in micro-expressions of infants with asd,” Journal of clinical medicine , vol. 10, no. 8, p. 1776, 2021

    2021

  32. [40]

    Can facial pose and expression be separated with weak perspective camera?

    E. Sariyanidi, C. J. Zampella, R. T. Schultz, and B. Tunc, “Can facial pose and expression be separated with weak perspective camera?” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 7173–7182

    2020

  33. [41]

    Ekman and E

    P. Ekman and E. L. Rosenberg, What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA, 1997

    1997

  34. [42]

    Learning to detect objects in images via a sparse, part-based representation,

    S. Agarwal, A. Awan, and D. Roth, “Learning to detect objects in images via a sparse, part-based representation,” IEEE transactions on pattern analysis and machine intelligence , vol. 26, no. 11, pp. 1475– 1490, 2004

    2004

  35. [43]

    Sparse coding for learning interpretable spatio-temporal primitives,

    T. Kim, G. Shakhnarovich, and R. Urtasun, “Sparse coding for learning interpretable spatio-temporal primitives,” Advances in neural information processing systems , vol. 23, 2010

    2010

  36. [44]

    The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,

    P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,” in 2010 ieee computer society conference on computer vision and pattern recognition-workshop...

    2010

  37. [45]

    Web-based database for facial expression analysis,

    M. Pantic, M. Valstar, R. Rademaker, and L. Maat, “Web-based database for facial expression analysis,” in 2005 IEEE international conference on multimedia and Expo . IEEE, 2005, pp. 5–pp

    2005

  38. [46]

    Disfa: A spontaneous facial action intensity database,

    S. M. Mavadati, M. H. Mahoor, K. Bartlett, P. Trinh, and J. F. Cohn, “Disfa: A spontaneous facial action intensity database,” IEEE Transactions on Affective Computing, vol. 4, no. 2, pp. 151–160, 2013

    2013

  39. [47]

    Development of the contextual assessment of social skills (cass): A role play measure of social skill for individuals with high- functioning autism,

    A. B. Ratto, L. Turner-Brown, B. M. Rupp, G. B. Mesibov, and D. L. Penn, “Development of the contextual assessment of social skills (cass): A role play measure of social skill for individuals with high- functioning autism,” Journal of Autism and Developmental Disorders , vol. ...

    2011

  40. [48]

    Evaluating the severity of depressive symptoms using upper body motion captured by rgb-depth sensors and machine learning in a clinical interview setting: a preliminary study,

    T. Horigome, B. Sumali, M. Kitazawa, M. Yoshimura, K.-c. Liang, Y . Tazawa, T. Fujita, M. Mimura, and T. Kishimoto, “Evaluating the severity of depressive symptoms using upper body motion captured by rgb-depth sensors and machine learning in a clinical interview setting: a pre...

    2020

  41. [49]

    Predicting autism from head movement patterns during naturalistic social interactions,

    D. Q. McDonald, E. Sariyanidi, C. J. Zampella, E. Dejardin, J. D. Herrington, R. T. Schultz, and B. Tunc, “Predicting autism from head movement patterns during naturalistic social interactions,” in Proceedings of the 2023 7th International Conference on Medical and Health Info...

    2023

  42. [50]

    Windowed cross- correlation and peak picking for the analysis of variability in the association between behavioral time series

    S. M. Boker, J. L. Rotondo, M. Xu, and K. King, “Windowed cross- correlation and peak picking for the analysis of variability in the association between behavioral time series.” Psychological methods , vol. 7, no. 3, p. 338, 2002

    2002

  43. [51]

    Can neurotypical individuals read autistic facial expressions? atypical production of emotional facial expressions in autism spectrum disorders,

    R. Brewer, F. Biotti, C. Catmur, C. Press, F. Happ ´e, R. Cook, and G. Bird, “Can neurotypical individuals read autistic facial expressions? atypical production of emotional facial expressions in autism spectrum disorders,” Autism Research, vol. 9, no. 2, pp. 262–271, 2016

    2016

  44. [52]

    The autism diagnostic observation schedule, module 4: revised algorithm and standardized severity scores,

    V . Hus and C. Lord, “The autism diagnostic observation schedule, module 4: revised algorithm and standardized severity scores,” Journal of autism and developmental disorders , vol. 44, pp. 1996–2012, 2014

    1996

  45. [53]

    A 3d face model for pose and illumination invariant face recognition,

    P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, “A 3d face model for pose and illumination invariant face recognition,” in 2009 sixth IEEE international conference on advanced video and signal based surveillance . Ieee, 2009, pp. 296–301

    2009

  46. [54]

    Expnet: Landmark-free, deep, 3d facial expressions,

    F.-J. Chang, A. Tran, T. Hassner, I. Masi, R. Nevatia, and G. Medioni, “Expnet: Landmark-free, deep, 3d facial expressions,” in 13th IEEE Conference on Automatic Face and Gesture Recognition , 2018

    2018

  47. [55]

    Towards fast, accurate and stable 3D dense face alignment,

    J. Guo, X. Zhu, Y . Yang, F. Yang, Z. Lei, and S. Z. Li, “Towards fast, accurate and stable 3D dense face alignment,” in European Conf. on Computer Vision (ECCV) , 2020

    2020

  48. [56]

    Face recognition in unconstrained videos with matched background similarity,

    L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity,” in CVPR 2011. IEEE, 2011, pp. 529–534

    2011

  49. [57]

    Large-scale celebfaces attributes (celeba) dataset,

    Z. Liu, P. Luo, X. Wang, and X. Tang, “Large-scale celebfaces attributes (celeba) dataset,” Retrieved August, vol. 15, no. 2018, p. 11, 2018

    2018

  50. [58]

    On quantifying facial expression-related atypicality of children with autism spectrum disorder,

    T. Guha, Z. Yang, A. Ramakrishna, R. B. Grossman, D. Hedley, S. Lee, and S. S. Narayanan, “On quantifying facial expression-related atypicality of children with autism spectrum disorder,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP)...

    2015

  51. [59]

    Objective mea- surement of head movement differences in children with and without autism spectrum disorder,

    K. B. Martin, Z. Hammal, G. Ren, J. F. Cohn, J. Cassell, M. Ogihara, J. C. Britton, A. Gutierrez, and D. S. Messinger, “Objective mea- surement of head movement differences in children with and without autism spectrum disorder,” Molecular autism, vol. 9, pp. 1–10, 2018

    2018

  52. [60]

    Detecting autism from head movements using kinesics,

    M. Gokmen, E. Sariyanidi, L. Yankowitz, C. J. Zampella, R. T. Schultz, and B. Tunc ¸, “Detecting autism from head movements using kinesics,” in Proceedings of the 26th International Conference on Multimodal Interaction, 2024, pp. 350–354

    2024

  53. [61]

    Abnormal classical eye- blink conditioning in autism,

    L. L. Sears, P. R. Finn, and J. E. Steinmetz, “Abnormal classical eye- blink conditioning in autism,” Journal of autism and developmental disorders, vol. 24, no. 6, pp. 737–751, 1994

    1994

  54. [62]

    Blink rate and facial orientation reveal distinctive patterns of attentional engagement in autistic toddlers: a digital phenotyping approach,

    P. R. Krishnappa Babu, V . Aikat, J. M. Di Martino, Z. Chang, S. Pero- chon, S. Espinosa, R. Aiello, K. LH Carpenter, S. Compton, N. Davis et al. , “Blink rate and facial orientation reveal distinctive patterns of attentional engagement in autistic toddlers: a digital phenotyp...

    2023

  55. [63]

    Computational measurement of social gaze during naturalistic conversations in autism,

    L. Yankowitz, M. K. Pargi, E. Dejardin, C. J. Zampella, W. Guthrie, J. Pandey, J. Bartley, G. K, D. Chen, D. Q. McDonald, A. Manakiwala, and et al., “Computational measurement of social gaze during naturalistic conversations in autism,” Dec 2024. [Online]. Available: osf.io/pr...

    2024

  56. [64]

    Quantifying atyp- icality in affective facial expressions of children with autism spectrum disorders,

    A. Metallinou, R. B. Grossman, and S. Narayanan, “Quantifying atyp- icality in affective facial expressions of children with autism spectrum disorders,” in 2013 IEEE international conference on multimedia and expo (ICME). IEEE, 2013, pp. 1–6

    2013

  57. [65]

    Stability of feature selection algorithm: A review,

    U. M. Khaire and R. Dhanalakshmi, “Stability of feature selection algorithm: A review,” Journal of King Saud University-Computer and Information Sciences, vol. 34, no. 4, pp. 1060–1073, 2022

    2022

  58. [66]

    Phace: Physics- based face modeling and animation,

    A.-E. Ichim, P. Kadle ˇcek, L. Kavan, and M. Pauly, “Phace: Physics- based face modeling and animation,” ACM Transactions on Graphics (TOG), vol. 36, no. 4, pp. 1–14, 2017

    2017

  59. [67]

    Algorithms for hierarchical clustering: an overview,

    F. Murtagh and P. Contreras, “Algorithms for hierarchical clustering: an overview,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 2, no. 1, pp. 86–97, 2012

    2012

  60. [68]

    Wing loss for robust facial landmark localisation with convolutional neural networks,

    Z.-H. Feng, J. Kittler, M. Awais, P. Huber, and X.-J. Wu, “Wing loss for robust facial landmark localisation with convolutional neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2235–2245

    2018

  61. [69]

    Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face prediction,

    H. Yang, H. Zhu, Y . Wang, M. Huang, Q. Shen, R. Yang, and X. Cao, “Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face prediction,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition , 2020, pp. 601–610

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.