Pith. sign in

REVIEW 3 major objections 6 minor 25 references

User independent Emotion Recognition with Residual Signal-Image Network

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a residual CNN on cvxEDA-decomposed EDA signal images, fused with music features, reaches 73.65% arousal and 73.43% valence accuracy on the PMEmo dataset and can serve as its user-independent classification baseline.

desk verdict A plausible first classification baseline on the PMEmo EDA dataset, but the missing subject- and song-disjoint cross-validation detail makes the headline accuracy numbers impossible to trust as 'user independent' yet. read the letter →

arxiv 1908.03692 v2 pith:RJZJWLYF submitted 2019-08-10 cs.CV cs.LG

Guanghao Yin , Shouqian Sun , Hui Zhang , Dian Yu , Chao Li , Kejun Zhang , Ning Zou This is my paper · ORCID
classification cs.CVcs.LG
keywords emotionrecognitionelectrodermalactivitycvxEDAsignal-to-imagetransformationRes-SINmulti-featurefusionPMEmodatasetuser-independentclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Res-SIN, an end-to-end pipeline for user-independent emotion recognition from Electrodermal Activity (EDA) signals, and reports numbers it offers as a baseline on the PMEmo dataset. The pipeline decomposes each EDA trace into tonic and phasic components with cvxEDA, converts the original plus the two components into grayscale images, and feeds those images into a residual convolutional subnet whose features are fused with music-derived features before a binary high/low arousal and valence decision. On 7962 EDA traces from 457 subjects, the method reaches 73.65% accuracy for arousal and 73.43% for valence, outperforming EDA-only and music-only inputs in the paper's experiments. A sympathetic reader would care because large-scale cross-user emotion recognition is an open problem, and binary labels are more interpretable than the dataset's original regression fits.

What carries the argument

The central object is the Residual Signal-Image Network (Res-SIN). Its load-bearing elements are cvxEDA, a convex-optimization method that splits a measured EDA trace $y$ into phasic $r$, tonic $t$, and noise; a signal-to-image transform that rearranges each 50 Hz channel into a $T\times 50$ grayscale image rescaled to $224\times224$; a residual signal-image subnet built from stacked $3\times3$ convolutional residual units that outputs a 512-dimensional vector; and a fusion classifier that concatenates this vector with z-scored music features and passes them through linear-ReLU-linear-softmax layers. The decomposition supplies denoising and data augmentation, the image transform makes the signals CNN-compatible while preserving curve shape, and the fusion combines subject-specific EDA features with a general music emotion benchmark.

What would settle it

Using the PMEmo subject IDs, re-run the identical Res-SIN pipeline with 10 folds built by subject so that no subject appears in both training and test sets, and compare accuracy with the reported 73.65% arousal and 73.43% valence; a large drop would show the earlier numbers came from within-subject overlap rather than user-independent learning.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that combining three transformations---cvxEDA decomposition into tonic and phasic components, resizing the three resulting one-dimensional signal channels into $224\times224$ grayscale images, and a residual CNN that outputs a 512-dimensional EDA feature vector fused with a 6373-dimensional music feature vector---produces a user-independent emotion classifier that beats either modality alone. The mixed three-channel input outperforms any single channel; tonic signals give the steadiest F1 score, phasic signals contribute dynamic short-term information, and the music stream accelerates convergence. The authors therefore position the accuracy figures as a classification baseline for PMEmo, filling the gap left by the dataset's regression-only baseline.

Load-bearing premise

The user-independence claim rests on the 10-fold cross-validation being subject-disjoint, but the paper never states that all samples from one subject stay in the same fold; if they do not, the reported accuracies would not demonstrate cross-user generalization.

Editorial extensions

If this is right

  • On PMEmo, the reported accuracies give affective computing a binary classification baseline for high/low arousal and valence, complementing the dataset's regression baseline.
  • Fusing the 512-dimensional EDA image features with the 6373-dimensional music features outperforms either modality alone, indicating the two streams carry complementary information for emotion classification.
  • Mixing the original, tonic, and phasic EDA channels improves accuracy and F1 over any single channel, so cvxEDA decomposition works as both denoising and feature augmentation.
  • The music feature stream also accelerates training convergence, which suggests it acts as an external constraint that regularizes the physiological classifier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never states whether the 10-fold cross-validation is subject-disjoint; if folds are drawn over the 7962 samples, traces from the same subject can appear in both training and test folds, so the user-independence numbers should be re-verified with subject-split folds.
  • A straightforward stress test is to train on a subset of the 457 subjects and test on held-out subjects, then compare with the reported random-fold accuracies; a large drop would indicate the model partly relies on subject-specific baselines.
  • Because the pipeline treats the physiological channel generically, the same decomposition-to-image-to-residual-CNN recipe could in principle be applied to other signals such as EEG or ECG, although transfer would need to be demonstrated and not assumed.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Res-SIN, an end-to-end CNN for user-independent binary emotion classification on the PMEmo dataset. It decomposes EDA signals with cvxEDA into original, phasic, and tonic channels, rescales them to 224x224 images, extracts 512-dimensional features with a ResNet-style subnet, and combines them with 6373-dimensional openSMILE music features in a two-layer classifier. Labels are generated per subject by k-means clustering of static valence/arousal annotations. The reported results are 73.65% arousal and 73.43% valence accuracy using 10-fold cross-validation, which the authors propose as a baseline for large-scale user-independent emotion recognition.

Significance. The paper addresses a real gap: PMEmo is the largest publicly available music-EDA emotion dataset, and a classification baseline on it is useful. The design choices (cvxEDA decomposition as data augmentation, signal-to-image transformation, residual CNN, fusion with music features) are clearly motivated, and the evaluation covers multiple signal components. The paper's value, however, hinges on whether the 10-fold protocol is leak-free; the current description does not guarantee this, so the headline numbers are not yet established. If the protocol is corrected and the numbers hold, this would be a valuable baseline.

major comments (3)
  1. [Section 3.2, Table 2] The 10-fold cross-validation is described only as "10-fold cross-validation method" and "tests of 10 folds" without stating that folds are grouped by subject or by song. Since PMEmo has 7,962 samples from only 457 subjects and 794 songs, a sample-level split will place multiple samples from the same subject (and the same song) in both training and test folds. Section 2.3 explicitly constructs per-subject EDA features, and the music feature vector is identical for every listener of the same song; either source of leakage can inflate the Table 2 accuracies, including the 73.65%/73.43% headline numbers and the claimed fusion improvement. The authors need to state and implement subject-disjoint folds for user-independent claims and song-disjoint folds for the music-fusion comparisons.
  2. [Table 1, Section 3.2 Correlation Analysis] The correlation pre-experiment reports Pearson r values of at most 0.088 (phasic/valence SVR) and roughly 0.04-0.07 for the cvxEDA components; these values are very small in absolute terms, so the claim that phasic and tonic signals "significantly improve correlation coefficient" is not supported by the reported magnitudes. Because this experiment is the only direct evidence for the cvxEDA decomposition's usefulness, the authors should report confidence intervals or significance tests and temper the wording, or their design motivation remains weak.
  3. [Section 2.3, Section 3.2] The experimental setup does not describe how hyperparameters (mini-batch size, initial learning rate, decay schedule, network depth) were chosen. If any of these choices were made after inspecting test-fold results, the reported accuracies are optimistic. The authors should state whether model selection was nested inside the 10-fold procedure and, if so, how the inner validation split was constructed; this is necessary for the results to serve as a trustworthy baseline.
minor comments (6)
  1. [Section 3.2] The word "flods" appears in "After tests of 10 flods" and should be "folds".
  2. [Section 2.3] There are typos in this section: "dosen't" should be "doesn't" and "follwed" should be "followed".
  3. [Table 2] Table 2 reports only point estimates; the authors should report mean and standard deviation over the 10 folds, because the fusion gains over music-only features (3.0 and 3.2 percentage points) may be within fold-to-fold variance.
  4. [Section 3.2, Multi-feature Fusion Analysis] The SVM + Music baseline is under-specified: the authors do not state the SVM kernel, hyperparameters, or whether the 6373-dimensional music features were reduced or selected, all of which are needed for reproducibility.
  5. [Figure 4] The label "Imput" in the figure should be "Input".
  6. [Section 3.2, Multi-feature Fusion Analysis] The sentence stating that music features "are actually meaningless out of subjective specificity" is confusing given that the SVM + Music baseline alone achieves 70.43%/71.49% accuracy in Table 2; the intended meaning should be clarified.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the central classification result is independently computed from labels, EDA images, and music features.

full rationale

The paper's central claim is that Res-SIN classifies high/low arousal and valence on PMEmo with 73.65% and 73.43% accuracy. The label generation in Section 2.2 uses per-subject k-means thresholds on the continuous V/A annotations (inspired by Yin et al. [14]); these labels are an input to training, not an output of the classifier, so the accuracy numbers are not self-definitional. The EDA signal images are produced by cvxEDA [17] decomposition and normalization, and the music benchmark is a 6373-dimensional openSMILE feature vector; neither is constructed from the classification labels. The comparison with Zhang et al. [13], a prior PMEmo paper by overlapping authors, is used only as a regression baseline (Table 1) and as the dataset source; the classification accuracy does not reduce to that baseline by construction. The only notable concern is that the 10-fold cross-validation in Section 3.2 is not described as subject-disjoint or song-disjoint, which could inflate user-independent accuracy; this is an experimental validity risk, not a circularity of the claimed derivation. No quoted equation or definition makes the prediction equal to its input, and no load-bearing argument rests on an unverified self-citation. Score 1 reflects the minor self-citation to the authors' own PMEmo baseline, which is not load-bearing.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the cvxEDA decomposition model (phasic+tonic+noise), on the per-subject k-means labeling, on the validity of the signal-to-image transformation, and on the openSMILE music features. No new physical entities are introduced. The hyperparameters of cvxEDA are not reported, which is a reproducibility gap.

free parameters (4)
  • cvxEDA hyperparameters (alpha, gamma, solver settings)
    The paper cites cvxEDA [17] but does not report the specific hyperparameters used for decomposition; results depend on them.
  • Network training hyperparameters (learning rate, batch size, decay) = lr=0.001, batch=100, decay factor 0.1 every 300 iterations
    Reported as empirically selected, no sensitivity analysis or tuning procedure is described.
  • k in k-means label generation = 2
    Binary high/low classes require k=2; the threshold is derived from each subject's own annotations.
  • Image resize dimension and interpolation = 224x224 bilinear
    Required for ResNet input; no sensitivity analysis for the resizing scheme.
assumptions (4)
  • domain assumption EDA signal is the additive sum of phasic, tonic, and zero-mean Gaussian noise (Eq. 1).
    Basis of cvxEDA decomposition, cited from [17] and [18], assumed for all signals.
  • domain assumption Per-subject k-means clustering with k=2 yields meaningful high/low emotion labels.
    Label generation method from [14] used without validating label quality on PMEmo.
  • domain assumption Signal-to-image transformation via bilinear interpolation preserves discriminative information.
    Justified visually (Fig. 3) but no quantitative comparison beyond classification results.
  • domain assumption openSMILE features represent external emotion benchmarks for the music clips.
    Used as the music modality without independent validation of feature-emotion correspondence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of User independent Emotion Recognition with Residual Signal-Image Network." pith.science (2026). https://pith.science/paper/RJZJWLYF

@misc{pith2026190803692,
  author       = {Pith},
  title        = {Pith review of: User independent Emotion Recognition with Residual Signal-Image Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RJZJWLYF}},
  note         = {Machine review of arXiv:1908.03692}
}
read the original abstract

User independent emotion recognition with large scale physiological signals is a tough problem. There exist many advanced methods but they are conducted under relatively small datasets with dozens of subjects. Here, we propose Res-SIN, a novel end-to-end framework using Electrodermal Activity(EDA) signal images to classify human emotion. We first apply convex optimization-based EDA (cvxEDA) to decompose signals and mine the static and dynamic emotion changes. Then, we transform decomposed signals to images so that they can be effectively processed by CNN frameworks. The Res-SIN combines individual emotion features and external emotion benchmarks to accelerate convergence. We evaluate our approach on the PMEmo dataset, the currently largest emotional dataset containing music and EDA signals. To the best of author's knowledge, our method is the first attempt to classify large scale subject-independent emotion with 7962 pieces of EDA signals from 457 subjects. Experimental results demonstrate the reliability of our model and the binary classification accuracy of 73.65% and 73.43% on arousal and valence dimension can be used as a baseline.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 23 canonical work pages

  1. [1]

    INTRODUCTION The emotion recognition is a research hotspot for the sig- nificant hallmark of intelligent human-computer interaction which has been extended in safe driving monitor[1], mental health[2], or etc.. Emotion recognition using physiological signals can guarantee the reliability because all those internal signals are reacted from the autonomic and...

    work page Pith review arXiv 1908

  2. [2]

    CvxEDA Algorithm We focus on processing large scale EDA data

    MATERIAL AND PROPOSED METHOD 2.1. CvxEDA Algorithm We focus on processing large scale EDA data. Accordingly, the strategy of EDA analysis must be efficient and convenient for processing. CvxEDA[17] provides a novel method us- ing convex optimization and prior probabilities to decompose EDA without preprocessing steps and heuristic solutions[4]. According t...

    work page

  3. [3]

    The re- sults and analysis can validate our idea and explain why our method achieves remarkable performance in large scale data

    EXPERIMENTS In this section, three experiments will be conducted. The re- sults and analysis can validate our idea and explain why our method achieves remarkable performance in large scale data. First, we introduce PMEmo dataset and evaluation criteria. 3.1. Dataset And Evaluation Criteria PMEmo Datase PMEmo is the currently largest dataset[13] with EDA a...

    work page 1940

  4. [4]

    For the specific significance of research, we convert emotion recognition to a binary classification

    CONCLUSION In this paper, we propose an end-to-end CNN framework (Res-SIN) for user-independent emotion recognition. For the specific significance of research, we convert emotion recognition to a binary classification. For conducting deep CNN, the 1-dimension EDA signals are transformed to 2- dimension images. Meanwhile, a novel data augmentation method of c...

    work page

  5. [5]

    Quantifying driver stress: Developing a system for col- lecting and processing bio-metric signals in natural sit- uations,

    Jennifer Healey, Justin Seger, and Rosalind Picard, “Quantifying driver stress: Developing a system for col- lecting and processing bio-metric signals in natural sit- uations,” Biomedical sciences instrumentation , vol. 35, pp. 193–198, 1999

    work page 1999

  6. [6]

    Pervasive and unobtrusive emotion sensing for human mental health,

    Rui Guo, Shuangjiang Li, Li He, Wei Gao, Hairong Qi, and Gina Owens, “Pervasive and unobtrusive emotion sensing for human mental health,” in Pervasive Com- puting Technologies for Healthcare (PervasiveHealth), 2013 7th International Conference on . IEEE, 2013, pp. 436–439

    work page 2013

  7. [7]

    Classification of emotional signals from the deap dataset,

    Giuseppe Placidi, Paolo Di Giamberardino, Andrea Petracca, Matteo Spezialetti, and Daniela Iacoviello, “Classification of emotional signals from the deap dataset,” in International Congress on Neurotechnol- ogy, Electronics and Informatics . SCITEPRESS, 2016, vol. 2, pp. 15–21

    work page 2016

  8. [8]

    Arousal and valence recog- nition of affective sounds based on electrodermal activ- ity,

    Alberto Greco, Gaetano Valenza, Luca Citi, and Enzo Pasquale Scilingo, “Arousal and valence recog- nition of affective sounds based on electrodermal activ- ity,” IEEE Sensors Journal, vol. 17, no. 3, pp. 716–725, 2017

    work page 2017

Show all 25 references
  1. [9]

    Emo- tion recognition based on the multiple physiological sig- nals,

    Ping Gong, Heather T Ma, and Yutong Wang, “Emo- tion recognition based on the multiple physiological sig- nals,” in Real-time Computing and Robotics (RCAR), IEEE International Conference on . IEEE, 2016, pp. 140–143

    2016

  2. [10]

    Eeg-based emotion classification using deep belief networks,

    Wei-Long Zheng, Jia-Yi Zhu, Yong Peng, and Bao- Liang Lu, “Eeg-based emotion classification using deep belief networks,” inMultimedia and Expo (ICME), 2014 IEEE International Conference on . IEEE, 2014, pp. 1– 6

    2014

  3. [11]

    Learning representations from eeg with deep recurrent-convolutional neural networks,

    Pouya Bashivan, Irina Rish, Mohammed Yeasin, and Noel Codella, “Learning representations from eeg with deep recurrent-convolutional neural networks,” arXiv preprint arXiv:1511.06448, 2015

    2015 arXiv

  4. [12]

    Deap: A database for emotion analysis; using physiological signals,

    Sander Koelstra, Christian Muhl, Mohammad So- leymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and Ioannis Pa- tras, “Deap: A database for emotion analysis; using physiological signals,” IEEE Transactions on Affective Computing, vol. 3, no....

    2012

  5. [13]

    http://bcmi.sjtu.edu.cn/ seed/,

    SEED Database, “http://bcmi.sjtu.edu.cn/ seed/,”

  6. [14]

    A multimodal database for affect recognition and implicit tagging,

    Mohammad Soleymani, Jeroen Lichtenauer, Thierry Pun, and Maja Pantic, “A multimodal database for affect recognition and implicit tagging,”IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 42–55, 2012

    2012

  7. [15]

    Yi-Hsuan Yang and Homer H Chen, Music emotion recognition, CRC Press, 2011

    2011

  8. [16]

    Affective video con- tent representation and modeling,

    Alan Hanjalic and Li-Qun Xu, “Affective video con- tent representation and modeling,” IEEE transactions on multimedia, vol. 7, no. 1, pp. 143–154, 2005

    2005

  9. [17]

    The pmemo dataset for music emo- tion recognition,

    Kejun Zhang, Hui Zhang, Simeng Li, Changyuan Yang, and Lingyun Sun, “The pmemo dataset for music emo- tion recognition,” in Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval . ACM, 2018, pp. 135–142

    2018

  10. [18]

    Cross-subject eeg feature selection for emotion recognition using transfer recursive feature elimination,

    Zhong Yin, Yongxiong Wang, Li Liu, Wei Zhang, and Jianhua Zhang, “Cross-subject eeg feature selection for emotion recognition using transfer recursive feature elimination,” Frontiers in neurorobotics, vol. 11, pp. 19, 2017

    2017

  11. [19]

    The circumplex model of affect: An integra- tive approach to affective neuroscience, cognitive devel- opment, and psychopathology,

    Jonathan Posner, James A Russell, and Bradley S Pe- terson, “The circumplex model of affect: An integra- tive approach to affective neuroscience, cognitive devel- opment, and psychopathology,” Development and psy- chopathology, vol. 17, no. 3, pp. 715–734, 2005

    2005

  12. [20]

    Deep learning and its applica- tions to signal and information processing [exploratory dsp],

    Dong Yu and Li Deng, “Deep learning and its applica- tions to signal and information processing [exploratory dsp],” IEEE Signal Processing Magazine, vol. 28, no. 1, pp. 145–154, 2011

    2011

  13. [21]

    cvxeda: A convex optimization approach to electrodermal activity processing,

    Alberto Greco, Gaetano Valenza, Antonio Lanata, Enzo Pasquale Scilingo, and Luca Citi, “cvxeda: A convex optimization approach to electrodermal activity processing,” IEEE Transactions on Biomedical Engi- neering, vol. 63, no. 4, pp. 797–804, 2016

    2016

  14. [22]

    Wolfram Boucsein, Electrodermal activity , Springer Science & Business Media, 2012

    2012

  15. [23]

    Developing a benchmark for emotional analysis of music,

    Anna Aljanaki, Yi-Hsuan Yang, and Mohammad Soley- mani, “Developing a benchmark for emotional analysis of music,” PloS one, vol. 12, no. 3, pp. e0173392, 2017

    2017

  16. [24]

    Deep residual learning for image recognition,

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  17. [25]

    Emotion recogni- tion based on physiological changes in music listening,

    Jonghwa Kim and Elisabeth Andr ´e, “Emotion recogni- tion based on physiological changes in music listening,” IEEE transactions on pattern analysis and machine in- telligence, vol. 30, no. 12, pp. 2067–2083, 2008

    2008

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.