{"id":"9ac2a4ce-96b3-4262-ab83-c0d80bc44036","arxiv_id":"1908.10623","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"On three acted speech emotion corpora, feature selection methods such as Fisher, ReliefF, ILFS, and the authors' AFS achieve similar or better unweighted average recall with substantially fewer acoustic features than the full set.","lead":"This paper tests four automatic feature selection methods for speech emotion recognition and finds that much smaller feature subsets can match or slightly beat full feature sets. The result matters for building emotion recognizers that run on low-power health monitoring devices.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported UARs are the best over feature counts/clusters selected on the same LOSO folds used for reporting, so the central 'smaller subset matches full set' claim lacks a held-out check.","rationale":"I read the paper as a comparative evaluation with a practical deployment motivation. The datasets are public, the feature sets are standard, and the method descriptions are detailed enough to reproduce. The central claim, however, rests on the equivalence of the selected-subset UAR to the full-set UAR in Table 3 and Figure 2. The weakest point is that all reported 'best' numbers are maxima over configurations evaluated on the same LOSO folds, so the expected value of the reported UAR is inflated relative to any fixed deployment policy. This is not a critique of the methods themselves; it is a correctness risk in the evaluation protocol. The reader's verdict already flags this, and my read supports it. A nested or held-out evaluation would settle the issue, and the public data make such a check straightforward.","tokens_in":13630,"tokens_out":4142,"duration_ms":46621,"concrete_test":"Run a nested LOSO evaluation on EmoDB (or all three corpora): for each held-out speaker, apply AFS, ILFS, ReliefF, and Fisher to the training speakers only; choose the number of features or SOM cluster by inner LOSO on those training speakers; then evaluate the chosen subset on the held-out speaker's utterances. Pool the per-speaker results into a mean UAR and compare it with the full-feature LOSO baseline. If the nested selected-subset UAR is not within roughly 1 UAR point of the full-feature baseline, or if the selected feature counts are much larger than about 30/88 and 100/988, the paper's headline claim is unsupported. Recompute Table 3 under this protocol and report whether the Fisher/ILFS/ReliefF/AFS ranking persists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 3's 'best UAR' entries and the abstract's 'around 30 of 88 / 100 of 988' claims are produced by looking at the same leave-one-subject-out evaluation that reports the numbers. For AFS this is explicit: Section 3.4 says the cluster with the highest validation accuracy is selected, and Figure 7 plots exactly that best cluster's UAR. For ILFS/ReliefF/Fisher, the rows appear to be the maximum of the LOSO curves in Figure 2 over number of features. This is selection on the test speakers: the reported UAR is a maximum over dozens or hundreds of candidate subsets, not the expected UAR of a fixed selection rule. Under the null that no subset generalizes better than the full set, the maximum of many correlated LOSO estimates will typically land above the full-set estimate, so the observed improvements and the method ranking (Fisher best in 3/6 cases, etc.) can be artifacts of this maximization. The load-bearing claim is that low-resource systems can use small feature subsets without sacrificing accuracy; selection bias directly threatens that claim because a practitioner cannot know in advance which feature count or cluster to deploy. The qualitative motivation and datasets are sound; the protocol needs a nested or independent evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates four feature selection methods—Infinite Latent Feature Selection (ILFS), ReliefF, the generalized Fisher score, and the authors' Active Feature Selection (AFS)—for emotion recognition from speech on three acted corpora (EmoDB, SAVEE, EMOVO) with two standard acoustic feature sets (eGeMAPs, emobase). Classification is performed with a linear SVM in leave-one-subject-out (LOSO) cross-validation, and performance is measured as unweighted average recall (UAR). The central claim is that substantially smaller feature subsets (around 30 of 88 eGeMAPs features and 100 of 988 emobase features) achieve UAR comparable to or better than the full feature set, with the generalized Fisher score providing the best average UAR and ReliefF best on a combined eight-class task.","tokens_in":13856,"tokens_out":3818,"duration_ms":39600,"significance":"If the quantitative claim were validated, the paper would make a useful practical contribution to low-resource and ambient-intelligence emotion recognition, where memory and compute constraints matter. The study has concrete strengths: it uses three public multilingual corpora, two standard acoustic feature sets, a class-imbalance-appropriate metric (UAR), a full-feature baseline, and a comparison of a recent method (AFS) with established ranking methods. The qualitative direction—that feature selection can prune many acoustics features without obvious loss—is plausible and worth investigating. However, as detailed below, the evaluation protocol selects the reported feature subset on the same LOSO data used to report performance, which invalidates the headline numbers as estimates of the accuracy a practitioner would obtain with a pre-specified selection rule.","major_comments":[{"comment":"The central claim that small feature subsets achieve the same or better UAR than the full feature set is not supported by the reported evaluation because the 'best' UAR for each method is selected on the same LOSO folds used to report accuracy. For ILFS, ReliefF, and Fisher, the Table 3 entries are the maximum of the LOSO UAR curves in Figure 2 over the number of features; for AFS, they are the maximum over the evaluated cluster counts. This amounts to choosing the feature subset on the test speakers. Under the null hypothesis that no subset generalizes better than the full set, the maximum of many positively correlated LOSO estimates will typically lie above the full-set estimate, so the observed improvements (e.g., 69.7% vs. 68.5%, 41.0% vs. 37.4%) and the ranking of methods can arise from selection bias alone. The paper needs a nested or independent evaluation: the feature count or cluster index should be selected on training partitions only, then evaluated on held-out folds, and the reported UAR should be the UAR of that fixed selection rule, not a maximum over configurations.","section":"Section 5, Table 3 and Figure 2"},{"comment":"For the AFS method, the protocol explicitly states that the cluster with the highest validation accuracy is selected, and Figure 7 plots the UAR of that best cluster. The AFS entries in Table 3 (e.g., 2 eGeMAPs features at 39.0% UAR for EMOVO, and 21 emobase features at 37.5% UAR for SAVEE) are therefore the maximum over the cluster sizes N = 5, 10, ..., 100. The improvement over baseline (37.4% and 38.1%, respectively) may be regression to the mean. Because a user cannot know in advance which cluster size will be best, the reported UAR does not estimate the performance of the AFS method as a deployable procedure. The authors should report the performance of a fixed rule, such as selecting N on an inner LOOCV loop or using a prespecified N, and then evaluating on the held-out speakers.","section":"Section 5, Table 3 and Section 3.4"},{"comment":"The comparison across methods is not apples-to-apples: for AFS the reported result is the best over 20 cluster sizes, while for the ranking methods it is the best over up to 88 or 988 feature-count thresholds. The number of candidate models searched differs by an order of magnitude, so the ranking of methods in Table 3 (Fisher best in 3 of 6 cases, ILFS in 2, ReliefF in 1, AFS second in 3) is confounded with the number of configurations evaluated. The paper should either compare methods under a common, fixed model-selection budget (e.g., inner cross-validation for both the feature count and the AFS cluster size) or report the full performance curves with a clear statement of how a practitioner would choose the operating point without access to test labels.","section":"Table 3 and Figures 2/7 (comparability of methods)"}],"minor_comments":[{"comment":"The sentence 'It involves clustering the data set into N clusters' is ambiguous on first reading; the paper later clarifies that the clustering is applied to feature dimensions, not data instances. The wording should be updated to avoid confusion.","section":"Section 3.4"},{"comment":"The method name is spelled 'reliefF' in most places but 'relifF' in Table 4; the spelling should be made consistent throughout.","section":"Table 4"},{"comment":"The abstract's claim that 'around 30 out of 88 eGeMAPs features and around 100 out of 988 emobase features suffice' does not correspond to the best configurations in Table 3 (e.g., AFS selects 2 eGeMAPs features for EMOVO, Fisher selects 25). The statement appears to refer to the visual curves in Figure 2; this should be stated explicitly and qualified with the selection-bias caveat.","section":"Abstract and Section 5"},{"comment":"The text states that the SAVEE best UAR is 42.9%, but Table 3 and the confusion matrix in Figure 5 show 42.4%. One of these is a typo and should be corrected.","section":"Section 5, EMOVO paragraph"},{"comment":"The text says the best AFS result for EMOVO is obtained with 70 clusters, while Figure 6 shows that the selected cluster contains only 2 features. The relationship between the number of clusters (70), the 2-feature cluster, and the UAR value of 39.0% should be spelled out in the caption or text.","section":"Section 5, AFS discussion"},{"comment":"The combined-dataset experiment in Table 4 uses a 7+1 emotion setup, but the text does not specify how the feature counts for each method are selected in that setting; the same best-over-LOSO selection issue applies there as well, and the table should state the selection rule.","section":"Section 5 and Table 4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting practical problem and the qualitative finding is plausible, but the evaluation protocol's best-over-LOSO selection is a load-bearing flaw in the reported numbers. I do not doubt the authors' good faith; the issue is methodological and appears fixable with nested or held-out model selection. I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nHave a look at this paper if you want a quick case study in how evaluation protocols can manufacture good results. The authors compare four feature-selection methods (ILFS, ReliefF, Fisher, and their own AFS) on three public emotion corpora with two acoustic feature sets, EmoDB, SAVEE, EMOVO and eGeMAPs/emobase. That comparison is genuinely new, and the paper is clearly written with a sensible low-resource motivation. The datasets and features are standard, so the setup is reproducible.\n\nThe problem is the protocol. The reported UARs are the maximum over feature-subset sizes for the ranking methods, and for AFS the best cluster is selected on the same leave-one-subject-out folds used to report the result. Table 3 and the abstract's \"around 30 of 88\" and \"100 of 988\" claims are read from curves like Figure 2 that plot LOSO performance against number of features. That is selection on test speakers. Under the null that no subset generalizes better than the full set, the maximum of many correlated LOSO estimates will routinely land above the full-set baseline. So the observed improvements and the method ranking (Fisher best in 3 of 6 cases) are uncertain. A practitioner cannot know in advance which feature count or cluster to deploy, which is exactly the setting the paper targets.\n\nI'm not saying the qualitative conclusion is false. It's quite plausible that small feature subsets give nearly the same accuracy as the full set on these acted corpora. But the numbers in the paper are not a clean test of that claim. The fix is straightforward: nested cross-validation, or a held-out test partition used only once to evaluate the selected feature subset. The authors should also report variance or a significance test; without it, differences of a few UAR points are hard to interpret.\n\nThis paper deserves a serious referee, but I'd send it back with a major-revision request, not because the idea is bad but because the central quantitative claim is not yet supported. With a corrected evaluation it could be a useful benchmark for practitioners working on low-resource emotion recognition.","headline":"Useful comparison of feature-selection methods for speech emotion recognition, but the headline accuracy numbers are selected on the same LOSO folds used to report them, so treat them as optimistic until a nested or independent evaluation is done.","tokens_in":14412,"tokens_out":2794,"would_cite":false,"duration_ms":26563,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that speech emotion recognition can match or exceed full-feature accuracy using a fraction of the acoustic features, around 30 of 88 eGeMAPs and 100 of 988 emobase features.","keywords":["feature selection","emotion recognition","speech processing","low-resource devices","eGeMAPs","emobase","unweighted average recall","self-organizing map"],"falsifier":"A direct test is to split the data by speaker, select feature subsets using only the training speakers, and then measure UAR on the held-out speakers. If a 30-feature eGeMAPs or 100-feature emobase subset no longer matches full-feature accuracy under this nested procedure, the claimed resource reduction would be called into question.","tokens_in":13396,"feed_emoji":"🎭","tokens_out":7269,"duration_ms":70690,"temperature":0.7,"pith_summary":"The paper investigates whether automatic emotion recognition from speech can run on low-resource hardware by shrinking acoustic feature sets before classification. It compares four feature-selection methods—Infinite Latent Feature Selection, ReliefF, the generalized Fisher score, and Active Feature Selection—on three acted corpora (German, English, Italian) using two standard acoustic feature sets, eGeMAPs and emobase. Its central finding is that substantially smaller subsets, around 30 of 88 eGeMAPs features and around 100 of 988 emobase features, deliver almost the same unweighted average recall as the full feature set. On individual corpus/feature-set combinations, the best selected subsets usually match or beat the full-set baseline. The paper concludes that pruning features is a viable route to reducing memory and computation in health-monitoring and ambient-intelligence applications.","feed_headline":"Around 30 of 88 speech features rival full emotion recognition","feed_subtitle":"Feature selection cuts speech emotion models to a fraction of their size at similar accuracy.","key_machinery":"The central object is the feature-selection comparison pipeline, built on leave-one-speaker-out (LOSO) cross-validation, a linear-kernel SVM classifier, and Unweighted Average Recall (UAR) as the performance measure. Three ranking-based selectors—ILFS, ReliefF, and the generalized Fisher score—score individual features and are evaluated at increasing feature-count cutoffs, while Active Feature Selection (AFS) clusters features with a self-organizing map and picks the cluster with the highest validation accuracy. The mechanism that carries the argument is the observation that UAR curves flatten well before the full feature count, meaning most of the discriminative information is concentrated in a small subset. This is what licenses the conclusion that a model trained on about 30 eGeMAPs or 100 emobase features can substitute for the full set on low-power devices.","core_discovery":"The paper's central claim is that feature selection can preserve or slightly improve emotion recognition accuracy while cutting the feature count by roughly two-thirds to nine-tenths. With eGeMAPs, around 30 of 88 features suffice; with emobase, around 100 of 988 suffice. Across the six corpus/feature-set combinations, the best subset for each method usually matched or exceeded the full-feature baseline, with a few exceptions, most notably Active Feature Selection on SAVEE. When all three datasets were pooled into an eight-class task, ReliefF gave the highest UAR (48.0% on emobase and 46.6% on eGeMAPs). The discovery is meant to show that model size and computational load can be reduced without sacrificing recognition performance.","pith_inferences":["An implication the authors leave implicit is that a system trained on the small subsets could run with much lower latency on wearable or ambient sensors, making real-time emotion monitoring in assisted-living settings more practical than the accuracy numbers alone suggest.","Because the best feature subset differs across corpora and languages, a single universal small acoustic feature set is unlikely; practical deployments may need to run feature selection on target-domain data.","The flatness of the UAR curves suggests that many of the 988 emobase features are redundant; a testable extension is to compare the selected subsets with random subsets of the same size, which would indicate whether the gains come from selector quality or from sheer redundancy.","The paper names fusing features from several AFS clusters as future work; one natural extension is to check whether combining the top two or three clusters improves on the single-cluster result."],"forward_implications":["Deploying emotion recognition on low-power devices becomes feasible: a classifier using roughly 30 eGeMAPs or 100 emobase features needs a fraction of the memory and computation of an 88- or 988-feature model.","Feature selection can serve as an alternative to feature transformation methods such as PCA when interpretability or resource constraints matter.","The best method depends on the data: the generalized Fisher score wins most per-corpus comparisons, while ReliefF wins when corpora from different languages are pooled, so method choice should be corpus-dependent.","Active Feature Selection can reach competitive accuracy with very small subsets (for example, 2 eGeMAPs features on EMOVO), suggesting strong redundancy in these acoustic feature sets.","Reducing features does not necessarily reduce accuracy, so model-size reduction and recognition performance are not in direct tension in this setting."],"supporting_citations":[{"why":"Introduces the Active Feature Selection method that is compared against the other selectors.","marker":"[16]"},{"why":"Supplies the ILFS method that is evaluated as one of the state-of-the-art feature selectors.","marker":"[39]"},{"why":"Supplies the generalized Fisher score method that is evaluated as one of the feature selectors.","marker":"[33]"},{"why":"Supplies the ReliefF method that is evaluated as one of the feature selectors.","marker":"[42]"},{"why":"Defines the eGeMAPs acoustic feature set used in all experiments.","marker":"[19]"},{"why":"Provides the openSMILE toolkit used to extract both eGeMAPs and emobase features.","marker":"[47]"},{"why":"Supplies the EmoDB corpus used as one of the three emotion recognition datasets.","marker":"[44]"},{"why":"Supplies the SAVEE corpus used as one of the three emotion recognition datasets.","marker":"[45]"},{"why":"Supplies the EMOVO corpus used as one of the three emotion recognition datasets.","marker":"[46]"}],"fun_headline_variants":["Feature selection cuts speech emotion models by up to 90%","Emotion recognition maintains accuracy with far fewer speech features","Tiny speech feature sets keep emotion detection robust","Cutting speech features to a fraction doesn't hurt emotion recognition","Spare speech features: emotion recognition still works"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains come from picking the feature subset that performed best on the same leave-one-speaker-out evaluation used to report accuracy, so the result assumes that this selection would also pick a good subset on new data.","fun_headline_variants_meta":{"raw":{"variants":["Feature selection cuts speech emotion models by up to 90%","Emotion recognition maintains accuracy with far fewer speech features","Tiny speech feature sets keep emotion detection robust","Cutting speech features to a fraction doesn't hurt emotion recognition","Spare speech features: emotion recognition still works"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001507,"raw_usage":{"total_tokens":6041,"prompt_tokens":941,"completion_tokens":5100,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":5022}},"tokens_in":557,"tokens_out":5100,"duration_ms":36402,"temperature":1.0,"reasoning_tokens":5022,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:37:34.020993+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to split the data by speaker, select feature subsets using only the training speakers, and then measure UAR on the held-out speakers. If a 30-feature eGeMAPs or 100-feature emobase subset no longer matches full-feature accuracy under this nested procedure, the claimed resource reduction would be called into question.","supporting_citations":[{"cited_title":"Haider, S","cited_arxiv_id":null,"evidence_quote":"Introduces the Active Feature Selection method that is compared against the other selectors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ILFS method that is evaluated as one of the state-of-the-art feature selectors."},{"cited_title":"Robnik- ˇSikonja, I","cited_arxiv_id":null,"evidence_quote":"Supplies the ReliefF method that is evaluated as one of the feature selectors."},{"cited_title":"Eyben, K","cited_arxiv_id":null,"evidence_quote":"Defines the eGeMAPs acoustic feature set used in all experiments."},{"cited_title":"Emotion Recognition in Low-Resource Settings: An Evaluation of Automatic Feature Selection Methods","cited_arxiv_id":"1908.10623","evidence_quote":"Provides the openSMILE toolkit used to extract both eGeMAPs and emobase features."},{"cited_title":"Burkhardt, A","cited_arxiv_id":null,"evidence_quote":"Supplies the EmoDB corpus used as one of the three emotion recognition datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SAVEE corpus used as one of the three emotion recognition datasets."},{"cited_title":"Costantini, I","cited_arxiv_id":null,"evidence_quote":"Supplies the EMOVO corpus used as one of the three emotion recognition datasets."}],"review_version":1}