{"id":"1054f9d9-8075-481a-aa39-d3fd212a7b34","arxiv_id":"2505.23427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Head-motion kineme features generalize across three depression datasets for classification and severity regression, outperforming raw head pose and facial cues in cross-dataset tests.","lead":"This paper tests whether simple head-motion patterns, called kinemes, can estimate depression severity consistently across three Western datasets recorded in Germany, Australia, and the US. The authors report that kineme features transfer across datasets better than raw head pose or facial features, and achieve competitive accuracy on the AVEC2013 benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unvalidated BDI/HRSD-to-QIDS label conversion makes the regression-based generalisability claim untestable as reported.","rationale":"The reader's weakest assumption (subject-disjoint splits) is a legitimate threat to the within-dataset and combined-dataset cross-validation results in Tables 2 and 4, but the paper's cross-dataset generalisability argument rests primarily on separate train-test experiments, where train and test are different datasets and no subject overlap exists. The more load-bearing vulnerability is the regression label conversion. The paper converts BDI and HRSD to QIDS-SR without presenting the conversion table, a reference, or validation; BDI and QIDS are different instruments with different item content, so a simple table-based mapping is unlikely to be clinically valid. This directly undermines the regression half of the abstract's central claim, because the MAE/RMSE values used for cross-method comparisons in Tables 5 and 7 depend on this unvalidated conversion. Additionally, the paper does not clearly state which label scale is used for the AVEC test-set comparison in Table 8; if Table 8 uses original BDI while Tables 4-5 use QIDS, the reported 'second-best MAE' is not directly comparable to the generalisability results. A concrete check - validating the conversion against paired clinical data and re-running the comparisons on each scale separately - would settle whether the regression generalisability claim holds. The verdict remains conditional, since these are addressable validity gaps rather than outright contradictions, but they need to be resolved before the claim can be accepted.","tokens_in":14535,"tokens_out":10955,"duration_ms":97917,"concrete_test":"Obtain paired BDI and QIDS-SR scores from a clinical sample (or from the original AVEC/Pitt data if item-level responses are available) and compute agreement (e.g., ICC, Bland-Altman) of the exact conversion table used; then re-run the regression experiments in Tables 5 and 7 with (a) the original BDI/HRSD labels and (b) the converted QIDS labels separately, reporting MAE/RMSE on each scale and verifying whether the relative ranking of kinemes versus the comparison methods is preserved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that kineme features are more generalisable than other visual cues for depression severity estimation (regression) rests on experiments in Tables 4, 5, and 7, which convert AVEC's BDI and Pitt's HRSD labels to QIDS-SR using an IDS/QIDS conversion table (Section 4). No validation, clinical reference, or error analysis for this conversion is provided, and BDI and QIDS are distinct instruments, so the mapping is not a standard cross-walk. The scale ambiguity is acute: Table 4 reports AVEC CV MAE 3.16, likely on QIDS (range 0-27), while Table 8 reports AVEC test MAE 5.68, presumably on BDI (range 0-63), and Table 7 claims to use evaluation settings identical to [31] without stating which label scale was used. If the conversion is invalid or applied inconsistently, the MAE differences in Tables 5 and 7 cannot be attributed to feature generalisability, and the regression half of the central claim collapses. The reader's subject-disjoint concern is valid but secondary: separate train-test cross-dataset experiments (Tables 3, 5, 6, 7) have no subject overlap, so they are unaffected by CV leakage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript examines whether head-motion-based \"kinemes\"—elementary motion atoms discovered via NMF and GMM clustering—can serve as generalisable depression biomarkers across three Western datasets (German AVEC2013, American Pitt, Australian Blackdog). Kinemes are learned only from low-depressed/healthy subjects, and all videos are represented by statistical descriptors of the reconstruction error between actual and kineme-reconstructed head pose segments. The authors evaluate binary classification and severity regression using k-fold cross-validation and cross-dataset train-test reuse, and claim that (1) kinemes are efficient biomarkers (second-best MAE on AVEC2013) and (2) kineme features generalise better than raw head pose descriptors for classification and than other visual cues for regression.","tokens_in":14800,"tokens_out":9056,"duration_ms":85555,"significance":"If the results are valid, this is a practically useful contribution: it suggests that a compact, interpretable head-motion representation can rival much more complex deep facial models on cross-cultural depression severity estimation, and it provides one of the few three-dataset generalisability studies in the field. The kineme construction is non-circular and transparent, and the use of three public datasets is a clear strength. However, unresolved issues around label scaling, subject-disjoint evaluation, and post-hoc model selection currently prevent the reported effect sizes from being taken at face value.","major_comments":[{"comment":"The conversion of AVEC BDI and Pitt HRSD labels to QIDS-SR via an IDS/QIDS conversion table is not validated and appears to be applied inconsistently. If Table 4's AVEC CV MAE of 3.16 is on the converted QIDS-SR scale (range 0-27) while Table 8's AVEC test MAE of 5.68 is compared against prior works that evaluate on the original BDI scale (range 0-63), the 'second best MAE' claim is not supported because the metrics are not commensurable. The authors should either (i) validate the cross-instrument mapping against clinical concordance data, (ii) rerun all regression experiments on the original instrument for each dataset, or (iii) restrict the claim to comparisons within the same label scale.","section":"Section 4, Tables 4-5 and 7-8"},{"comment":"The k-fold cross-validation splits videos, not participants, and the manuscript does not state that folds are subject-disjoint. Since AVEC contains 150 videos from 82 participants and Pitt contains 148 videos from 49 participants, the same person can appear in both training and test folds, which inflates the reported within-dataset F1/MAE values (e.g., AVEC F1 = 0.93 in Table 2) and weakens the claim that kinemes are efficient biomarkers for same-distribution data. The cross-dataset separate train-test experiments (Tables 3 and 5) are not affected by this leakage, but the paper should either report subject-disjoint folds or explicitly qualify the within-dataset results.","section":"Section 5.3, Tables 2 and 4"},{"comment":"The model and chunk-size selection is post hoc: Section 5.4 states 'we report the best results obtained' among Random Forest, SVM, and XGB, and Section 6 states that results are compiled with a chunk size of 60s after trying 60/75/90/120s. Without a predetermined held-out validation protocol, these choices can overfit the test data and bias the comparisons in Tables 6-8. Please describe the selection procedure (e.g., nested cross-validation or a separate validation fold) and separate it from the final evaluation on the test set.","section":"Sections 5.3-5.4, Tables 2-5"},{"comment":"The key classification comparison with prior work is not metric-aligned: the authors report accuracy, F1, precision, and recall for their kineme method, but the baseline column from Alghowinem et al. [30] only provides 'Average Recall.' The claim that kineme features are more generalisable than raw head pose features for classification therefore rests on comparing F1 or accuracy with average recall, which is not a like-for-like comparison. Please re-evaluate the prior method on the same metrics (or obtain the same metrics from the original implementation) or explicitly justify why a cross-metric comparison is meaningful.","section":"Section 6.1, Table 6"}],"minor_comments":[{"comment":"'5-repetitions of 10-fold cross-validation (50 runs)' is imprecise; five repeats of a 10-fold CV yields 50 fold evaluations, not 50 independent runs.","section":"Section 5.3"},{"comment":"The dataset name appears as 'A VEC' in the first paragraph of Section 4 and in several tables, but as 'AVEC' elsewhere; please standardise the spelling.","section":"Section 4 and throughout"},{"comment":"The F1 value 0.77 attributed to Table 3 for the AVEC-conf condition actually appears in Table 2 (row 'All three'); the cross-referencing should be corrected.","section":"Section 6, third bullet"},{"comment":"There is a typo in the title: 'Comparison of Beck depression inventories-ia and-ii' should read 'Comparison of Beck Depression Inventories-IA and -II.'","section":"Reference [34]"},{"comment":"The captions state 'Lowest RMSE/MAE values per row are denoted in bold,' but no bold formatting appears in the provided text; please ensure the final typeset version matches the captions.","section":"Tables 5 and 7 captions"}],"recommendation":"major_revision","confidential_remarks":"The label conversion issue is the most serious technical obstacle; if the authors cannot provide a clinical validation of the BDI/HRSD-to-QIDS mapping, the regression generalisability claim should be reduced to per-dataset results. The paper also needs to address subject-disjoint splits and the post-hoc selection protocol. These are correctable within the scope of a revision, so I recommend major_revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, it is a legitimate extension of the authors' own kineme work: they add ordinal severity regression and a three-dataset cross-corpus evaluation, which the prior paper didn't do. Second, the headline claim about regression generalisability is not fully supported as reported, because the severity labels for AVEC and Pitt are converted to QIDS-SR via an IDS/QIDS table with no validation, and the cross-validation splits are not stated to be subject-disjoint. The separate train-test experiments are the more trustworthy part.\n\nThe kineme pipeline is inherited from [20], but the new content is real: a systematic comparison across AVEC2013, Blackdog, and Pitt, with both k-fold and model-reuse protocols. The design of learning kinemes only from healthy/low-depressed subjects and then computing reconstruction errors for both classes is clean and avoids circularity in feature construction. The AVEC test MAE of 5.68 beats head-pose baselines and is competitive with facial-feature methods. The separate train-test results (Tables 3, 5, 6, 7) involve no subject overlap, so those numbers are not affected by the leakage concern.\n\nThe label conversion is the load-bearing issue. BDI and HRSD are different instruments from QIDS-SR; the paper does not provide any clinical validation or error analysis for the mapping. Tables 4, 5, and 7 rely on this conversion for the regression claims. As reported, you cannot tell whether the MAE differences reflect feature generalisability or scale/psychometric artifacts. The stress-test note is right: Table 4's AVEC CV MAE of 3.16 (presumably QIDS-scale, range 0-27) versus Table 8's test MAE of 5.68 (presumably BDI, range 0-63) is a red flag for scale inconsistency. The CV leakage is real but secondary: the paper says AVEC has 150 videos from 82 subjects and Pitt has 148 from 49, and never states that the 10-fold splits are subject-disjoint. That inflates the within-dataset numbers in Tables 2 and 4. Also, the authors report the best results over several chunk sizes and models without correction, which is a minor post-hoc selection issue. No code or data is released, which makes the conversion and the splits hard to verify.\n\nFor a reader in affective computing or digital health, this is a useful paper to know about, but I wouldn't rely on the regression generalisability claim until the label conversion is justified or removed. The classification results and the separate train-test protocol are the more solid contributions.\n\nI would send it to peer review: the question is important, the method is coherent, and the flaws are addressable. A referee should ask for subject-disjoint CV, a defended label conversion, and either code/data or a clear statement on splits and scales. If the authors can address them, the paper could be a reasonable contribution.","headline":"A useful extension with a clean feature-construction story, but the regression generalisability claim rests on an unvalidated label conversion and possibly leaked cross-validation; the classification half is the stronger half.","tokens_in":15311,"tokens_out":2731,"would_cite":false,"duration_ms":24538,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Head-motion 'kinemes' learned from healthy subjects transfer across three Western depression datasets, matching or beating facial cues.","keywords":["kinemes","head motion","depression severity estimation","binary depression classification","cross-corpus generalisability","reconstruction error features","NMF-GMM clustering","nonverbal depression biomarkers"],"falsifier":"Re-run the k-fold and separate train-test experiments with strictly subject-exclusive splits, ensuring all videos of one participant appear in only one fold or one set. If the cross-dataset F1 and MAE gaps shrink toward chance, or if the AVEC-derived kineme advantage disappears, the claimed generalisability is an artifact of participant leakage across splits.","tokens_in":14361,"feed_emoji":"🎥","tokens_out":4948,"duration_ms":52302,"temperature":0.7,"pith_summary":"This paper argues that elementary head-motion units called kinemes are efficient and generalisable biomarkers for depression severity. Kinemes are learned only from head movements of healthy or low-depressed people, then used to reconstruct the movements of both low- and high-depressed groups. Across three culturally distinct Western datasets (German AVEC2013, Australian Blackdog, American Pitt), kineme-based features match or beat raw head-pose features for classifying low versus severe depression and outperform other visual cues for severity regression. The strongest single result is an MAE of 5.68 on the AVEC2013 test set, second only to a facial-keypoint method. A reliable head-motion-only marker would matter because it is cheap to extract from ordinary video and reveals less about identity than facial appearance.","feed_headline":"Kineme head motion beats raw pose and facial cues across 3 datasets","feed_subtitle":"Learned from healthy head motion, 24 descriptors estimate depression severity across German, Australian, and US recordings.","key_machinery":"The kineme dictionary is a set of k = 16 prototypical five-second head-pose segments learned via non-negative matrix factorisation (to obtain a basis and coefficient matrix) followed by Gaussian mixture model clustering in the coefficient space, built only from healthy or low-depressed subjects. Any subject's head motion is then expressed as a sequence of nearest kinemes, and the signed difference between original and reconstructed pose, summarised as minimum, maximum, range, mean, median, standard deviation, skewness, and kurtosis over each of pitch, yaw, and roll, forms the feature vector. This reconstruction-error design is what forces the model to encode departures from normal head motion, and the control-only learning is what makes the features a candidate biomarker rather than a within-corpus artifact.","core_discovery":"The paper's central discovery is that the mismatch between observed head-pose segments and a small learned dictionary of typical head-motion patterns (kinemes) carries depression-severity information that transfers across datasets. Kinemes are discovered by NMF decomposition and GMM clustering of overlapping five-second yaw, pitch, and roll segments taken only from healthy controls; eight statistical descriptors of the per-axis reconstruction error yield a 24-dimensional feature vector. In both k-fold cross-validation and separate train-test reuse experiments, kinemes derived from AVEC generalise best, and the AVEC-derived set achieves an F1 score of 0.77 across all three datasets while outperforming raw head-pose features for classification and facial-feature CNN baselines for regression. On the AVEC2013 test set, XGBoost regression on these features gives an MAE of 5.68 and RMSE of 7.57.","pith_inferences":["Because AVEC contains 150 videos from 82 participants and Pitt contains 148 videos from 49 participants, the claimed generalisability depends on train-test splits being subject-disjoint; the paper does not state this, so a subject-exclusive resplit is the first test I would run.","The Blackdog labels (healthy controls versus depressed patients) differ in meaning from the AVEC and Pitt labels (low versus severe depression), which likely explains the weaker transfer onto Blackdog; calibrating kineme features to symptom-severity scales rather than diagnostic class could make cross-dataset transfer more clinically meaningful.","The finding that AVEC-derived kinemes generalise better than interview-derived kinemes suggests task context matters more than culture, a hypothesis that could be tested directly by learning kinemes from two interview datasets and evaluating on a third.","The reconstruction-error framing could yield an interpretable clinical readout, such as per-axis error profiles showing which head-motion patterns deviate most, although the paper does not pursue that visualisation."],"forward_implications":["Head-pose-only pipelines are viable for depression screening, avoiding dependency on facial appearance features and their privacy costs.","Kineme features transfer across recording setups (self-administered tasks versus clinical interviews) and across language and cultural groups within Western data.","Kinemes discovered from self-directed task recordings such as AVEC can serve as a shared dictionary for future cross-corpus depression studies.","Regression to a common severity scale (QIDS-equivalent) works from head motion alone, with error rates competitive with facial-feature deep learning models.","Because the final feature set is only 24 numbers, the pipeline is lightweight enough for near-real-time screening with classical machine learning classifiers."],"supporting_citations":[{"why":"Supplies the kineme formulation, reconstruction-error feature set, and the initial depression-detection framework that this paper extends.","marker":"[20]"},{"why":"Provides the AVEC2013 dataset, its BDI labels, and the baseline SVR performance on head and facial appearance features.","marker":"[15]"},{"why":"Provides the cross-cultural classification baseline using head pose and eye gaze that the kineme classification results are compared against.","marker":"[30]"},{"why":"Provides the CNN-based facial-feature regression baselines on AVEC and Blackdog that the kineme regression results are compared against.","marker":"[31]"},{"why":"Supplies the OpenFace toolkit used to extract the 3D head pose angles (pitch, yaw, roll) from all videos.","marker":"[32]"},{"why":"Provides the NMF-plus-GMM approach for discovering head-motion patterns in a learned subspace.","marker":"[33]"},{"why":"Provides the Blackdog dataset, including its QIDS-SR labels and structured interview setting.","marker":"[9]"},{"why":"Provides the Pitt dataset, its HRSD labels, and the clinical interview setting.","marker":"[16]"},{"why":"Provides the prior head-pose CNN approach on AVEC2013 that the kineme MAE of 5.68 is compared against.","marker":"[47]"}],"fun_headline_variants":["Kineme head motion predicts depression across 3 cultures","Kineme features beat raw pose and facial cues for depression","Cross-dataset depression severity from head motion kinemes","Head motion kinemes generalize depression severity across cultures"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the train and test splits never put the same person in both sets; the paper does not state that cross-validation or train-test splits are subject-disjoint, and both AVEC and Pitt contain multiple videos per participant.","fun_headline_variants_meta":{"raw":{"variants":["Kineme head motion predicts depression across 3 cultures","Kineme features beat raw pose and facial cues for depression","Cross-dataset depression severity from head motion kinemes","Head motion kinemes generalize depression severity across cultures"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00032,"raw_usage":{"total_tokens":1813,"prompt_tokens":965,"completion_tokens":848,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":785}},"tokens_in":581,"tokens_out":848,"duration_ms":9394,"temperature":1.0,"reasoning_tokens":785,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:45:43.937241+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the k-fold and separate train-test experiments with strictly subject-exclusive splits, ensuring all videos of one participant appear in only one fold or one set. If the cross-dataset F1 and MAE gaps shrink toward chance, or if the AVEC-derived kineme advantage disappears, the claimed generalisability is an artifact of participant leakage across splits.","supporting_citations":[{"cited_title":"Explainable depression detection via head motion patterns,","cited_arxiv_id":null,"evidence_quote":"Supplies the kineme formulation, reconstruction-error feature set, and the initial depression-detection framework that this paper extends."},{"cited_title":"Avec 2013: the continuous audio/visual emotion and depression recognition challenge,","cited_arxiv_id":null,"evidence_quote":"Provides the AVEC2013 dataset, its BDI labels, and the baseline SVR performance on head and facial appearance features."},{"cited_title":"Cross-cultural detection of depression from nonverbal behaviour,","cited_arxiv_id":null,"evidence_quote":"Provides the cross-cultural classification baseline using head pose and eye gaze that the kineme classification results are compared against."},{"cited_title":"CNN depression severity level estimation from upper body vs. face-only images,","cited_arxiv_id":null,"evidence_quote":"Provides the CNN-based facial-feature regression baselines on AVEC and Blackdog that the kineme regression results are compared against."},{"cited_title":"OpenFace: An open source facial behavior analysis toolkit,","cited_arxiv_id":null,"evidence_quote":"Supplies the OpenFace toolkit used to extract the 3D head pose angles (pitch, yaw, roll) from all videos."},{"cited_title":"On the role of head motion in affective expression,","cited_arxiv_id":null,"evidence_quote":"Provides the NMF-plus-GMM approach for discovering head-motion patterns in a learned subspace."},{"cited_title":"Multimodal depression detection: fusion analysis of paralinguistic, head pose and eye gaze behav- iors,","cited_arxiv_id":null,"evidence_quote":"Provides the Blackdog dataset, including its QIDS-SR labels and structured interview setting."},{"cited_title":"Detecting depression sever- ity from vocal prosody,","cited_arxiv_id":null,"evidence_quote":"Provides the Pitt dataset, its HRSD labels, and the clinical interview setting."},{"cited_title":"Spectral repre- sentation of behaviour primitives for depression analysis,","cited_arxiv_id":null,"evidence_quote":"Provides the prior head-pose CNN approach on AVEC2013 that the kineme MAE of 5.68 is compared against."}],"review_version":1}