{"id":"c1197d7d-981c-4e9c-b1fd-b43dafa09a20","arxiv_id":"2501.00038","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Audio-only recognition of touch gestures and emotional arousal/valence is feasible on the Pepper robot using a 0.24-million-parameter CNN, with accuracies around 84% for gestures and 54% for five-way arousal-valence states.","lead":"This paper tests whether the sounds of a person touching a robot can reveal the gesture and the emotion behind it, using a tiny audio-recognition model that runs on low-power hardware. It matters because many social robots lack full-body touch sensors, and sound-only sensing could give them a privacy-friendly way to sense how people touch them.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Participant-level data leakage likely inflates all reported accuracies; without a participant-independent split the central claim is not established.","rationale":"The reader's weakest assumption correctly identifies the data-split protocol as the most load-bearing issue. The abstract and conclusion claim that the model effectively recognizes touch gestures and arousal/valence states, and the strongest claim is that this holds for varying audio lengths. All of those claims rest on the accuracy numbers in Tables I and III. If the same participant appears in both training and test, the model can learn person-specific cues (e.g., recording levels, hand shape, microphone position) rather than generalizable gesture/emotion patterns. Since the paper does not describe a group-wise split, the burden is on the authors to show that the split is participant-independent. The paper also uses the validation set to select the optimal input length in Table II, but then never reports test-set accuracies at 6 s and 7 s, so the conclusion about optimal length is weaker than stated. These issues are correctable in revision, hence CONDITIONAL rather than REJECT or ACCEPT. I found no other concern as consequential: the MTRCNN architecture is clearly described, the FLOPs/parameter comparison is useful, and the claim that MTRCNN is competitive with PANNs is plausible if the split issue is resolved.","tokens_in":9320,"tokens_out":2690,"duration_ms":26352,"concrete_test":"Inspect the public repository's data-loading script to determine whether clips are grouped by participant before splitting. Then re-run the gesture classification (Table I and III) with leave-one-participant-out cross-validation, or a 70/10/20 split at the participant level, using the same MTRCNN hyperparameters. If the mean test accuracy for gestures falls below the reported 82.14±3.46 by more than the standard error, the original numbers are inflated by participant leakage; also report the Aro-Val joint accuracy under the same split.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper never states whether the train/validation/test split in Section IV-A is participant-independent. With 28 participants each contributing three 10-s clips per gesture/emotion, a random clip-level split almost certainly places clips from the same participant in both training and test. The model can then exploit participant-specific recording conditions, hand acoustics, or microphone idiosyncrasies to inflate accuracy. This affects every headline number in Tables I and III, and also the optimal-length analysis in Table II (which is computed on the same validation split). Since the central claim is that the model recognizes gestures and emotions from touch sounds in general, the reported numbers must survive a split that keeps all clips of a participant in one partition. The paper also does not report test-set results for the claimed optimal 6–7 s input lengths; Table II is restricted to the validation set, so the best-length conclusion is not tested on held-out data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a lightweight audio-only model (MTRCNN) that recognizes touch gestures (hold, pat, poke, tickle, tap, rub) and emotion arousal/valence states from sounds produced during human-robot interaction with the Pepper robot. The model uses multi-branch dilated convolutions and a fusion layer, with 0.24M parameters and 0.7G FLOPs. Experiments on a dataset of 28 participants report test accuracies around 82% for gestures and 54% for joint arousal-valence classification, with an analysis claiming optimal input lengths of 6–7 seconds. The paper compares MTRCNN against several larger baselines, including pretrained PANNs, and concludes that the model is suitable for real-time deployment on Pepper.","tokens_in":9413,"tokens_out":4578,"duration_ms":40379,"significance":"If the reported results hold, this is a valuable contribution to privacy-preserving HRI, offering a low-cost, non-intrusive alternative to vision- or tactile-sensor-based emotion and gesture recognition. The paper provides a dataset, code, and models, which supports reproducibility and follow-up research. The lightweight architecture and low latency are concrete strengths for embedded deployment. However, the evaluation has several gaps that currently undermine the central quantitative claims, especially the unclear split protocol and missing input-length specification in comparisons. The practical significance is high if these issues are resolved.","major_comments":[{"comment":"The train/validation/test split protocol is not specified at the participant level. With 28 participants each contributing three clips per gesture/emotion, a random clip-level split will almost certainly place clips from the same participant in both training and test, allowing the model to exploit participant-specific recording conditions or hand acoustics. This would inflate all reported accuracies in Tables I and III, as well as the validation-based conclusions in Table II. The authors must state explicitly whether all clips from a given participant are confined to a single partition, and if not, re-run the experiments with a participant-independent split.","section":"Section IV-A"},{"comment":"The gesture clip counts are inconsistent. For six gestures with 84 clips each, the total should be 504 clips, but the reported training/validation/test numbers sum to 366+42+84=492, leaving 12 clips unaccounted for. The manuscript should explain how these clips were discarded and confirm that the remaining clips are balanced across gestures and participants. Without this, the reported gesture accuracy may not reflect the full dataset.","section":"Section IV-A"},{"comment":"Table III does not state the input audio length used for each model. Since the paper concludes in RQ2 that 6s and 7s are the optimal lengths for gesture and Aro-Val classification and declares these as defaults, Table III must specify the input length for each method and use a consistent length across models. The current ambiguity is material: MTRCNN's gesture accuracy in Table III (84.17%) differs from the 10s result in Table I (82.14%), so it is unclear whether the table uses the optimal lengths or something else.","section":"Table III"},{"comment":"The optimal-length analysis in Table II is performed only on the validation set, and the selected lengths (6s for gestures, 7s for Aro-Val) are never evaluated on the held-out test set. The conclusion that these lengths are optimal is therefore not supported by test-set evidence. The authors should report test-set accuracies for the selected lengths, as well as for nearby lengths (e.g., 5s, 6s, 7s, 8s), to confirm the choice on unseen data.","section":"Section IV-B, RQ2"},{"comment":"The paired t-tests comparing accuracies at different input lengths may be statistically invalid. The paper states that models were trained '10 times without a fixed seed,' meaning the runs for different input lengths are independent, not paired. A paired t-test requires paired observations (e.g., identical initialization across conditions). If the runs are independent, an unpaired test or a repeated-measures design with fixed seeds should be used. This affects the significance claims for the optimal length selection.","section":"Section IV-B, RQ2"}],"minor_comments":[{"comment":"The sentence 'participants first expressed gestures 10 emotions independently' is missing a conjunction; it should read 'gestures and 10 emotions.'","section":"Section IV-A"},{"comment":"The confusion matrices in Fig. 5 are reported only on the test set, but it is not stated which input length was used for the model when generating them. For consistency with RQ2, the figure should specify whether the default 6s/7s lengths were used.","section":"Section IV-B, RQ3"},{"comment":"The architecture description omits the pooling sizes and the exact placement of pooling operations, despite the minimum input length of 1.10s being derived from pooling. The GitHub link is useful, but the paper should include these details for self-contained reproducibility.","section":"Section III-B"},{"comment":"Several references are incomplete or inconsistent, e.g., [12] lacks a volume/issue, and [41] uses only a journal name without volume. Please harmonize the reference style throughout.","section":"References"},{"comment":"The abstract claims the model 'achieves similar results as well-known pretrained audio neural networks (PANNs)' but the statistical test for this claim is only reported for Aro-Val, not for gesture. Clarify the comparison or add a significance test for gesture accuracy.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The dataset and several prior works (including reference [33]) are from the authors' group, so the novelty claim ('first attempt to recognise touch gestures and emotions based on sounds produced during HRI') should be carefully checked against the dataset paper. The evaluation gaps—especially the participant-independent split and the missing input-length specification in Table III—are serious but fixable. If the authors can re-run experiments with a proper split and report test-set results for the selected lengths, the paper would be suitable for publication. I would not recommend acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, it's a genuinely new idea: using the sounds of touching a robot to classify both touch gestures and emotional arousal/valence, rather than tactile skin or vision. Useful because social robots like Pepper lack full-body tactile sensing, and cameras raise privacy issues. Second, the reported numbers are probably too good to trust as reported, because the paper never says the train/test split is participant-independent. With 28 people each producing three clips per gesture, a random clip-level split almost guarantees the same person appears in both training and test, which inflates accuracy through personal acoustics or recording conditions.\n\nWhat's actually new and done well: the MTRCNN is clearly described, small (0.24M params), and respects Pepper's FLOPs budget. The comparison against PANNs with and without AudioSet pretraining is a sensible efficiency benchmark. The dataset and code are promised to be public, which is a real plus. The analysis of how accuracy varies with input length is useful, if it holds up.\n\nSoft spots, in order of severity.\n\n1. The split protocol. Section IV-A gives counts but never says participants stay within one partition. This affects every table and the optimal-length analysis. If they confirm a subject-independent split and re-run, the numbers become credible; if not, they're suspect. To be clear, this isn't curve-fitting to the test set; the test set is real. The problem is that 'held-out' may only mean 'different clips,' not 'different people.'\n\n2. Arithmetic mismatch: 366+42+84 = 492 for gesture classification, but 28 participants × 3 clips × 6 gestures = 504. That's 12 clips unaccounted for. The emotion totals add up (660+80+100 = 840), so this is likely an oversight, but it needs explaining.\n\n3. Table III doesn't state the input length used for the baseline models. If baselines got 10s while the proposed model got its tuned 6–7s, the comparison isn't fair. Also, the optimal 6–7s lengths were selected on the validation set and never re-tested on the test set.\n\n4. The conclusion says the model is 'ideal for real-time applications on robots such as Pepper,' but inference time was measured on a Tesla T4, not on Pepper's Atom processor. The FLOPs argument is plausible, but actual deployment is future work.\n\nBottom line: this is a novel, motivated idea with a plausible method and a promised public dataset. The central claim is not established until the split issue is resolved. It deserves a serious referee—send it out—but the review should demand subject-independent evaluation and matched-length comparisons.","headline":"Novel and motivated idea for audio-only touch gesture and emotion recognition on a robot, but the reported accuracies likely leak participant identity because the paper never states that the train/test split is participant-independent.","tokens_in":9993,"tokens_out":3675,"would_cite":false,"duration_ms":31642,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that the sound of touching a robot carries enough signal to recognize six touch gestures and to classify emotions by arousal and valence, using a 0.24-million-parameter audio model that can run on Pepper's onboard computer.","keywords":["touch gesture recognition","emotion recognition","arousal-valence model","human-robot interaction","audio-only classification","multi-temporal-resolution CNN","Pepper robot","MTRCNN"],"falsifier":"Rerun MTRCNN under a strict leave-one-participant-out protocol: train on clips from 27 participants and test on the held-out 28th participant. If six-way gesture accuracy falls from the reported 82-90% toward the 17% chance level, or five-way arousal-valence accuracy falls toward the 20% chance level, then the model is learning participant-specific sound profiles rather than the named gestures and emotion states.","tokens_in":9071,"feed_emoji":"🤖","tokens_out":11301,"duration_ms":88611,"temperature":0.7,"pith_summary":"The paper's goal is to give humanoid robots a privacy-friendly way to feel touch: instead of installing full-body tactile skin or relying on cameras that raise data-protection concerns, it asks whether the sounds people make when touching the robot can reveal both the gesture and the emotion behind it. The authors propose MTRCNN, a multi-temporal-resolution convolutional network with 0.24 million parameters, 0.94 MB size, and 0.708 G FLOPs, and evaluate it on recordings of 28 participants touching the Pepper robot's forearm. They report that the model distinguishes six touch gestures and places emotions in an arousal-valence space, with accuracy that holds when the input clip varies from about 2 seconds up to an optimum of 6–7 seconds. It matters because this is the first attempt to decode both gesture and emotion from the non-speech sounds of touch during human-robot interaction, and the model's size and latency fit the constraints of Pepper's onboard brain.","feed_headline":"Six touch gestures, five emotion states decoded from sound alone","feed_subtitle":"A lightweight audio-only network matches far larger pretrained models while needing no camera and no tactile skin.","key_machinery":"The carrying object is MTRCNN, a multi-temporal-resolution convolutional neural network. It runs three parallel convolutional branches with kernel sizes (3,3), (5,5), and (7,7), applies hybrid dilated convolutions with dilation rates (1,1), (2,1), and (3,1) along the time axis to enlarge the receptive field, pools so that the minimum input is 1.10 seconds, and fuses three 64-dimensional embeddings into a 192-dimensional representation. Four softmax heads consume that representation for arousal, valence, joint arousal-valence, and gesture classification; the key property is that the small kernel set and dilated time-axis filters let a 0.24M-parameter model capture long-range rhythm and contact patterns in variable-length clips.","core_discovery":"The central claim, stated on the paper's own terms, is that touch sounds are informative enough to support joint gesture and emotion recognition on a real robot platform. On 10-second clips the model reaches 82.14% accuracy across the six gestures and 53.93% across the five joint arousal-valence categories; on the validation set the best performance is 90.24% for gestures with 6-second clips and 58.54% for arousal-valence with 7-second clips, with paired t-tests reported as confirming that those lengths beat neighboring clip lengths. The same 0.24M-parameter network achieves higher gesture accuracy than PANNs pretrained on AudioSet, and is not significantly different from it on arousal-valence, while using far fewer parameters, FLOPs, and memory.","pith_inferences":["The paper does not state whether the train/validation/test split keeps each participant's clips in a single partition; with 28 participants and three clips each, a clip-level random split would almost surely place the same person in both training and test, so a strict leave-one-participant-out evaluation is needed to confirm that the model generalizes to new people.","If the split-leakage risk is real, the absolute accuracies in Tables I and III are likely optimistic, although the relative ordering of clip lengths and model comparisons would still be informative.","The same architecture and touch-sound idea could plausibly extend to other non-verbal contact sounds in human-robot interaction, such as footsteps, handshakes, or object taps, where on-robot microphones already record audio; the paper does not test this.","A natural next test is to feed a continuous interaction stream and let the model segment its own windows, since the paper evaluates only pre-segmented clips and the optimal 6–7 second length may depend on how the clips were cut."],"forward_implications":["A robot with a microphone but no tactile skin can classify six touch gestures in real time, since inference on a clip takes about 7 milliseconds.","Because the model needs only 0.708 G FLOPs, it fits the computing ceiling of Pepper's onboard processor and can run on-device without cloud support.","Large-scale audio pretraining is not required for this task: the PANNs baseline pretrained on AudioSet is not significantly better than MTRCNN on arousal-valence joint classification, and MTRCNN is better on gesture classification.","The effective interaction window is about 6 seconds for gestures and 7 seconds for emotions, and clips down to about 2 seconds still work, so future deployments can segment interaction audio accordingly."],"supporting_citations":[{"why":"Identifies Pepper and defines the computing constraints that motivate the lightweight model design.","marker":"[7]"},{"why":"Supplies Russell's circumplex model of affect, whose arousal-valence dimensions become the paper's emotion label space.","marker":"[23]"},{"why":"Provides the hybrid dilated convolution scheme used in MTRCNN to enlarge the receptive field without gridding artifacts.","marker":"[30]"},{"why":"Documents the data collection from 28 participants touching Pepper, producing the touch-sound dataset the model is trained on.","marker":"[33]"},{"why":"Introduces PANNs and the mel-filter feature setting, serving as the large pretrained baseline and the feature extraction reference.","marker":"[34]"},{"why":"Supplies the Shapiro-Wilk normality test used to justify applying paired t-tests to the clip-length accuracy comparisons.","marker":"[39]"},{"why":"Supplies the paired t-test whose significant results support the claimed 6-second and 7-second optimal clip lengths.","marker":"[40]"},{"why":"Defines AudioSet, the 5800-hour corpus whose pretrained weights give the PANNs baseline its advantage and which MTRCNN matches without using.","marker":"[43]"}],"fun_headline_variants":["Hear the touch: audio-only AI reads gestures and feelings","No camera, no tactile skin: touch sounds tell all","Sound-only robot touch recognition beats heavy pretrained models","Lightweight audio model decodes touch gestures and emotions","Touch sounds enable privacy-friendly emotion AI for robots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracies assume that clips from the same person never appear in both the training and the test sets, but the paper does not say whether the data were split that way.","fun_headline_variants_meta":{"raw":{"variants":["Hear the touch: audio-only AI reads gestures and feelings","No camera, no tactile skin: touch sounds tell all","Sound-only robot touch recognition beats heavy pretrained models","Lightweight audio model decodes touch gestures and emotions","Touch sounds enable privacy-friendly emotion AI for robots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000866,"raw_usage":{"total_tokens":3753,"prompt_tokens":943,"completion_tokens":2810,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":2732}},"tokens_in":559,"tokens_out":2810,"duration_ms":17413,"temperature":1.0,"reasoning_tokens":2732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:49:17.395308+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun MTRCNN under a strict leave-one-participant-out protocol: train on clips from 27 participants and test on the held-out 28th participant. If six-way gesture accuracy falls from the reported 82-90% toward the 17% chance level, or five-way arousal-valence accuracy falls toward the 20% chance level, then the model is learning participant-specific sound profiles rather than the named gestures and emotion states.","supporting_citations":[{"cited_title":"Pepper: The first machine of its kind,","cited_arxiv_id":null,"evidence_quote":"Identifies Pepper and defines the computing constraints that motivate the lightweight model design."},{"cited_title":"A circumplex model of affect.,","cited_arxiv_id":null,"evidence_quote":"Supplies Russell's circumplex model of affect, whose arousal-valence dimensions become the paper's emotion label space."},{"cited_title":"Understanding convolution for semantic segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the hybrid dilated convolution scheme used in MTRCNN to enlarge the receptive field without gridding artifacts."},{"cited_title":"Comparisons of various types of normality tests,","cited_arxiv_id":null,"evidence_quote":"Supplies the Shapiro-Wilk normality test used to justify applying paired t-tests to the clip-length accuracy comparisons."},{"cited_title":"Con- tinuous variable analyses: t-test, mann–whitney, wilcoxin rank,","cited_arxiv_id":null,"evidence_quote":"Supplies the paired t-test whose significant results support the claimed 6-second and 7-second optimal clip lengths."},{"cited_title":"AudioSet: An ontology and human-labeled dataset for audio events,","cited_arxiv_id":null,"evidence_quote":"Defines AudioSet, the 5800-hour corpus whose pretrained weights give the PANNs baseline its advantage and which MTRCNN matches without using."}],"review_version":1}