{"id":"cba32a6d-8223-4869-8eec-fc784f17d974","arxiv_id":"2506.16677","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A performance-guided fusion of ECG, GSR, and EMG reaches 81.1% accuracy in three-level and 74.3% in seven-level trust classification during human-robot construction tasks.","lead":"The paper introduces a machine-learning model that combines heart, skin, and muscle signals with a measure of construction performance to predict how much a person trusts a robot teammate. If validated, it could enable continuous, safety-relevant trust monitoring in human-robot collaboration on construction sites.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported accuracy is unverifiable because no train/test split is specified; window-overlap and same-subject leakage could explain the gains.","rationale":"The paper's contribution is an empirical claim: CP guidance improves physiological trust classification to 81.1% / 74.3%, beating Transformer by 6.7%. That claim is only as strong as the evaluation protocol that produced it. The manuscript nowhere states the split. This is not a stylistic omission: with 30 participants, short overlapping windows, block-constant labels, and a high-capacity transformer, there are multiple concrete leakage routes. A random split lets the same participant appear in train and test, so the model can exploit stable individual physiology rather than trust; overlapping windows and block-constant labels create near-duplicate training/test instances. Under such leakage, the 6.7% margin and the seven-level result are not evidence of a general PPTP mechanism. I agree partly with the Reader: the constant-label ground truth is a real validity concern, but it would more likely blur trust dynamics than create the reported separability. The missing split is more urgent because it can directly manufacture the reported accuracy. The proposed test--strict leave-one-subject-out with no overlap and significance testing--would settle it.","tokens_in":10108,"tokens_out":5267,"duration_ms":64798,"concrete_test":"Re-run the full comparison under leave-one-subject-out cross-validation: assign all windows of each participant to one fold; ensure no overlapping windows cross folds; within each fold, train only on other participants' windows and report per-fold accuracy, balanced accuracy, and a paired significance test (e.g., Wilcoxon) against the best Transformer baseline. Also repeat with random-window split and compare; if subject-independent accuracy drops materially or no longer beats Transformer by more than 3%, the reported gains are leakage or memorization rather than generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is not the label construction but the missing evaluation protocol. Section IV reports Table I accuracy means/stds across 30 participants but never states whether train/test partitions are subject-independent, task-independent, window-independent, or random. In Section III-C, signals are segmented into short windows with 108ms overlap, and trust labels are held constant across each inter-block interval. Under a random window split, near-identical overlapping windows and windows from the same block with the same label can appear in both train and test; the model can also memorize participant-specific physiological baselines because the same subject contributes to both sets. Given a deep cross-attention Transformer and N=30 participants, these leak paths can plausibly account for the 6.7% margin over Transformer and for the surprisingly high 7-level accuracy. Without the split specification, the 81.1% and 74.3% numbers cannot be interpreted as evidence for PPTP.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PPTP, a Transformer-based multimodal fusion model that combines ECG, GSR, and EMG with a Collaboration Performance (CP) metric computed from block-stacking skewness and failure risk, to classify human trust in a human-robot construction task. Data from 30 participants across three difficulty conditions are segmented into short windows, and each window is assigned the trust score of the surrounding inter-questionnaire interval. The paper reports 81.1% three-level and 74.3% seven-level classification accuracy, ablation results across modality combinations, and comparisons with baseline models.","tokens_in":10267,"tokens_out":5340,"duration_ms":61046,"significance":"If the evaluation protocol is sound, the paper makes a useful contribution: it addresses a realistic construction HRC setting, integrates an interpretable CP measure as guidance, and explores multi-level trust classification beyond the common three-level setup. The CP-guided attention mechanism is a reasonable design, and the modality-combination ablations are informative. However, the absence of a clear data-partitioning protocol leaves the headline accuracies unverifiable, and the label-construction assumption limits what the numbers can support. Subject-independent evaluation and controls for task difficulty would make the results credible to the HRC and affective computing communities. No code or data are provided, so reproducibility currently rests on the textual description.","major_comments":[{"comment":"The manuscript never states how training and test sets were formed. Section III-C segments signals into 3 s (ECG/GSR) and 216 ms (EMG) windows with 108 ms overlap and assigns each window the trust score of the surrounding inter-questionnaire interval. If windows are assigned to train/test randomly, overlapping windows from the same block can appear in both sets, and because all windows of a participant share that participant's physiological baseline, the model can memorize subject-specific patterns with only N=30 participants. This could account for the 81.1% three-level and 74.3% seven-level accuracies and for the 6.7% margin over the Transformer baseline. Please specify whether partitioning was subject-independent, task-independent, window-independent, or random, and report accuracy under a strict subject-independent split. This is required to interpret Table I and Table IV.","section":"Section IV (Tables I–IV) and Section III-C"},{"comment":"Ground-truth trust labels are constructed by holding each post-step Muir questionnaire score constant across the entire inter-block interval. Section III-C explicitly states, \"we assumed constant trust levels between consecutive recording points.\" The paper therefore assumes that trust does not change within a block, even though the introduction and Section IV-E motivate continuous monitoring of trust dynamics. If a participant's trust changes during a block, the labels are wrong, and the reported classification accuracy overstates the model's ability to track dynamic trust. Please quantify the distribution of label durations, discuss the temporal-resolution limits of this labeling, or provide an auxiliary analysis (e.g., using only the first or second half of each interval) to justify the assumption.","section":"Section III-C and Section IV-E"},{"comment":"The three difficulty conditions produce significantly different trust scores (Section IV-A), and the CP metric in Eq. (2) is computed from block skewing and failure events that are likely affected by the same difficulty manipulation (e.g., inverted camera views in the HD condition). Consequently, CP guidance may serve as a proxy for task difficulty rather than for collaboration performance per se, making the causal interpretation that performance guides physiology ambiguous. Please report an ablation that includes task condition (LD/MD/HD) as an input feature or otherwise controls for difficulty, and discuss the sensitivity of the result to the discount factor gamma=0.8.","section":"Section III-B, Section IV-A, Eq. (2)"}],"minor_comments":[{"comment":"The text states that CP guidance \"significantly improves\" performance (e.g., Fig. 6), but no statistical test or confidence interval accompanies the per-subject accuracy comparisons; the 2.6-point gain over all physiological signals in Table I is within one standard deviation. Please add paired tests or confidence intervals.","section":"Section IV-B and IV-C"},{"comment":"There are several typos and formatting issues: \"correponding\" in Section I, \"Data Acquasition\" in Fig. 1, and \"ANOV A\" in Section IV-A. Please proofread the manuscript.","section":"Sections I, III-A, IV-A"},{"comment":"The claim of being \"a first in trust prediction research\" for seven-level classification needs a literature search or at least a delimiting statement, since seven-level labels from a Likert questionnaire are a natural extension and prior work may have used them.","section":"Section IV-E"},{"comment":"Baseline implementations (CPM-Nets, RNN, LSTM, Transformer) are not described beyond a CP embedding module; please provide architecture and training details so that the 6.7% margin can be reproduced.","section":"Section IV-D"},{"comment":"The sampling rates and window lengths are given, but the number of windows per participant, the class distribution after labeling, and any class-balancing strategy are not reported. Please add these details.","section":"Section III-C and IV-A"}],"recommendation":"major_revision","confidential_remarks":"The central evaluation gap is the missing train/test split, which is load-bearing for the reported accuracies. Because this is fixable with a subject-independent evaluation and accompanying statistical tests, I recommend major revision rather than rejection. Please also verify the 'first seven-level trust prediction' claim against prior literature before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:2506.16677. The genuinely new piece is using collaboration performance (CP) as a cross-attention guide over physiological signal patches, and pushing trust classification to seven levels. The experiment is also real: 30 participants doing block-stacking with a physical robot, synchronized ECG/GSR/EMG, and a CP metric that is more graded than binary success/failure. That part is worth reading, and the citations look appropriate.\n\nThe soft spots are real, and the stress-test note holds up. Section IV reports mean/stdev accuracy across 30 subjects but never says whether the train/test split is subject-independent, task-independent, window-independent, or random. Given the 108 ms window overlap and the fact that each block's trust label is held constant over the entire inter-block interval, a random window split lets near-identical windows from the same participant appear in both train and test. With a deep cross-attention Transformer and N=30, that leakage path alone could plausibly account for the 81.1% and 74.3% numbers. This is the load-bearing gap.\n\nThe label-construction issue is secondary. Holding a Muir questionnaire score constant across a short block is a reasonable approximation; if trust drifts mid-block, the labels get noisy, but that would attenuate accuracy rather than inflate it. I would not make that the main complaint.\n\nThere are also smaller reporting problems. The 2.6-point gain from adding CP (78.5 to 81.1) is called \"significant\" but no significance test is reported, and the same is true for the 6.7-point margin over the Transformer baseline. The \"first seven-level trust prediction\" claim needs a citation check; I cannot verify it from the text. On the positive side, the CP metric is deterministic from block positions and a hand-set discount factor, not fitted to trust labels, so the circularity concern is unfounded.\n\nWho is this for? People working on multimodal physiological trust prediction and HRC evaluation. The architecture is a reasonable engineering combination, not a conceptual breakthrough, but it is a legitimate extension. If the split is subject-independent and the results hold, this is a solid contribution. As presented, I would not cite the headline accuracy.\n\nRecommendation: send it to a serious referee, but with a pointed request for the exact split protocol, participant independence, and significance tests. Those are fixable in revision, and the underlying experiment deserves the engagement.","headline":"A useful, well-situated HRC trust experiment with a sensible CP-guided fusion idea, but the missing train/test split details currently make the headline accuracies unverifiable.","tokens_in":10778,"tokens_out":2047,"would_cite":false,"duration_ms":25638,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper demonstrates that collaboration performance can serve as a guiding signal for physiological trust prediction, achieving 81.1% three-level and 74.3% seven-level classification accuracy in a human-robot construction task.","keywords":["trust prediction","human-robot collaboration","physiological signals","collaboration performance","multimodal fusion","cross-modality attention","construction robotics"],"falsifier":"Collect trust ratings at multiple points inside a single block, for example with a continuous slider or repeated short questionnaires, then retrain or re-evaluate the model against those finer labels. If within-block accuracy is substantially lower than the 81.1% and 74.3% figures obtained with held-constant labels, the central claim of dynamic trust tracking fails; if accuracy holds, the constant-label assumption is benign.","tokens_in":9900,"feed_emoji":"🤖","tokens_out":6465,"duration_ms":70907,"temperature":0.7,"pith_summary":"Trust is hard to measure continuously: questionnaires sample before and after a task, while physiological signals are continuous but vary wildly between people. This paper tries to close that gap by using collaboration performance—how well the human-robot team is actually building—as a guiding cue for interpreting ECG, GSR, and EMG signals. In a block-stacking construction task with three difficulty levels, the proposed PPTP framework classifies trust into three levels with 81.1% accuracy and into seven finer levels with 74.3% accuracy, the first reported seven-level trust classification in this line of work. The reason to care is practical: if trust can be read at fine granularity during a task, a robot could adjust its autonomy or request help before trust becomes miscalibrated.","feed_headline":"Performance data guides trust prediction to 81% accuracy","feed_subtitle":"Fusing ECG, GSR and EMG with task-performance cues reads human trust in robot teammates, down to seven levels.","key_machinery":"The two-part mechanism is a collaboration-performance evaluator and a cross-modality fusion block. The evaluator estimates the skewing of every placed block from camera-detected block centers, treats the bottom layer as stable, and converts the skewing history into a discounted failure-risk vector $F \\in \\mathbb{R}^{1 \\times 10}$ using a weighted moving average with discount factor $\\gamma = 0.8$; a failed placement or collapse enters as $-1$ or $-2$. The fusion module embeds this performance vector and uses it as key/value guidance in a cross-attention layer inserted between the self-attention and feed-forward sections of each block of a Transformer-style physiological signal processor, so performance information steers which physiological patches the model attends to. This design is what lets standardized task outcomes compensate for individual differences in physiological response.","core_discovery":"The central claim is that physiological trust signals become substantially more readable when feature extraction is guided by a standardized measure of how well the collaboration is going. The paper constructs a continuous collaboration performance score from the skewing of each stacked block and a discounted history of failure risk, then feeds that score into an attention-based fusion module as a cross-modal guide for selecting relevant patches of ECG, GSR, and EMG. Across 30 participants, this raises three-level trust classification to 81.1% accuracy, 6.7 percentage points above the best baseline, and yields 74.3% accuracy on a seven-level trust scale, whereas the same physiological signals without performance guidance reach 78.5% and 72.9% accuracy respectively. The authors present the seven-level result as the first high-resolution trust prediction in human-robot collaboration research.","pith_inferences":["The same performance-guided fusion recipe could transfer to any collaborative task with a measurable outcome, such as assembly quality, surgical steadiness, or driving errors, where trust is currently inferred from physiology alone; testing that transfer is a natural next step.","If extremely high trust is physiologically saturated, as the label-7 confusions suggest, then the seven-level scale may be capturing roughly five or six distinguishable physiological states, and a finer subjective scale would not automatically yield more distinct signal patterns.","A testable extension the paper does not run is to feed predicted trust back into robot control online, changing autonomy mid-task and comparing task outcomes against a no-adaptation control; that would show whether 81% classification accuracy is enough to improve collaboration in practice."],"forward_implications":["If the 81.1% three-level result holds up, physiological trust monitors for human-robot collaboration can be built around wearable ECG, GSR, and EMG without laboratory EEG equipment.","The 74.3% seven-level result implies that trust is not just low, medium, and high: finer distinctions are accessible, which could support graded robot responses rather than coarse mode switches.","Performance guidance improves accuracy across nearly every signal combination tested, suggesting that standardized task metrics are a general corrective to person-to-person physiological variability.","The model stays above 73% accuracy even when two physiological modalities are missing, so deployment can degrade gracefully when some sensors fail or are removed."],"supporting_citations":[{"why":"Supplies the physiological trust-prediction baseline in a construction-like task that motivates the ECG and GSR choices.","marker":"[10]"},{"why":"Shows EEG and GSR can classify trust states, the approach this work replaces with wearable signals plus performance guidance.","marker":"[14]"},{"why":"Uses robot performance history for trust inference, the binary success/failure approach the continuous collaboration-performance evaluator is built to improve on.","marker":"[24]"},{"why":"Provides the block-center detection used to compute per-block skewing for the collaboration performance score.","marker":"[29]"},{"why":"Provides the trust questionnaire that supplies ground-truth trust labels for training and evaluation.","marker":"[31]"},{"why":"Supplies the weighted moving average technique used to turn block-skewing history into the failure-risk vector.","marker":"[34]"},{"why":"Provides the attention architecture that the multimodal processing module modifies with cross-modality guidance.","marker":"[37]"},{"why":"Serves as a state-of-the-art multi-view learning baseline for comparison.","marker":"[41]"}],"fun_headline_variants":["Performance-guided signals lift trust prediction to 81%","First 7-level trust prediction in human-robot collaboration","ECG, GSR, EMG plus performance cues read trust levels","Task performance guides physiology to predict trust at 81%","Seven-level trust prediction achieved with physiological cues"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Trust is assumed to stay constant between consecutive questionnaire responses, so each post-block questionnaire score labels the entire preceding window of physiological data; if a participant's trust shifts within a block, the ground-truth labels are wrong and the reported accuracy overstates how well the model tracks trust as it changes.","fun_headline_variants_meta":{"raw":{"variants":["Performance-guided signals lift trust prediction to 81%","First 7-level trust prediction in human-robot collaboration","ECG, GSR, EMG plus performance cues read trust levels","Task performance guides physiology to predict trust at 81%","Seven-level trust prediction achieved with physiological cues"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000814,"raw_usage":{"total_tokens":3547,"prompt_tokens":900,"completion_tokens":2647,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":2567}},"tokens_in":516,"tokens_out":2647,"duration_ms":21783,"temperature":1.0,"reasoning_tokens":2567,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:38:16.474008+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect trust ratings at multiple points inside a single block, for example with a continuous slider or repeated short questionnaires, then retrain or re-evaluate the model against those finer labels. If within-block accuracy is substantially lower than the 81.1% and 74.3% figures obtained with held-constant labels, the central claim of dynamic trust tracking fails; if accuracy holds, the constant-label assumption is benign.","supporting_citations":[{"cited_title":"Predicting human trust in human- robot collaborations using machine learning and psychophysiological responses,","cited_arxiv_id":null,"evidence_quote":"Supplies the physiological trust-prediction baseline in a construction-like task that motivates the ECG and GSR choices."},{"cited_title":"A classification model for sensing human trust in machines using eeg and gsr,","cited_arxiv_id":null,"evidence_quote":"Shows EEG and GSR can classify trust states, the approach this work replaces with wearable signals plus performance guidance."},{"cited_title":"Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations,","cited_arxiv_id":null,"evidence_quote":"Uses robot performance history for trust inference, the binary success/failure approach the continuous collaboration-performance evaluator is built to improve on."},{"cited_title":"A review of yolo algo- rithm developments,","cited_arxiv_id":null,"evidence_quote":"Provides the block-center detection used to compute per-block skewing for the collaboration performance score."},{"cited_title":"Trust in automation: Part i. theoretical issues in the study of trust and human intervention in automated systems,","cited_arxiv_id":null,"evidence_quote":"Provides the trust questionnaire that supplies ground-truth trust labels for training and evaluation."},{"cited_title":"The weighted moving average technique,","cited_arxiv_id":null,"evidence_quote":"Supplies the weighted moving average technique used to turn block-skewing history into the failure-risk vector."},{"cited_title":"Deep partial multi-view learning,","cited_arxiv_id":null,"evidence_quote":"Serves as a state-of-the-art multi-view learning baseline for comparison."}],"review_version":1}