{"id":"47777719-dfac-4f06-844b-1803bc611487","arxiv_id":"2506.20212","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A federated stress-prediction model trained on device matched centralized accuracy and local fine-tuning improved per-person accuracy in a 33-subject assembly task.","lead":"This paper reports that a federated-learning stress model, trained from physiological signals on workers' own devices, matches a centralized model in a 33-person factory assembly study. A smart generalist would read it because it sketches a privacy-preserving route to personalized robot adaptation to worker stress.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's personalized-model column is evaluated on the same data used for fine-tuning, so the reported RMSE improvement over the global model does not establish personalization.","rationale":"The reader flagged the proxy ground truth (one STAI-Y1 score per assembly normalized per subject) as the weakest assumption. That is a serious validity concern, but the more load-bearing and more directly checkable problem is the personalization evaluation in Table II: the fine-tuned model appears to be evaluated on the same data on which it was fine-tuned, and the global model used for comparison includes the test subject among its clients. This makes the reported personalization improvement an artifact of in-sample evaluation rather than evidence of personalized generalization. The ground-truth concern affects both centralized and federated training equally, so it weakens the absolute stress-prediction validity but not specifically the FL-versus-centralized or personalization comparisons. The in-sample personalization issue, by contrast, uniquely undermines the paper's second headline claim. I am not alleging misconduct; I am noting that no held-out split is described, and with no code or data released, the result cannot be independently checked. The appropriate response is to require the authors to provide a within-subject held-out evaluation before the personalization claim is accepted. Since the reader's verdict was already CONDITIONAL, my concern reinforces that condition rather than moving the verdict.","tokens_in":11919,"tokens_out":2614,"duration_ms":29885,"concrete_test":"Re-evaluate the personalization claim with a within-subject held-out protocol. For each subject with five assembly sessions, fine-tune the global model on assemblies 1–4 and evaluate on assembly 5, and compare against the global model evaluated on that same held-out assembly; alternatively, use leave-one-assembly-out cross-validation within each subject. Additionally, ensure the global model in the same comparison is also evaluated under a leave-one-subject-out protocol so that both columns are strictly out-of-sample. If the personalized RMSE is not lower than the global RMSE on held-out data, the Table II improvement is due to in-sample evaluation. This test directly settles whether the personalization claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest claim includes per-subject fine-tuning reducing RMSE to 0.25 ± 0.06 (Table II). Section VIII states that, during FL training, each client's fine-tuned model was stored locally after every round, and the personalized model is then compared with the global model 'on that particular subject,' where the global model 'includes the specific subject as one of the clients.' There is no mention of any held-out split for the fine-tuned model. If the personalized model is trained on all of a subject's available samples and then evaluated on those same samples, a lower RMSE can be explained by overfitting or memorization, not by genuine personalization. In addition, the comparison in Table II is not on equal footing: the global model in that table includes the test subject among its training clients, whereas the headline FL-versus-centralized comparison in Table I uses leave-one-subject-out evaluation. The Table II result is therefore not a valid demonstration that FL 'allows for enhancing personalization,' which is a central advertised contribution. This concern does not invalidate the Table I LOSO comparison between FL and centralized training, but it directly undermines the personalization component of the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a federated learning (FL) framework for personalized mental-state (stress) evaluation in human-robot collaboration, using multimodal physiological signals (EEG, ECG, EDA, EMG, RESP). The system is deployed on an Android/STM32 edge device, with on-device training and Flower-based FedAvg aggregation. Experiments with 33 subjects performing an aircraft-engine assembly task report that the federated global model achieves lower leave-one-subject-out RMSE (0.26 ± 0.05) than a centralized model (0.38 ± 0.076), and that per-subject fine-tuning further reduces RMSE (0.25 ± 0.06). The paper also describes the hardware, data synchronization, and the overall HRC framework.","tokens_in":12078,"tokens_out":4264,"duration_ms":45994,"significance":"If validated, the work would make a useful contribution by demonstrating that FL can train a stress-prediction model for industrial HRC without sharing raw physiological data, and by providing a real-world deployment with a cobot. The direct LOSO comparison between federated and centralized training on a non-public industrial dataset is a reasonable design, and the on-device training infrastructure is a nontrivial engineering achievement. However, the evaluation has important weaknesses: the ground-truth labels are extremely coarse (one STAI-Y1 score per assembly stretched over many windows), the personalization comparison in Table II lacks any held-out split, and no significance tests are reported. These issues affect the two central claims of the paper: the FL-versus-centralized comparison and the personalization benefit.","major_comments":[{"comment":"The regression target is a single STAI-Y1 score per assembly, min-max normalized per subject, and this constant label is assigned to every 60-second physiological window of that assembly. This ignores within-assembly stress dynamics and reduces the effective number of independent labels to five per subject. Moreover, because the min-max normalization is subject-specific and is computed before the LOSO split, the test subject's normalized labels depend on that subject's own data, which is a form of leakage and makes the RMSE values not directly comparable across subjects. This issue is load-bearing for both Table I and Table II, and the authors should either use a label that varies within the assembly (e.g., per-segment self-report or a validated continuous stress signal) or clearly restrict the claim to predicting average assembly-level stress, with normalization fitted only on training subjects.","section":"Section V-B and Section VII"},{"comment":"The personalized models are fine-tuned on a subject's data during FL training and then evaluated 'on that particular subject,' with the global model that 'includes the specific subject as one of the clients.' No held-out partition is described for the fine-tuned model. Therefore the comparison in Table II is between a model fine-tuned and evaluated on the same samples and a global model that has already seen the test subject; the reported RMSE reduction from 0.34 to 0.25 can be explained by overfitting or memorization, not by genuine personalization. The authors should evaluate personalization on a held-out assembly or session of each subject and compare against a global model trained without that subject, as in Table I.","section":"Section VIII, Table II"},{"comment":"The central claim that FL outperforms centralized training rests on the difference between RMSE 0.38 ± 0.076 and 0.26 ± 0.05, but no significance test is reported. Additionally, the manuscript states that the RMSE of the 'best epoch/round' is reported, which is a form of test-set selection; a principled early-stopping or fixed-budget criterion is required. The training budgets are also not matched: the centralized model trains for 500 epochs on all subjects per epoch, while the FL model trains for 50 rounds with 8 random clients per round and 5 local epochs, so the comparison conflates algorithm with computational budget. Please report per-subject paired errors, a paired significance test (e.g., Wilcoxon signed-rank), and either match the total number of gradient updates or justify the chosen budgets.","section":"Section VIII, Table I"},{"comment":"Several hyperparameters—Huber loss delta (0.05), learning rate, number of local epochs, number of clients per round, window length, and architecture sizes—are fixed without sensitivity analysis or a validation-based selection procedure. The reported advantage of FL over centralized training may depend on these specific choices. Furthermore, the subject-specific min-max normalization of both features and labels (Section VII) is performed before the LOSO split; for the test subject this uses statistics of the test data and should be replaced by normalization fitted only on training subjects. The authors should provide at least a small sensitivity study or justify the choices by reference to prior work.","section":"Section V-C and Section VIII"}],"minor_comments":[{"comment":"The organization paragraph says 'Section V explains ... while Section IV analyze ...', but Section IV is presented before Section V; please correct the order.","section":"Section I"},{"comment":"The inventory is called 'Stress Trait Anxiety Inventory-Y1', but the standard name is the State-Trait Anxiety Inventory (STAI-Y1); please use the correct name.","section":"Section V-B"},{"comment":"The global-model RMSE in Table II (0.34) differs from the distributed model RMSE in Table I (0.26); the manuscript should clarify that Table II reports the final-round global model evaluated with the test subject included among the clients, so the two quantities are not the same.","section":"Table II caption"},{"comment":"There are several formatting issues, such as '31.94±11.95', 'delay of8samples', '500epochs', and 'FedAvgAndroid' without a description; please standardize spacing and define all abbreviations.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is more of a systems/application paper, and the evaluation rigor is below what I would expect for a journal submission. The personalization claim in Table II is not supported by the current evaluation protocol, and the ground-truth weakness is a concern for both main tables. If the authors can provide a proper held-out personalization evaluation and significance tests, the FL-versus-centralized comparison may still be salvageable; otherwise the contribution is primarily an engineering demonstration, which might be better suited to a conference venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: the paper has one solid experiment and one broken table. The FL-versus-centralized LOSO comparison in Table I is worth a look; the personalization comparison in Table II does not establish what it claims, because the fine-tuned model is evaluated on the same subject data used for fine-tuning, and the global model in that table includes the test subject as a client. That's not an apples-to-apples comparison, so the advertised personalization benefit is not supported by the evidence.\n\nWhat's genuinely new: a 33-subject aircraft-engine assembly experiment with multimodal physiological signals (EEG, ECG, EDA, EMG, respiration), a real on-device training pipeline on Android phones using Flower, and a working integration with a Fanuc cobot. That's a nontrivial engineering effort, and the paper gives a clear description of the synchronization and data-flow. The FL-vs-centralized LOSO result (RMSE 0.26 vs 0.38) is interesting if the ground truth holds up.\n\nWhere the soft spots are, in order of severity. First, the ground truth: one STAI-Y1 questionnaire score per assembly is stretched over all 60-second windows of that assembly. That's a very coarse label, and it's not clear the model is learning stress dynamics rather than task phase. Second, subject-specific min-max normalization is applied before the LOSO split, which can leak information about the test subject's label distribution into the features. That affects Table I as well. Third, Table II: as described in Section VIII, the personalized model is fine-tuned on a subject's data and then evaluated 'on that particular subject,' and the global model includes that subject as a client. No held-out split is mentioned. So the personalization result is likely overfitting, not personalization. Fourth, no significance tests are reported; given the small sample and the coarse labels, the gap between 0.26 and 0.38 needs a confidence statement.\n\nThe privacy claim is modest and mostly fine: they share model weights, not raw data, and they don't overclaim differential privacy. The citation pattern looks reasonable for a conference paper.\n\nWho this is for: researchers working on FL for physiological sensing or HRC. It's a useful proof-of-concept, but the evaluation needs a rework before the personalization claim can be trusted.\n\nMy recommendation: send it to peer review, but expect heavy revision. The data collection and integration are real; the evaluation design is not yet sound enough for the headline claims.","headline":"The FL-vs-centralized LOSO comparison is worth a look, but the personalization claim in Table II is not supported because it evaluates on the same data used for fine-tuning.","tokens_in":12681,"tokens_out":2821,"would_cite":false,"duration_ms":29253,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated learning can train a multimodal physiological stress predictor for human-robot collaboration that outperforms a centralized model while keeping raw biosignals on the operator's device, with per-user fine-tuning adding further…","keywords":["Federated Learning","Human-Robot Collaboration","Mental State Estimation","Stress Prediction","Physiological Signals","Personalization","Multimodal Deep Learning","Industry 5.0"],"falsifier":"Recompute the leave-one-subject-out RMSE using an evaluation protocol where the subject-wise min-max normalization is fitted only on training subjects and applied to the left-out subject's labels; if the federated advantage over centralized training disappears or reverses, the reported comparison was an artifact of label normalization. Alternatively, replace the single per-assembly STAI label with per-window labels derived from continuous physiological stress markers and check whether the 0.26 versus 0.38 gap persists.","tokens_in":11658,"feed_emoji":"🤖","tokens_out":3224,"duration_ms":35243,"temperature":0.7,"pith_summary":"This paper claims that federated learning can train a multimodal physiological stress predictor for human-robot collaboration that is more accurate per person than a centralized model, while the raw biosignals never leave the operator's device. In leave-one-subject-out tests with 33 people assembling an aircraft engine, the federated global model reached an RMSE of 0.26 ± 0.05, against 0.38 ± 0.076 for centralized training. Adding per-subject fine-tuning brought the RMSE to 0.25 ± 0.06, beating the shared global model's 0.34 ± 0.07. If correct, this means privacy-preserving, locally trained models can also be the better personalized models for adaptive robot behavior.","feed_headline":"Federated stress model tops centralized, keeps biosignals private","feed_subtitle":"In a 33-person aircraft assembly test, the federated global model reached RMSE 0.26 while centralized scored 0.38.","key_machinery":"The machinery is a multimodal regression network with an EEG convolutional backbone and a biophysical multilayer-perceptron backbone, fused late and fed to a small MLP regressor, trained by Federated Averaging across Android devices through the Flower framework. The regression target is a subject-wise min-max normalized STAI-Y1 anxiety score assigned to each 60-second window of features gathered during one assembly run. On-device training uses the Huber loss with a threshold of 0.05, and personalization is achieved by keeping a locally fine-tuned copy of the global model for each user.","core_discovery":"The central discovery is that a model trained by Federated Averaging, whose weights are aggregated from on-device training on each operator's local device, predicts self-reported stress in a collaborative assembly task better than a model trained centrally on all the same data pooled together. The paper attributes the gain to the federated procedure combined with per-client fine-tuning: the global model provides a shared baseline, and each client keeps a locally fine-tuned personalized version that is more accurate for that specific operator. This is demonstrated by the leave-one-subject-out RMSE comparison between centralized and distributed training, and by the comparison between the global and personalized models on each client, using synchronized EEG, ECG, EDA, EMG, and respiration features as inputs.","pith_inferences":["The reported federated advantage over centralized training may hinge on the subject-wise min-max normalization of the self-report labels; a replication using normalization fitted only on training subjects would clarify whether the comparison is an artifact.","A testable extension is validating the STAI-Y1-derived label against continuous physiological stress markers, since the paper's conclusions would be stronger if per-window labels agreed with an independent stress signal.","The robot adaptation loop is described but not end-to-end evaluated; measuring task outcomes such as completion time, errors, or subjective workload under personalized versus global stress estimates would test the claimed practical benefit."],"forward_implications":["Federated stress prediction can be deployed in industrial settings without transferring raw physiological recordings, easing privacy and consent constraints for workers.","Personalized local models yield better per-operator stress estimates than a shared global model, enabling robot behavior adaptation that responds to an individual's actual mental state.","The framework only requires transmission of model weights, so a baseline mental-state model can keep improving as operators use the system, without storing biometric data centrally.","The same pipeline could be extended to other mental states or additional modalities, such as facial expressions or speech, to improve stress detection accuracy."],"supporting_citations":[{"why":"Introduces Federated Averaging, the core algorithm used for aggregating locally trained model weights into the global model.","marker":"[7]"},{"why":"Provides the Flower framework used to implement the federated training and aggregation across Android clients.","marker":"[30]"},{"why":"Defines the STAI-Y1 questionnaire that supplies the ground-truth stress labels for each assembly.","marker":"[35]"},{"why":"Supplies the cvxEDA algorithm that splits electrodermal activity into phasic and tonic components for feature extraction.","marker":"[34]"},{"why":"Shows a federated learning approach for cognitive workload estimation, the related work this paper extends to stress prediction and personalization.","marker":"[21]"}],"fun_headline_variants":["Federated learning personalizes stress detection for human-robot teams","Privacy-preserving AI improves stress prediction in human-robot work","On-device training yields better stress models for adaptive robots","Federated stress model beats centralized, keeps biosignals private","Personalized stress evaluation via federated learning in HRC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire RMSE comparison rests on treating one post-assembly STAI-Y1 questionnaire score, min-max normalized per subject, as the true stress label for every 60-second window of that assembly; if that proxy is not a valid high-resolution stress signal, the reported numbers lose their meaning.","fun_headline_variants_meta":{"raw":{"variants":["Federated learning personalizes stress detection for human-robot teams","Privacy-preserving AI improves stress prediction in human-robot work","On-device training yields better stress models for adaptive robots","Federated stress model beats centralized, keeps biosignals private","Personalized stress evaluation via federated learning in HRC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000865,"raw_usage":{"total_tokens":3715,"prompt_tokens":877,"completion_tokens":2838,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":2766}},"tokens_in":493,"tokens_out":2838,"duration_ms":24316,"temperature":1.0,"reasoning_tokens":2766,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:54:55.154801+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the leave-one-subject-out RMSE using an evaluation protocol where the subject-wise min-max normalization is fitted only on training subjects and applied to the left-out subject's labels; if the federated advantage over centralized training disappears or reverses, the reported comparison was an artifact of label normalization. Alternatively, replace the single per-assembly STAI label with per-window labels derived from continuous physiological stress markers and check whether the 0.26 versus 0.38 gap persists.","supporting_citations":[{"cited_title":"Communication-Efficient Learning of Deep Networks from Decentral- ized Data,","cited_arxiv_id":null,"evidence_quote":"Introduces Federated Averaging, the core algorithm used for aggregating locally trained model weights into the global model."},{"cited_title":"Spielberger and R","cited_arxiv_id":null,"evidence_quote":"Defines the STAI-Y1 questionnaire that supplies the ground-truth stress labels for each assembly."},{"cited_title":"cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing,","cited_arxiv_id":null,"evidence_quote":"Supplies the cvxEDA algorithm that splits electrodermal activity into phasic and tonic components for feature extraction."},{"cited_title":"Federated Learning for Privacy- aware Cognitive Workload Estimation,","cited_arxiv_id":null,"evidence_quote":"Shows a federated learning approach for cognitive workload estimation, the related work this paper extends to stress prediction and personalization."}],"review_version":1}