REVIEW 4 major objections 4 minor 35 references
Personalized Mental State Evaluation in Human-Robot Interaction using Federated Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Federated learning can train a multimodal physiological stress predictor for human-robot collaboration that outperforms a centralized model while keeping raw biosignals on the operator's device, with per-user fine-tuning adding further…
desk verdict The FL-vs-centralized LOSO comparison is worth a look, but the personalization claim in Table II is not supported because it evaluates on the same data used for fine-tuning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a multimodal regression network with an EEG convolutional backbone and a biophysical multilayer-perceptron backbone, fused late and fed to a small MLP regressor, trained by Federated Averaging across Android devices through the Flower framework. The regression target is a subject-wise min-max normalized STAI-Y1 anxiety score assigned to each 60-second window of features gathered during one assembly run. On-device training uses the Huber loss with a threshold of 0.05, and personalization is achieved by keeping a locally fine-tuned copy of the global model for each user.
What would settle it
Recompute the leave-one-subject-out RMSE using an evaluation protocol where the subject-wise min-max normalization is fitted only on training subjects and applied to the left-out subject's labels; if the federated advantage over centralized training disappears or reverses, the reported comparison was an artifact of label normalization. Alternatively, replace the single per-assembly STAI label with per-window labels derived from continuous physiological stress markers and check whether the 0.26 versus 0.38 gap persists.
Extended reading notes
Core claim
The central discovery is that a model trained by Federated Averaging, whose weights are aggregated from on-device training on each operator's local device, predicts self-reported stress in a collaborative assembly task better than a model trained centrally on all the same data pooled together. The paper attributes the gain to the federated procedure combined with per-client fine-tuning: the global model provides a shared baseline, and each client keeps a locally fine-tuned personalized version that is more accurate for that specific operator. This is demonstrated by the leave-one-subject-out RMSE comparison between centralized and distributed training, and by the comparison between the global and personalized models on each client, using synchronized EEG, ECG, EDA, EMG, and respiration features as inputs.
Load-bearing premise
The entire RMSE comparison rests on treating one post-assembly STAI-Y1 questionnaire score, min-max normalized per subject, as the true stress label for every 60-second window of that assembly; if that proxy is not a valid high-resolution stress signal, the reported numbers lose their meaning.
Editorial extensions
If this is right
- Federated stress prediction can be deployed in industrial settings without transferring raw physiological recordings, easing privacy and consent constraints for workers.
- Personalized local models yield better per-operator stress estimates than a shared global model, enabling robot behavior adaptation that responds to an individual's actual mental state.
- The framework only requires transmission of model weights, so a baseline mental-state model can keep improving as operators use the system, without storing biometric data centrally.
- The same pipeline could be extended to other mental states or additional modalities, such as facial expressions or speech, to improve stress detection accuracy.
Reading between the lines
- The reported federated advantage over centralized training may hinge on the subject-wise min-max normalization of the self-report labels; a replication using normalization fitted only on training subjects would clarify whether the comparison is an artifact.
- A testable extension is validating the STAI-Y1-derived label against continuous physiological stress markers, since the paper's conclusions would be stronger if per-window labels agreed with an independent stress signal.
- The robot adaptation loop is described but not end-to-end evaluated; measuring task outcomes such as completion time, errors, or subjective workload under personalized versus global stress estimates would test the claimed practical benefit.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a federated learning (FL) framework for personalized mental-state (stress) evaluation in human-robot collaboration, using multimodal physiological signals (EEG, ECG, EDA, EMG, RESP). The system is deployed on an Android/STM32 edge device, with on-device training and Flower-based FedAvg aggregation. Experiments with 33 subjects performing an aircraft-engine assembly task report that the federated global model achieves lower leave-one-subject-out RMSE (0.26 ± 0.05) than a centralized model (0.38 ± 0.076), and that per-subject fine-tuning further reduces RMSE (0.25 ± 0.06). The paper also describes the hardware, data synchronization, and the overall HRC framework.
Significance. If validated, the work would make a useful contribution by demonstrating that FL can train a stress-prediction model for industrial HRC without sharing raw physiological data, and by providing a real-world deployment with a cobot. The direct LOSO comparison between federated and centralized training on a non-public industrial dataset is a reasonable design, and the on-device training infrastructure is a nontrivial engineering achievement. However, the evaluation has important weaknesses: the ground-truth labels are extremely coarse (one STAI-Y1 score per assembly stretched over many windows), the personalization comparison in Table II lacks any held-out split, and no significance tests are reported. These issues affect the two central claims of the paper: the FL-versus-centralized comparison and the personalization benefit.
major comments (4)
- [Section V-B and Section VII] The regression target is a single STAI-Y1 score per assembly, min-max normalized per subject, and this constant label is assigned to every 60-second physiological window of that assembly. This ignores within-assembly stress dynamics and reduces the effective number of independent labels to five per subject. Moreover, because the min-max normalization is subject-specific and is computed before the LOSO split, the test subject's normalized labels depend on that subject's own data, which is a form of leakage and makes the RMSE values not directly comparable across subjects. This issue is load-bearing for both Table I and Table II, and the authors should either use a label that varies within the assembly (e.g., per-segment self-report or a validated continuous stress signal) or clearly restrict the claim to predicting average assembly-level stress, with normalization fitted only on training subjects.
- [Section VIII, Table II] The personalized models are fine-tuned on a subject's data during FL training and then evaluated 'on that particular subject,' with the global model that 'includes the specific subject as one of the clients.' No held-out partition is described for the fine-tuned model. Therefore the comparison in Table II is between a model fine-tuned and evaluated on the same samples and a global model that has already seen the test subject; the reported RMSE reduction from 0.34 to 0.25 can be explained by overfitting or memorization, not by genuine personalization. The authors should evaluate personalization on a held-out assembly or session of each subject and compare against a global model trained without that subject, as in Table I.
- [Section VIII, Table I] The central claim that FL outperforms centralized training rests on the difference between RMSE 0.38 ± 0.076 and 0.26 ± 0.05, but no significance test is reported. Additionally, the manuscript states that the RMSE of the 'best epoch/round' is reported, which is a form of test-set selection; a principled early-stopping or fixed-budget criterion is required. The training budgets are also not matched: the centralized model trains for 500 epochs on all subjects per epoch, while the FL model trains for 50 rounds with 8 random clients per round and 5 local epochs, so the comparison conflates algorithm with computational budget. Please report per-subject paired errors, a paired significance test (e.g., Wilcoxon signed-rank), and either match the total number of gradient updates or justify the chosen budgets.
- [Section V-C and Section VIII] Several hyperparameters—Huber loss delta (0.05), learning rate, number of local epochs, number of clients per round, window length, and architecture sizes—are fixed without sensitivity analysis or a validation-based selection procedure. The reported advantage of FL over centralized training may depend on these specific choices. Furthermore, the subject-specific min-max normalization of both features and labels (Section VII) is performed before the LOSO split; for the test subject this uses statistics of the test data and should be replaced by normalization fitted only on training subjects. The authors should provide at least a small sensitivity study or justify the choices by reference to prior work.
minor comments (4)
- [Section I] The organization paragraph says 'Section V explains ... while Section IV analyze ...', but Section IV is presented before Section V; please correct the order.
- [Section V-B] The inventory is called 'Stress Trait Anxiety Inventory-Y1', but the standard name is the State-Trait Anxiety Inventory (STAI-Y1); please use the correct name.
- [Table II caption] The global-model RMSE in Table II (0.34) differs from the distributed model RMSE in Table I (0.26); the manuscript should clarify that Table II reports the final-round global model evaluated with the test subject included among the clients, so the two quantities are not the same.
- [Throughout] There are several formatting issues, such as '31.94±11.95', 'delay of8samples', '500epochs', and 'FedAvgAndroid' without a description; please standardize spacing and define all abbreviations.
Circularity Check
The personalized-model comparison in Table II evaluates the model on the same subject data used for fine-tuning, so the reported personalization gain is a training-set artifact rather than a demonstrated prediction improvement.
-
fitted input called prediction
[Section VIII, Table II (personalized vs global model comparison)]
"During FL training, each client’s fine-tuned model was stored locally after every round for the respective subject. ... In Tab. II we compared the RMSE obtained as the mean performance of the global model, which includes the specific subject as one of the clients, and the results obtained from the model personalized on that specific subject. As expected, the personalized model improves the performance on that particular subject, without ever sharing the subject’s data."
The personalized model is obtained by fine-tuning the global model on a specific subject's local data (Section III: 'it is fine-tuned on the specific subject data'; Section VIII: fine-tuned model stored 'for the respective subject'). Table II then reports that this personalized model outperforms the global model 'on that particular subject,' but the global model in this comparison explicitly 'includes the specific subject as one of the clients.' No held-out split, validation set, or separate test partition is described for Table II. The RMSE is therefore computed on data that were used to fine-tune the personalized model, so the lower RMSE (0.25 vs 0.34) can be explained by memorization or overfitting rather than by genuine personalization.
full rationale
The paper's main FL-versus-centralized comparison (Table I) uses leave-one-subject-out validation and is not circular: the distributed and centralized models are evaluated under the same held-out protocol, and the reported RMSE values are empirical outcomes rather than constructions. However, the personalization claim (Table II) is evaluated on the same subject data used for local fine-tuning, with no held-out partition described, and the global baseline in that table includes the test subject as a training client. The observed personalization improvement is therefore by construction and cannot support the conclusion that FL 'allows for enhancing personalization.' The per-subject min-max normalization of labels and features before the split adds a further evaluation confound, but the primary circular step is the Table II training-set evaluation. The self-citations in the paper are background references and are not load-bearing for the central derivation.
Assumptions & free parameters
free parameters (7)
- Huber loss delta =
0.05
- Learning rate =
1e-4
- FL rounds and local epochs =
50 rounds, 5 local epochs
- Minimum clients per round =
8
- Model architecture sizes =
Conv2D(32,32), MLP hidden 64/128/64
- Window lengths =
60s biophysical, 5s EEG
- Subject-specific min-max normalization =
per-subject min/max of STAI and features
assumptions (5)
- domain assumption STAI-Y1 score after each assembly is a valid continuous stress label for all 60-second windows within that assembly.
- domain assumption Per-subject min-max normalization of features and labels does not leak test-subject statistics in LOSO.
- standard math FedAvg converges to a useful global model under client heterogeneity and intermittent availability.
- domain assumption The 33 recruited participants, mostly from the authors' facility, represent industrial operators.
- domain assumption Hardware synchronization via initial timestamp plus sampling period and re-sync every 10 seconds correctly aligns all sensor streams.
Cite this review
Pith. "Pith review of Personalized Mental State Evaluation in Human-Robot Interaction using Federated Learning." pith.science (2026). https://pith.science/paper/6YF3XTL5
@misc{pith2026250620212,
author = {Pith},
title = {Pith review of: Personalized Mental State Evaluation in Human-Robot Interaction using Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/6YF3XTL5}},
note = {Machine review of arXiv:2506.20212}
}
read the original abstract
With the advent of Industry 5.0, manufacturers are increasingly prioritizing worker well-being alongside mass customization. Stress-aware Human-Robot Collaboration (HRC) plays a crucial role in this paradigm, where robots must adapt their behavior to human mental states to improve collaboration fluency and safety. This paper presents a novel framework that integrates Federated Learning (FL) to enable personalized mental state evaluation while preserving user privacy. By leveraging physiological signals, including EEG, ECG, EDA, EMG, and respiration, a multimodal model predicts an operator's stress level, facilitating real-time robot adaptation. The FL-based approach allows distributed on-device training, ensuring data confidentiality while improving model generalization and individual customization. Results demonstrate that the deployment of an FL approach results in a global model with performance in stress prediction accuracy comparable to a centralized training approach. Moreover, FL allows for enhancing personalization, thereby optimizing human-robot interaction in industrial settings, while preserving data privacy. The proposed framework advances privacy-preserving, adaptive robotics to enhance workforce well-being in smart manufacturing.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Ergonomic human-robot collaboration in industry: A review,
M. Lorenzini, M. Lagomarsino, L. Fortini, S. Gholami, and A. Ajoudani, “Ergonomic human-robot collaboration in industry: A review,”Frontiers in Robotics and AI, vol. 9, 2023
work page 2023
-
[2]
A. Arif, Z. Zakeri, A. Omurtag, P. Breedon, and A. Khalid, “Neu- rophysiological approach for psychological safety: Enhancing mental health in human–robot collaboration in smart manufacturing setups using neuroimaging,”Information, vol. 15, no. 10, 2024
work page 2024
-
[3]
A. Bussolan, S. Baraldo, L. M. Gambardella, and A. Valente, “Assessing the impact of human-robot collaboration on stress levels and cognitive load in industrial assembly tasks,” inISR Europe 2023; 56th Interna- tional Symposium on Robotics, pp. 78–85, 2023
work page 2023
-
[4]
A vision-based human digital twin modeling approach for adaptive human–robot collaboration,
J. Fan, P. Zheng, and C. K. M. Lee, “A vision-based human digital twin modeling approach for adaptive human–robot collaboration,”Journal of Manufacturing Science and Engineering, vol. 145, p. 121002, 07 2023
work page 2023
-
[5]
Robotics in manufacturing: A review of advances in automation and workforce implications,
R. A. Adebayo, N. C. Obiuto, I. C. Festus-Ikhuoria, and O. K. Olajiga, “Robotics in manufacturing: A review of advances in automation and workforce implications,”International Journal of Advanced Multidisci- plinary Research and Studies, 2024
work page 2024
-
[6]
On the impact of robot personalization on human-robot interaction: A review,
J. Yang, C. Vindolet, J. R. G. Olvera, and G. Cheng, “On the impact of robot personalization on human-robot interaction: A review,” Jan. 2024. arXiv:2401.11776 [cs]
arXiv 2024
-
[7]
Communication-Efficient Learning of Deep Networks from Decentral- ized Data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentral- ized Data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics(A. Singh and J. Zhu, eds.), vol. 54 ofProceedings of Machine Learning Research, pp. 1273–1282, PMLR, 20–22 Apr 2017
work page 2017
-
[8]
Advances and open problems in federated learning,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, R. G. L. D’Oliveira, H. Eichner, S. E. Rouayheb, D. Evans, J. Gardner, Z. Garrett, A. Gasc ´on, B. Ghazi, P. B. Gibbons, M. Gruteser, Z. Harchaoui, C. He, L. He, Z. Huo, B. Hutchinson, J. Hsu, M. Jaggi, T. Javidi, G. Joshi, M. Khodak...
work page 2021
Show all 35 references
-
[9]
Federated learning of socially appropriate agent behaviours in simulated home environments,
S. Checker, N. Churamani, and H. Gunes, “Federated learning of socially appropriate agent behaviours in simulated home environments,” 2024
2024
-
[10]
Federated continual learning for socially aware robotics,
L. Guerdan and H. Gunes, “Federated continual learning for socially aware robotics,” in2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1522–1529, 2023
2023
-
[11]
Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach,
A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach,” inProceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, (Red Hook, NY , USA), Curran ...
2020
-
[12]
Multi- modal fusion stress detector for enhanced human-robot collaboration in industrial assembly tasks,
A. Bussolan, S. Baraldo, L. M. Gambardella, and A. Valente, “Multi- modal fusion stress detector for enhanced human-robot collaboration in industrial assembly tasks,” in2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), pp. 978– 984, 2024
2024
-
[13]
Friend or foe? understanding assembly workers’ acceptance of human- robot collaboration,
A. Meissner, A. Tr ¨ubswetter, A. S. Conti-Kufner, and J. Schmidtler, “Friend or foe? understanding assembly workers’ acceptance of human- robot collaboration,”J. Hum.-Robot Interact., vol. 10, July 2020
2020
-
[14]
Advancing Human- Robot Collaboration by Robust Speech Recognition in Smart Manu- facturing,
O. Avram, C. Fasana, S. Baraldo, and A. Valente, “Advancing Human- Robot Collaboration by Robust Speech Recognition in Smart Manu- facturing,” inEuropean Robotics Forum 2024, pp. 168–173, Springer Nature Switzerland, 2024
2024
-
[15]
Federated learning in robotic and autonomous systems,
Y . Xianjia, J. P. Queralta, J. Heikkonen, and T. Westerlund, “Federated learning in robotic and autonomous systems,”Procedia Computer Sci- ence, vol. 191, pp. 135–142, 2021. The 18th International Conference on Mobile Systems and Pervasive Computing (MobiSPC), The 16th Inter-...
2021
-
[16]
Asynchronous federated learning system for human–robot touch interaction,
J. J. Gamboa-Montero, F. Alonso-Martin, S. Marques-Villarroya, J. Se- queira, and M. A. Salichs, “Asynchronous federated learning system for human–robot touch interaction,”Expert Systems with Applications, vol. 211, p. 118510, 2023
2023
-
[17]
A systematic survey on the application of federated learning in mental state detection and human activity recognition,
A. Grataloup and M. Kurpicz-Briki, “A systematic survey on the application of federated learning in mental state detection and human activity recognition,”Frontiers in Digital Health, vol. 6, 2024
2024
-
[18]
Federated Learning for Affective Computing Tasks,
K. Somandepalli, H. Qi, B. Eoff, A. Cowen, K. Audhkhasi, J. Belanich, and B. Jou, “Federated Learning for Affective Computing Tasks,” in2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII), (Nara, Japan), pp. 1–8, IEEE, Oct. 2022
2022
-
[19]
Privacy-preserving speech emotion recognition through semi-supervised federated learning,
V . Tsouvalas, T. Ozcelebi, and N. Meratnia, “Privacy-preserving speech emotion recognition through semi-supervised federated learning,” in 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Work- shops), ...
2022
-
[20]
Multimodal federated learning on iot data,
Y . Zhao, P. Barnaghi, and H. Haddadi, “Multimodal federated learning on iot data,” in2022 IEEE/ACM Seventh International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 43–54, 2022
2022
-
[21]
Federated Learning for Privacy- aware Cognitive Workload Estimation,
D. Fenoglio, D. Josifovski, A. Gobbetti, M. Formo, H. Gjoreski, M. Gjoreski, and M. Langheinrich, “Federated Learning for Privacy- aware Cognitive Workload Estimation,” inProceedings of the 22nd In- ternational Conference on Mobile and Ubiquitous Multimedia, (Vienna Austria), ...
2023
-
[22]
Commission, D.-G
E. Commission, D.-G. for Research, Innovation, M. Breque, L. De Nul, and A. Petridis,Industry 5.0 – Towards a sustainable, human-centric and resilient European industry. Publications Office of the European Union, 2021
2021
-
[23]
Promoting operator’s wellbeing in Industry 5.0: detecting mental and physical fatigue,
V . Villani, M. Gabbi, and L. Sabattini, “Promoting operator’s wellbeing in Industry 5.0: detecting mental and physical fatigue,” in2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), (Prague, Czech Republic), pp. 2030–2036, IEEE, Oct. 2022
2022
-
[24]
PRO- MIND: Proximity and Reactivity Optimisation of robot Motion to tune safety limits, human stress, and productivity in INDustrial settings,
M. Lagomarsino, M. Lorenzini, E. D. Momi, and A. Ajoudani, “PRO- MIND: Proximity and Reactivity Optimisation of robot Motion to tune safety limits, human stress, and productivity in INDustrial settings,” Sept. 2024. arXiv:2409.06864 [cs]
2024 arXiv
-
[25]
Relieving operators’ workload: Towards affective robotics in industrial scenarios,
C. T. Landi, V . Villani, F. Ferraguti, L. Sabattini, C. Secchi, and C. Fantuzzi, “Relieving operators’ workload: Towards affective robotics in industrial scenarios,”Mechatronics, vol. 54, pp. 144–154, Oct. 2018
2018
-
[26]
Mental State Assessment and Validation Using Personalized Physiological Biometrics,
A. N. Patel, M. D. Howard, S. M. Roach, A. P. Jones, N. B. Bryant, C. S. H. Robinson, V . P. Clark, and P. K. Pilly, “Mental State Assessment and Validation Using Personalized Physiological Biometrics,”Frontiers in Human Neuroscience, vol. 12, p. 221, June 2018
2018
-
[27]
Acceptability of robotic manipulators in shared working environments through human-like re- dundancy resolution,
A. M. Zanchettin, L. Bascetta, and P. Rocco, “Acceptability of robotic manipulators in shared working environments through human-like re- dundancy resolution,”Applied Ergonomics, vol. 44, pp. 982–989, Nov. 2013
2013
-
[28]
Deliberative robotics – a novel interactive control framework enhancing human-robot collaboration,
A. Valente, G. Pavesi, M. Zamboni, and E. Carpanzano, “Deliberative robotics – a novel interactive control framework enhancing human-robot collaboration,”CIRP Annals, vol. 71, no. 1, pp. 21–24, 2022
2022
-
[29]
Generalized behavior frame- work for mobile robots teaming with humans in harsh environments,
O. Avram, S. Baraldo, and A. Valente, “Generalized behavior frame- work for mobile robots teaming with humans in harsh environments,” Frontiers in Robotics and AI, vol. 9, pp. 1–20, 2022
2022
-
[30]
Flower: A friendly federated learning research framework,
D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y . Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusm ˜ao,et al., “Flower: A friendly federated learning research framework,”arXiv preprint arXiv:2007.14390, 2020
2007 arXiv
-
[31]
Benchmarking robustness of ai-enabled multi-sensor fusion systems: Challenges and opportunities,
X. Gao, Z. Wang, Y . Feng, L. Ma, Z. Chen, and B. Xu, “Benchmarking robustness of ai-enabled multi-sensor fusion systems: Challenges and opportunities,” inProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software En...
2023
-
[32]
rwisdm: Repaired wisdm, a public dataset for human activity recognition,
M. Heydarian and T. E. Doyle, “rwisdm: Repaired wisdm, a public dataset for human activity recognition,”arXiv preprint arXiv:2305.10222, 2023
2023 arXiv
-
[33]
A time-synchronized framework for bluetooth low energy wireless sensor networks,
A. Pignata, V . Fra, E. Macii, and G. Urgese, “A time-synchronized framework for bluetooth low energy wireless sensor networks,” in2024 IEEE 18th International Conference on Application of Information and Communication Technologies (AICT), pp. 1–6, IEEE, 2024
2024
-
[34]
cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing,
A. Greco, G. Valenza, A. Lanata, E. Scilingo, and L. Citi, “cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing,” IEEE Transactions on Biomedical Engineering, pp. 1–1, 2016
2016
-
[35]
Spielberger and R
C. Spielberger and R. Gorsuch,Manual for the State-trait Anxiety Inventory (form Y) (”self-evaluation Questionnaire”). Consulting Psy- chologists Press, 1983
1983
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.