Recognition: unknown
Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation
Pith reviewed 2026-05-09 21:00 UTC · model grok-4.3
The pith
A selective correlation method for knowledge distillation produces compact models that estimate ground reaction forces more accurately from noisy insole sensor data than prior approaches.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors introduce Selective Correlation Based Knowledge Distillation (SCKD), which selects features that respect temporal characteristics when constructing correlation maps for transferring knowledge from a large teacher network to a compact student network, and they show through experiments on insole sensor recordings at varied speeds and window sizes that the resulting models outperform existing methods in estimating ground reaction forces.
What carries the argument
Selective Correlation Based Knowledge Distillation (SCKD), a distillation process that restricts correlation-map construction to temporally selected features in order to transfer knowledge efficiently between teacher and student models.
If this is right
- Compact student models become practical for real-time processing on portable devices without laboratory force plates.
- The same distillation setup can be retrained on recordings taken at multiple speeds and still maintain higher accuracy than baselines.
- Interpretability of the transferred knowledge increases because the correlation maps are built only from temporally coherent features.
- The method supplies a concrete way to balance model size against estimation error when sensor data are high-dimensional and noisy.
Where Pith is reading between the lines
- The temporal-selection step inside the correlation maps may transfer usefully to other noisy time-series sensor tasks such as joint-angle prediction or muscle-activity estimation.
- If the accuracy gains persist on clinical populations with gait impairments, the approach could support continuous monitoring outside controlled environments.
- The framework implicitly suggests that future work could test whether the same selective mechanism improves distillation when the teacher and student differ in architecture rather than just size.
Load-bearing premise
That choosing features according to their temporal properties when building correlation maps improves accuracy and reduces noise effects enough to beat standard distillation for ground reaction force estimation.
What would settle it
Running the same teacher-student pairs on an independent insole-sensor dataset collected at a new range of walking speeds and checking whether the reported accuracy advantage over non-selective distillation disappears.
Figures
read the original abstract
Wearable sensor-based human gait analysis holds great promise in healthcare, rehabilitation, clinical diagnosis and monitoring, and sports activities. Specifically, ground reaction force (GRF) provides essential insights into the body's interaction with the ground during movement and is typically measured using instrumented treadmills equipped with force plates. However, such equipment is expensive and restricted to laboratory environments. To enable a more portable solution, wearable insole sensors have been used to measure GRF. These sensors, however, are prone to noise and external interference, which reduces measurement accuracy. Deep learning methodologies could be adopted to address these issues, but they often require significant computing resources to achieve high accuracy, limiting their applicability for real-time analysis on portable devices. To overcome these limitations, we propose Selective Correlation Based Knowledge Distillation (SCKD) for estimating GRF from data collected by insole sensors. Our proposed method utilizes selected features considering temporal characteristics in the process of extracting correlation maps for knowledge transfer, enhancing interpretability and mitigating issues in high dimensional data processing. We demonstrate the effectiveness of the compact models generated by our distillation framework through comparison with existing methods. Various configurations of teacher-student architectures and training approaches are examined based on multiple evaluation criteria, utilizing data collected at different walking speeds and with different window sizes. Experimental results confirm that our approach outperforms existing methods in estimating GRF from wearable insole sensor data. Therefore, our approach offers a reliable and resource-efficient solution for human gait analysis.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Selective Correlation Based Knowledge Distillation (SCKD) to estimate ground reaction force (GRF) from noisy wearable insole sensor data. The core idea is to select features that incorporate temporal characteristics when constructing correlation maps for knowledge transfer from a teacher network to a compact student network. Multiple teacher-student configurations and training regimes are tested on data collected at different walking speeds and window sizes, with the central claim being that the resulting models outperform existing methods in GRF estimation accuracy while remaining suitable for resource-constrained portable devices.
Significance. If the reported results hold, the work offers a practical route to portable, real-time GRF estimation outside laboratory force-plate setups, with direct relevance to clinical gait analysis, rehabilitation, and sports monitoring. The explicit ablation of the selective-correlation step and the use of multiple evaluation criteria (RMSE/MAE across speeds and windows) provide concrete evidence of the method's contribution beyond generic distillation. The approach also addresses high-dimensional sensor noise in a manner that may improve both efficiency and interpretability.
minor comments (3)
- Abstract: the outperformance claim is stated without any numerical support (RMSE/MAE values, baselines, or error bars). Although the experimental results section supplies these details, the abstract should include at least one or two key quantitative findings so that the central claim can be assessed at a glance.
- Experimental results section: the ablation isolating the selective-correlation component is described as showing consistent gains; it would be helpful to report the exact delta in RMSE/MAE and any statistical significance tests for that ablation to strengthen the causal link to the proposed mechanism.
- The manuscript mentions 'various configurations of teacher-student architectures' but does not tabulate the exact network sizes, parameter counts, or inference latencies; adding a compact table with these metrics would better substantiate the resource-efficiency claim.
Simulated Author's Rebuttal
We thank the referee for their positive evaluation of our manuscript on Selective Correlation Based Knowledge Distillation for ground reaction force estimation. We appreciate the recognition of the method's practical value for portable, real-time GRF analysis and the recommendation for minor revision. We will prepare a revised version incorporating any minor editorial or presentational improvements.
Circularity Check
No significant circularity
full rationale
The paper presents an empirical ML framework (SCKD) for GRF estimation from insole sensors, with the central claims resting on experimental outperformance via RMSE/MAE metrics, ablations isolating the selective correlation step, and comparisons to external baselines across walking speeds and window sizes. No derivation chain reduces by construction to fitted parameters, self-definitions, or self-citation load-bearing premises; the temporal feature selection for correlation maps is a standard preprocessing choice evaluated externally rather than assumed or renamed as a prediction. The manuscript is self-contained against reported quantitative results without invoking uniqueness theorems or ansatzes from prior author work.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Knowledge distillation from a larger teacher model can produce a smaller student model that retains high accuracy on the target task
- domain assumption Temporal characteristics in time-series sensor data can be captured effectively through selective correlation maps to improve feature transfer
Reference graph
Works this paper leans on
-
[1]
D. Chen, Y . Cai, X. Qian, R. Ansari, W. Xu, K.-C. Chu, M.-C. Huang, Bring gait lab to everyday life: Gait analysis in terms of activities of daily living, IEEE Internet of Things Journal 7 (2) (2019) 1298–1312
2019
-
[2]
Y . Cai, X. Qian, H. Cao, J. Zheng, W. Xu, M.-C. Huang, mhealth technologies toward active health information collection and tracking in daily life: A dynamic gait monitoring example, IEEE Internet of Things Journal 9 (16) (2022) 15077–15088
2022
-
[3]
L. M. Dang, K. Min, H. Wang, M. J. Piran, C. H. Lee, H. Moon, Sensor-based and vision-based human activity recognition: A comprehensive survey, Pattern Recognition 108 (2020) 107561
2020
-
[4]
Z. Yang, C. Song, F. Lin, J. Langan, W. Xu, A smart environment-adapting timed-up-and-go system powered by sensor-embedded insoles, IEEE Internet of Things Journal 6 (2) (2018) 1298–1305
2018
-
[5]
Brunelli, N
S. Brunelli, N. Gentileschi, M. Iosa, F. R. Fusco, V . Grossi, S. Duri, C. Foti, M. Traballesi, Early balance train- ing with a computerized stabilometric platform in persons with mild hemiparesis in subacute stroke phase: A randomized controlled pilot study, Restorative Neurology and Neuroscience 38 (6) (2020) 467–475. 20
2020
-
[6]
R. Li, Y . Zhang, Y . Jiang, M. Wang, W. H. D. Ang, Y . Lau, Rehabilitation training based on virtual reality for patients with parkinson’s disease in improving balance, quality of life, activities of daily living, and depressive symptoms: a systematic review and meta-regression analysis, Clinical Rehabilitation 35 (8) (2021) 1089–1102
2021
-
[7]
Rajasekaran, E
V . Rajasekaran, E. López-Larraz, F. Trincado-Alonso, J. Aranda, L. Montesano, A. J. Del-Ama, J. L. Pons, V olition-adaptive control for gait training using wearable exoskeleton: preliminary tests with incomplete spinal cord injury individuals, Journal of neuroengineering and rehabilitation 15 (2018) 1–15
2018
-
[8]
D. Su, Z. Hu, J. Wu, P. Shang, Z. Luo, Review of adaptive control for stroke lower limb exoskeleton rehabilitation robot based on motion intention recognition, Frontiers in Neurorobotics 17 (2023) 1186175
2023
-
[9]
Xiang, Z
L. Xiang, Z. Gao, A. Wang, V . Shim, G. Fekete, Y . Gu, J. Fernandez, Rethinking running biomechanics: a critical review of ground reaction forces, tibial bone loading, and the role of wearable sensors, Frontiers in Bioengineering and Biotechnology 12 (2024) 1377383
2024
-
[10]
T. J. Buurke, L. van de Venis, R. den Otter, J. Nonnekes, Keijsers, Comparison of ground reaction force and marker-based methods to estimate mediolateral center of mass displacement and margins of stability during walking, Journal of biomechanics 146 (2023) 111415
2023
-
[11]
J. An, I. Lee, Artificial neural network-based ground reaction force estimation and learning for dynamic-legged robot systems, PeerJ Computer Science 9 (2023) e1720
2023
-
[12]
Key, The analysis of movement, in: J
J. Key, The analysis of movement, in: J. Key (Ed.), Back Pain - A Movement Problem, Elsevier, 2010, pp. 37–54
2010
-
[13]
Sakamoto, Y
S.-I. Sakamoto, Y . Hutabarat, D. Owaki, M. Hayashibe, Ground reaction force and moment estimation through EMG sensing using long short-term memory network during posture coordination, Cyborg Bionic Syst. 4 (2023) 0016
2023
-
[14]
Kluitenberg, S
B. Kluitenberg, S. W. Bredeweg, S. Zijlstra, W. Zijlstra, I. Buist, Comparison of vertical ground reaction forces during overground and treadmill running. a validation study, BMC musculoskeletal disorders 13 (2012) 1–8
2012
-
[15]
A. M. Howell, T. Kobayashi, H. A. Hayes, K. B. Foreman, S. J. M. Bamberg, Kinetic gait analysis using a low-cost insole, IEEE Transactions on Biomedical Engineering 60 (12) (2013) 3284–3290
2013
-
[16]
Y . Shi, L. Du, X. Chen, X. Liao, Z. Yu, Z. Li, C. Wang, S. Xue, Robust gait recognition based on deep cnns with camera and radar sensor fusion, IEEE Internet of Things Journal 10 (12) (2023) 10817–10832
2023
-
[17]
G. T. Burns, J. Deneweth Zendler, R. F. Zernicke, Validation of a wireless shoe insole for ground reaction force measurement, Journal of sports sciences 37 (10) (2019) 1129–1138
2019
-
[18]
J. Lee, G. Li, W. F. Christensen, G. Collins, M. Seeley, A. E. Bowden, D. T. Fullwood, J. Goldsmith, Functional data analyses of gait data measured using in-shoe sensors, Statistics in biosciences 11 (2019) 288–313
2019
-
[19]
J. Chen, Y . Qin, P. Lin, J. Li, Y . Xue, H. Ma, Center of pressure estimation by analyzing walking videos, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 3460–3464
2024
-
[20]
Gehlhar, J.-H
R. Gehlhar, J.-H. Yang, A. D. Ames, Powered prosthesis locomotion on varying terrains: Model-dependent control with real-time force sensing, IEEE Robot. Autom. Lett. 7 (2) (2022) 5151–5158
2022
-
[21]
P. S. Dyer, S. J. M. Bamberg, Instrumented insole vs. force plate: A comparison of center of plantar pressure, in: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2011, pp. 6805–6809
2011
-
[22]
Masani, M
K. Masani, M. Kouzaki, T. Fukunaga, Variability of ground reaction forces during treadmill walking, Journal of applied physiology 92 (5) (2002) 1885–1890. 21
2002
-
[23]
G. E. Hinton, R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313 (5786) (2006) 504–507
2006
-
[24]
Huang, Z
G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in: Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700–4708
2017
-
[25]
H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, P.-A. Muller, Deep learning for time series classification: a review, Data Mining and Knowledge Discovery 33 (4) (2019) 917–963
2019
-
[26]
A. Khan, A. Sohail, U. Zahoora, A. S. Qureshi, A survey of the recent architectures of deep convolutional neural networks, Artificial Intelligence Review 53 (8) (2020) 5455–5516
2020
-
[27]
H. S. Saad, J. F. Zaki, M. M. Abdelsalam, Employing of machine learning and wearable devices in healthcare system: tasks and challenges, Neural Computing and Applications (2024) 1–21
2024
- [28]
-
[29]
Z. Li, F. Liu, W. Yang, S. Peng, J. Zhou, A survey of convolutional neural networks: Analysis, applications, and prospects, IEEE Transactions on Neural Networks and Learning Systems 33 (12) (2022) 6999–7019
2022
- [30]
-
[31]
Hinton, O
G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, in: Proceedings of the NeurIPS Deep Learning and Representation Learning Workshop, V ol. 2, 2015
2015
-
[32]
E. S. Jeon, A. Som, A. Shukla, K. Hasanaj, M. P. Buman, P. Turaga, Role of data augmentation strategies in knowledge distillation for wearable sensor data, IEEE Internet of Things Journal 9 (14) (2022) 12848–12860
2022
-
[33]
J. Gou, B. Yu, S. J. Maybank, D. Tao, Knowledge distillation: A survey, International Journal of Computer Vision 129 (6) (2021) 1789–1819
2021
-
[34]
F. Tung, G. Mori, Similarity-preserving knowledge distillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1365–1374
2019
-
[35]
I. M. Johnstone, D. M. Titterington, Statistical challenges of high-dimensional data (2009)
2009
-
[36]
Zhang, Improved three-dimensional inception networks for hyperspectral remote sensing image classification, IEEE Access 11 (2023) 32648–32658
X. Zhang, Improved three-dimensional inception networks for hyperspectral remote sensing image classification, IEEE Access 11 (2023) 32648–32658
2023
-
[37]
Y . Tian, X. Wang, W. Chen, Z. Liu, L. Li, Adaptive multiple classifiers fusion for inertial sensor based human activity recognition, Cluster Computing 22 (4) (2019) 8141–8154
2019
-
[38]
J. Chen, Y . Qin, P. Lin, J. Li, Y . Xue, H. Ma, Center of pressure estimation by analyzing walking videos, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 3460–3464
2024
-
[39]
Manttari, S
J. Manttari, S. Broome, J. Folkesson, H. Kjellstrom, Interpreting video features: A comparison of 3d convolu- tional networks and convolutional lstm networks, in: Proceedings of the Asian Conference on Computer Vision (ACCV), 2020
2020
-
[40]
Mardanpour, M
M. Mardanpour, M. Sepahvand, F. Abdali-Mohammadi, M. Nikouei, H. Sarabi, Human activity recognition based on multiple inertial sensors through feature-based knowledge distillation paradigm, Information Sciences 640 (2023) 119073
2023
-
[41]
Jeong, J
Y . Jeong, J. Park, D. Cho, Y . Hwang, S. B. Choi, I. S. Kweon, Lightweight depth completion network with local similarity-preserving knowledge distillation, Sensors 22 (19) (2022). 22
2022
-
[42]
D. P. Kingma, M. Welling, Auto-encoding variational bayes, Proceedings of the International Conference on Learning Representations (2014)
2014
-
[43]
Fabius, J
O. Fabius, J. R. van Amersfoort, D. P. Kingma, Variational recurrent auto-encoders, in: Proceedings of the International Conference on Learning Representations Workshops, 2015
2015
-
[44]
Tolstikhin, O
I. Tolstikhin, O. Bousquet, S. Gelly, B. Schölkopf, Wasserstein auto-encoders, in: Proceedings of the Interna- tional Conference on Learning Representations, 2018
2018
-
[45]
Bucilu ˇa, R
C. Bucilu ˇa, R. Caruana, A. Niculescu-Mizil, Model compression, in: Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining (KDD), 2006, pp. 535–541
2006
-
[46]
Zagoruyko, N
S. Zagoruyko, N. Komodakis, Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer, in: Proceedings of the International Conference on Learning and Repre- sentations (ICLR), 2017, pp. 1–13
2017
-
[47]
X. Li, H. Xiong, X. Li, X. Wu, X. Zhang, J. Liu, J. Bian, D. Dou, Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond, Knowledge and Information Systems 64 (12) (2022) 3197–3234
2022
-
[48]
Räuker, A
T. Räuker, A. Ho, S. Casper, D. Hadfield-Menell, Toward transparent ai: A survey on interpreting the inner structures of deep neural networks, in: Proceedings of the IEEE Conference on Secure and Trustworthy Machine Learning (SATML), IEEE, 2023, pp. 464–483
2023
-
[49]
J. Fan, F. Han, H. Liu, Challenges of big data analysis, National science review 1 (2) (2014) 293–314
2014
-
[50]
D. Tran, L. Bourdev, R. Fergus, L. Torresani, M. Paluri, Learning spatiotemporal features with 3d convolutional networks, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497
2015
-
[51]
Carreira, A
J. Carreira, A. Zisserman, Quo vadis, action recognition? a new model and the kinetics dataset, in: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308
2017
-
[52]
D. Tran, H. Wang, L. Torresani, J. Ray, Y . LeCun, M. Paluri, A closer look at spatiotemporal convolutions for action recognition, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 6450–6459
2018
-
[53]
URLhttps://en.wikipedia.org/w/index.php?title=Pearson_correlation_ coefficient&oldid=1183832791
Wikipedia contributors, Pearson correlation coefficient, accessed 08-April-2025. URLhttps://en.wikipedia.org/w/index.php?title=Pearson_correlation_ coefficient&oldid=1183832791
2025
-
[54]
Scholkopf, K.-K
B. Scholkopf, K.-K. Sung, C. J. Burges, F. Girosi, P. Niyogi, T. Poggio, V . Vapnik, Comparing support vec- tor machines with gaussian kernels to radial basis function classifiers, IEEE transactions on Signal Processing 45 (11) (1997) 2758–2765
1997
-
[55]
M. Ring, B. M. Eskofier, An approximation of the gaussian rbf kernel for efficient classification with svms, Pattern Recognition Letters 84 (2016) 107–113
2016
-
[56]
Jeong, J
Y . Jeong, J. Park, D. Cho, Y . Hwang, S. B. Choi, I. S. Kweon, Lightweight depth completion network with local similarity-preserving knowledge distillation, Sensors 22 (19) (2022) 7388
2022
-
[57]
S. Ji, W. Xu, M. Yang, K. Yu, 3d convolutional neural networks for human action recognition, IEEE transactions on pattern analysis and machine intelligence 35 (1) (2012) 221–231
2012
-
[58]
Huang, Z
X. Huang, Z. Cai, A review of video action recognition based on 3d convolution, Computers and Electrical Engineering 108 (2023) 108713
2023
-
[59]
M. N. Orlin, T. G. McPoil, Plantar pressure assessment, Physical therapy 80 (4) (2000) 399–409. 23
2000
-
[60]
Huang, S
T. Huang, S. You, F. Wang, C. Qian, C. Xu, Knowledge distillation from a stronger teacher, Advances in Neural Information Processing Systems 35 (2022) 33716–33727
2022
-
[61]
C. Wang, D. Chen, J.-P. Mei, Y . Zhang, Y . Feng, C. Chen, Semckd: Semantic calibration for cross-layer knowledge distillation, IEEE Transactions on Knowledge and Data Engineering 35 (6) (2023) 6305–6319. doi:10.1109/TKDE.2022.3171571
-
[62]
G. Sejnova, M. Vavrecka, K. Stepanova, Benchmarking multimodal variational autoencoders: Cdsprites+ dataset and toolkit (2023).arXiv:2209.03048
-
[63]
Daunhawer, T
I. Daunhawer, T. M. Sutter, K. Chin-Cheong, E. Palumbo, J. E. V ogt, On the limitations of multimodal vaes, in: Proceedings of the International Conference on Learning Representations, 2022
2022
-
[64]
J. H. Cho, B. Hariharan, On the efficacy of knowledge distillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4794–4802
2019
-
[65]
C. H. Martin, M. W. Mahoney, Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning, Journal of Machine Learning Research 22 (165) (2021) 1–73
2021
-
[66]
X. Lan, X. Zhu, S. Gong, Self-referenced deep learning, in: Proceedings of the Asian Conference on Computer Vision, Springer, 2019, pp. 284–300
2019
-
[67]
E. H. Frank, Regression modeling strategies with applications to linear models, logistic and ordinal regression, and survival analysis (2015)
2015
-
[68]
M. P. Naeini, G. Cooper, M. Hauskrecht, Obtaining well calibrated probabilities using bayesian binning, in: Proceedings of the AAAI conference on artificial intelligence, V ol. 29, 2015
2015
-
[69]
C. Guo, G. Pleiss, Y . Sun, K. Q. Weinberger, On calibration of modern neural networks, in: Proceedings of the International Conference on Machine Learning (ICML), 2017, pp. 1321–1330
2017
-
[70]
E. S. Jeon, S. Lohit, R. Anirudh, P. Turaga, Robust time series recovery and classification using test-time noise simulator networks, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2023, pp. 1–5
2023
-
[71]
Lohit, Q
S. Lohit, Q. Wang, P. Turaga, Temporal transformer networks: Joint learning of invariant and discriminative time warping, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 12426–12435. 24
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.