REVIEW 4 major objections 6 minor 34 references
DWnet: Deep-Wide Network for 3D Action Recognition
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Replacing a deep network's final layers with a broad learning readout preserves 3D action accuracy and cuts test time by 100x.
desk verdict The BLS-plus-CNN architecture is a plausible engineering idea, but the paper's headline 100x speedup claim is not supported by the timing evidence, which appears to measure only the BLS readout, not the full pipeline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the PruHCN feature mapping feeding a broad learning readout. The feature mapping writes $Z_i = \theta(P(\mathrm{ReLU}(X \otimes W_{ei} + \beta_{ei})))$; the enhancement nodes are $H_j = \xi(Z W_{hj} + \beta_{hj})$ with random $W_{hj}, \beta_{hj}$ and a tansig activation; the final output is the concatenation of $Z$ and all $H_j$ multiplied by ridge-regression weights. This machine carries the argument because the random expansion is what compensates for the two pruned layers, and the closed-form solution is what makes testing fast.
What would settle it
Train DWnet while choosing enhancement-node counts on a validation split rather than the test set, then test under the same preprocessing. If accuracy falls below HCN's by a margin comparable to the reported gains (for example, more than 2.5% on SBU), the claim that random enhancement restores the pruned layers' approximation is refuted. A complementary check is to train a linear classifier directly on the same Fc6 features; if it already matches DWnet, the enhancement nodes add little.
Extended reading notes
Core claim
The central claim is that replacing the Dropout layer and the final Fc7 softmax layer of a trained HCN with a broad learning system readout preserves learning capacity and improves inference efficiency. The pruned trunk still learns local spatial-temporal and co-occurrence features; the broad readout expands the Fc6 feature vector through random nonlinear enhancement nodes and connects both to an output layer whose weights are obtained in closed form. The reported numbers are 97.39% versus 94.83% on SBU and 84.07% versus 84.30% on NTU cross-subject, with per-sample test times falling from about 4.12e-2 seconds to 2.17e-4 seconds on SBU and from about 1.1e-2 seconds to 2.26e-4 seconds on NTU. The paper interprets this as evidence that the width of the broad readout can restore the approximation ability removed by pruning.
Load-bearing premise
The load-bearing premise is that a single trained PruHCN's Fc6 feature vector, expanded by randomly initialized enhancement nodes and solved with ridge regression, can maintain or improve the classification power of the original HCN's Dropout and Fc7 layers.
Editorial extensions
If this is right
- The last fully connected layers of a trained deep CNN can be swapped for a ridge-regression readout without hurting accuracy, so the expensive softmax classifier is not a necessary part of the model.
- Skeleton action recognition can run near real time on modest hardware: per-sample times near 2e-4 seconds move the bottleneck from classification to feature extraction.
- The enhancement-node count is a dataset-dependent knob; the paper reports optimal values of 550 on SBU and 7500 or 12500 on NTU, so accuracy and speed trade off through a single width parameter.
- Because only the readout changes, the same extracted features can be reclassified after retraining just the readout, avoiding a full end-to-end retraining pass.
Reading between the lines
- The recipe generalizes beyond skeletons: any CNN that produces a dense feature vector could have its classifier pruned and replaced by a broad readout for faster inference, but the paper does not test this.
- The reported 100x speed gain is tied to the specific enhancement-node counts; for very wide enhancement layers the ridge regression itself may dominate the runtime and weaken the advantage.
- The success of random enhancement nodes suggests the Fc6 representation is already close to linearly separable; a simple probe such as a linear classifier on the same Fc6 features could reveal how much of the accuracy comes from the broad readout rather than the deep trunk.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DWnet, which combines a pruned Hierarchical Co-occurrence Network (PruHCN) with a Broad Learning System (BLS) readout for 3D skeleton-based action recognition. The deep part is HCN with the final Dropout and Fc7 layers removed, and the BLS readout consists of randomly generated enhancement nodes followed by a ridge-regression output layer. On SBU Kinect, the authors report 97.39% accuracy for DWnet versus 94.83% for HCN; on NTU RGB+D they report near-identical accuracies (84.07% vs 84.30% CS; 89.84% vs 89.90% CV). The headline claim is an approximately 100x reduction in per-sample testing time (e.g., 4.12e-2 s vs 2.17e-4 s on SBU). The paper also compares against a plain BLS and an HCNBLS variant with untrained PruHCN modules.
Significance. If the efficiency claim were properly supported, the idea of using a pretrained deep feature extractor with a broad-learning readout for skeleton-based action recognition would be of moderate interest, especially for latency-sensitive applications. The architecture is clearly described, and the experiments cover two standard datasets with a direct comparison to HCN. However, the paper provides no code, no repeated-run statistics, and only one strong deep-learning baseline, so the empirical support for the 'state-of-the-art' conclusion is weak. Crucially, the central speedup claim is not yet established by the reported measurements.
major comments (4)
- [Section III.C, Tables II and III] The roughly 100x speedup claim is not supported by the evidence because the paper does not state whether the reported times are end-to-end or only for the readout layers. DWnet retains all of HCN's convolutional layers and Fc6 (Fig. 2), so replacing only Dropout and Fc7 with a BLS readout cannot reduce total inference time by 190x on SBU if both timings include the same convolutional forward pass. Additionally, Dropout is not active at inference time in standard HCN, so pruning it has no effect on test-time cost. Please report end-to-end timings for both models with the same hardware/software stack, and state exactly which operations are included in each measurement.
- [Section IV.C, Figs. 5-7] The number of enhancement nodes m is selected by maximizing test-set accuracy: Fig. 5 reports the best average testing accuracy at m=550, and Figs. 6-7 select 7500 and 12500 by the same criterion. This selection on the test set makes the accuracies in Tables I and III optimistic relative to what would be expected on unseen data. Please tune m on a validation split (or report accuracy as a function of m without claiming the maximum as expected performance) and provide repeated-run mean ± std to establish the stability of the reported improvements.
- [Section III.C] The complexity argument is incomplete and does not by itself imply the claimed speedup. On NTU RGB+D, the BLS readout with 7500 or 12500 enhancement nodes multiplies a feature vector of dimension 256 plus the enhancement dimension by the 60-class output matrix, yielding far more multiply-adds than HCN's Fc7 (256x60). On SBU, 614x8 operations for the BLS readout exceed 64x8 for Fc7. Thus the statement that 'the computational complexity of the BLS is lower than that of pruned layer of HCN' is not true in terms of multiply-add count for the configurations actually used, and the reported timing difference likely reflects differences in implementation, batching, or measurement scope rather than the architecture change.
- [Section V and Section IV.D] The conclusion that DWnet 'can achieve the state-of-the-art performance' is unsupported by the comparisons presented. The only deep-learning baseline is HCN itself, and on NTU RGB+D DWnet is slightly worse than HCN (89.84% vs 89.90% CV; 84.07% vs 84.30% CS). Please either temper the claim to 'comparable to HCN' or add the relevant state-of-the-art baselines from the skeleton-based action recognition literature.
minor comments (6)
- [Section III.A and III.B] There are several typos: 'irregualr' in Section III.A, 'spapatial-temporal' and 'sapatial-temporal' in Section III.B, and 'pruned' used as a verb in Fig. 2 caption.
- [Section III.B] The text says 'There are 6 layers including 4 convolutional layers and 2 fully connected layers in the PruHCN,' but Fig. 2 shows PruHCN includes five convolutional layers (Conv1-Conv5) and one fully connected layer (Fc6); please correct the layer counts.
- [Eq. (2) and Eq. (3)] Equation (2) defines Z_i for i=1,...,n but the paper then uses Z without a subscript in Eq. (3); please clarify whether Z denotes Z_1 given that n=1.
- [Table III] The table header is confusing: the columns are labeled 'CV' and 'CS' with 'Testing Time' straddling the accuracy and time columns, making it unclear which number is accuracy and which is time. Please use explicit column headers such as 'CV Acc.' and 'CV Time'.
- [References] References [23] and [34] cite the same NTU RGB+D dataset paper; please merge them to avoid duplicate citations.
- [Section IV.C] The paper says DWnet 'almost achieves real-time testing' but does not define real-time relative to any frame-rate requirement; please specify the target frame rate and the measurement protocol.
Circularity Check
No circular derivation: DWnet accuracy and speedup are empirical comparisons; no equation reduces to its inputs.
full rationale
The paper makes an empirical architecture claim: a trained PruHCN feature extractor followed by a BLS ridge-regression readout. Section III-C defines the feature mapping (Eq. 2) and enhancement nodes (Eq. 3), but no equation defines the output accuracy or testing time in terms of these quantities; accuracy and speed are measured on test sets (Tables I-III). The speed comparison is questionable because both HCN and DWnet share the convolutional stack and Fc6, so the ~100x speedup cannot follow from replacing Dropout/Fc7 with BLS alone; however, this is a correctness or measurement concern, not a circular reduction. The number of enhancement nodes is tuned on test-set curves (Figs. 5-7), which weakens generalization but is not a fitted quantity renamed as a prediction. The one self-citation [28] concerns cement compressive strength and is not load-bearing. The Conclusion's admission that DWnet is non-end-to-end is a limitation, not a circular step. No load-bearing derivation step reduces to its own inputs.
Assumptions & free parameters
free parameters (3)
- Number of BLS enhancement nodes m =
550 (SBU), 7500 (NTU CS), 12500 (NTU CV)
- Number of feature nodes (PruHCN modules) =
1
- BLS random weights W_hj and biases beta_hj =
Not specified
assumptions (3)
- domain assumption HCN is an effective feature extractor for skeleton co-occurrence
- domain assumption BLS ridge regression can restore the approximation ability lost by pruning layers
- ad hoc to paper Randomly generated nonlinear enhancement nodes provide a sufficiently rich feature expansion
Cite this review
Pith. "Pith review of DWnet: Deep-Wide Network for 3D Action Recognition." pith.science (2026). https://pith.science/paper/QI2TXYWA
@misc{pith2026190811036,
author = {Pith},
title = {Pith review of: DWnet: Deep-Wide Network for 3D Action Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/QI2TXYWA}},
note = {Machine review of arXiv:1908.11036}
}
read the original abstract
We propose in this paper a deep-wide network (DWnet) which combines the deep structure with the broad learning system (BLS) to recognize actions. Compared with the deep structure, the novel model saves lots of testing time and almost achieves real-time testing. Furthermore, the DWnet can capture better features than broad learning system can. In terms of methodology, we use pruned hierarchical co-occurrence network (PruHCN) to learn local and global spatial-temporal features. To obtain sufficient global information, BLS is used to expand features extracted by PruHCN. Experiments on two common skeletal datasets demonstrate the advantage of the proposed model on testing time and the effectiveness of the novel model to recognize the action.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks,
I. Lee, D. Kim, S. Kang, and S. Lee, “Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks,” in Proc. ICCV, Venice, Italy, Oct. 2017, pp. 1012–1020
work page 2017
-
[2]
Deep progressive reinforcement learning for skeleton- based action recognition,
Y. Tang, Y. Tian, J. Lu, P. Li, and J. Zhou, “Deep progressive reinforcement learning for skeleton- based action recognition,” inProc. CVPR, Salt Lake City, UT, USA , Jun. 2018, pp. 5323–5332
work page 2018
-
[3]
A new representation of skeleton sequences for 3d action recognition,
Q. Ke, M. Bennamoun, S. An, F. Sohel, and F. Bous- said, “A new representation of skeleton sequences for 3d action recognition,” in Proc. CVPR, Los Alamitos, CA, USA , Jul. 2017, pp. 4570–4579
work page 2017
-
[4]
Skeleton- based action recognition with convolutional neural networks,
C. Li, Q. Zhong, D. Xie, and S. Pu, “Skeleton- based action recognition with convolutional neural networks,” in Proc. ICMEW, Hong Kong, China , Jul. 2017, pp. 597–600. 11
work page 2017
-
[5]
Skeleton-based action recognition with spatial rea- soning and temporal stack learning,
C. Si, Y. Jing, W. Wang, L. Wang, and T. Tan, “Skeleton-based action recognition with spatial rea- soning and temporal stack learning,” in Proc. 15th ECCV, Munich, Germany , vol. 11205, Sep. 2018, pp. 106–121
work page 2018
-
[6]
Graph Edge Convolutional Neural Networks for Skeleton Based Action Recognition
X. Zhang, C. Xu, X. Tian, and D. Tao, “Graph edge convolutional neural networks for skeleton based ac- tion recognition,” CoRR, vol. abs/1805.06184, 2018
work page Pith review arXiv 2018
-
[7]
W. Zhu, C. Lan, J. Xing, W. Zeng, Y. Li, L. Shen, and X. Xie, “Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks,” in Proc. 30th AAAI Press., Phoenix, Arizona, USA. , vol. abs/1603.07772, Feb. 2016, pp. 3697–3704
work page Pith review arXiv 2016
-
[8]
Space- time representation of people based on 3d skeletal data: A review,
F. Han, B. Reily, W. Hoff, and H. Zhang, “Space- time representation of people based on 3d skeletal data: A review,” Comput. Vis. Image Underst. , vol. 158, pp. 85–105, May. 2017
work page 2017
Show all 34 references
-
[9]
Graph based skeleton motion representation and similarity measurement for action recognition,
P. Wang, C. Yuan, W. Hu, B. Li, and Y. Zhang, “Graph based skeleton motion representation and similarity measurement for action recognition,” in Proc. 14th ECCV, Amsterdam, The Netherlands , 2016, pp. 370–385
2016
-
[10]
Spatio-temporal naive-bayes nearest-neighbor (st-nbnn) for skeleton- based action recognition,
J. Weng, C. Weng, and J. Yuan, “Spatio-temporal naive-bayes nearest-neighbor (st-nbnn) for skeleton- based action recognition,” in Proc. CVPR, Hon- olulu, HI, USA , Jul. 2017, pp. 445–454
2017
-
[11]
View invari- ant human action recognition using histograms of 3d joints,
L. Xia, C. Chen, and J. K. Aggarwal, “View invari- ant human action recognition using histograms of 3d joints,” in Proc. CVPR W, Providence, RI, USA, Jun. 2012, pp. 20–27
2012
-
[12]
Graves, Supervised Sequence Labelling with Re- current Neural Networks, ser
A. Graves, Supervised Sequence Labelling with Re- current Neural Networks, ser. Stud. Comput. Intell., Jan. 2012, vol. 385
2012
-
[13]
Speech recognition with deep recurrent neural networks,
A. Graves, A. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Proc. ICASSP, Vancouver, BC, Canada, May. 2013, pp. 6645–6649
2013
-
[14]
Convolutional, long short-term memory, fully con- nected deep neural networks,
T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, “Convolutional, long short-term memory, fully con- nected deep neural networks,” in Proc. IEEE Conf. ICASSP, South Brisbane, Queensland, Australia , Apr. 2015, pp. 4580–4584
2015
-
[15]
Differential recurrent neural networks for action recognition,
V. Veeriah, N. Zhuang, and G. Qi, “Differential recurrent neural networks for action recognition,” in Proc. ICCV, Santiago, Chile , vol. abs/1504.06678, 2015, pp. 4041–4049
2015 arXiv
-
[16]
Skeleton optical spectra-based action recognition using convolutional neural networks,
Y. Hou, Z. Li, P. Wang, and W. Li, “Skeleton optical spectra-based action recognition using convolutional neural networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 28, no. 3, pp. 807–811, Mar. 2018
2018
-
[17]
Skeleton based action recognition with convolutional neural network,
Y. Du, Y. Fu, and L. Wang, “Skeleton based action recognition with convolutional neural network,” in Proc. 3rd ACPR, Kuala Lumpur, Malaysia , Nov. 2015, pp. 579–583
2015
-
[18]
Recurrent highway networks with language cnn for image captioning,
J. Gu, G. Wang, and T. Chen, “Recurrent highway networks with language cnn for image captioning,” Dec. 2016
2016
-
[19]
Co-occurrence feature learning from skeleton data for action recog- nition and detection with hierarchical aggregation,
C. Li, Q. Zhong, D. Xie, and S. Pu, “Co-occurrence feature learning from skeleton data for action recog- nition and detection with hierarchical aggregation,” in Proc. 27th IJCAI, Stockholm, Sweden , Jul. 2018, pp. 786–792
2018
-
[20]
Broad learning system: An effective and efficient incremental learning sys- tem without the need for deep architecture,
C. L. P. Chen and Z. Liu, “Broad learning system: An effective and efficient incremental learning sys- tem without the need for deep architecture,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 29, no. 1, pp. 10–24, Jan. 2018
2018
-
[21]
Discriminative graph regularized broad learning system for image recognition,
J. Jin, Z. Liu, and C. L. P. Chen, “Discriminative graph regularized broad learning system for image recognition,” Sci. China-Inf. Sci. , vol. 61, no. 11, Nov. 2018
2018
-
[22]
Broad learn- ing system: Feature extraction based on k-means clustering algorithm,
Z. Liu, J. Zhou, and C. L. P. Chen, “Broad learn- ing system: Feature extraction based on k-means clustering algorithm,” in Proc. 4th ICCSS, Dalian, China, Jul. 2017, pp. 683–687
2017
-
[23]
Ntu rgb+d: A large scale dataset for 3d human activity analysis,
A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, “Ntu rgb+d: A large scale dataset for 3d human activity analysis,” in Proc. CVPR, Las Vegas, America, Jun. 2016
2016
-
[24]
Spatio- temporal lstm with trust gates for 3d human action recognition,
J. Liu, A. Shahroudy, D. Xu, and G. Wang, “Spatio- temporal lstm with trust gates for 3d human action recognition,” in Proc. 14th ECCV, Amsterdam, The Netherlands, Oct. 2016, pp. 816–833
2016
-
[25]
3d-based deep convolutional neural network for action recognition with depth sequences,
Z. Liu, C. Zhang, and Y. Tian, “3d-based deep convolutional neural network for action recognition with depth sequences,” Image Vis. Comput., vol. 55, pp. 93–100, Nov. 2016
2016
-
[26]
Stochastic choice of basis func- tions in adaptive function approximation and the functional-link net,
Y.-H. P. B. Igelnik, “Stochastic choice of basis func- tions in adaptive function approximation and the functional-link net,” IEEE Trans. Neural Netw. , vol. 6, no. 6, pp. 1320–1329, 1995
1995
-
[27]
Universal approximation capability of broad learning system and its structural variations,
C. L. P. Chen, Z. Liu, and S. Feng, “Universal approximation capability of broad learning system and its structural variations,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 30, no. 4, pp. 1191–1204, Apr. 2019
2019
-
[28]
Estimating cement compressive strength from microstructure images using broad learning system,
Y. Dang, L. Wang, J. Yin, X. Zhu, Z. Feng, and J. Guo, “Estimating cement compressive strength from microstructure images using broad learning system,” in Proc. IEEE Conf. SMC, Miyazaki, Japan, Oct. 2018, pp. 4230–4235
2018
-
[29]
Outdoor shadow estimating using multiclass geo- metric decomposition based on bls,
Z. Chen, T. Gao, B. Sheng, P. Li, and C. L. P. Chen, “Outdoor shadow estimating using multiclass geo- metric decomposition based on bls,” IEEE Trans. Cybern., pp. 1–14, 2018
2018
-
[30]
Student body gesture recognition based on fisher broad learning system,
Y. Shi, Y. Wei, D. Pan, W. Deng, H. Yao, T. Chen, G. Zhao, M. Tong, and Q. Liu, “Student body gesture recognition based on fisher broad learning system,” Int. J. Wavelets Multiresolut. Inf. Process., vol. 17, no. 01, p. 1950001, Jan. 2019
2019
-
[31]
Local log-euclidean multivariate gaussian descriptor and its application to image classification,
P. Li, Q. Wang, H. Zeng, and L. Zhang, “Local log-euclidean multivariate gaussian descriptor and its application to image classification,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 39, no. 4, pp. 803– 817, Apr. 2017
2017
-
[32]
Graph neural networks: A review of methods and applications,
J. Zhou, G. Cui, Z. Zhang, C. Yang, and M. Sun, “Graph neural networks: A review of methods and applications,” CoRR, vol. abs/1812.08434, 2018
2018 arXiv
-
[33]
Two-person interaction detec- tion using body-pose features and multiple instance 12 learning,
K. Yun, J. Honorio, D. Chattopadhyay, T. L. Berg, and D. Samaras, “Two-person interaction detec- tion using body-pose features and multiple instance 12 learning,” in Proc. CVPR W, Providence, RI, USA , Jun. 2012, pp. 28–35
2012
-
[34]
NTU RGB+D: A large scale dataset for 3d human activity analysis,
A. Shahroudy, J. Liu, T. Ng, and G. Wang, “NTU RGB+D: A large scale dataset for 3d human activity analysis,” in Proc. CVPR, Las Vegas, NV, USA , Jun. 2016
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.