REVIEW 4 major objections 4 minor 1 cited by
Enhanced 3D convolutional networks for crowd counting
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that stacking temporal channel-aware blocks—3D convolutions plus channel attention—lets a fully convolutional network exploit motion between frames and thereby count crowds more accurately than per-frame CNN methods.
desk verdict The TCA/E3D architecture is a genuinely new application of 3D convolutions and channel attention to crowd counting, but the reported SOTA margins are selected on test data and the training setup is undocumented. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The temporal channel-aware (TCA) block is the load-bearing component. Its mainstream branch computes $O$ from the input $X$ with two 3D convolutional layers; a channel descriptor $v$ is formed by averaging $O$ over the depth, height, and width dimensions, then $u=\sigma(W_2\delta(W_1 v))$ produces per-channel modulation weights, and the block output is $\tilde X = X + u\odot O$. The 3D kernels supply local spatio-temporal features, while the global average pooling and bottleneck supply scene-level context that decides which channels matter. Stacking alternating downsampling and non-downsampling TCA blocks gives E3D; replacing every 3D kernel by its 2D counterpart gives E2D for still-image datasets. The design is fully convolutional, so input resolution is arbitrary.
What would settle it
Feed E3D the same test frames in shuffled or reversed temporal order so the 3D kernels see no real motion: if MAE and MSE stay essentially unchanged, the 3D temporal modeling is not what carries the gain. A more direct check would train a count-loss variant that regresses the summed count rather than the density image on identical clips; if it matches or beats E3D, the density-map proxy, not the TCA block, is responsible.
Extended reading notes
Core claim
The central claim is that modeling temporal interdependencies explicitly—inside the feature extractor rather than as a post-processing step—produces more accurate density maps for crowd counting. The E3D network takes 16 consecutive frames as a single $D\times H\times W$ volume; a first $7\times 7\times 7$ convolution mixes them, and each TCA block applies two 3D convolutions, squeezes the whole spatio-temporal volume to a per-channel descriptor by global average pooling, recalibrates the channels through a bottleneck with sigmoid activation, and adds the rescaled features back to the input. This local-plus-global mechanism lowers MAE on UCSD from 1.02 to 0.93 and on Mall from 1.82 to 1.64, gives the best average MAE on WorldExpo'10 (8.32), and E2D reaches a GAME0 of 2.88 on TRANCOS. The paper interprets these numbers as evidence that temporal 3D convolutions plus channel attention help in diverse scenes, and that the block remains effective even when temporal input is unavailable.
Load-bearing premise
The load-bearing assumption is that a blurred dot map is a good stand-in for a head count: if reducing pixel error on Gaussian blobs does not reduce counting error, the reported gains may measure map quality rather than counting.
Editorial extensions
If this is right
- Counting accuracy on video benchmarks should improve whenever a model consumes several frames at once, because the temporal dimension carries information about occluded or ambiguous pedestrians.
- The channel-recalibration mechanism should transfer across scene types; the same block trained on walking crowds was applied directly to mall interiors and WorldExpo'10 scenes with different densities.
- On datasets without temporal continuity, the 2D variant remains competitive, so the block's value is not limited to video; E2D reports a GAME0 of 2.88 on TRANCOS.
- Longer input clips (12–16 frames at 10 fps) are needed before temporal benefits appear; on faster or slower frame rates the optimal clip length should scale with actual motion, not with frame count.
- Stack depth has an optimum (8 TCA blocks in the experiments); deeper is not automatically better for density-map regression.
Reading between the lines
- If temporal modeling is what improves counts, the same block should smooth count estimates across time, producing more stable per-frame counts in surveillance streams; that is a measurable prediction the paper does not report.
- The density-map proxy is the weakest link: a testable extension is to replace or augment the Gaussian target with a count-aware loss (direct regression of the summed map or a Poisson count term) and see whether the temporal advantage persists.
- The architecture suggests that 3D convolutions could help other dense prediction tasks on video, such as cell or nuclei counting in time-lapse microscopy and traffic flow estimation, where repeated objects move between frames.
- The ablation shows E3D without global context still beats E2D without global context, so the 3D convolution carries part of the gain independently of channel attention; a clean experiment would isolate how much of the gain is temporal structure versus added parameters.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes temporal channel-aware (TCA) blocks that combine 3D convolutions with channel-wise attention, and stacks them into an "enhanced 3D convolutional network" (E3D) for video crowd counting. A 2D variant, E2D, is used for static-image counting. The authors evaluate E3D on UCSD, Mall, and WorldExpo'10 and E2D on TRANCOS, reporting state-of-the-art or near-state-of-the-art results, and include ablation studies on UCSD over the global-context branch, input frame length, and number of TCA blocks.
Significance. If the reported results hold, this would be a useful contribution: it is among the first attempts to use 3D convolutions for crowd counting, and the TCA block's combination of local spatio-temporal features with channel recalibration is a plausible and clearly described architectural idea. The paper also provides ablation evidence that both temporal modeling (3D vs 2D) and the global-context branch contribute to accuracy. However, the empirical support for the state-of-the-art claims is weakened by test-set-based hyperparameter selection, an internal numerical inconsistency, an inaccurate textual claim about WorldExpo'10 scene-wise results, and missing training details that prevent reproduction.
major comments (4)
- [Sec. 3.1, Sec. 4.2, Sec. 4.3] The state-of-the-art comparisons are not out-of-sample. Section 3.1 states that the number of TCA blocks is "chosen according to the performance on all testing datasets," and Section 4.3 selects the frame length (16) and block count (8) by evaluating on the UCSD test frames (frames 0-600 and 1401-2000, with training on 601-1400). No held-out validation set is used anywhere in the paper. Because Tables 2-5 report exactly these test-selected configurations, the reported margins are optimistically biased and cannot support the "state-of-the-art" claim as stated. The experiments should be rerun with a clean validation procedure, or the claimed state-of-the-art status should be substantially qualified.
- [Table 2 vs Table 6] There is a numerical inconsistency for the same E3D configuration on UCSD: Table 2 reports MAE 0.93 / MSE 1.17, while Table 6 (the E3D row in the component-analysis ablation) reports MAE 0.93 / MSE 1.13. Since the paper uses MSE as a headline metric, the source of this discrepancy must be explained; if the training or evaluation procedure differs, the tables should be reconciled, and if it is an error, the corrected values should be reported consistently.
- [Sec. 4.2, WorldExpo'10 paragraph and Table 4] The text states that E3D "achieves the best accuracy in 4 out of 5 scenes" on WorldExpo'10, but Table 4 shows that E3D has the strictly best MAE only in scene S4 (10.2), ties with SCNet in S5 (3.2), and is worse than at least one method in S1, S2, and S3. The claim should be corrected to describe the actual scene-wise results; the average MAE is indeed the best (8.32 vs 8.4 for SCNet), but the difference is very small and the "4 out of 5" statement is factually wrong.
- [Sec. 3.1, Sec. 4] The training setup is underspecified. The loss function used to train the network is never stated, and no optimizer, learning rate, batch size, or number of epochs is reported anywhere in the paper. Section 3.1 only says the network "can be optimized via an end-to-end training scheme." Without these details, and without code, the quantitative results in Tables 2-6 cannot be independently reproduced, which is a load-bearing gap for an empirical architecture paper.
minor comments (4)
- [Abstract and Sec. 1] Minor wording issues: "defacto" should be "de facto," and "inculding" should be "including." In Sec. 1, "throughout study" should be "thorough study."
- [Sec. 4.2, Mall paragraph] The sentence "It's difficult to find the underlying relationship between the head size and density map" could be clarified; the point is presumably about perspective and scale variation, not just lighting and reflection.
- [Sec. 4.3] The ablation section reports results without error bars or multiple runs. Given the small performance differences involved (e.g., MAE 0.93 vs 1.00 in the component study), a statement about run-to-run variance, or at least a note on whether results are from a single seed, would increase confidence in the ablations.
- [Table 1] For Mall, the table lists "Color" as RGB, but the original Mall dataset is often described as grayscale; this should be verified and corrected if needed.
Circularity Check
No significant circularity: the paper's results are benchmark measurements rather than a derivation that reduces to its inputs; the test-set tuning concern is a validation-protocol issue, not a definitional circularity.
full rationale
This is an empirical architecture paper. The claimed contribution is a new 3D temporal-channel-aware block and its measured performance on standard benchmarks. There is no analytical derivation chain in which an output quantity is defined in terms of the same quantity, nor is any reported 'prediction' equivalent to a fitted parameter by construction. The Gaussian density-map ground truth (Eq. 6) is normalized so that its integral equals the count, but this is the standard regression-target definition for crowd counting; the network must still learn the mapping from input frames to those maps, and the reported MAE/MSE are computed on test frames. The only load-bearing weakness is that Section 3.1 says the number of TCA blocks is 'chosen according to the performance on all testing datasets' and Section 4.3 ablates frame length and block count on the UCSD test split, so the final tables are optimistically biased by test-set-based model selection. That is a statistical validity and reproducibility concern, not circularity of derivation: no equation in the paper reduces a predicted quantity to a fitted parameter or to a self-citation, and no load-bearing claim depends on the authors' own prior work. Therefore no circular step is identified under the specified definitions.
Assumptions & free parameters
free parameters (5)
- frame_length =
16
- number_of_TCA_blocks =
8
- channel_reduction_ratio_r =
4
- geometry_adaptive_beta =
0.3
- geometry_adaptive_k =
3
assumptions (4)
- domain assumption Ground-truth density maps are generated by convolving head annotations with Gaussian kernels, and the total sum of a density map equals the true count.
- domain assumption MAE and MSE computed by summing predicted density maps over the whole image are the accepted way to measure counting accuracy.
- domain assumption Benchmark train/test splits, ROIs, and evaluation protocols follow prior work exactly.
- domain assumption The loss used to train the network against density maps, although not stated, is an effective proxy for count accuracy.
Cite this review
Pith. "Pith review of Enhanced 3D convolutional networks for crowd counting." pith.science (2026). https://pith.science/paper/4OFYZPYX
@misc{pith2026190804121,
author = {Pith},
title = {Pith review of: Enhanced 3D convolutional networks for crowd counting},
year = {2026},
howpublished = {\url{https://pith.science/paper/4OFYZPYX}},
note = {Machine review of arXiv:1908.04121}
}
read the original abstract
Recently, convolutional neural networks (CNNs) are the leading defacto method for crowd counting. However, when dealing with video datasets, CNN-based methods still process each video frame independently, thus ignoring the powerful temporal information between consecutive frames. In this work, we propose a novel architecture termed as "temporal channel-aware" (TCA) block, which achieves the capability of exploiting the temporal interdependencies among video sequences. Specifically, we incorporate 3D convolution kernels to encode local spatio-temporal features. Furthermore, the global contextual information is encoded into modulation weights which adaptively recalibrate channel-aware feature responses. With the local and global context combined, the proposed block enhances the discriminative ability of the feature representations and contributes to more precise results in diverse scenes. By stacking TCA blocks together, we obtain the deep trainable architecture called enhanced 3D convolutional networks (E3D). The experiments on three benchmark datasets show that the proposed method delivers state-of-the-art performance. To verify the generality, an extended experiment is conducted on a vehicle dataset TRANCOS and our approach beats previous methods by large margins.
Figures
Forward citations
Cited by 1 Pith paper
-
Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark
E-MAC, with density-guided masked modeling and optical-flow temporal fusion, achieves state-of-the-art MAE on four video counting benchmarks, including the new DroneBird bird dataset.
Reference graph
Works this paper leans on
-
[1]
Deepak Babu Sam, Shiv Surya, and R. Venkatesh Babu. Switching convolutional neural network for crowd counting. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
work page 2017
- [2]
-
[3]
Scale aggregation network for accurate and efficient crowd counting
Xinkun Cao, Zhipeng Wang, Yanyun Zhao, and Fei Su. Scale aggregation network for accurate and efficient crowd counting. In The European Conference on Computer Vision (ECCV), September 2018
work page 2018
-
[4]
A. B. Chan and N. Vasconcelos. Bayesian poisson regression for crowd counting. In 2009 IEEE 12th International Conference on Computer Vision , pages 545–551, Sep
work page 2009
-
[5]
A. B. Chan, , and N. Vasconcelos. Privacy preserving crowd monitoring: Counting people without people models or tracking. In 2008 IEEE Conference on Computer Vision and Pattern Recognition , pages 1–7, June 2008. doi: 10.1109/CVPR.2008. 4587569
-
[6]
Feature mining for localised crowd counting
Ke Chen, Chen Change Loy, Shaogang Gong, and Tony Xiang. Feature mining for localised crowd counting. In BMVC, 2012
work page 2012
-
[7]
Histograms of Oriented Gradients for Human Detection
Navneet Dalal and Bill Triggs. Histograms of Oriented Gradients for Human Detection. In International Conference on Computer Vision & Pattern Recognition (CVPR ’05) , volume 1, pages 886–893, June 2005. doi: 10.1109/CVPR.2005.177. ZOU,SHAO: ENHANCED 3D CONVOLUTIONAL NETWORKS FOR CROWD COUNTING 11
-
[8]
An aggregated multicolumn dilated convolution network for perspective-free counting
Diptodip Deb and Jonathan Ventura. An aggregated multicolumn dilated convolution network for perspective-free counting. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018
work page 2018
Show all 38 references
-
[9]
Crowd counting using multiple local features
C.Fookes D.Ryan, S.Deman and S.Sridharan. Crowd counting using multiple local features. Digital Image Computing:Techniques and Applications , pages 81–88.IEEE, Feb 2009. doi: 2009.DICTA’09
2009
-
[10]
Extremely overlapping vehicle counting
Ricardo Guerrero-Gómez-Olmedo, Beatriz Torre-Jiménez, Roberto López-Sastre, Sat- urnino Maldonado-Bascón, and Daniel Oñoro-Rubio. Extremely overlapping vehicle counting. In Roberto Paredes, Jaime S. Cardoso, and Xosé M. Pardo, editors, Pattern Recognition and Image Analysis , ...
2015
-
[11]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[12]
D. Kang, Z. Ma, and A. B. Chan. Beyond counting: Comparisons of density maps for crowd analysis tasks - counting, detection, and tracking. IEEE Transactions on Circuits and Systems for Video Technology , pages 1–1, 2018. ISSN 1051-8215. doi: 10.1109/TCSVT.2018.2837153
2018
-
[13]
Crowd counting by adaptively fusing predictions from an image pyramid
Di Kang and Antoni Chan. Crowd counting by adaptively fusing predictions from an image pyramid. In The British Machine Vision Conference, 2018
2018
-
[14]
M. Li, Z. Zhang, K. Huang, and T. Tan. Estimating the number of people in crowded scenes by mid based foreground segmentation and head-shoulder detection. In 2008 19th International Conference on Pattern Recognition , pages 1–4, Dec 2008. doi: 10.1109/ICPR.2008.4761705
2008
-
[15]
Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes.CoRR, abs/1802.10062, 2018
Yuhong Li, Xiaofan Zhang, and Deming Chen. Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes.CoRR, abs/1802.10062, 2018
2018 arXiv
-
[16]
Hauptmann
Jiang Liu, Chenqiang Gao, Deyu Meng, and Alexander G. Hauptmann. Decidenet: Counting varying density crowds through attention guided detection and density esti- mation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
2018
-
[17]
Bagdanov
Xialei Liu, Joost van de Weijer, and Andrew D. Bagdanov. Leveraging unlabeled data for crowd counting by learning to rank. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
2018
-
[18]
Direct shape regression networks for end-to-end face alignment
Xin Miao, Xiantong Zhen, Xianglong Liu, Cheng Deng, Vassilis Athitsos, and Heng Huang. Direct shape regression networks for end-to-end face alignment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018
2018
-
[19]
López-Sastre
Daniel Oñoro-Rubio and Roberto J. López-Sastre. Towards perspective-free object counting with deep learning. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016 , pages 615–629, Cham, 2016. Springer Inter- national Publishing. ISBN 97...
2016
-
[20]
Learning short-cut connections for object counting
Daniel Oñoro-Rubio, Mathias Niepert, and Roberto J López-Sastre. Learning short-cut connections for object counting. arXiv preprint arXiv:1805.02919, 2018
2018 arXiv
-
[21]
V . Pham, T. Kozakaya, O. Yamaguchi, and R. Okada. Count forest: Co-voting uncertain number of targets using random forest for crowd density estimation. In 2015 IEEE International Conference on Computer Vision (ICCV) , pages 3253–3261, Dec 2015. doi: 10.1109/ICCV .2015.372
2015 doi
-
[22]
Iterative crowd counting
Viresh Ranjan, Hieu Le, and Minh Hoai. Iterative crowd counting. In The European Conference on Computer Vision (ECCV), September 2018
2018
-
[23]
Sheng, C
B. Sheng, C. Shen, G. Lin, J. Li, W. Yang, and C. Sun. Crowd counting via weighted vlad on a dense attribute feature map. IEEE Transactions on Circuits and Systems for Video Technology, 28(8):1788–1797, Aug 2018. ISSN 1051-8215. doi: 10.1109/ TCSVT.2016.2637379
2018
-
[24]
Crowd counting with deep negative correlation learning
Zenglin Shi, Le Zhang, Yun Liu, Xiaofeng Cao, Yangdong Ye, Ming-Ming Cheng, and Guoyan Zheng. Crowd counting with deep negative correlation learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018
2018
-
[25]
Sindagi and Vishal M
Vishwanath A. Sindagi and Vishal M. Patel. Generating high-quality crowd density maps using contextual pyramid cnns. In The IEEE International Conference on Com- puter Vision (ICCV), Oct 2017
2017
-
[26]
Learning spatiotemporal features with 3d convolutional networks
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. In The IEEE In- ternational Conference on Computer Vision (ICCV) , December 2015
2015
-
[27]
A closer look at spatiotemporal convolutions for action recognition
Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. CoRR, abs/1711.11248, 2017
2017 arXiv
-
[28]
Paul Viola and Michael J. Jones. Robust real-time face detection. International Journal of Computer Vision , 57(2):137–154, May 2004. ISSN 1573-1405. doi: 10.1023/B: VISI.0000013087.49260.fb
2004
-
[29]
Wang and Y
Y . Wang and Y . Zou. Fast visual object counting via example-based density estimation. In 2016 IEEE International Conference on Image Processing (ICIP), pages 3653–3657, Sep. 2016. doi: 10.1109/ICIP.2016.7533041
2016
-
[30]
In defense of single-column networks for crowd counting
Ze Wang, Zehao Xiao, Kai Xie, Qiang Qiu, Xiantong Zhen, and Xianbin Cao. In defense of single-column networks for crowd counting. CoRR, abs/1808.06133, 2018
2018 arXiv
-
[31]
Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors
Bo Wu and Ram Nevatia. Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. International Journal of Computer Vision , 75(2):247–266, Nov 2007. ISSN 1573-1405. doi: 10.1007/s11263-006-0027-7
2007 doi
-
[32]
Spatiotemporal modeling for crowd counting in videos
Feng Xiong, Xingjian Shi, and Dit-Yan Yeung. Spatiotemporal modeling for crowd counting in videos. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017. ZOU,SHAO: ENHANCED 3D CONVOLUTIONAL NETWORKS FOR CROWD COUNTING 13
2017
-
[33]
Cross-scene crowd counting via deep convolutional neural networks
Cong Zhang, Hongsheng Li, Xiaogang Wang, and Xiaokang Yang. Cross-scene crowd counting via deep convolutional neural networks. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 833–841, June 2015. doi: 10.1109/CVPR.2015.7298684
2015
-
[34]
Costeira, and José M
Shanghang Zhang, Guanhang Wu, João P. Costeira, and José M. F. Moura. Fcn-rlstm: Deep spatio-temporal neural networks for vehicle counting in city cameras. CoRR, abs/1707.09476, 2017
2017 arXiv
-
[35]
Fully convolutional adaptation networks for semantic segmentation
Yiheng Zhang, Zhaofan Qiu, Ting Yao, Dong Liu, and Tao Mei. Fully convolutional adaptation networks for semantic segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
2018
-
[36]
Single-image crowd counting via multi-column convolutional neural network
Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma. Single-image crowd counting via multi-column convolutional neural network. In The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , June 2016
2016
-
[37]
Z. Zou, X. Su, X. Qu, and P. Zhou. Da-net: Learning the fine-grained density distribu- tion with deformation aggregation network.IEEE Access, 6:60745–60756, 2018. ISSN 2169-3536. doi: 10.1109/ACCESS.2018.2875495
2018
-
[2009]
doi: 10.1109/ICCV .2009.5459191
2009
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.