REVIEW 3 major objections 5 minor 24 references
MTCNET: Multi-task Learning Paradigm for Crowd Count Estimation
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper's central claim is that adding a crowd-count group classification task to a dilated-convolution density estimator lowers counting error, with the best reported MAE on ShanghaiTech and UCF_CC_50.
desk verdict Decent MTL-for-crowd-counting paper with new SOTA numbers, but the ablation doesn't fully isolate the auxiliary task and λ is chosen on the test set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the shared VGG-16 front-end followed by a two-branch split: the density branch uses dilated convolutions to regress the density map, while the auxiliary branch classifies the shared features into ten equal-width crowd-count groups. The auxiliary branch's ground-truth labels are obtained by binning each image's count into one of ten equal intervals between the minimum and maximum training counts. This classifier is what carries the argument, because it forces the shared features to separate images by density scale, and the combined MSE plus weighted cross-entropy loss propagates that scale information back into the feature extractor.
What would settle it
Train a single-task density estimator with the same number of parameters and the same optimization schedule as MTCNet—for instance, keep the classifier branch architecture but set its loss weight to zero or freeze it—and compare MAE on ShanghaiTech Part A. If this control reaches the same 63.7 MAE, the auxiliary task is not the cause of the improvement.
Extended reading notes
Core claim
MTCNet is a multi-task network whose main branch is a CSRNet-style dilated-convolution density estimator and whose auxiliary branch is a ten-class crowd-count group classifier. Both branches share the first ten layers of VGG-16. The auxiliary branch takes the 128-feature-map block X2, passes it through convolutional and fully connected layers, and outputs a softmax over ten count ranges; the main branch concatenates X1 (512 maps) with X2 and regresses a density map using dilation rate two. The combined loss is the density MSE plus λ times the classification cross-entropy, with λ=1e-3 giving the best result. In the paper's ablation, the joint model reaches 63.7 MAE on ShanghaiTech Part A while the main task alone reaches 68.2, and the auxiliary classifier's accuracy rises from 56.6% standalone to 79.7% in joint training.
Load-bearing premise
The paper's central assumption is that the counting improvement comes from the auxiliary classification supervision itself; the ablation does not control for the extra parameters of the classifier branch, so the gain could in principle come from a larger or differently trained network.
Editorial extensions
If this is right
- A coarse count-range classifier can be added to existing dilated-convolution density estimators as a drop-in auxiliary head and reduce MAE on both dense and sparse scenes.
- Joint training with the auxiliary task works without data augmentation, so the gain is not dependent on extra training data.
- The optimal weight of the auxiliary loss is small (1e-3), so the classification loss helps most as a weak regularizer rather than as a dominating objective.
- The auxiliary classifier's standalone accuracy of 56.6% vs 79.7% under joint training implies density regression supplies useful signal back to the classification task, making the two tasks mutually reinforcing.
Reading between the lines
- The paper leaves implicit whether the benefit comes from the classification labels per se or from the architectural regularization of adding a small output branch; a testable extension is to replace the ten-class classifier with an auxiliary branch predicting a scalar count via regression and compare MAE.
- Because the auxiliary branch consumes only X2, the count-group classifier may be blind to the deeper X1 features; attaching the classifier to the concatenated features might sharpen scale separation further.
- The ten-way binning is arbitrary; using ordinal regression or fewer bins might retain most of the gain with fewer parameters, which would help distinguish the task-supervision effect from capacity effects.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MTCNet, a multi-task learning architecture for crowd counting. The main task is density-map regression with a VGG-16 front-end and a dilated convolutional backend; the auxiliary task is a ten-class crowd-count-group classifier that shares the front-end. The combined loss is MSE plus a weighted cross-entropy term (Eq. 6). The authors report lower MAE and MSE than prior state-of-the-art methods on ShanghaiTech Part A/B and UCF_CC_50, and they include a weight-factor sensitivity study (Section 4.4) and an ablation showing that joint training improves the main-task MAE from 68.2 to 63.7 on ShanghaiTech Part A (Section 4.5).
Significance. If the reported results hold, the paper demonstrates a simple and effective use of multi-task learning for crowd counting, with claimed MAE improvements of 5.8%–14.9% over competing methods on standard benchmarks. The architectural idea is clearly presented, and the paper includes an ablation, a sensitivity analysis, and comparisons on two widely used datasets. The main strengths are the straightforward integration of an auxiliary count-group classifier and the consistency of the reported gains across three benchmark settings. However, the current experimental design does not yet isolate the effect of the auxiliary task: the ablation in Section 4.5 compares models that differ in architecture and supervision, and the hyperparameter λ is selected using test-set MAE in Section 4.4. Additional control experiments and a proper validation procedure are needed before the central claim can be considered established.
major comments (3)
- [Section 4.5, 'Effect of MTL setting'] The ablation compares a main-task-only MAE of 68.2 with a joint-training MAE of 63.7, but the model is not held fixed between the two settings. The joint model adds a classifier head consisting of three dense layers and a second loss term, and the paper does not specify whether the standalone model retains the X2 feature pathway and the X1-X2 concatenation when the auxiliary branch is removed. A capacity-matched control—for example, training the same two-branch architecture with the auxiliary loss applied to randomly shuffled or fixed class labels, or with the auxiliary branch present but its loss detached—is necessary to determine whether the 4.5-point MAE improvement is due to the count-group supervision itself or to the extra parameters and changed optimization landscape. Without such a control, the paper's central claim that the auxiliary task improves density estimation is not established.
- [Section 4.4, 'Effect of the weight factor λ'] Table 3 reports MAE and MSE on ShanghaiTech Part A for five values of λ and selects λ = 1e-3 as optimal because it gives the lowest MAE, but the paper does not describe any validation split or nested cross-validation procedure. The test set is therefore used for model selection, which makes the reported 63.7 MAE an optimistic, test-set-selected number. The claim of a 5.8% improvement over CSRNet may be inflated by this selection effect. Please use a held-out validation split to choose λ and report the final test performance only once, together with an estimate of variability across runs.
- [Section 4.3, Evaluation on UCF_CC_50] Table 2 reports only a single MAE and MSE for the 5-fold cross-validation on UCF_CC_50, which contains only 50 images. Given the small dataset and the known sensitivity of crowd-counting results to the fold split and to random initialization, a single aggregate number does not support the claim of a 10.5% improvement over the second-best method. Please report per-fold results and the mean ± standard deviation across folds, and preferably across multiple random seeds as well.
minor comments (5)
- [Section 2.3, Eq. (3)] Equation (3) is rendered illegibly; the count-group label formula should be typeset properly, for example as class_label = min(round(9 * (C_i^GT - C_min)/(C_max - C_min)), 9). Please also clarify whether C_min and C_max are computed on the training split only and how test counts outside that range are handled.
- [Figure 2 and Section 2] The flow of X1 and X2 into the density branch and the placement of the AdaptiveMaxPool2d layer are ambiguous. A short textual description of the tensor shapes at each concatenation and pooling step would help readers reproduce the architecture.
- [Section 2.4, Training details] The training details are incomplete: the paper states a fixed learning rate of 1e-7 but does not report batch size, number of epochs, weight decay, or whether all hyperparameters were held fixed across the λ values in Table 3. These details are needed for independent verification.
- [Section 4.4, Table 3] The paper reports the auxiliary-task classification accuracy only for the selected λ (56.6% standalone vs. 79.7% in MTL). Showing the auxiliary-task accuracy for each λ value would help confirm that the auxiliary branch actually learns the count-group structure and would strengthen the interpretation of the λ sensitivity results.
- [Section 4.2, Table 1] The comparison with prior methods does not state whether the competing methods used data augmentation. Since the paper emphasizes that its own results are obtained without data augmentation, a note on the augmentation settings of the baselines would make the comparison fairer.
Circularity Check
No circularity: MTCNet's auxiliary task is an independent supervised signal, and its contribution is tested by an ablation rather than assumed.
full rationale
The paper's central claim is that adding crowd-count group classification as an auxiliary task improves density estimation. The density-map target and the count-group label are both generated from the same head annotations, but they are different supervised signals: one is a per-pixel Gaussian-blurred density map and the other is a single discrete class derived from the total count range. Neither signal is defined in terms of the other in a way that would make the main-task prediction equivalent to the auxiliary-task input by construction. The auxiliary task's contribution is tested in Section 4.5 by comparing standalone main-task training (68.2 MAE) with joint MTL training (63.7 MAE), which is an ablation rather than a fitted parameter renamed as a prediction. The reported results are benchmark comparisons on external test sets, not predictions fitted to the measured outcome. The weight factor lambda is selected empirically in Section 4.4, which may raise reproducibility or overfitting concerns, but that is not circularity because the selected value is not derived from the main-task output and does not force the final MAE by definition. No load-bearing self-citation or imported uniqueness theorem appears; the cited prior work supplies standard baselines, ground-truth generation settings, and architectural choices such as dilated convolution, none of which define the claimed MTL improvement. Overall, the derivation chain is self-contained and no step reduces to its own inputs.
Assumptions & free parameters
free parameters (3)
- Loss weight lambda =
1e-3
- Number of auxiliary count groups =
10
- Ground-truth density kernel parameters =
beta=0.3, k=3 (Part A, UCF); sigma=3 (Part B)
assumptions (4)
- domain assumption Pre-trained VGG-16 features transfer to crowd scenes.
- domain assumption Auxiliary count-group classification captures scale information useful for density estimation.
- domain assumption Blurring head annotations with a Gaussian or geometry-adaptive kernel produces an appropriate regression target.
- domain assumption Dilation rate two is appropriate for the density branch.
Cite this review
Pith. "Pith review of MTCNET: Multi-task Learning Paradigm for Crowd Count Estimation." pith.science (2026). https://pith.science/paper/N5PSJWWQ
@misc{pith2026190808652,
author = {Pith},
title = {Pith review of: MTCNET: Multi-task Learning Paradigm for Crowd Count Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/N5PSJWWQ}},
note = {Machine review of arXiv:1908.08652}
}
read the original abstract
We propose a Multi-Task Learning (MTL) paradigm based deep neural network architecture, called MTCNet (Multi-Task Crowd Network) for crowd density and count estimation. Crowd count estimation is challenging due to the non-uniform scale variations and the arbitrary perspective of an individual image. The proposed model has two related tasks, with Crowd Density Estimation as the main task and Crowd-Count Group Classification as the auxiliary task. The auxiliary task helps in capturing the relevant scale-related information to improve the performance of the main task. The main task model comprises two blocks: VGG-16 front-end for feature extraction and a dilated Convolutional Neural Network for density map generation. The auxiliary task model shares the same front-end as the main task, followed by a CNN classifier. Our proposed network achieves 5.8% and 14.9% lower Mean Absolute Error (MAE) than the state-of-the-art methods on ShanghaiTech dataset without using any data augmentation. Our model also outperforms with 10.5% lower MAE on UCF_CC_50 dataset.
Reference graph
Works this paper leans on
-
[1]
It could hel p to prevent traffic congestion and stampedes at crowded events
INTRODUCTION Recently, there has been an a ccelerated growth in research on crowd understanding due to its varied applications including crowd management, public safety, video surveillance, crow d flow analysis etc. It could hel p to prevent traffic congestion and stampedes at crowded events. In the last few years, researchers have worked on various crowd...
-
[2]
PROPOSED METHOD The proposed network MTCNet (Figure 1) is deep-cascaded convolutional neural networks for two related tasks: Crowd Density Estimation ( main task ) and Crowd -Count Group Classification ( auxiliary task ). Both tasks share the same frontend, which consists of the first ten layers from the VGG-16 [13] network. The input to the MTCNet networ...
-
[3]
DATASET 3.1. ShanghaiTech dataset The ShanghaiTech dataset [9] comprises 1198 annotat ed images with to tal 330,165 people in them . This dataset consists of two parts- Part A and Part B. Part A contains 482 randomly crawled images from internet and have high crowd density. Part B comprises 716 images and has relatively sparse crowd density. The train-tes...
-
[4]
EXPERIMENTAL RESULTS AND DISCUSSION We demonstrate the performance of MTCNet for crowd count estimation as compared to state-of-the-art methods . MTCNet outperforms them all on two benchmark datasets in terms of Mean Absolute Error (MAE) and Mean Squa red Error (MSE). 4.1. Evaluation metrics The MAE and the MSE used for evaluating and comparing MTCNet and...
-
[5]
CONCLUSION We have proposed a novel end -to-end model, MTCNet, based on the multi-task learning paradigm to jointly train the crowd density map (main task) and the count group classification (auxiliary task) . By combining two related tasks (main & auxiliary), the model is able to implicitly learn the scale factor of the given crowd scene and hence able t...
-
[6]
Histograms of oriented gradients for human detection,
Navneet Dalal and Bill Triggs, “Histograms of oriented gradients for human detection,” in Computer Vision and Pattern Recog nition, 2005. CVPR 2005. IEEE Computer Society Conference on. IEEE, 2005, vol. 1, pp. 886–893
work page 2005
-
[7]
Bayesian poisson regression for crowd counting,
Antoni B Chan and Nuno Vasconcelos, “Bayesian poisson regression for crowd counting,” in Computer Vision, 2009 IEEE 12th International Conference on . IEEE, 2009 , pp. 545–551
work page 2009
-
[8]
Crowd density estimation based on rich features and random projection forest,
Bolei Xu and Guoping Qiu, “Crowd density estimation based on rich features and random projection forest,” in Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016, pp. 1–8
work page 2016
Show all 24 references
-
[9]
Learning to count objects in images,
Victor Lempitsky and Andrew Zisserman , “Learning to count objects in images,” in Advances in Neural Information Processing Systems, 2010, pp. 1324–1332
2010
-
[10]
COUNT Forest: CO -Voting Uncertain Number of Targets Using Random Forest for Crowd Density Estimation,
V. Pham, T. Kozakaya, O. Yamaguchi and R. Okada, "COUNT Forest: CO -Voting Uncertain Number of Targets Using Random Forest for Crowd Density Estimation," 2015 IEEE International Conference on Computer Vision (ICCV) , Santiago, 2015, pp. 3253-3261
2015
-
[11]
Deep people counting in extremely dense crowds,
C. Wang, H. Zhang, L. Yang, S. Liu, and X. Cao, “Deep people counting in extremely dense crowds,” in Proceedings of the 2015 ACM on Multimedia Confe rence, pp. 1299–1302, 2015
2015
-
[12]
Learning to count with cnn boosting,
E. Walach and L. Wolf, “Learning to count with cnn boosting,” in European Conference on Computer Vision , pp. 660–676. Springer, 2016
2016
-
[13]
Cross-scene crowd counting via deep co nvolutional neural networks,
Cong Zhang, Hongsheng Li, X. Wang and Xiaokang Yang, "Cross-scene crowd counting via deep co nvolutional neural networks," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Boston, MA, 2015, pp. 833 - 841
2015
-
[14]
Single - Image Crowd Counting via Multi -Column Convolutional Neural Network,
Y. Zhang, D. Zhou, S. Chen, S. Gao and Y. Ma, "Single - Image Crowd Counting via Multi -Column Convolutional Neural Network," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Las Vegas, NV, 2016, pp. 589-597
2016
-
[15]
Switching Convolutional Neural Network for Crowd Counting,
D. B. Sam, S. Surya and R. V. Babu, "Switching Convolutional Neural Network for Crowd Counting," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 4031 -4039
2017
-
[16]
CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes,
Y. Li, X. Zhang and D. Chen, "CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , Salt L ake City, UT, 2018, pp. 1091-1100
2018
-
[17]
Multitask learning,
R.Caruana, “Multitask learning,” Machine learning , vol. 28(1), pp.41-75, 1997
1997
-
[18]
Very deep convolutional networks for large -scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large -scale image recognition,” i n Proceedings of International Conference on Learning Representations, 2015
2015
-
[19]
Multi -scale context aggregation by dilated convolutions,
F. Yu and V. Koltun, “Multi -scale context aggregation by dilated convolutions,” in Proceedings of International Conference on Learning Representations, 2016
2016
-
[20]
Exploiting Unrelated Tasks in Multi -Task Learning,
B. R. Paredes, A. Argyriou, N. Berthouze, M. Pontil, "Exploiting Unrelated Tasks in Multi -Task Learning," Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics , vol. 22, pp. 951 -959, 2012
2012
-
[21]
Mul ti-source Multi-scale Counting in Extremely Dense Crowd Images,
H. Idrees, I. Saleemi, C. Seibert and M. Shah, "Mul ti-source Multi-scale Counting in Extremely Dense Crowd Images," 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, 2013, pp. 2547 -2554
2013
-
[22]
Multi -scale convolutional neural networks fo r crowd counting,
L. Zeng, X. Xu, B. Cai, S. Qiu and T. Zhang, "Multi -scale convolutional neural networks fo r crowd counting," 2017 IEEE International Conference on Image Processing (ICIP) , Beijing, 2017, pp. 465-469
2017
-
[23]
Leveraging Unlabeled Data for Crowd Counting by Learning to Rank,
X. Liu, J. van de Weijer and A. D. Bagdanov, "Leveraging Unlabeled Data for Crowd Counting by Learning to Rank," 2018 IEEE/CVF Conference on Comp uter Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 7661-7669
2018
-
[24]
Crowd Counting with Fully Convolutional Neural Network,
Ming Liu, Jue Jiang, Zhenqei Guo, Zenan Wang and Yang Liu, "Crowd Counting with Fully Convolutional Neural Network," 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, 2018, pp. 953-957
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.