{"id":"c58a2f79-a4fc-4f55-b146-8664c5679868","arxiv_id":"2412.02250","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Vision transformers are competitive but not superior to ResNets for weakly-supervised microorganism counting when trained from scratch.","lead":"This paper tests whether vision transformers can count microbes in microscope images using only total counts as labels. It finds ResNets still win overall, but some transformers come close and one runs faster.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"From-scratch training protocol biases the ViT comparison; the unqualified conclusion that ResNets outperform ViTs is not supported without pretrained ViT baselines.","rationale":"The reader identified the from-scratch training protocol as the weakest assumption, and the paper's own Discussion section supports this concern by acknowledging that most ViT studies use pretrained weights and that TransCrowd's pretrained version behaves differently. This is the most load-bearing issue because the central claim is a comparative one: if the protocol systematically disadvantages one family, the headline result may be an artifact. The paper has real strengths: it provides a reproducible benchmark, a new synthetic dataset, and a clear experimental structure across four datasets. The concern does not invalidate the paper's value as a from-scratch comparison, but it does require the conclusions to be explicitly scoped to that setting. The verdict should remain CONDITIONAL, requiring either pretrained ViT baselines or a qualified conclusion; hence no change to the reader's verdict is needed.","tokens_in":13087,"tokens_out":7650,"duration_ms":83018,"concrete_test":"Fine-tune ImageNet-pretrained Vanilla ViT-B/16, CrossViT-Ti, and TransCrowd (as in their original papers) on the same four datasets with the same regression heads and evaluation splits. Compare their MAE/RMSE to the from-scratch ResNet50/101 values in Table III, running at least 5 seeds and reporting mean±std. If any pretrained ViT reaches or beats the best ResNet MAE on the majority of datasets, the conclusion 'ResNets perform better overall' is an artifact of the from-scratch protocol. If ResNets still win, the claim survives once the qualifier is restored.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central conclusion that ResNets perform better overall is obtained under a protocol that systematically disadvantages ViTs: all models are trained from scratch with no pre-trained weights and near-identical hyperparameters (Section IV-A). ViTs are known to be data-hungry and in practice rely on pretraining; the paper itself concedes in Section VI that 'most studies use pre-trained weights and special fine-tuning for ViT-based approaches' and that TransCrowd, after pretraining and fine-tuning in its original use case, showed state-of-the-art performance 'contrary to the results of our study.' Thus Table III may reflect the training budget rather than architectural merit. Because the abstract and conclusion state the ResNet-over-ViT result without the from-scratch qualifier, the central claim overreaches. The absence of variance estimates compounds this: with no standard deviations reported, it is unclear whether gaps such as Vanilla ViT's 1.886 MAE versus ResNet50's 1.225 MAE on VGG-Cells are meaningful. The load-bearing fix is to test pretrained ViT baselines before claiming ResNet superiority.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a comparative empirical study of vision transformer (ViT) backbones versus traditional CNN and ResNet architectures for weakly-supervised microorganism counting. Four microscopy datasets are used, including a newly created synthetic fluorescent bacteria dataset. All models are trained from scratch with near-identical hyperparameters, and performance is measured by MAE, RMSE, and FLOPs. The central claim is that, while ResNets achieve the best overall accuracy, ViTs—especially CrossViT—are competitive and occasionally superior on homogeneously distributed data. The manuscript also reports that the TransCrowd model, a state-of-the-art ViT-based counting method, underperforms when trained from scratch, in contrast to its pretrained performance reported in the literature.","tokens_in":13261,"tokens_out":3293,"duration_ms":33884,"significance":"The study addresses a practical and under-explored problem: using weakly-supervised counting for microorganism enumeration, where only global counts are available as labels. Its main strengths are the breadth of architectures compared, the introduction of a synthetic fluorescent bacteria dataset with released generation code, and the clear statement of the from-scratch training protocol, which supports reproducibility. If the comparison were properly controlled, the result that from-scratch ViTs are generally worse than ResNets for this task would be a useful negative result. However, because the protocol deliberately excludes pretraining—which is known to be crucial for ViTs—and because the paper itself concedes this limitation in Section VI, the current evidence does not support the unqualified conclusion stated in the abstract and conclusion. The absence of variance estimates further weakens the significance of the reported ranking. The paper is a plausible starting point for a more rigorous benchmarking study, but as it stands the central claim is not fully supported.","major_comments":[{"comment":"The central claim that ResNets outperform ViTs is drawn from a protocol where all models are trained from scratch and the paper itself acknowledges in Section VI that 'most studies use pre-trained weights and special fine-tuning for ViT-based approaches' and that TransCrowd, after pretraining, shows state-of-the-art performance 'contrary to the results of our study.' This self-acknowledged limitation means the MAE/RMSE gap in Table III may reflect the training budget rather than architectural merit. To support the abstract's unqualified statement, the authors should either add pretrained ViT baselines (at least for one or two representative architectures) or explicitly restrict the conclusion to the from-scratch setting in both the abstract and the conclusion.","section":"Section IV-A and Table III"},{"comment":"No standard deviations, confidence intervals, or significance tests are reported for any MAE/RMSE value, despite the statement in Section IV-A that 'Experiments were run with different randomization seeds.' With multiple seeds, variance estimates are cheap and necessary to determine whether gaps such as Vanilla ViT's MAE of 1.886 versus ResNet50's 1.225 on VGG-Cells are meaningful. Without such estimates, the performance ranking in Table III is not statistically grounded.","section":"Table III and Section IV-A"},{"comment":"The training protocol is not actually 'nearly identical' because the batch size is confounded with the architecture: 128 for CNNs, 64 for Vanilla ViT, CrossViT, TransCrowd, and ResNets, and 32 for Parallel ViT, DeepViT, and XCiT. Batch size affects optimization dynamics, regularization, and final accuracy, so the comparison is not controlled across architectures. The authors should either use a common batch size (if memory permits) or justify the choice and analyze its effect; otherwise, any performance difference cannot be attributed solely to the architecture.","section":"Section IV-A"},{"comment":"The train/validation split procedure is not described. For each dataset, it is not stated how the augmented images were partitioned, what ratio was used, whether the split was stratified by count, or whether the same split was used for all models. This is especially important for the Human Cancer Cells dataset, which has only 1463 augmented images and where results are likely sensitive to the split. The authors should specify the split procedure and ideally run multiple splits or cross-validation to ensure the ranking is robust.","section":"Section IV-B"}],"minor_comments":[{"comment":"The citation for CCTrans is incorrect: the text says 'named CCTrans [16]', but reference [16] is the CounTR paper; CCTrans should cite reference [40] (Tian et al., CCTrans). Please correct the citation.","section":"Section II-C"},{"comment":"The column header 'MLP Dim.' for CNNs is described as 'CONVOLUTIONAL OUTPUT'; this is confusing because MLP dimension is not a standard term for CNN channels. Please rename the column, e.g., 'Feature channels' for CNNs and keep 'MLP dim' for ViTs.","section":"Table I"},{"comment":"The paper reports 'average FLOPS (floating point operations per second)' in Table III and Section V-C, but the units (10^8) indicate a count of floating-point operations per inference, not operations per second. If throughput is intended, please report inference time or FPS; if it is FLOPs, please correct the terminology.","section":"Section IV-A"},{"comment":"The 'Data augmentation' column states 'yes' for three datasets but does not describe what augmentation was applied. Please specify the augmentation operations (e.g., random crops, flips, rotations) in the text or in a footnote.","section":"Table II"},{"comment":"The phrase 'ViTs performance demonstrates competent results' should be 'ViTs' performance demonstrates competent results' (add apostrophe). In addition, the sentence is grammatically awkward; consider rephrasing.","section":"Section I"},{"comment":"The text states 'To achieve higher model complexity without compromising parameter and compute neutrality, Parallel ViT [50] proposes parallelizing the MHSA and feed-forward blocks.' The term 'compute neutrality' is unclear; please rephrase to 'without increasing the number of parameters or FLOPs' for precision.","section":"Section III-B-2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical study but the central claim overreaches given the from-scratch protocol. I would urge the editor to ask for pretrained baselines or a clearly qualified conclusion. The synthetic dataset generation code is a positive contribution, but the representativeness of the synthetic bacteria images should be validated against real imagery before drawing strong conclusions about that dataset. The reference error for CCTrans is minor but should be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely useful benchmark for a practical niche, with code and a synthetic dataset released, but the central \"ResNets are better\" claim is only true under a from-scratch training protocol that systematically disadvantages ViTs, and the paper states it without that qualifier. Worth a referee, but it needs revisions.\n\nWhat's actually new: the first systematic comparison of ViT backbones for weakly-supervised microorganism counting, spanning four datasets including a newly generated synthetic fluorescent bacteria set. The code and dataset generation tool are public, which makes the comparison independently reproducible. That's real value. The paper also does the honest work of including TransCrowd as a SOTA reference and reporting FLOPs, and it openly acknowledges in Section VI that most ViT work relies on pretraining and fine-tuning, and that TransCrowd's original results contradict what they see from scratch.\n\nSoft spots, in order of importance. First, no error bars or standard deviations anywhere, despite saying experiments used different seeds. The gap between Vanilla ViT (1.886 MAE) and ResNet50 (1.225 MAE) on VGG-Cells might be meaningful or noise; we can't tell. That is a basic reporting requirement for a benchmark paper. Second, the from-scratch protocol: batch sizes vary across models (128 for CNNs, 64 for ViTs/ResNets, 32 for larger ViTs), and fixed 400 epochs with a patience-based scheduler can favor architectures that converge faster. The paper's own discussion concedes the pretraining point, so the abstract's unqualified \"ResNets perform better overall\" overreaches. It should either say \"under from-scratch training\" or add a pretrained ViT baseline. Third, the synthetic bacteria dataset is simple: random Gaussian ellipses on real background patches. It serves as a controlled stress test, but it's not a substitute for real labelled fluorescence data. That's a minor point because the authors say as much.\n\nThe citation pattern looks appropriate. The math is straightforward regression; no claims of derivations. The paper doesn't oversell novelty, except the abstract's \"innovative approaches\" phrasing is a bit much.\n\nWho gets value: practitioners in microbial image analysis choosing between CNN and ViT backbones, and crowd-counting researchers who want a clean multi-dataset comparison in a different domain. I'd probably not cite it in my own work unless I was working on microorganism counting, but I'd send it to a competent referee. With error bars and a pretrained ViT baseline (or a tempered claim), it's a solid workshop or short conference paper.","headline":"Solid, reproducible benchmark; the headline ResNet-over-ViT claim is protocol-dependent and needs qualifiers or pretrained baselines.","tokens_in":13773,"tokens_out":3070,"would_cite":false,"duration_ms":32479,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Transformer vision models can count microorganisms from whole-image labels, though ResNets still lead.","keywords":["weakly-supervised counting","vision transformers","microorganism enumeration","ResNet","CrossViT","regression counting","fluorescent microscopy","cell counting"],"falsifier":"Run the same four datasets with an ImageNet-pretrained CrossViT and an ImageNet-pretrained ResNet50 under identical fine-tuning schedules: if the pretrained CrossViT still fails to beat ResNet50 on the artificial bacteria dataset, the paper's case for ViT competitiveness would be weakened; if it does beat it, the from-scratch protocol would be the reason for the reported gap.","tokens_in":12891,"feed_emoji":"🦠","tokens_out":7978,"duration_ms":71892,"temperature":0.7,"pith_summary":"The paper tries to establish that weakly-supervised counting — regressing a whole-image microorganism count directly from image features, with no localization labels — can be carried out by vision transformer backbones at a level close to that of ResNets. It compares several ViT variants, the TransCrowd counting head, and CNN/ResNet baselines on four microscopy datasets, training all models from scratch under the same schedule. The paper argues that while ResNet50 and ResNet101 achieve the lowest errors overall, CrossViT is competitive, wins on their homogeneous artificial bacteria dataset, and is the most compute-efficient ViT tested. This matters because microorganism enumeration is labor-intensive, and a method that needs only global count labels would cut annotation cost while supporting automation.","feed_headline":"Transformer vision models nearly match ResNets at microbe counting","feed_subtitle":"CrossViT wins on uniform bacteria images while using far less compute than other ViTs.","key_machinery":"The central object is the backbone-plus-regression-head pipeline for weakly-supervised counting: a feature extractor (ResNet, CNN, or vision transformer) maps the image to embeddings, and a single fully connected layer regresses those embeddings to a total count. For ViTs, images are divided into patches ($16\\times16$ or $32\\times32$) and processed by multi-head self-attention; CrossViT adds a dual-branch design with cross-attention that fuses features from two patch scales. The authors also constructed a synthetic dataset by placing Gaussian-ellipse fluorescent bacteria onto real backgrounds, yielding 12,000 images with counts up to 1,855, to cover dense and homogeneously sparse scenes. Training every architecture from scratch with near-identical hyperparameters is what lets the paper attribute observed differences to architecture choice.","core_discovery":"On all four datasets, ResNet backbones reach the lowest mean absolute error, with ResNet101 best on fluorescent neurons (MAE 1.400) and ResNet50 best on VGG-cells (MAE 1.225), human cancer cells (MAE 27.206), and the artificial bacteria dataset (MAE 22.030). The ViT family is not far behind in most settings: CrossViT captures the best overall result on the artificial bacteria dataset (MAE 20.011) with the lowest transformer FLOPS ($50.91\\times 10^8$), TransCrowd-Token matches ResNet-level accuracy on fluorescent neurons and artificial bacteria, DeepViT is the best ViT on fluorescent neurons, and vanilla ViT is best on VGG-cells. The paper reads this as evidence that ViTs are viable feature extractors for weakly-supervised microorganism enumeration, especially on homogeneous data, and that their gap to ResNets may shrink with pretraining and fine-tuning.","pith_inferences":["Because ViTs are known to depend on pretraining, the paper's from-scratch comparison likely understates their ceiling; a fairer test would compare ImageNet-pretrained ViTs against from-scratch ResNets.","The CrossViT advantage on homogeneous data may extend to other uniform-texture counting problems, such as colony counting on agar plates or particle counting in environmental samples.","A testable follow-up is to vary patch size and fusion strategy on the artificial dataset to isolate whether multi-scale cross-attention or parameter efficiency drives CrossViT's win.","Weakly-supervised counting with global labels could combine with active learning to further reduce real-world labeling effort."],"forward_implications":["If ViTs can count microorganisms from global counts, laboratories could skip instance-level annotation and still automate enumeration for contamination monitoring and health-standard checks.","CrossViT's result on the artificial bacteria dataset suggests multi-scale patch fusion is a promising direction for dense, uniformly distributed microorganism images.","The from-scratch protocol leaves open that pretrained ViTs, fine-tuned in the usual way, could close or reverse the gap to ResNets.","The released artificial dataset generator gives other researchers a controlled benchmark for weakly-supervised counting with known density."],"supporting_citations":[{"why":"Supplies TransCrowd, the state-of-the-art ViT-based weakly-supervised counting model with token and GAP heads that the paper adapts and benchmarks.","marker":"[17]"},{"why":"Defines the vanilla ViT backbone and patch-based self-attention mechanism used for most transformer baselines.","marker":"[18]"},{"why":"Provides the CrossViT dual-branch cross-attention architecture whose efficiency and multi-scale fusion drive the best ViT results.","marker":"[49]"},{"why":"Provides the ResNet50 and ResNet101 residual backbones that achieve the best overall accuracy in the comparison.","marker":"[52]"},{"why":"Supplies the Fluorescent Neuronal Cells dataset used as the low-density neuron counting task.","marker":"[19]"},{"why":"Supplies the VGG-Cells dataset used as the high-density cell counting task and the density-estimation context.","marker":"[20]"},{"why":"Supplies the Human Cancer Cells dataset used as the small-data counting task with global-count labels.","marker":"[21]"},{"why":"Provides the prior ResNet-based weakly-supervised cell counting architecture that motivates the baseline choice.","marker":"[33]"}],"fun_headline_variants":["ViTs narrow gap to ResNets in microbe counting","CrossViT tops bacteria enumeration, cuts compute","Transformer backbones viable for microbe counting","ResNets lead microbe counting, ViTs trail closely","CrossViT wins on uniform bacteria with low compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that training all architectures from scratch with nearly identical hyperparameters is a fair way to judge them, even though vision transformers are known to need pretraining and large datasets to reach their potential.","fun_headline_variants_meta":{"raw":{"variants":["ViTs narrow gap to ResNets in microbe counting","CrossViT tops bacteria enumeration, cuts compute","Transformer backbones viable for microbe counting","ResNets lead microbe counting, ViTs trail closely","CrossViT wins on uniform bacteria with low compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000336,"raw_usage":{"total_tokens":1855,"prompt_tokens":933,"completion_tokens":922,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":845}},"tokens_in":549,"tokens_out":922,"duration_ms":9342,"temperature":1.0,"reasoning_tokens":845,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:39:20.904088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four datasets with an ImageNet-pretrained CrossViT and an ImageNet-pretrained ResNet50 under identical fine-tuning schedules: if the pretrained CrossViT still fails to beat ResNet50 on the artificial bacteria dataset, the paper's case for ViT competitiveness would be weakened; if it does beat it, the from-scratch protocol would be the reason for the reported gap.","supporting_citations":[{"cited_title":"TransCrowd: weakly-supervised crowd counting with transformers","cited_arxiv_id":"2104.09116","evidence_quote":"Supplies TransCrowd, the state-of-the-art ViT-based weakly-supervised counting model with token and GAP heads that the paper adapts and benchmarks."},{"cited_title":"Crossvit: Cross-attention multi-scale vision transformer for image classification,","cited_arxiv_id":null,"evidence_quote":"Provides the CrossViT dual-branch cross-attention architecture whose efficiency and multi-scale fusion drive the best ViT results."},{"cited_title":"Automating cell counting in fluorescent microscopy through deep learning with c-resunet,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fluorescent Neuronal Cells dataset used as the low-density neuron counting task."},{"cited_title":"Learning to count objects in images,","cited_arxiv_id":null,"evidence_quote":"Supplies the VGG-Cells dataset used as the high-density cell counting task and the density-estimation context."},{"cited_title":"Microscope images of human cancer cell lines (u2os and hl-60),","cited_arxiv_id":null,"evidence_quote":"Supplies the Human Cancer Cells dataset used as the small-data counting task with global-count labels."},{"cited_title":"Deep learning and transfer learning for automatic cell counting in microscope images of human cancer cell lines,","cited_arxiv_id":null,"evidence_quote":"Provides the prior ResNet-based weakly-supervised cell counting architecture that motivates the baseline choice."}],"review_version":1}