{"id":"876dc8db-b5d0-4a9a-94c3-e0e11f706f8b","arxiv_id":"2507.21364","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"On a four-species African wildlife dataset, ViT-H/14 reaches 99% accuracy versus 67% for the best CNN (DenseNet-201), but at far greater computational cost.","lead":"This paper compares four pretrained deep learning models for classifying African wildlife images from a public dataset of 1,504 photos. A Vision Transformer achieved 99% accuracy, far above the best convolutional network at 67%, but with much higher compute cost.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy trade-off claim is confounded: ViT-H/14 is the only model evaluated at 518×518 while all CNNs run at 64×64, so no controlled comparison supports an architectural conclusion.","rationale":"The reader and I identify the same weakest assumption: resolution is not matched across model families. The paper's own methodology sections make the confound explicit. The comparison can be saved only by showing that DenseNet remains near 67% at 518×518 or that ViT drops sharply at 224×224; neither experiment appears in the manuscript. I would not change the reader's REJECT verdict. Credit is due for a working Gradio deployment and for acknowledging domain-shift and proof-of-concept limitations, and those are real artifacts; but the deployed DenseNet is not evidence for the headline accuracy trade-off. The proposed test is a small, well-defined experiment that would settle whether the concern lands: if the best CNN matches ViT at matched resolution, the central claim falls; if not, the architecture explanation gains support.","tokens_in":7732,"tokens_out":5814,"duration_ms":70777,"concrete_test":"Re-run DenseNet-201 (and, if feasible, ResNet-152 and EfficientNet-B4) with frozen ImageNet features on the same train/test split, using the exact §III.B protocol (Adam, lr 1e-3, batch 32, 10 epochs) but with input resolution 518×518 instead of 64×64. If DenseNet-201 accuracy rises from 67% toward 99%, the reported ViT advantage is a resolution effect and the paper's trade-off conclusion is not supported; if it stays near 67%, the resolution confound is not decisive. Repeat over at least five random splits and report mean ± std, since the test set has only 301 images.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the paper's main claim that ViT-H/14's 99% accuracy versus DenseNet-201's 67% reveals a model-family trade-off, the accuracy gap would have to be attributable to architecture. The reported setup prevents that inference. Section III.A states that all images were resized to 64×64 for CNNs, but 'for the vision transformer training, the images were resized to 518 × 518 pixels as that is the minimum size that the model can take [19]'; Section III.B applies the same 64×64 (518×518 for ViT) protocol. Input resolution and architecture are therefore perfectly confounded in the only cross-family comparison. A 64×64 image has 4,096 pixels; a 518×518 image has 268,324, roughly 65 times as many. Species-defining cues (horns, tusks, trunk, stripe patterns) are largely destroyed at 64×64, so the 32-point gap could be caused by resolution alone. No matched-resolution control is reported, and the paper's discussion attributes the gap to transformers without addressing this. The ViT GFLOPs entry in Table II (1016.7 while the table states GFLOPs are based on 224×224 input) is also inconsistent and reinforces that the ViT was run under a different regime. The cited torchvision page does not establish a 518×518 minimum. This is the load-bearing weakness of the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates four ImageNet-pretrained, frozen-feature classifiers (DenseNet-201, ResNet-152, EfficientNet-B4, and ViT-H/14) on the public African Wildlife dataset of four species (1,504 images). The authors fine-tune only the classification head for 10 epochs, reporting test accuracy and macro F1. They find DenseNet-201 best among CNNs (67% accuracy), while ViT-H/14 achieves 99% accuracy at much higher compute cost, and they use this comparison to argue that DenseNet provides the best accuracy-to-deployability trade-off. A Hugging Face Gradio demo is described.","tokens_in":8029,"tokens_out":3955,"duration_ms":44361,"significance":"If the comparison were properly controlled, the paper would provide a modest empirical data point for model selection on small, low-budget wildlife datasets, but it would not be a major advance. The paper does offer some useful infrastructure (W&B tracking, reproducibility-oriented logging, and a deployed demo), as well as a thoughtful ethics/limitations section. However, the central claim of an architecture-level accuracy trade-off is not supported by the evidence because input resolution and architecture are confounded; the work as presented does not meet the standard for a journal publication.","major_comments":[{"comment":"The central comparison is confounded by input resolution: all CNNs are trained and evaluated at 64×64, while ViT-H/14 is evaluated at 518×518 (Sections III.A and III.B). A 64×64 image has 4,096 pixels versus 268,324 at 518×518, and species-defining cues such as horns, tusks, and stripe patterns may be largely destroyed at the lower resolution. The abstract and Section V.a attribute the 67% vs 99% accuracy gap to the architecture family, but no matched-resolution control is reported (e.g., CNNs at 518×518, or ViT at 64×64). The gap could therefore be fully explained by resolution alone. This is load-bearing because the paper's central claim of an accuracy-cost trade-off across model families rests on this comparison.","section":"III.A, III.B, V.a"},{"comment":"The ViT-H/14 row in Table II reports 1016.7 GFLOPs while the table header says GFLOPs are based on a 224×224 input. In the cited torchvision implementation, ViT-H/14 at 224×224 has roughly 167 GFLOPs; the reported 1016.7 GFLOPs corresponds to a much larger resolution. This internal inconsistency reinforces that the ViT was evaluated under a different regime and invalidates the FLOPs comparison in the table. In addition, Section III.A's claim that 518×518 is \"the minimum size that the model can take\" is not supported by the cited torchvision documentation, which also accepts 224×224. The factual claim should be corrected and all FLOPs numbers recomputed consistently.","section":"Table II and III.A"},{"comment":"The experimental report lacks statistical grounding. Each model is run only once, with no multiple seeds, confidence intervals, standard deviations, or documented random split/index seed; the split is described only as an 80/20 partition of 1,504 images. On a 301-image test set, the differences among CNNs (48%, 57%, 67%) could easily be within run-to-run noise. This weakens the specific claim that DenseNet-201 is the best CNN and the cross-model ranking in Table I. The authors should report mean and standard deviation over at least several seeds, describe the exact split procedure, and state the per-class test-set sizes.","section":"III.A and IV (Table I)"}],"minor_comments":[{"comment":"The paper's own evaluation notes that performance declined sharply on smartphone-captured field images (Section V.b) and that the model is \"a proof-of-concept and not yet robust enough for critical decision-making\" (Section VII); these statements should be reflected earlier in the abstract, which currently claims feasibility of real-time field use without that caveat.","section":"V.b and VII"},{"comment":"References [7] and [20] are duplicates of the same DenseNet paper; they should be unified to avoid citation inconsistencies.","section":"References"},{"comment":"The text refers to \"Appendix 2\" in Section V.a, but the appendix is not numbered; the reference should be to a specific figure (e.g., Figure 2) for the GPU power plot.","section":"Appendix"},{"comment":"Table I would be easier to read with consistent decimal formatting (67.0%, 48.0%, etc.) and with the number of test images per class stated.","section":"Table I"},{"comment":"The informal phrasing \"a dataset gotten from Africa\" in Section II should be replaced with a more precise description of the dataset's provenance and collection conditions.","section":"III.A"}],"recommendation":"reject","confidential_remarks":"This is a clear reject: the main quantitative comparison is invalid because input resolution is perfectly confounded with model architecture, and the FLOPs table contains an obvious inconsistency. The paper also has no error bars, so even the CNN-to-CNN comparison is not reliable. The authors should be advised that a controlled experiment (e.g., all models at the same resolution, multiple seeds) is necessary before any architecture-level conclusion can be drawn. I see no sign of misconduct; the W&B tracking and the deployed demo are legitimate efforts, and the ethics section is a positive addition."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the central comparison in this paper is confounded by input resolution, and the headline result doesn't survive contact with the methods. The authors train all CNNs at 64×64 but give ViT-H/14 518×518 — a 65× difference in pixels — and then attribute the 99% vs 67% gap to architecture. That attribution is not supported. The cited torchvision page doesn't mandate 518×518, and there's no matching-resolution control. So the accuracy/compute trade-off they advertise is also unsupported.\n\nTo give credit: the paper is clearly written, uses a public African wildlife dataset, reports per-class F1 scores, and the authors actually deployed a DenseNet Gradio demo and wrote a sensible limitations/ethics section. The related work is fine. None of that is new — [16] already used the same Kaggle dataset and [2] compared DenseNet/ResNet against YOLO — but the execution is tidy.\n\nThe soft spots beyond the resolution issue: single runs, no seeds, no confidence intervals, no code or data release. Table II says GFLOPs are based on 224×224, yet ViT's 1016.7 GFLOPs can't be at that size; it's another sign the ViT ran in a different regime. And the 10-epoch frozen-feature protocol is the bare minimum.\n\nI can't see a reader who gets reliable guidance from this. The central empirical claim is confounded, and the small dataset limits the generality. It would be a useful cautionary exercise for a class, but not a paper I'd cite or send to a serious referee. If the authors did a resolution-matched control (even one extra CNN at 518×518) and reported seeds and variance, it could become a modest benchmark note. As is, I'd desk reject.","headline":"The ViT vs CNN comparison is confounded by a 65× input-resolution gap, so the paper's central trade-off claim doesn't hold.","tokens_in":8576,"tokens_out":3392,"would_cite":false,"duration_ms":38892,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Vision Transformer reaches 99% accuracy on four African wildlife classes, while the best CNN reaches 67%, and the lightweight CNN is argued to be the more practical deployment choice.","keywords":["African wildlife","image classification","DenseNet","Vision Transformer","transfer learning","model deployment","computational cost","camera trap"],"falsifier":"Train DenseNet-201 on the same African Wildlife dataset at 224×224 or 518×518 with the same frozen-feature-extractor protocol and identical hyperparameters; if accuracy rises substantially above 67%, the ViT's advantage is at least partly a resolution artifact rather than a pure architectural superiority.","tokens_in":7559,"feed_emoji":"🦓","tokens_out":8516,"duration_ms":84055,"temperature":0.7,"pith_summary":"The paper sets out to determine which pretrained deep-learning model is most suitable for classifying images of four African savanna species: buffalo, elephant, rhinoceros, and zebra. Using transfer learning with frozen feature extractors on a balanced public dataset of 1,504 images, it reports that the best convolutional network, DenseNet-201, reaches 67% test accuracy, while the Vision Transformer ViT-H/14 reaches 99%. The paper argues that this accuracy gap is a trade-off with computational cost, and that the lightweight DenseNet-201 is the more realistic choice for real-time conservation deployment. The reader should take away that lightweight CNNs remain viable in low-resource settings, and that the ViT's near-perfect accuracy is purchased with a much larger model and training budget.","feed_headline":"ViT scores 99% on African wildlife; best CNN 67%","feed_subtitle":"Four-species benchmark shows a wide accuracy gap, but the paper argues lightweight DenseNet is the practical field choice.","key_machinery":"The comparison is carried by transfer learning with frozen feature extractors: each ImageNet-pretrained backbone is kept frozen and only the final classification layer is fine-tuned, isolating the feature representations and keeping training cheap. The decisive technical detail is the input-resolution asymmetry: the CNN models receive 64×64 images, while ViT-H/14 receives 518×518 images because that is the model's minimum accepted input size. This asymmetry, rather than the attention-versus-convolution distinction alone, is what the reported comparison actually tests.","core_discovery":"The paper's central claim is that on the African Wildlife dataset, ViT-H/14 classifies buffalo, elephant, rhinoceros, and zebra with 99% accuracy and a macro F1-score of 0.99, while the best convolutional model, DenseNet-201, reaches 67% accuracy and a macro F1-score of 0.67. ResNet-152 and EfficientNet-B4 sit below DenseNet, at 57% and 48% respectively. DenseNet-201 has 20 million parameters, trains in under two minutes on the reported GPU, and is shipped as an interactive web application; ViT-H/14 has 632 million parameters, requires roughly an hour of training, and is described as resource-intensive. The paper concludes that for field deployment in conservation settings, the lightweight CNN is the practical default, while the transformer's accuracy represents a target for future compression or distillation.","pith_inferences":["A matched-resolution control experiment (feeding the same 224×224 or 518×518 images to the CNNs and the ViT) would likely shrink the 32-point accuracy gap, because the ViT's advantage is entangled with about an 8× linear increase in input size.","The paper's practical recommendation to use DenseNet could survive even if the accuracy gap is mostly resolution, but the justification would shift from 'the CNN architecture is well suited' to 'the CNN architecture is cheap enough to run at field-friendly resolutions.'","The unusually low 64×64 CNN input may underuse modern backbones, so EfficientNet-B4's 48% result may reflect resolution rather than a fundamental weakness of compound scaling.","If CNN accuracy rises sharply at higher resolution, the reported trade-off becomes a tunable knob: conservation teams could choose accuracy or speed by adjusting input size rather than switching architecture."],"forward_implications":["DenseNet-201 at 64×64 input gives a working four-species classifier at 67% accuracy with 20 million parameters, trainable in under two minutes on the reported hardware.","ViT-H/14's 99% accuracy comes with 632 million parameters and more than an hour of training, so it is not directly deployable on low-resource edge devices.","The accuracy–cost gap means model selection for field use should treat deployability as a first-class constraint, not just a footnote.","A real-world deployment test shows a sharp accuracy drop on smartphone-captured images, which the paper attributes to domain shift between curated training data and field conditions.","Future gains are expected from larger, more diverse datasets and from compressing or distilling the transformer, rather than from tuning the frozen CNN backbones."],"supporting_citations":[{"why":"Supplies the African Wildlife dataset of 1,504 images across the four species used in all experiments.","marker":"[6]"},{"why":"Defines the DenseNet architecture whose dense connectivity the paper hypothesizes suits small wildlife datasets.","marker":"[7]"},{"why":"Defines ResNet-152, one of the CNN baselines in the comparison.","marker":"[8]"},{"why":"Defines EfficientNet-B4, one of the CNN baselines in the comparison.","marker":"[9]"},{"why":"Defines ViT-H/14, the model that achieves 99% accuracy.","marker":"[10]"},{"why":"Motivates the 64×64 resizing and normalization preprocessing applied to the CNN inputs.","marker":"[18]"},{"why":"States the 518×518 minimum input size for ViT-H/14, creating the resolution asymmetry.","marker":"[19]"},{"why":"Documents domain shift in wildlife recognition, used to explain the deployed model's drop on field images.","marker":"[12]"}],"fun_headline_variants":["ViT 99% vs CNN 67% on wildlife: accuracy vs practicality","Transformer tops wildlife benchmark, but CNN wins on deployability","Vision transformer 99% vs CNN 67% on African wildlife","Accuracy gap: ViT 99% vs DenseNet 67%—but lightweight wins in practice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that comparing CNNs at 64×64 input with a Vision Transformer at 518×518 input is a fair architecture comparison; if the resolution difference drives most of the accuracy gap, the architecture-level conclusion does not follow.","fun_headline_variants_meta":{"raw":{"variants":["ViT 99% vs CNN 67% on wildlife: accuracy vs practicality","Transformer tops wildlife benchmark, but CNN wins on deployability","Vision transformer 99% vs CNN 67% on African wildlife","Accuracy gap: ViT 99% vs DenseNet 67%—but lightweight wins in practice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001043,"raw_usage":{"total_tokens":4392,"prompt_tokens":959,"completion_tokens":3433,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":3349}},"tokens_in":575,"tokens_out":3433,"duration_ms":25408,"temperature":1.0,"reasoning_tokens":3349,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:49:13.662750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train DenseNet-201 on the same African Wildlife dataset at 224×224 or 518×518 with the same frozen-feature-extractor protocol and identical hyperparameters; if accuracy rises substantially above 67%, the ViT's advantage is at least partly a resolution artifact rather than a pure architectural superiority.","supporting_citations":[{"cited_title":"African wildlife dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the African Wildlife dataset of 1,504 images across the four species used in all experiments."},{"cited_title":"Densely connected convolutional networks,","cited_arxiv_id":null,"evidence_quote":"Defines the DenseNet architecture whose dense connectivity the paper hypothesizes suits small wildlife datasets."},{"cited_title":"Deep residual learning for image recognition,","cited_arxiv_id":null,"evidence_quote":"Defines ResNet-152, one of the CNN baselines in the comparison."},{"cited_title":"Efficientnet: Rethinking model scaling for convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines EfficientNet-B4, one of the CNN baselines in the comparison."},{"cited_title":"An image is worth 16x16 words: Transformers for image recognition at scale,","cited_arxiv_id":null,"evidence_quote":"Defines ViT-H/14, the model that achieves 99% accuracy."},{"cited_title":"A survey on data augmentation for deep learning,","cited_arxiv_id":null,"evidence_quote":"Motivates the 64×64 resizing and normalization preprocessing applied to the CNN inputs."},{"cited_title":"torchvision.models.vit h 14,","cited_arxiv_id":null,"evidence_quote":"States the 518×518 minimum input size for ViT-H/14, creating the resolution asymmetry."},{"cited_title":"Recognition in terra incognita: Wildlife object classification in unseen domains,","cited_arxiv_id":null,"evidence_quote":"Documents domain shift in wildlife recognition, used to explain the deployed model's drop on field images."}],"review_version":1}