REVIEW 3 major objections 4 minor 1 cited by
The DongNiao International Birds 10000 Dataset
T0 review · 3 major / 4 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read A new bird dataset spans 10,922 species and claims full global coverage.
desk verdict A genuinely large new bird dataset, but the label accuracy is unmeasured and the archive is fragile; worth peer review only if the authors add validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the collection pipeline. For each species, the authors query image search engines with the species' scientific and/or common names, keep only pictures whose titles contain those names, and then pass the kept images through an object-detection model that crops each bird and resizes the crop to a 300x300 thumbnail while preserving the bird's aspect ratio, padding with black frames where necessary. This pipeline is what turns the 10.1 world bird checklist into the claimed category-to-image mapping. The checklist itself is the other structural component, because it fixes what counts as a species and therefore what 'all bird species in the world' means.
What would settle it
Sample roughly 200 thumbnails across a range of species, have an independent bird specialist identify the species from each image, and compare with the dataset label; if the disagreement rate is above a few percent, the title-filtering label pipeline is refuted and the dataset's classification value is substantially weakened.
Extended reading notes
Core claim
The central claim is that DIB-10K contains 4,876,536 thumbnails of 10,922 distinct bird species and, because the categories enumerate the 10.1 edition of the standard world bird checklist, it covers all bird species in the world. The authors describe the dataset as the largest current bird image dataset by number of categories and as a challenge for fine-grained classification: common species such as the Western Jackdaw have thousands of images while extinct or very rare species such as the Mauritius Blue Pigeon have roughly twenty. They also state that DIB-10K is not an object-detection benchmark, because every thumbnail is centered on one bird, and that it distributes thumbnails only, without owning image copyrights, in the same thumbnail-only spirit as the large-scale reference datasets it discusses.
Load-bearing premise
The label of each image is only as reliable as the text in the search-engine result title, and the paper reports no visual check that the image actually shows the named species, so systematic label error would go undetected.
Editorial extensions
If this is right
- If the central claim is correct, DIB-10K gives fine-grained classifiers a testbed with 10,922 classes, roughly an order of magnitude more than typical vision benchmarks.
- The centered-thumbnail format means the dataset is suitable for classification and fine-grained recognition but not for object detection, so it should be compared with bird benchmarks rather than detection suites.
- Because the category set is tied to a living checklist, any future update to that checklist changes the dataset's coverage and class structure.
- The severe imbalance between abundant and near-extinct species makes long-tail and few-shot learning the central methodological problems the dataset poses.
Reading between the lines
- An independent label audit is a natural next step: sample thumbnails across species, compare visual similarity among labels, and publish a per-species label-noise estimate, since the paper reports no such verification.
- The search-title pipeline could be reused for other fine-grained taxa, but its accuracy would need to be validated first because bird image titles may not be representative of how well search titles identify other organisms.
- A standard few-shot classification protocol over the full 10,922-class label space would be a sensible extension, because the strong class imbalance makes naive train/test splits potentially misleading.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper announces DongNiao International Birds 10000 (DIB-10K), an image dataset of 4,876,536 thumbnails organized into 10,922 bird species, intended for fine-grained classification and ornithology. Images were crawled from the web, filtered by a Faster R-CNN bird detector, cropped and resized to 300x300 thumbnails, and labeled by querying image search engines with scientific or common names and retaining images whose titles match those names. The authors state that the dataset follows IOC 10.1 taxonomy, covers all bird species worldwide, is bird-centered, has an unprecedentedly large number of categories, and is severely class-imbalanced, with examples ranging from over 4,000 images for the Western Jackdaw to 20 images for the extinct Mauritius Blue Pigeon. The paper contains no experiments, no label-precision measurements, and no downloadable species manifest.
Significance. If the taxonomic coverage and label accuracy were verified, DIB-10K would be a uniquely large fine-grained image dataset, with more than an order of magnitude more categories than CUB-200 and with explicit IOC taxonomy alignment. The scale and the bird-centered thumbnail design could support extreme-classification research and some ornithological applications. The paper honestly states that it does not own image copyrights and provides thumbnails rather than full images. However, the scientific value depends entirely on the correctness of species labels, and the paper provides no evidence for that correctness, so the significance is conditional rather than demonstrated.
major comments (3)
- [Section 2 (Dataset Specification and Collection)] The label-assignment procedure relies on search-engine titles: images are kept only if their title text contains the scientific or common name. This is not a visual verification of the species in the image. Search-engine titles can describe the containing web page rather than the image content, common names are ambiguous across species and languages, and many bird species are visually near-identical to congeners. The claimed central result, that DIB-10K contains 10,922 correctly labeled IOC species, is therefore unsupported. The paper should report a stratified human audit of label precision (e.g., per-species or per-rarity-stratum estimates with confidence intervals), compare a sample against existing verified datasets such as CUB-200 or iNaturalist, and ideally include a baseline classification experiment with a fixed train/test split. The Mauritius Blue Pigeon example, with only 20 images, is a useful stress case: title-only labeling is most fragile precisely for rare and extinct species.
- [Section 2 and Table 1] The manuscript contains an internal inconsistency: the abstract and text state 4,876,536 thumbnails of 10,922 species, while Table 1 reports 4,800,000 images and 10,000 categories. Additionally, the claim of complete IOC 10.1 coverage cannot be checked because the paper provides no species list, per-species image counts, or IOC identifiers. The authors should release or include a manifest with the full species list and per-category counts, and correct the numerical discrepancy, so that readers can independently verify the stated coverage and imbalance.
- [Section 2 (thumbnail content claim)] The paper states that each thumbnail 'contains only one object and it is in the centre of the thumbnail,' but the detection-and-crop step is not evaluated. No detection precision/recall numbers, no audit of false positives, multiple-bird crops, or partially cropped birds are provided. Because the bird-centered property is advertised as making classification easier and as distinguishing DIB-10K from datasets like ImageNet and COCO, this property should be measured on a sample, especially given that the detection tool itself could introduce species-correlated errors (e.g., rare species with few training examples).
minor comments (4)
- [Throughout] There are several typos and wording issues: 'extinguished' should be 'extinct' (Section 2), 'status quo' should be 'state of the art' (Section 2), and 'visual the distribution' should be 'visualize the distribution' (Figure 1 caption).
- [References] Reference [4] has the author order reversed and inconsistent formatting; it should be presented as Gill, F., Donsker, D., & Rasmussen, P. (Eds.), IOC World Bird List (v10.1), 2020. The reference list should also be checked for formatting consistency across entries.
- [Figures] Figure 2 is cited in the text ('as Fig 2 shows') but appears only at the end of the paper and has no descriptive caption beyond 'Example of five bird images from each of random categories.' A caption explaining what the figure demonstrates and the species shown would improve readability.
- [Table 1] Table 1 uses approximate values (4,800,000 and 10,000) that conflict with the exact counts in the text. The table should either state exact numbers or clearly label all entries as rounded, and the conflict should be resolved.
Circularity Check
No circularity: DIB-10K is a dataset resource with no fitted parameters or predictive derivation; label provenance is external and any noise is a validation concern, not a circular reduction.
full rationale
This paper is a dataset description, not a derivation or prediction. It makes no fitted-parameter claims, no uniqueness claims, and no equations whose outputs reduce to inputs. The central factual claims are the dataset size (4,876,536 thumbnails), the category count (10,922 species), and the assertion that the categories follow the IOC 10.1 taxonomy. The IOC list is an external taxonomic authority cited from the published IOC World Bird List, and the taxonomy is not defined in terms of the dataset itself. The collection pipeline uses search-engine queries with scientific/common names as external keys and filters by image titles; this could produce label noise, and the paper explicitly acknowledges the mechanism without claiming visual verification. Label noise is a correctness/validation gap, not circularity: no quantity in the paper is constructed from its own output. The references to TensorFlow, Faster R-CNN, ImageNet, COCO, and PASCAL VOC are standard tools or prior datasets used as comparison points, not load-bearing self-citations. No argument in the paper relies on the authors' prior work to justify the dataset's assumptions. The only self-referential aspect is that the dataset's utility is assumed, which is not a technical circularity. Therefore no circular step is present, and the honest finding is a score of 0.
Assumptions & free parameters
assumptions (3)
- domain assumption The IOC World Bird List v10.1 provides a complete and correct taxonomy of all bird species.
- ad hoc to paper Image search engine results, filtered by title text, yield correct species labels.
- domain assumption The TensorFlow Faster R-CNN bird detector correctly detects and crops individual birds.
Cite this review
Pith. "Pith review of The DongNiao International Birds 10000 Dataset." pith.science (2026). https://pith.science/paper/COCDOJL6
@misc{pith2026201006454,
author = {Pith},
title = {Pith review of: The DongNiao International Birds 10000 Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/COCDOJL6}},
note = {Machine review of arXiv:2010.06454}
}
read the original abstract
DongNiao International Birds 10000 (DIB-10K) is a challenging image dataset which has more than 10 thousand different types of birds. It was created to enable the study of machine learning and also ornithology research. DIB-10K does not own the copyright of these images. It only provides thumbnails of images, in a way similar to ImageNet.
Forward citations
Cited by 1 Pith paper
-
Quantifying Avian Morphological Evolution through Deep Representation Learning
Features extracted from a 10,000-species bird classifier form a visual morphospace that shows a rapid early burst of morphological disparity after the K-Pg extinction.
Reference graph
Works this paper leans on
-
[1]
Tensorflow: A system for large-scale machine learning
Mart \' n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large-scale machine learning. In 12th \ USENIX \ Symposium on Operating Systems Design and Implementation ( \ OSDI \ 16) , pages 265--283, 2016
work page 2016
-
[2]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database . In CVPR09 , 2009
work page 2009
-
[3]
Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision , 88:303--338, 2009
work page 2009
-
[4]
D Donsker Gill F and P Rasmussen (Eds). IOC World Bird List (v10.1). 2020
work page 2020
-
[5]
Microsoft COCO: Common Objects in Context
Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. ArXiv , abs/1405.0312, 2014
work page Pith review arXiv 2014
-
[6]
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 39:1137--1149, 2015
work page 2015
-
[7]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge . International Journal of Computer Vision (IJCV) , 115(3):211--252, 2015
work page 2015
-
[8]
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge J. Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010
work page 2010
Show all 9 references
-
[9]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.