Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Self-supervised pretraining on 2.5 million wheat images yields a crop-specific vision backbone that beats a general-purpose foundation model on all ten field tasks evaluated, with the biggest gains in pixel-level organ segmentation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Training a vision transformer on 2.5 million wheat images outperforms general-domain backbones across ten crop vision tasks.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Large wheat dataset and model, but missing train/test de-duplication analysis undercuts the 'consistent outperformance' claim. the 3 major comments →

arxiv 2509.06907 v1 pith:JFW3BFOH submitted 2025-09-08 cs.CV

FoMo4Wheat: Toward reliable crop vision foundation models with globally curated data

Bing Han , Chen Zhu , Dong Han , Rui Yu , Songliang Cao , Jianhui Wu , Scott Chapman , Zijian Wang
show 17 more authors
Bangyou Zheng Wei Guo Marie Weiss Benoit de Solan Andreas Hund Lukas Roth Kirchgessner Norbert Andrea Visioni Yufeng Ge Wenjuan Li Alexis Comar Dong Jiang Dejun Han Fred Baret Yanfeng Ding Hao Lu Shouyang Liu
This is my paper
classification cs.CV
keywords wheat phenotypingvision foundation modelself-supervised pretrainingagricultural computer visionsemantic segmentationmasked image modelingcontrastive learningcross-crop transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a vision foundation model pretrained exclusively on wheat imagery—rather than on general-purpose image collections—produces representations that transfer across agricultural vision tasks and even to other crops. It introduces ImAg4Wheat, a 2.5-million-image global wheat dataset spanning ten countries, and FoMo4Wheat, a family of ViT backbones pretrained on it with masked-image modeling and contrastive self-supervision. Across ten downstream tasks, FoMo4Wheat consistently beats matching DINOv2-backed models when the backbone is frozen and only lightweight adapters and task heads are trained. The largest gains are in organ-level semantic segmentation, including a roughly six-point mIoU improvement on wheat organ segmentation and double-digit gains on multi-crop and crop-weed segmentation. If correct, this means domain-specific data scale and curation, not just architecture, are what unlock reliable field monitoring and a path toward universal crop foundation models.

Core claim

The central discovery is that domain-matched pretraining is the deciding factor: a vision backbone trained self-supervised on 2.5 million wheat images learns features that a general-purpose backbone does not, and those features transfer across tasks, platforms, and even species. FoMo4Wheat Giant improves wheat organ-segmentation mIoU to 76.49, about six points above the 70.57 of the same model built on a DINOv2 backbone, and exceeds the previously reported best of 73.66; in spike detection it surpasses the best reported AP by 1.8 points. The pattern is consistent across ten tasks, with small wins in counting and large wins in per-pixel segmentation, and it persists under data-scarce training

What carries the argument

The load-bearing mechanism is a two-stage pipeline. First, FoMo4Wheat's ViT backbones are pretrained on ImAg4Wheat with a DINOv2-style teacher-student objective that combines masked-image reconstruction at the patch level with contrastive view alignment at the image level; the Giant is initialized from DINOv2 weights and the Large and Base variants are distilled from the Giant. Second, the pretrained backbone is frozen and only lightweight adapters and task heads are trained for each downstream task. Because the same heads, adapters, and training protocols are used for FoMo4Wheat and the DINOv2 baselines, the measured differences are attributed to the representations learned from wheat-domai

Load-bearing premise

The load-bearing premise is that pretraining and test images are disjoint: the paper integrates the Swiss FIP 1.0 set into its pretraining corpus while the GWFSS organ-segmentation tests include Swiss images, and its own UAV and leaf benchmarks come from the same Chinese sites used for pretraining, so without de-duplication the reported gains could be inflated by overlap.

What would settle it

Run near-duplicate retrieval between ImAg4Wheat and the test partitions of GWFSS, the UAV spike sets, the wheat and rice leaf sets, and RiceSEG—for example, via perceptual hashing or cosine similarity of pretrained embeddings at a high threshold. If a nontrivial fraction of test images match pretraining images, the reported advantages are suspect; if no matches are found, the domain-pretraining explanation is directly supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Replacing a frozen general-purpose backbone with FoMo4Wheat in existing agricultural pipelines should improve accuracy without changing downstream heads; the paper demonstrates this with identical heads for classification, detection, counting, and segmentation.
  • Growth-stage and disease classifiers reach comparable or better accuracy with roughly 25–30% of annotated training samples, so field-monitoring systems can be deployed with much less labeled data.
  • Models trained on ground-level wheat images transfer to UAV imagery at coarser resolutions, supporting low-cost aerial monitoring without new annotation campaigns.
  • A wheat-only pretrained backbone improves rice leaf counting, rice organ segmentation, multi-crop segmentation, and crop-weed segmentation, pointing toward a single crop foundation model shared across species.
  • The distilled Large and Base variants preserve most of the gains, so the approach is not tied to billion-parameter models and can be adapted to lighter, more deployable settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the gains are genuinely caused by domain-matched pretraining rather than data overlap, the same recipe should transfer to other major crops whenever comparable global datasets exist; the paper's own FoMo4Crop trajectory depends on this assumption.
  • A concrete way to separate the two explanations is near-duplicate retrieval between ImAg4Wheat and each downstream test set; if no near-duplicates are found, the 30-site diversity explanation is strongly supported.
  • The long tail in crop-weed segmentation suggests the hardest cases are rare weed species with little visual mass; a wheat foundation model may help most where background and canopy structure dominate, and help least on extremely rare classes.
  • Because the backbone is frozen rather than fine-tuned, its public release can serve as a drop-in encoder; measuring gains on data collected after 2024 gives an independent, naturally disjoint evaluation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ImAg4Wheat, a 2.5M-image wheat pretraining corpus aggregated from 30 international sites, and FoMo4Wheat, a family of ViT backbones (Base/Large/Giant, 80M/300M/1.1B) pretrained on it with DINOv2-style objectives. After freezing the backbone, the authors attach task-specific adapters and heads and evaluate ten in-field vision tasks: wheat growth-stage and disease classification, ground-/UAV spike detection, wheat/rice leaf counting, wheat/rice organ segmentation, multi-crop segmentation, and crop-weed segmentation. The central claim is that FoMo4Wheat consistently outperforms the DINOv2-based state of the art with identical task heads, and also exceeds published best results on spike detection (+1.8 AP) and wheat organ segmentation (+2.83 mIoU), while transferring to rice and other crops. The comparison protocol is clean in design, but the paper does not document de-duplication between pretraining and test splits, which is the key risk to the headline claim.

Significance. If the results hold, this is a substantial step for agricultural vision: a large, globally curated wheat dataset; a family of domain-specific foundation models with parameter-efficient adaptation; and a broad benchmark covering classification, detection, counting, and segmentation across wheat, rice, and weed scenarios. The head-to-head comparison with DINOv2 under identical heads is a strength, and the cross-crop/weed transfer tasks are genuine out-of-distribution tests. The release of models and data is a useful community resource. However, the missing pretraining/test de-duplication and the absence of variance estimates for several small-margin results currently limit how strongly the 'consistent outperformance' claim can be asserted.

major comments (3)
  1. [Methods/ImAg4Wheat; Supplementary Table 5; Supplementary Notes 5.3, 6.2, 7.2] The benchmark protocol assumes downstream test sets are disjoint from the 2.5M-image pretraining corpus, but the paper never reports de-duplication. The Discussion states 'A carefully selected subset of FIP 1.0 has been integrated into ImAg4Wheat', and Supplementary Table 5 includes the Swiss Eschikon FIP site. Supplementary Note 7.2 says the GWFSS wheat-organ test set was collected from institutions 'across Switzerland, Belgium, UK, China...', so Swiss test images may overlap with pretraining images. Similarly, the UAV spike test (Note 5.3: Yangling, Nanchong) and leaf-counting test (Note 6.2: Xuzhou, Baima, Henan, Jurong) come from sites that also supply pretraining images (Yangling, Baima, Jurong, Xuzhou in Table 5). Exact or cropped near-duplicates would inflate FoMo4Wheat's gains, especially where gains are small (0.6 AP spike detection; 0.01-0.03 R² counting). Please provide a spli
  2. [Supplementary Tables 9-10; Results] Several headline comparisons are within the range of typical head-training noise, e.g., ground spike detection AP 36.0 vs 36.6 for Giant, UAV 6m AP 34.2 vs 35.4, and leaf-counting R² 0.88 vs 0.89 (Giant) or 0.86 vs 0.86 (Large). Unlike the classification results (Supplementary Table 8), the detection and counting results are not accompanied by variance over seeds or statistical tests. The abstract's 'consistently outperform' therefore rests, for several tasks, on margins that are not shown to be reproducible. Please report mean±std over at least three seeds (or paired error bars over test images) for detection/counting, and state explicitly which comparisons are significant.
  3. [Discussion, third paragraph; Fig. 2] The claim that organ segmentation emerges 'even without explicit task-specific training' is not supported by any quantitative experiment. Figure 2 shows PCA projections of backbone features, not segmentation outputs. A quantitative zero-shot or oracle segmentation evaluation is required before making an 'emergent capability' claim. At a minimum, the claim should be removed or explicitly qualified as a qualitative visualization rather than an emergent segmentation result.
minor comments (4)
  1. [Supplementary Table 7; Supplementary Note 7.3] The Rice Organ row lists Test = 216, while the text says 616 testing images; 2462 + 616 = 3078 matches the total. Please correct the table.
  2. [Abstract; Supplementary Fig. 5] The main text says 30 global sites in 10 countries, while the supplementary figure caption says 29 regions across 9 countries. Clarify the counting convention.
  3. [Supplementary Table 7] The UAV spike rows are formatted as '100 / 100' and '120 / 120', which is ambiguous. Use explicit image-number, training, and test columns and mark these as test-only sets.
  4. [Results; Supplementary Tables 9 and 11] The term 'SOTA' is used both for DINOv2-with-identical-heads and for published best results (GWHD, GWFSS). Add a clear caption-level distinction so readers can separate the head-to-head comparison from the literature comparison.

Circularity Check

0 steps flagged

No significant circularity: the central claim is an empirical benchmark against external tasks and a fixed DINOv2 baseline; the only caveat is a data-provenance/overlap risk, not a definitional reduction.

full rationale

The paper's central claim is empirical: DINOv2-initialized ViTs are pretrained on ImAg4Wheat, then the backbone is frozen and task-specific adapters/heads are trained, and downstream accuracy is compared with the same protocol using DINOv2 backbones. No predicted metric is defined as a fitted parameter, no uniqueness theorem is invoked, and no result is derived from an equation that reduces to its own inputs. The benchmarks use external datasets (GWHD, GWFSS, VegAnn, CropAndWeed, WGSP, public disease sets) and identical task heads, so the comparisons are not constructed to match a fitted quantity. The closest caveat is data provenance rather than circularity: ImAg4Wheat integrates FIP 1.0, a Swiss field-image dataset with overlapping authorship, while the GWFSS organ-segmentation test set includes Swiss images, and self-collected UAV/leaf test sets come from the same Chinese sites (Yangling, Baima, Jurong, Xuzhou) listed in Supplementary Table 5 as pretraining sources. If exact or cropped near-duplicates of test images were in pretraining, the reported gains could be inflated; the paper does not describe de-duplication. However, this is a leakage/validity risk, not a definitional circularity: no step in the paper exhibits a quantity that is equal to its input by construction, and no load-bearing inference is authorized solely by a self-citation. The FIP and GWFSS citations are used as data and benchmark sources, not as authorities that force the conclusion. Therefore the circularity score is 1: essentially self-contained and non-circular, with a minor self-citation/data-provenance caveat.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper's claims rest on two premises: the DINOv2 pretraining objectives transfer to crop imagery, and the evaluation benchmarks are uncontaminated. The latter is questionable given potential overlap between pretraining and downstream datasets.

axioms (2)
  • domain assumption DINOv2 self-supervised objectives (MIM + contrastive learning) are suitable for learning crop-specific representations
    The pretraining pipeline is inherited from DINOv2 (Ref 12) without ablation showing these objectives are optimal for dense, repetitive crop structures; the paper only adjusts the Sinkhorn-Knopp loss weight.
  • ad hoc to paper Downstream benchmark test sets are disjoint from the ImAg4Wheat pretraining set
    The paper does not describe any de-duplication. FIP 1.0 is integrated into ImAg4Wheat while GWFSS test images include Swiss sites; self-collected UAV and leaf datasets come from the same Chinese sites as pretraining.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of FoMo4Wheat: Toward reliable crop vision foundation models with globally curated data." pith.science (2026). https://pith.science/paper/JFW3BFOH

@misc{pith2026250906907,
  author       = {Pith},
  title        = {Pith review of: FoMo4Wheat: Toward reliable crop vision foundation models with globally curated data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JFW3BFOH}},
  note         = {Machine review of arXiv:2509.06907}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Vision-driven field monitoring is central to digital agriculture, yet models built on general-domain pretrained backbones often fail to generalize across tasks, owing to the interaction of fine, variable canopy structures with fluctuating field conditions. We present FoMo4Wheat, one of the first crop-domain vision foundation model pretrained with self-supervision on ImAg4Wheat, the largest and most diverse wheat image dataset to date (2.5 million high-resolution images collected over a decade at 30 global sites, spanning >2,000 genotypes and >500 environmental conditions). This wheat-specific pretraining yields representations that are robust for wheat and transferable to other crops and weeds. Across ten in-field vision tasks at canopy and organ levels, FoMo4Wheat models consistently outperform state-of-the-art models pretrained on general-domain dataset. These results demonstrate the value of crop-specific foundation models for reliable in-field perception and chart a path toward a universal crop foundation model with cross-species and cross-task capabilities. FoMo4Wheat models and the ImAg4Wheat dataset are publicly available online: https://github.com/PheniX-Lab/FoMo4Wheat and https://huggingface.co/PheniX-Lab/FoMo4Wheat. The demonstration website is: https://fomo4wheat.phenix-lab.com/.

Figures

Figures reproduced from arXiv: 2509.06907 by Alexis Comar, Andreas Hund, Andrea Visioni, Bangyou Zheng, Benoit de Solan, Bing Han, Chen Zhu, Dejun Han, Dong Han, Dong Jiang, Fred Baret, Hao Lu, Jianhui Wu, Kirchgessner Norbert, Lukas Roth, Marie Weiss, Rui Yu, Scott Chapman, Shouyang Liu, Songliang Cao, Wei Guo, Wenjuan Li, Yanfeng Ding, Yufeng Ge, Zijian Wang.

Figure 1
Figure 1. Figure 1: Overview of ImAg4Wheat dataset and FoMo4Wheat model. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of features extracted by FoMo4Wheat and DINOv2. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance comparison of FoMo4Wheat and SOTA models on wheat canopy level classification tasks. Model performance for classification is evaluated using mean average precision (mAP). a, Performance across varying training sample sizes for the growth stage classification task, with error bars showing variability across five random samplings for the training. b, 2D t-SNE visualizations of feature embeddings … view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison of FoMo4Wheat and SOTA models for wheat organ level tasks. Model performance on wheat spike detection is evaluated using Average Precision (AP) metrics. a, Performance on the ground-level dataset (GSD 0.4 mm). b, Performance on the 6 m-altitude UAV data (GSD 0.6 mm). c, Performance on the 10 m-altitude UAV data (GSD 1.2 mm). Detection results are compared across two examples per reso… view at source ↗
Figure 5
Figure 5. Figure 5: Performance comparison of FoMo4Wheat and SOTA models for other crops with weeds tasks. a, Rice leaf counting performance, evaluated using the coefficient of determination (R², expressed as percentage) metrics. Four example cases are used to compare the counting results, with ground truth, predict and omission. b-d, Model performance on segmentation tasks is evaluated using Mean Intersection over Union (mIo… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SPROUT: A Scalable Diffusion Foundation Model for Agricultural Vision

    cs.CV 2026-03 conditional novelty 5.0

    SPROUT, a pixel-space diffusion transformer pre-trained on 2.6M unlabeled agricultural images with effective-rank timestep selection, outperforms web-pretrained foundation models on dense plant phenotyping tasks.

  2. 3D Reconstruction and Knowledge Distillation to Improve Multi-View Image Models to Explore Spike Volume Estimation in Wheat

    cs.CV 2026-05 unverdicted novelty 4.0

    Knowledge distillation from a rigid-invariant 3D point cloud network into a regulated multi-view Transformer yields lower-error, faster wheat spike volume estimates from 2D images.

Reference graph

Works this paper leans on

77 extracted references · 52 canonical work pages · cited by 2 Pith papers · 4 internal anchors

  1. [1]

    & Antle, J

    Basso, B. & Antle, J. Digital agriculture to design sustainable agricultural systems. Nat. Sustain. 3, 254–256 (2020)

    work page 2020

  2. [2]

    El Jarroudi, M. et al. Leveraging edge artificial intelligence for sustainable agriculture. Nat. Sustain. 7, 846–854 (2024)

    work page 2024

  3. [3]

    Meemken, E.-M. et al. Digital innovations for monitoring sustainability in food systems. Nat. Food 5, 656–660 (2024)

    work page 2024

  4. [4]

    Bian, L. et al. A broadband hyperspectral image sensor with high spatio -temporal resolution. Nature 635, 73–81 (2024)

    work page 2024

  5. [5]

    C., Knoch, D., Junker, A

    Heuermann, M. C., Knoch, D., Junker, A. & Altmann, T. Natural plant growth and development achieved in the IPK PhenoSphere by dynamic environment simulation. Nat. Commun. 14, 5783 (2023)

    work page 2023

  6. [6]

    Wang, M. et al. Variation in TaSPL6 -D confers salinity tolerance in bread wheat by activating TaHKT1;5-D while preserving yield-related traits. Nat. Genet. 56, 1257–1269 (2024)

    work page 2024

  7. [7]

    Welcker, C. et al. Physiological adaptive traits are a potential allele reservoir for maize genetic progress under challenging conditions. Nat. Commun. 13, 3225 (2022)

    work page 2022

  8. [9]

    M., Ludwig, E., Gutierrez, J

    Murphy, K. M., Ludwig, E., Gutierrez, J. & Gehan, M. A. Deep Learning in Image -Based Plant Phenotyping. Annu. Rev. Plant Biol. 75, 771–795 (2024)

    work page 2024

  9. [10]

    Bommasani, R. et al. On the Opportunities and Risks of Foundation Models. Preprint at https://doi.org/10.48550/arXiv.2108.07258 (2022). 26

    Pith/arXiv arXiv doi:10.48550/arxiv.2108.07258 2022

  10. [11]

    Deng, J. et al. ImageNet: A large -scale hierarchical image database. in 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255 (2009). doi:10.1109/CVPR.2009.5206848

    arXiv 2009

  11. [12]

    Oquab, M. et al. DINOv2: Learning Robust Visual Features without Supervision. Preprint at https://doi.org/10.48550/arXiv.2304.07193 (2024)

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.07193 2024

  12. [13]

    Dong, X. et al. CLIP Itself is a Strong Fine -tuner: Achieving 85.7% and 88.0% Top -1 Accuracy with ViT-B and ViT-L on ImageNet. Preprint at https://doi.org/10.48550/arXiv.2212.06138 (2022)

    Pith/arXiv arXiv doi:10.48550/arxiv.2212.06138 2022

  13. [14]

    Russakovsky, O. et al. ImageNet Large Scale Visual Recognition Challenge. Preprint at https://doi.org/10.48550/arXiv.1409.0575 (2015)

    Pith/arXiv arXiv doi:10.48550/arxiv.1409.0575 2015

  14. [15]

    Jiang, Y . & Li, C. Convolutional Neural Networks for Image -Based High -Throughput Plant Phenotyping: A Review. Plant Phenomics 2020, (2020)

    work page 2020

  15. [16]

    & Hund, A

    Walter, A., Liebisch, F. & Hund, A. Plant phenotyping: from bean weighing to image analysis. Plant Methods 11, 14 (2015)

    work page 2015

  16. [17]

    Yu, Y . et al. Crop/Plant Modeling Supports Plant Breeding: I. Optimization of Environmental Factors in Accelerating Crop Growth and Development for Speed Breeding. Plant Phenomics 5, 0099 (2023)

    work page 2023

  17. [18]

    Meraj, T. et al. Computer vision-based plants phenotyping: A comprehensive survey. iScience 27, (2024)

    work page 2024

  18. [19]

    Singh, A. et al. Challenges and Opportunities in Machine -Augmented Plant Stress Phenotyping. Trends Plant Sci. 26, 53–69 (2021)

    work page 2021

  19. [20]

    Hao, M. et al. Large-scale foundation model on single-cell transcriptomics. Nat. Methods 21, 1481– 1491 (2024)

    work page 2024

  20. [21]

    Xiang, J. et al. A vision –language foundation model for precision oncology. Nature 1–10 (2025) 27 doi:10.1038/s41586-024-08378-w

    doi:10.1038/s41586-024-08378-w 2025

  21. [22]

    Zhao, T. et al. A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nat. Methods 22, 166–176 (2025)

    work page 2025

  22. [23]

    Hong, D. et al. SpectralGPT: Spectral Remote Sensing Foundation Model. IEEE Trans. Pattern Anal. Mach. Intell. 46, 5227–5244 (2024)

    work page 2024

  23. [24]

    Jakubik, J. et al. TerraMind: Large-Scale Generative Multimodality for Earth Observation. Preprint at https://doi.org/10.48550/arXiv.2504.11171 (2025)

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.11171 2025

  24. [25]

    Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023)

    work page 2023

  25. [26]

    & Manso, G

    Thompson, N., Greenewald, K., Lee, K. & Manso, G. F. The Computational Limits of Deep Learning. in Ninth Computing within Limits 2023 (LIMITS, Virtual). doi:10.21428/bf6fb269.1f033948

    doi:10.21428/bf6fb269.1f033948 2023

  26. [27]

    Li, J. et al. Foundation models in smart agriculture: Basics, opportunities, and challenges. Comput. Electron. Agric. 222, 109032 (2024)

    work page 2024

  27. [28]

    Shen, Y . et al. WeedNet: A Foundation Model -Based Global -to-Local AI Approach for Real -Time Weed Species Identification and Classification. Preprint at https://doi.org/10.48550/arXiv.2505.18930 (2025)

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.18930 2025

  28. [29]

    Xu, Y . et al. Smart breeding driven by big data, artificial intelligence, and integrated genomic - enviromic prediction. Mol. Plant 15, 1664–1695 (2022)

    work page 2022

  29. [30]

    iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species

    Jain, N., Joshi, A. & Earles, M. iNatAg: Multi-Class Classification Models Enabled by a Large -Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species. Preprint at https://doi.org/10.48550/arXiv.2503.20068 (2025). 28

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2503.20068 2025

  30. [31]

    Mehrabi, Z. et al. The global divide in data-driven farming. Nat. Sustain. 4, 154–160 (2021)

    work page 2021

  31. [32]

    Dosovitskiy, A. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Preprint at https://doi.org/10.48550/arXiv.2010.11929 (2021)

    Pith/arXiv arXiv doi:10.48550/arxiv.2010.11929 2010

  32. [33]

    Zhou, J. et al. iBOT: Image BERT Pre -Training with Online Tokenizer. Preprint at https://doi.org/10.48550/arXiv.2111.07832 (2022)

    Pith/arXiv arXiv doi:10.48550/arxiv.2111.07832 2022

  33. [34]

    & Hinton, G

    Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A Simple Framework for Contrastive Learning of Visual Representations. Preprint at https://doi.org/10.48550/arXiv.2002.05709 (2020)

    Pith/arXiv arXiv doi:10.48550/arxiv.2002.05709 2002

  34. [35]

    Gao, Y . et al. Bridging real and simulated data for cross-spatial- resolution vegetation segmentation with application to rice crops. ISPRS J. Photogramm. Remote Sens. 218, 133–150 (2024)

    work page 2024

  35. [36]

    Roth, L. et al. The FIP 1.0 Data Set: Highly Resolved Annotated Image Time Series of 4,000 Wheat Plots Grown in Six Years. 2024.10.04.616624 Preprint at https://doi.org/10.1101/2024.10.04.616624 (2025)

    work page doi:10.1101/2024.10.04.616624 2024

  36. [38]

    Wang, Z. et al. The Global Wheat Full Semantic Organ Segmentation (GWFSS) Dataset. Plant Phenomics 100084 (2025) doi:10.1016/j.plaphe.2025.100084

    arXiv 2025

  37. [39]

    J., Ghosh, T., Sheikhi, F

    Al Nahian, M. J., Ghosh, T., Sheikhi, F. & Maleki, F. Agri-FM+: A Self-Supervised Foundation Model for Agricultural Vision. in 5511–5523 (2025)

    work page 2025

  38. [40]

    Wei, J. et al. Emergent Abilities of Large Language Models. Preprint at https://doi.org/10.48550/arXiv.2206.07682 (2022)

    Pith/arXiv arXiv doi:10.48550/arxiv.2206.07682 2022

  39. [41]

    Zhu, C. et al. Genotype × environment × management analysis to define allometric rules between 29 leaves and stems in wheat. J. Exp. Bot. 75, 6388–6404 (2024)

    work page 2024

  40. [42]

    Upadhyay, A. et al. Advances in ground robotic technologies for site-specific weed management in precision agriculture: A review. Comput. Electron. Agric. 225, 109363 (2024)

    work page 2024

  41. [43]

    Vaswani, A. et al. Attention is All you Need. in Advances in Neural Information Processing Systems vol. 30 (Curran Associates, Inc., 2017)

    work page 2017

  42. [44]

    -W., Lee, K

    Devlin, J., Chang, M. -W., Lee, K. & Toutanova, K. BERT: Pre -training of Deep Bidirectional Transformers for Language Understanding. in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (eds Burstein, J., Doran, C. & Solorio, T.)...

    doi:10.18653/v1/n19-1423 2019

  43. [45]

    Simplifying Transformer Blocks

    He, B. & Hofmann, T. Simplifying Transformer Blocks. Preprint at https://doi.org/10.48550/arXiv.2311.01906 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2311.01906 2024

  44. [46]

    L., Kiros, J

    Ba, J. L., Kiros, J. R. & Hinton, G. E. Layer Normalization. Preprint at https://doi.org/10.48550/arXiv.1607.06450 (2016)

    Pith/arXiv arXiv doi:10.48550/arxiv.1607.06450 2016

  45. [47]

    GLU Variants Improve Transformer

    Shazeer, N. GLU Variants Improve Transformer. Preprint at https://doi.org/10.48550/arXiv.2002.05202 (2020)

    Pith/arXiv arXiv doi:10.48550/arxiv.2002.05202 2002

  46. [49]

    Li, H. et al. CPSAA: Accelerating Sparse Attention using Crossbar -based Processing -In-Memory Architecture. Preprint at https://doi.org/10.48550/arXiv.2210.06696 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2210.06696 2023

  47. [50]

    E., Hinton, G

    Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. 30 Nature 323, 533–536 (1986)

    work page 1986

  48. [51]

    & Girshick, R

    He, K., Gkioxari, G., Dollár, P . & Girshick, R. Mask R-CNN. in 2017 IEEE International Conference on Computer Vision (ICCV) 2980–2988 (2017). doi:10.1109/ICCV.2017.322

    doi:10.1109/iccv.2017.322 2017

  49. [54]

    & Dean, J

    Hinton, G., Vinyals, O. & Dean, J. Distilling the Knowledge in a Neural Network. Preprint at https://doi.org/10.48550/arXiv.1503.02531 (2015)

    Pith/arXiv arXiv doi:10.48550/arxiv.1503.02531 2015

  50. [55]

    Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114, 3521–3526 (2017). 31 Supplementary Notes

    work page 2017

  51. [56]

    To train the FoMo4Wheat Giant model, we compared two weight initialization strategies: random initialization and initialization by DINOv2 pretrained weights

    Training from scratch or with DINOv2 pretrained weights for the Giant model Weight initialization significantly affects the performance of the foundation model 1. To train the FoMo4Wheat Giant model, we compared two weight initialization strategies: random initialization and initialization by DINOv2 pretrained weights. The DINOv2 pre-trained weights may o...

    work page

  52. [57]

    Similarly, the two segmentation tasks (wheat organ segmentation and crop multi- class segmentation) were repurposed as benchmarks

    Direct training or distillation for Large/Base variants Once the FoMo4Wheat Giant model was trained, there are two possible strategies to train the subsequent Large and Base variants: i) direct training from scratch on the whole dataset or ii) distillation from the Giant model. Similarly, the two segmentation tasks (wheat organ segmentation and crop multi...

    work page

  53. [58]

    Implementation Details Our experiments were conducted on an HPC cluster equipped with NVIDIA Ampere GPU infrastructure, where intra-server GPU communication utilizes NVLink and high -speed interconnects to establish a multi-node deep learning environment optimized for efficient training. We employ a mixed -precision training strategy to minimize time cons...

    work page

  54. [59]

    We first evaluate on image-level classification tasks

    Classification Tasks Classification Task Head. We first evaluate on image-level classification tasks. Given the FoMo4Wheat model backbone, we follow the linear probing protocol where a linear classifier is used as the task head while the backbone parameters are kept frozen. We evaluate on two wheat classification tasks. To highlight the performance differ...

    work page 2018

  55. [60]

    Here we evaluate the adaptability of the backbone on the detection tasks where a model should output bounding boxes to indicate what objects are and where they are

    Detection Tasks Detection Task Head. Here we evaluate the adaptability of the backbone on the detection tasks where a model should output bounding boxes to indicate what objects are and where they are. We adopt the Mask R-CNN head8 for detection, and the backbone parameters were frozen. The constructed baseline is identical the ViT-Adapter9. According to ...

    work page 2016

  56. [61]

    Different from detection, counting does not need to predict where objects are, but how many they are

    Counting Tasks Counting Task Head. Different from detection, counting does not need to predict where objects are, but how many they are. We adopt the Point quEry Transformer (PET)11 as our baseline. The PET model employs a point -based regression paradigm to solve dense object counting. PET redefines dense counting as a decomposable point -query process, ...

    work page 2020

  57. [62]

    Segmentation requires per-pixel label prediction

    Segmentation Tasks Segmentation Task Head . Segmentation requires per-pixel label prediction. We adopt the Mask2Former head13 for segmentation (Supplementary Figure 4 ). The Mask2Former head includes a pixel decoder used to generate multi -scale features and a transformer decoder used to predict object class and object mask. The mask head and the adapter ...

    work page 2020

  58. [64]

    & Bengio, Y

    Glorot, X. & Bengio, Y . Understanding the difficulty of training deep feedforward neural networks. in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics 249–256 (JMLR Workshop and Conference Proceedings, 2010)

    work page 2010

  59. [65]

    Shen, L. et al. GSP-AI: An AI -Powered Platform for Identifying Key Growth Stages and the Vegetative-to-Reproductive Transition in Wheat Using Trilateral Drone Imagery and Meteorological Data. Plant Phenomics 6, 0255 (2024)

    work page 2024

  60. [66]

    Long, M., Hartley, M., Morris, R. J. & Brown, J. K. M. Classification of wheat diseases using deep learning networks with field and glasshouse images. Plant Pathol. 72, 536–547 (2023)

    work page 2023

  61. [67]

    Genaev, M. A. et al. Image-based wheat fungi diseases identification by deep learning. Plants 10, 1500 (2021)

    work page 2021

  62. [68]

    Automated Wheat Disease Detection using a ROS-based Autonomous Guided UAV

    Safarijalal, B., Alborzi, Y . & Najafi, E. Automated Wheat Disease Detection using a ROS -based Autonomous Guided UAV. Preprint at http://arxiv.org/abs/2206.15042 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  63. [69]

    Mao, R. et al. DAE-Mask: a novel deep-learning-based automatic detection model for in-field wheat diseases. Precis. Agric. 25, 785–810 (2024)

    work page 2024

  64. [70]

    M., Singh, A

    Goyal, L., Sharma, C. M., Singh, A. & Singh, P . K. Leaf and spike wheat disease detection & classification using an improved deep convolutional architecture. Inform. Med. Unlocked 25, 100642 (2021)

    work page 2021

  65. [71]

    & Girshick, R

    He, K., Gkioxari, G., Dollár, P . & Girshick, R. Mask R -CNN. Preprint at https://doi.org/10.48550/arXiv.1703.06870 (2018)

    Pith/arXiv arXiv doi:10.48550/arxiv.1703.06870 2018

  66. [72]

    Chen, Z. et al. Vision Transformer Adapter for Dense Predictions. Preprint at https://doi.org/10.48550/arXiv.2205.08534 (2023). 59

    Pith/arXiv arXiv doi:10.48550/arxiv.2205.08534 2023

  67. [73]

    David, E. et al. Global Wheat Head Detection 2021: An Improved Dataset for Benchmarking Wheat Head Detection Methods. Plant Phenomics 2021, 2021/9846158 (2021)

    work page 2021

  68. [74]

    & Liu, T

    Liu, C., Lu, H., Cao, Z. & Liu, T. Point-Query Quadtree for Crowd Counting, Localization, and More. in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) 1676–1685 (IEEE, Paris, France, 2023). doi:10.1109/iccv51070.2023.00161

    arXiv 2023

  69. [75]

    Li, Y . et al. Self-Supervised Plant Phenotyping by Combining Domain Adaptation with 3D Plant Model Simulations: Application to Wheat Leaf Counting at Seedling Stage. Plant Phenomics 5, 0041 (2023)

    work page 2023

  70. [76]

    G., Kirillov, A

    Cheng, B., Misra, I., Schwing, A. G., Kirillov, A. & Girdhar, R. Masked -attention Mask Transformer for Universal Image Segmentation. in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 1280–1289 (IEEE, New Orleans, LA, USA, 2022). doi:10.1109/cvpr52688.2022.00135

    arXiv 2022

  71. [77]

    Wang, Z. et al. The Global Wheat Full Semantic Organ Segmentation (GWFSS) Dataset. 2025.03.18.642594 Preprint at https://doi.org/10.1101/2025.03.18.642594 (2025)

    work page doi:10.1101/2025.03.18.642594 2025

  72. [78]

    Zhou, J. et al. Global Rice Multiclass Segmentation Dataset (RiceSEG): Comprehensive and Diverse High-Resolution RGB -Annotated Images for the Development and Benchmarking of Rice Segmentation Algorithms. Plant Phenomics 100099 (2025) doi:10.1016/j.plaphe.2025.100099

    arXiv 2025

  73. [79]

    Madec, S. et al. VegAnn, Vegetation Annotation of multi-crop RGB images acquired under diverse conditions for segmentation. Sci. Data 10, 302 (2023)

    work page 2023

  74. [80]

    & Widhalm, V

    Steininger, D., Trondl, A., Croonen, G., Simon, J. & Widhalm, V. The CropAndWeed Dataset: a Multi- Modal Learning Approach for Efficient Crop and Weed Manipulation. in 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 3718–3727 (IEEE, Waikoloa, HI, USA, 2023). 60 doi:10.1109/WACV56688.2023.00372

    arXiv 2023

  75. [81]

    https://www.kaggle.com/datasets/olyadgetch/wheat-leaf-dataset

    Wheat Leaf dataset. https://www.kaggle.com/datasets/olyadgetch/wheat-leaf-dataset

    work page

  76. [82]

    https://www.kaggle.com/datasets/gauravduttakiit/cgiar - wheat-growth-stage-challenge

    CGIAR Wheat Growth Stage Challenge. https://www.kaggle.com/datasets/gauravduttakiit/cgiar - wheat-growth-stage-challenge

    work page

  77. [2024]

    N 119°10'00.00

    The dataset contains four categories for wheat organ segmentation. The images cover a variety of genotypes at different growth stages and include data from collaborative institutions across Switzerland, Belgium, UK, China , Mexico, Australia, France, Morocco, Japan, Canada collected under various planting densities and environmental conditions. The datase...

    work page 2012

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.