Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Multimodal large language model for wheat breeding: a new exploration of smart breeding

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a multimodal large language model adapted to wheat breeding with SFT, RAG, and RLHF outperforms general-purpose chatbots on breeding tasks, and that fusing remote sensing, phenotyping, weather, and germplasm data…

desk verdict Useful wheat-breeding dataset and systematic MLLM comparison, but benchmark leakage and a missing yield-prediction protocol undercut the headline claims. read the letter →

arxiv 2411.15203 v1 pith:37EAGCLR submitted 2024-11-20 cs.LG cs.AIcs.CL

Guofeng Yang , Yu Li , Yong He , Zhenjiang Zhou , Lingzhen Ye , Hui Fang , Yiqi Luo , Xuping Feng This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords multimodallargelanguagemodelwheatbreedingsupervisedfine-tuningretrieval-augmentedgenerationreinforcementlearningfromhumanfeedbackyieldpredictionUAVremotesensingcross-domaindatafusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an open-source multimodal large language model can become a practical wheat-breeding assistant when domain knowledge is injected through three complementary techniques: supervised fine-tuning (SFT), retrieval-augmented generation (RAG), and reinforcement learning from human feedback (RLHF). The authors build several WBLMs on different open-source vision-language models and report that the InternVL2-8B version leads on a new evaluation benchmark they designed, outperforming commercial chatbots such as GPT-4o, Gemini, Qwen, and ERNIE Bot on breeding-specific tasks. They further report that using remote sensing, phenotyping, weather, and germplasm data together yields the best wheat-yield predictions, with $R^2=0.821$ and RMSE $=489.254$ kg/ha. If these results hold, breeders could get integrated, traceable decision support from a single conversational tool instead of assembling data and code from many specialized sources.

What carries the argument

The load-bearing machinery is the three-stage knowledge-injection pipeline for building WBLM, together with a new evaluation benchmark. SFT uses LoRA fine-tuning on a 75k-question dataset; RAG retrieves from an external knowledge base (germplasm, cultivation, plant protection, seed price) through a vector-database pipeline with BGE-M3 embeddings and reranking; RLHF trains a reward model and optimizes the policy with PPO. The evaluation benchmark defines five wheat-breeding tasks and scores models on accuracy, stability, and reasoning; it is the instrument that carries the paper's comparative performance claims.

What would settle it

Evaluate WBLM on an independently built wheat-breeding question set that has no overlap with the SFT, reward-model, or PPO training data, with answers scored by breeders blind to model identity; if WBLM's accuracy and stability advantages over commercial MLLMs disappear, the benchmark-driven claim is falsified. Also test WBLM yield predictions on a new site or season absent from the training split; an $R^2$ well below 0.821 would show the cross-domain fusion result does not generalize.

Watch

Extended reading notes

Core claim

The central claim is that a domain-adapted multimodal large language model, WBLM, built by applying supervised fine-tuning, retrieval-augmented generation, and reinforcement learning from human feedback to an open-source InternVL2-8B backbone, achieves leading performance on a purpose-built wheat-breeding evaluation benchmark spanning phenotyping estimation, environmental stress assessment, target germplasm screening, cultivation technique recommendation, and seed price query. The paper reports that this WBLM outperforms both other open-source WBLMs and closed-source commercial MLLMs (GPT-4o, Gemini, Qwen, ERNIE Bot) on accuracy and stability, and that it gives the best yield prediction when remote sensing, phenotyping, weather, and germplasm data are fused, with $R^2=0.821$ and RMSE $=489.254$ kg/ha. The authors also claim that the three knowledge-injection techniques are complementary: SFT teaches expected answer formats, RAG supplies traceable external facts, and RLHF aligns answers with breeder preferences, so the combined model produces more professional, stable, and less hallucinated answers than the base MLLM alone.

Load-bearing premise

The load-bearing premise is that the paper's self-constructed wheat-breeding evaluation benchmark is a valid, unbiased test, even though the same team built both the models and the benchmark and reserved some test questions from the SFT training pool.

Editorial extensions

If this is right

  • If WBLM's performance is real, breeders can query one multimodal assistant for phenotyping estimates, stress assessment, germplasm screening, cultivation advice, and seed prices instead of moving between databases, papers, and custom code.
  • Cross-domain fusion of UAV remote sensing, field phenotyping, weather, and germplasm data becomes the recommended input set for yield prediction, since the paper reports it beats any single source or pair.
  • The ablation results imply a reusable recipe: SFT, RAG, and RLHF contribute differently, so future domain-adapted agricultural models should combine all three rather than pick one.
  • The released benchmark and dataset give other teams a common testbed for comparing future breeding MLLMs.
  • Off-the-shelf closed-source commercial MLLMs would remain weak on specialized breeding questions unless they allow fine-tuning or retrieval augmentation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: an independent, externally curated benchmark with no overlap with the training data would provide the cleanest test of whether WBLM's edge over commercial chatbots is due to domain knowledge rather than benchmark familiarity.
  • My inference: the yield-prediction pipeline should be tested on a new site and season to see whether the $R^2=0.821$ achieved on the two experimental stations transfers to unseen environments.
  • My inference: the same SFT+RAG+RLHF recipe is portable to other crops and to dynamic knowledge bases, and continuous refresh of the price database would turn the seed-price query subtask into a real-time service.
  • My inference: using the released dataset, one could isolate how much of the gain comes from RAG's traceable facts versus SFT's formatting by testing retrieval-only and fine-tuning-only variants.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes building wheat-breeding multimodal large language models (WBLMs) by combining supervised fine-tuning (SFT), retrieval-augmented generation (RAG), and reinforcement learning from human feedback (RLHF) on top of open-source MLLMs such as Qwen-VL, InternVL, and DeepSeek-VL. The authors construct a cross-domain dataset (UAV remote sensing, phenotyping, weather, germplasm, price) and a self-built evaluation benchmark with five task families. They report that the WBLM based on InternVL2-8B outperforms both other WBLMs and closed-source commercial MLLMs, that ablation experiments show the combination of SFT, RAG, and RLHF improves generation quality, and that using all cross-domain data simultaneously yields the best wheat yield prediction with R2=0.821 and RMSE=489.254 kg/ha.

Significance. If the claims were substantiated, the paper would demonstrate a practical integration of heterogeneous agricultural data into a multimodal language model, with a publicly released dataset and reproducible training pipeline. The strengths are the extensive field data collection, the use of standard SFT/RM/PPO formulations, and the decision to release the data on Zenodo. However, the central comparative claims rest entirely on a self-constructed benchmark whose evaluation protocol has serious validity threats, and the headline yield-prediction number is not verifiable from the manuscript. The significance is therefore conditional on fixing these load-bearing evaluation issues.

major comments (4)
  1. [Section 3.1] The accuracy evaluation for the phenotyping-estimation and environmental-stress tasks uses 20% of the SFT question-answer dataset as a test set. Because this SFT dataset is generated from the same plot-level field measurements (yield, SPAD, LAI, CH, CV, WH, PL, WL, FVC) that the benchmark asks the model to predict, a question-level random split does not prevent the same experimental plot, variety, or imaging date from appearing in both training and test pairs. The manuscript does not state whether the split is stratified by experimental unit or whether near-duplicate questions were removed. Without this information, the reported R2 and accuracy values in Section 4.1 could reflect memorization of plot-specific values rather than domain competence, which would invalidate the central comparison across MLLMs and the ablation conclusions.
  2. [Section 4.2.2] The yield-prediction result (R2=0.821, RMSE=489.254 kg/ha) is presented as the key achievement of cross-domain data fusion, but the manuscript gives no description of how the WBLM produces a numeric yield, what input features are provided in what representation, what train/test split or cross-validation scheme is used, or what objective is optimized. Figure 7 is labelled a cross-validation scatter plot, yet no cross-validation procedure is described. This number cannot be checked or reproduced from the manuscript as written, and it is a central claim in the abstract and conclusion.
  3. [Section 3.2] The benchmark relies on manual scoring for accuracy, stability, and reasoning, with no reported number of annotators, no inter-annotator agreement measure, and no external validation panel. Since the same team that built and tuned the models also constructed the benchmark and scored the outputs, the scores for the accuracy, stability, and reasoning comparisons are at risk of subjective bias. The paper should at minimum report a detailed scoring protocol and agreement statistics; without these, the comparative rankings in Section 4.1 are not independently verifiable.
  4. [Section 4.1] The comparison with closed-source commercial MLLMs is confounded by the fact that only the WBLM has access to the authors' knowledge base and fine-tuning data. Tasks such as seed-price query and target-germplasm screening inherently require the private knowledge base, so the poor performance of ChatGPT, Gemini, Qwen, and ERNIE Bot on those tasks is expected and does not demonstrate superior reasoning ability of WBLM. The paper should either restrict comparative claims to tasks that do not require the private knowledge base or provide an additional baseline that receives the same retrieved context through RAG.
minor comments (5)
  1. [Section 3.1] The text repeatedly uses 'STF' where 'SFT' is intended; please correct this typo throughout the section.
  2. [Section 3.2] The sentence 'This benchmark aims to integrate challenges from various wheat breeding tasks' appears twice in succession.
  3. [Section 4.1] The statement 'ChatGPT shows leading performance among open-source models' is contradictory; ChatGPT is a closed-source model in the paper's list. It should likely read 'among closed-source models.'
  4. [Equations (1)–(6)] The mathematical notation contains formatting artifacts (e.g., 𝐸𝐸, 𝜋𝜋𝜃𝜃, 𝐷𝐷𝐾𝐾𝐾𝐾) that should be cleaned up to standard notation.
  5. [Figure 7 and Section 4.2.2] The term 'cross-validation scatter plot' is used without a description of the cross-validation scheme; please either describe the procedure or rename the figure.

Circularity Check

1 steps flagged · score 4.0 of 10

Benchmark accuracy for phenotyping and stress tasks is computed on a held-out slice of the SFT training dataset, making those headline numbers partly a self-consistency check rather than an independent prediction.

  1. fitted input called prediction [Section 3.1 (Construction of WBLM), dataset D description]
    "The SFT dataset (75k) is created using multi-source datasets from the field, which contains questions and answers for training the SFT model. 80% of this dataset is used for training, and 20% is used for accuracy testing of wheat breeding model evaluation benchmark (phenotyping estimation task and environmental stress assessment task)."

    The paper's headline accuracy claims for phenotyping estimation and environmental stress assessment are evaluated on a 20% slice of the same SFT dataset used to train WBLM. The SFT objective in Eq. (2) maximizes log p_theta(y|x) over this dataset, so the 'benchmark' answers are drawn from the authors' own QA-generation pipeline built from the same field measurements (yield, SPAD, LAI, CH, CV, WH, PL, WL, FVC). For these tasks the evaluation is not an external test of wheat-breeding knowledge: a model can succeed by memorizing plot-level answer patterns, and the comparison against closed-source models that were never trained on this dataset conflates the effect of domain adaptation with exposure to the benchmark distribution.

full rationale

The central claim is not wholly circular. WBLM is also compared with closed-source MLLMs that were not trained on the authors' SFT data, the SFT/RM/PPO equations in Section 3.1 are standard, and the RAG-based tasks (germplasm screening, cultivation recommendations, seed-price queries) are evaluated against the authors' external knowledge base rather than against the SFT training pool. The main circularity is localized: for the phenotyping-estimation and environmental-stress tasks, the evaluation benchmark explicitly reuses a held-out part of the SFT dataset, so the accuracy and stability advantages reported for WBLM on those tasks reflect in-distribution fitting to the authors' own question-answer construction as much as genuine breeding competence. The Section 4.2.2 yield-prediction result (R2=0.821, RMSE=489.254 kg/ha) is reported without a train/test split, input encoding, or validation protocol, so it cannot be checked from the manuscript; this is a reproducibility concern rather than a demonstrated identity between input and prediction. Self-citations (e.g., Zhu et al. 2022 for wheat-head detection) are tool citations and not load-bearing for the main benchmark claim. Overall score 4: partial circularity through the SFT-derived benchmark, with substantial independent content remaining.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper's performance claims rest on assumptions about benchmark validity, clean data splits, and the accuracy of UAV-derived phenotype features. None of these are independently verified in the manuscript, and the benchmark shares a data-generating process with the SFT training set. No new physical entities are introduced.

free parameters (2)
  • Stability and price evaluation tolerance = +/-10%
    Section 3.2 defines consistency and seed-price correctness as agreement within 10%. This hand-chosen threshold directly determines the reported stability scores and price-query accuracy.
  • LoRA hyperparameters = not reported (SWIFT defaults)
    Section 3.1 says the SWIFT recommended training script is used with 5 epochs and no further hyperparameter adjustment, but rank, alpha, learning rate, batch size, and other values are not given. These choices affect the fine-tuned model and the reported performance.
assumptions (5)
  • domain assumption The authors' manually constructed evaluation benchmark is a valid measure of wheat-breeding question-answering ability.
    The benchmark is introduced in Section 3.2 with no external validation, no inter-annotator agreement, and standard answers written by the same team that built and tuned the models.
  • domain assumption The held-out 20% of the SFT dataset is a clean, non-overlapping test split for phenotyping and stress tasks.
    Section 3.1 describes an 80/20 split of the SFT dataset, but does not state that plots, varieties, or dates in the test split are disjoint from training, and no external test set is used.
  • domain assumption UAV-derived remote-sensing features (VIs, FVC, CH, CV, WH, WL) accurately represent the corresponding phenotypes.
    Section 2.3 derives these features from imagery and point clouds, but no accuracy metrics are reported for FVC, WL, WH, or CV; CH was manually verified without reporting agreement statistics.
  • domain assumption The RLHF reward model captures genuine breeder preferences.
    Section 3.1 trains a reward model on breeder rankings, but the ranking dataset is not released and no reliability statistics are given.
  • standard math Standard autoregressive language-model loss and PPO training objectives are applicable.
    Equations (1) through (6) are standard SFT, reward-model, and PPO objectives with no mathematical novelty or internal inconsistency.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal large language model for wheat breeding: a new exploration of smart breeding." pith.science (2026). https://pith.science/paper/37EAGCLR

@misc{pith2026241115203,
  author       = {Pith},
  title        = {Pith review of: Multimodal large language model for wheat breeding: a new exploration of smart breeding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37EAGCLR}},
  note         = {Machine review of arXiv:2411.15203}
}
read the original abstract

UAV remote sensing technology has become a key technology in crop breeding, which can achieve high-throughput and non-destructive collection of crop phenotyping data. However, the multidisciplinary nature of breeding has brought technical barriers and efficiency challenges to knowledge mining. Therefore, it is important to develop a smart breeding goal tool to mine cross-domain multimodal data. Based on different pre-trained open-source multimodal large language models (MLLMs) (e.g., Qwen-VL, InternVL, Deepseek-VL), this study used supervised fine-tuning (SFT), retrieval-augmented generation (RAG), and reinforcement learning from human feedback (RLHF) technologies to inject cross-domain knowledge into MLLMs, thereby constructing multiple multimodal large language models for wheat breeding (WBLMs). The above WBLMs were evaluated using the newly created evaluation benchmark in this study. The results showed that the WBLM constructed using SFT, RAG and RLHF technologies and InternVL2-8B has leading performance. Then, subsequent experiments were conducted using the WBLM. Ablation experiments indicated that the combination of SFT, RAG, and RLHF technologies can improve the overall generation performance, enhance the generated quality, balance the timeliness and adaptability of the generated answer, and reduce hallucinations and biases. The WBLM performed best in wheat yield prediction using cross-domain data (remote sensing, phenotyping, weather, germplasm) simultaneously, with R2 and RMSE of 0.821 and 489.254 kg/ha, respectively. Furthermore, the WBLM can generate professional decision support answers for phenotyping estimation, environmental stress assessment, target germplasm screening, cultivation technique recommendation, and seed price query tasks.

Figures

Figures reproduced from arXiv: 2411.15203 by the authors.

Figure 1
Figure 1. (a) Experimental sites. (b) Multi-source data acquisition. 2.2. Data acquisition 2.2.1. UAV data For the experimental field, we used multiple sensors and multiple UAVs to acquire hyperspectral (HS), lidar, multispectral (MS), and RGB (red, green, blue) data on the same day and compiled them into a RS dataset (Fig. 1b). The Matrice 300 RTK (DJI, China) was selected as the flight platform to carry the Zenmuse L1 (DJI,… view at source ↗
Figure 2
Figure 2. A workflow for the construction and application of a multimodal large language model for wheat breeding (WBLM). (a) Multi-source dataset construction. (b) External domain knowledge base construction. (c) The WBLM with domain knowledge is constructed using supervised fine-tuning, retrieval augmented generation, and reinforcement learning from human feedback. (d) The user's question (image and text) is sent to WBLM. (… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Agent Geospatial Copilots for Remote Sensing Workflows

    cs.LG 2025-01 conditional novelty 5.0 of 10

    A hybrid multi-agent orchestrator (composition plus iterative reassessment) reports 60.3% agentic correctness on generated remote sensing workflows, about 17 percentage points above the single-agent GeoLLM-Engine baseline.

Reference graph

Works this paper leans on

2 extracted references · cited by 1 Pith paper

  1. [204]

    Automatic estimation of optimal UA V flight parameters for real-time wide areas monitoring

    https://doi.org/10.1016/j.compag.2022.107558 Avola, D., Cinque, L., Fagioli, A., Foresti, G.L., Pannone, D., Piciarelli, C., 2021. Automatic estimation of optimal UA V flight parameters for real-time wide areas monitoring. Multimed. TOOLS Appl. 80, 25009–25031. https://doi.org/10.1007/s11042-021-10859-3 Badgley, G., Field, C.B., Berry, J.A., 2017. Canopy ...

    arXiv 2024

  2. [2023]

    https://doi.org/10.1093/database/baad051 Xu, T., Wang, F., Shi, Z., Xie, L., Yao, X., 2023. Dynamic estimation of rice aboveground biomass based on spectral and spatial information extracted from hyperspect ral remote sensing images at different combinations of growth stages. ISPRS J. Photogramm. REMOTE Sens. https://doi.org/10.1016/j.isprsjprs.2023.05.02...

    arXiv 2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.