{"id":"c737c6ba-31ee-4e48-b996-1f6b66be7d13","arxiv_id":"2505.21544","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The paper presents a YOLOv8 + RAG + LLM system for coffee leaf disease detection and remedy suggestions, with detection metrics but no validation of the language output.","lead":"A paper describes a farm assistant that uses YOLOv8 to detect coffee leaf diseases and a RAG-backed language model to suggest remedies. It reports a detection mAP of 0.681 but gives no evaluation of the generated text, and the dataset description is internally contradictory.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Sections 5.1 and 7.1 contradict which dataset version produced Table 1, leaving the reported mAP uninterpretable.","rationale":"The reader's weakest assumption identifies exactly the dataset ambiguity, which is the most load-bearing issue. The abstract and Section 1 make the system's usefulness depend on the detection component; the RAG/LLM text output is completely unevaluated, but the dataset mismatch is the sharpest concrete inconsistency. A detailed review shows that Section 5.1 explicitly says the original version performed best while Section 7.1 explicitly says the curated version was used for the reported metrics. Since the paper provides no split details and no code, the Table 1 numbers cannot be independently checked. Thus the paper's central claim is not yet falsified, but it is unsupported as written. I agree with the reader's identification and recommend CONDITIONAL, requiring the dataset/split clarification, released knowledge base/code, and an evaluation of the RAG/LLM output quality.","tokens_in":7909,"tokens_out":1377,"duration_ms":12027,"concrete_test":"Request the exact data.yaml and training/validation split used for the reported run. Recompute Table 1 by training YOLOv8n for 100 epochs at 640x640 on (a) the original BRACOL version and (b) the curated version, using the same split as Section 7.1. If only one version reproduces the reported mAP@0.5 of 0.681 within a small tolerance, the ambiguity is resolved; if neither reproduces it, the headline detection claim is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim is the YOLOv8 detection mAP@0.5 of 0.681 reported in Table 1, which then conditions the usefulness of the whole RAG-LLM system. The paper never resolves which dataset produced those numbers. Section 5.1 states the best performance was obtained when the model was trained on the original version of the BRACOL dataset. Section 7.1 then reports results 'after training the YOLOv8n model on the curated and re-annotated BRACOL dataset' with a validation set of 228 images and 402 annotated instances. These are two different dataset regimes, and no training/validation/test split is defined in Section 6. Every class-wise and overall metric inherits this ambiguity. Additionally, the claim in Section 7.1 that 'classes with a higher number of representative annotations' (Miner and Phoma) yield better results is internally inconsistent with the curated dataset distribution in Section 5.2 (rust 6,013; phoma 1,671; leaf miner 341; cercospora 201), since Rust has the most annotations but the lowest mAP@0.5 of 0.415 and Miner the highest mAP@0.5 of 0.894 with only 341 annotations. The missing dataset-version resolution is load-bearing because the entire evaluation rests on the validity of the training/validation setup.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript describes a coffee-leaf disease diagnosis system that combines a fine-tuned YOLOv8n detector with a Retrieval-Augmented Generation (RAG) pipeline and an LLM (accessed through the Groq API). The detection module is evaluated on the BRACOL dataset, reporting an overall mAP@0.5 of 0.681 on a validation set. The language module is illustrated through a Streamlit interface with example disease explanations, remedy suggestions, and follow-up question handling. The paper's stated contribution is an end-to-end tool that grounds LLM responses in a domain-specific knowledge base in order to reduce hallucination and provide actionable, reference-backed agricultural advice.","tokens_in":8131,"tokens_out":5171,"duration_ms":45581,"significance":"If the detection numbers are reliable and the RAG component demonstrably improves answer quality, the system would be a useful practical integration for precision agriculture. The authors provide a transparent account of the training hyperparameters and use a public benchmark dataset (BRACOL), which helps reproducibility. The class-wise detection metrics in Table 1 are a reasonable starting point for assessing the YOLOv8n component. However, at present the evidence for the central claims is incomplete: the detection evaluation is ambiguous because of the dataset-version contradiction, the RAG/LLM component is evaluated only through qualitative screenshots, no comparison is made against a non-RAG baseline or against existing integrated systems such as FarmTalk-Nexus, and the knowledge base is entirely author-created. These gaps prevent the paper from currently supporting its stated claims of hallucination mitigation and satisfactory integrated performance.","major_comments":[{"comment":"Section 5.1 states that the best performance was obtained when the model was trained using the original version of the BRACOL dataset, whereas Section 7.1 reports all validation results \"after training the YOLOv8n model on the curated and re-annotated BRACOL dataset.\" These are two different annotation regimes (1,899 vs. 8,226 annotations). Section 6 does not define the exact training/validation split or state which dataset version produced the numbers in Table 1. Because every detection metric in Table 1 inherits this ambiguity, the central quantitative result of the paper is currently uninterpretable. The authors must state which dataset version, which split, and which annotation counts were used for the numbers in Table 1, and, if both versions were used, report results separately.","section":"§5.1, §6, §7.1, Table 1"},{"comment":"The sentence \"it is evident that classes with a higher number of representative annotations (Miner and Phoma) yield better results\" is contradicted by the authors' own dataset statistics in Section 5.2: Rust has the highest annotation count (6,013) yet the lowest mAP@0.5 (0.415), while Miner has 341 annotations (fewer than Phoma's 1,671 and close to Cercospora's 201) yet the highest mAP@0.5 (0.894). The explanation in terms of annotation count cannot be correct as stated. The authors should either remove this claim or provide a different, evidence-based explanation, for example involving visual similarity to healthy tissue, bounding-box size, or annotation quality.","section":"§7.1 vs. §5.2"},{"comment":"The RAG/LLM component is not evaluated at all. Section 7.2 presents screenshots and example interactions, but there are no quantitative or structured qualitative measurements of retrieval accuracy, answer faithfulness, hallucination rate, or user satisfaction, and no comparison with the same LLM without RAG. The abstract and Section 1 claim that the system \"mitigate[s] the hallucination issue of LLM,\" but the paper provides no evidence for this claim. At minimum, the authors should include a small evaluation set with expert-validated gold answers, report retrieval hit rates and answer-groundedness scores, and compare RAG vs. no-RAG outputs.","section":"§7.2, §4.2–4.3"},{"comment":"The knowledge base was created by the authors themselves (\"We created a detailed document that serves as a knowledge base...\"), and no external sources, citations, or expert validation are given for its content. This makes the \"grounded\" answers self-referential in the sense that the LLM is constrained only by text that the authors wrote. To support the claim of factual grounding, the authors should cite the agricultural sources used to build the knowledge base and/or have the content reviewed by a plant pathology expert.","section":"§4.2"},{"comment":"The paper does not compare the proposed YOLOv8n model with other detectors or with the BRACOL baselines reported in the literature, including the YOLO-based results in Reference [16]. Adding such a comparison would contextualize the reported 0.681 mAP@0.5 and support the claim of satisfactory performance. Without a baseline, the reader cannot assess whether the proposed system improves over readily available alternatives.","section":"§2, §7.1"}],"minor_comments":[{"comment":"The Abstract contains wording such as \"model AI\" that should be \"AI models,\" and the opening sentence about humans as social beings is tangential to the technical content; the abstract would be stronger if it moved directly to the problem and contribution.","section":"Abstract"},{"comment":"The notation \"YOLOV8\" appears in the keywords and in some headings, while the body uses \"YOLOv8\"; please use a single consistent spelling.","section":"Throughout"},{"comment":"The text writes \"F AISS\" with a space; this should be \"FAISS\".","section":"§4.2, §7.2"},{"comment":"The phrase \"most affection one\" should be \"most favored one\" or \"most popular one.\"","section":"§1"},{"comment":"Reference [2] is malformed (\"al., L.C.C.:\") and should be completed with the author names and report title; Reference [9] is a GitHub link with no version or access date, which makes it difficult to verify.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript currently reads more like a system demonstration than a fully validated research contribution. The dataset-version contradiction in Sections 5.1 and 7.1 is the most urgent issue because it undermines the headline detection numbers. The absence of any evaluation of the RAG/LLM component is also a serious gap for a paper whose central claim is hallucination mitigation. In addition, the related work already contains FarmTalk-Nexus, which integrates YOLOv8 with GPT-3.5 Turbo using RAG; the authors should position their contribution more explicitly against that system. I do not see evidence of intentional misreporting, but the self-authored knowledge base should be clearly disclosed as such in the text."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a straightforward engineering integration: YOLOv8n for coffee leaf disease detection, RAG over a self-written knowledge base, and Llama-3 via Groq, wrapped in a Streamlit app. The authors cite FarmTalk-Nexus [9], which already combines YOLOv8 with a RAG-backed LLM for plant disease, so the architecture is not new. What is new is the specific application to coffee leaves with the BRACOL dataset and the attempt to report class-wise detection metrics.\n\nThe detection section is the only part with numbers, and the numbers look like a real training run: overall mAP@0.5 of 0.681 on a validation set of 228 images. The paper does a decent job describing the training setup, dataset versions, and the modular pipeline. Credit where due: they do not claim to beat SOTA and they list the prior integrated system.\n\nThe soft spots are real and load-bearing. Section 5.1 says the best performance came from training on the original BRACOL version (1,899 annotations), while Section 7.1 reports results after training on the curated and re-annotated version (8,226 annotations). The paper never resolves which dataset produced Table 1. Every metric inherits that ambiguity. The stress-test note also has a good catch: Section 7.1 says classes with more annotations performed better, but the curated distribution gives Rust 6,013 annotations and the lowest mAP@0.5 (0.415), while Miner with 341 annotations gets 0.894. That statement is internally inconsistent with the paper's own numbers.\n\nThe RAG/LLM half has no quantitative evaluation. The knowledge base was written by the authors, so the grounded answers are self-referential rather than checked against external authority. No code, no knowledge base, and no data are released. For an applied systems paper, that is a deficiency, but not a fatal one if the detection evaluation is fixed.\n\nWho is this for? Practitioners wanting a template for a crop-specific YOLOv8+RAG assistant, and possibly as a baseline for coffee-leaf detection. It is not a methods paper. With the dataset contradiction resolved and a basic evaluation of remedy quality added, it could be a modestly useful applied paper. As it stands, the central detection claim is uninterpretable.\n\nRecommendation: conditional acceptance path. A serious editor could send it to peer review if the venue is applied and the authors can clarify the dataset split and release the assets; otherwise it is a desk reject. I would engage with a revision.","headline":"A modest engineering integration of YOLOv8+RAG+LLM for coffee leaf disease that has a real detection table but a load-bearing dataset contradiction; fix the split ambiguity and it could be a useful applied paper.","tokens_in":8705,"tokens_out":1578,"would_cite":false,"duration_ms":10128,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that chaining YOLOv8 leaf-disease detection with retrieval-augmented generation can give farmers reference-grounded diagnoses and remedies, and reports mAP@0.5 of 0.681 on a coffee-leaf validation set.","keywords":["LLM","RAG","NLP","YOLOv8","Precision Agriculture","Coffee leaf disease detection","Retrieval-Augmented Generation"],"falsifier":"Re-train YOLOv8n under each of the two BRACOL dataset versions described in Section 5, evaluate both on the same held-out 228-image, 402-instance validation set, and check whether either configuration reproduces the reported overall mAP@0.5 of 0.681; separately, compare remedy answers generated with retrieval enabled versus disabled and count unsupported statements to test the hallucination-reduction claim.","tokens_in":7656,"feed_emoji":"☕","tokens_out":7426,"duration_ms":62581,"temperature":0.7,"pith_summary":"This paper sets out to combine computer vision and language generation into one agricultural assistant: a YOLOv8 model detects coffee leaf diseases from a photo, a retrieval-augmented generation (RAG) pipeline pulls matching passages from a curated knowledge base, and a large language model turns both into a plain-language explanation with remedies. The author argues that chaining these three components addresses what neither part can do alone, because detectors label regions but cannot explain why or what to do, while language models answer fluently but hallucinate without external grounding. On the detection side, the paper reports an overall mAP@0.5 of 0.681 on a validation set of 228 images, with the best class (leaf miner) reaching 0.894. If the approach holds, a farmer could upload a leaf photo and receive a diagnosis plus source-referenced treatment suggestions in one interface.","feed_headline":"RAG-grounded YOLOv8 spots coffee leaf disease and explains it","feed_subtitle":"No more guesswork: the same photo yields a detection, an explanation, and treatment advice grounded in a curated knowledge base.","key_machinery":"The load-bearing mechanism is the retrieval-then-read chain. A YOLOv8 detector first emits predicted disease labels and bounding boxes from the uploaded image; each detected label is then turned into a query against a vector store built by chunking and embedding a hand-written knowledge base of coffee leaf diseases and remedies. The top retrieved passages are appended to the user's prompt, and a large language model generates the final explanation and treatment advice from that augmented context. A short-term conversational memory keeps follow-up questions in context, so the system functions as a dialogue rather than a one-shot classifier.","core_discovery":"The central claim is that a three-part pipeline called the RAG-Augmented YOLOv8 Framework can identify coffee leaf diseases and generate explainable, reference-backed remedies in a single system. The author presents the design as neurosymbolic: YOLOv8 and the large language model are the neural components, while an external knowledge base of disease symptoms, causes, and treatments is the symbolic component that grounds the language output. The paper states that this retrieval-then-read mechanism mitigates the hallucination problem of large language models and enables fine-grained text output and conversational follow-up questions. It reports that the YOLOv8n detector, trained for 100 epochs at 640 by 640 resolution, achieves an overall mAP@0.5 of 0.681 and an overall mAP@0.5:0.95 of 0.454 on the validation set, with the class imbalance appearing to hurt rust detection most.","pith_inferences":["A testable extension would be to measure the RAG component's effect directly by running the same user questions with and without retrieved passages and counting unsupported statements; the paper describes the mechanism but reports no such evaluation.","Because the knowledge base is a single hand-written document, the quality of the generated advice is bounded by that document's coverage, so a formal comparison against expert-annotated remedy pairs would show whether the hallucination-mitigation claim holds in practice.","The class imbalance in the curated dataset, with rust far outnumbering cercospora, suggests the reported per-class mAP would shift under a rebalanced validation set, and severity estimation, a natural next step for farmer assistance, is not addressed.","If the detector's false negatives correlate with visual similarity to healthy leaf texture, as the paper says for rust, then a healthy-leaf rejection stage before the LLM query could reduce wasted retrievals and improve trust in the assistant."],"forward_implications":["If the reported metrics are reproducible, YOLOv8n at 640 by 640 resolution is enough for usable coffee leaf disease detection on a consumer GPU, making on-farm deployment plausible.","If RAG grounding works as claimed, the remedy text will track the curated knowledge base, so updating that document updates the advice without retraining the detector or the language model.","The same three-part chain can be re-targeted to other crops by swapping the detector's training data and the knowledge-base document.","The conversational memory component turns a one-shot diagnosis into a follow-up dialogue, which is the intended path toward adaptive treatment plans."],"supporting_citations":[{"why":"Supplies the YOLOv8 implementation and default training configuration used throughout the experiments.","marker":"[4]"},{"why":"Reports YOLOv8-based leaf disease detection at 98% mAP, forming the performance context the paper expects to approach.","marker":"[5]"},{"why":"Is the closest prior system combining YOLOv8, RAG, and an LLM for plant disease remediation, which the framework extends.","marker":"[9]"},{"why":"Provides the retrieval-generation synergy result that motivates grounding the LLM with retrieved passages.","marker":"[12]"},{"why":"Establishes the baseline that RAG outperforms fine-tuning alone, supporting the hallucination-mitigation design choice.","marker":"[14]"},{"why":"Supplies the original BRACOL images and 1,899 annotations, the version the paper says trained best.","marker":"[15]"},{"why":"Supplies the expert-reannotated 8,226-annotation version used for the reported validation results.","marker":"[16]"}],"fun_headline_variants":["RAG and YOLOv8 join forces to diagnose coffee leaf diseases","One photo, two AIs: coffee disease detection plus grounded treatment advice","Explainable coffee disease ID: YOLOv8 spots it, RAG explains it","Neurosymbolic AI for farmers: detect coffee leaf disease and get remedies","From leaf photo to treatment plan: a RAG-YOLOv8 pipeline for coffee"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation's validity rests on knowing which version of the BRACOL dataset produced Table 1, because Section 5.1 says the original 1,899-annotation version trained best while Section 7.1 reports results after training on the re-annotated 8,226-annotation version, and the paper never resolves this discrepancy.","fun_headline_variants_meta":{"raw":{"variants":["RAG and YOLOv8 join forces to diagnose coffee leaf diseases","One photo, two AIs: coffee disease detection plus grounded treatment advice","Explainable coffee disease ID: YOLOv8 spots it, RAG explains it","Neurosymbolic AI for farmers: detect coffee leaf disease and get remedies","From leaf photo to treatment plan: a RAG-YOLOv8 pipeline for coffee"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1626,"prompt_tokens":1045,"completion_tokens":581,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":478}},"tokens_in":661,"tokens_out":581,"duration_ms":4034,"temperature":1.0,"reasoning_tokens":478,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:24:20.509259+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train YOLOv8n under each of the two BRACOL dataset versions described in Section 5, evaluate both on the same held-out 228-image, 402-instance validation set, and check whether either configuration reproduces the reported overall mAP@0.5 of 0.681; separately, compare remedy answers generated with retrieval enabled versus disabled and count unsupported statements to test the hallucination-reduction claim.","supporting_citations":[{"cited_title":"Ultralytics (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the YOLOv8 implementation and default training configuration used throughout the experiments."},{"cited_title":"github (2025)","cited_arxiv_id":null,"evidence_quote":"Is the closest prior system combining YOLOv8, RAG, and an LLM for plant disease remediation, which the framework extends."},{"cited_title":"Applied Sciences 15(9) (2025) 14","cited_arxiv_id":null,"evidence_quote":"Supplies the expert-reannotated 8,226-annotation version used for the reported validation results."}],"review_version":1}