{"id":"7cc97214-2d32-4dba-8e44-19df11090113","arxiv_id":"2501.09363","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A six-layer custom CNN achieves 98.4 to 99.7 percent test accuracy on three medicinal plant leaf datasets, including a new but unreleased 10-class dataset.","lead":"A custom convolutional neural network was tested on three medicinal leaf image sets, including a newly collected set of 10 plant classes, and reached test accuracies around 98.5 to 99.7 percent. The paper is an incremental application of a standard CNN architecture rather than a methodological advance, and its new dataset is not publicly released.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pre-split data augmentation may cause train/test leakage, invalidating the reported accuracies.","rationale":"The central claim is that the proposed CNN achieves ~99% test accuracy on three medicinal leaf datasets. For that claim to hold, the test set must contain images unseen during training. The manuscript's ordering of operations creates a concrete risk that this condition is violated: Section 3.1 describes applying augmentation to 'the dataset' and generating four images per source, and Section 3.2 lists the 80/10/10 split after augmentation in the preprocessing sequence. If so, test images are augmented siblings of training images, and the reported accuracies are inflated. This is a load-bearing issue because it attacks the internal validity of the numbers, not just their external generalization. The reader's subset-representativeness concern is real but secondary; no subset can rescue a contaminated test set. The concern is not an accusation of fraud; it is an ambiguity that the authors can resolve by specifying the split order and releasing data/code. If augmentation was applied only to the training partition, the concern disappears. Given that the issue is potentially severe and currently unresolved, the paper should remain conditional pending that clarification, so I do not change the reader's verdict. A concrete check—comparing augment-then-split vs split-then-augment on a public dataset—would settle it.","tokens_in":8955,"tokens_out":8863,"duration_ms":72267,"concrete_test":"Replicate the pipeline on a public dataset (e.g., Indian Medicinal Leaves [10]) using the same augmentation (flip, zoom 1.0, 10° rotation, four outputs per source) and the same 6-layer CNN, with two protocols: (A) augment then split 80/10/10; (B) split then augment only the training partition. Compare test accuracies. Also compute near-duplicate overlap (e.g., perceptual hash) between train and test in protocol A. If protocol A yields substantially higher accuracy (e.g., >2 percentage points) or if near-duplicates cross the split, leakage is confirmed. Ask the authors to release their exact split and state the augmentation order.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's preprocessing pipeline applies data augmentation (random flip, random zoom, 10° rotation) to 'the dataset', generating four images per source leaf (Section 3.1, Fig. 3), and only then describes the 80/10/10 train/validation/test split (Section 3.2). The text never states that augmentation is restricted to the training set. If augmentation precedes the split, augmented copies of the same original image can appear in both training and test sets. The test accuracies in Table 2 (99.55%, 98.49%, 99.74%) would then reflect near-duplicate recognition rather than generalization to unseen leaves. This is more fundamental than the subset-representativeness concern: even the reported subset accuracies would be invalid, and the abstract's full-dataset claims would be unsupported. The ambiguity is directly visible in the manuscript and is testable.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a custom convolutional neural network with six convolution layers for the identification of traditional medicinal plant leaves, and evaluates it on three datasets: the authors' self-curated dataset, the Indian Medicinal Leaves Image Dataset, and the MED117 dataset. The model is reported to achieve test accuracies of 99.55%, 98.49%, and 99.74% respectively (Table 2), with the self-curated dataset containing 42,250 images of 50 plant species. The authors compare several optimizers (Adam, RMSprop, SGD with momentum) and claim the proposed model is effective for medicinal plant identification. The evaluation, however, is performed on subsets of each dataset, and the experimental protocol as described leaves open the possibility of data leakage from pre-split augmentation.","tokens_in":9193,"tokens_out":4540,"duration_ms":41181,"significance":"If the reported results were valid, the paper would provide a simple and effective CNN baseline for medicinal plant leaf classification, along with a potentially useful self-curated dataset for a region (Assam, India) that is underrepresented in existing benchmarks. The practical motivation is clear, and the reported accuracy numbers are competitive with prior work. However, the current evidence for these claims is undermined by experimental-protocol ambiguities and reporting gaps: the augmentation pipeline appears to be applied before the train/test split, the evaluation uses only subsets of the full datasets without justification, results are single-run with no error bars, and the abstract presents subset accuracies as though they apply to the full datasets. These issues must be resolved before the central claims can be accepted. The paper does not release code or the self-curated dataset publicly, which further limits reproducibility.","major_comments":[{"comment":"The data augmentation pipeline is described in Section 3.1 ('Data augmentation is applied to the dataset to avoid overfitting... four images in total were generated from one pre-processed plant leaf image'), and only in Section 3.2 is the dataset split into 80% train, 10% test, and 10% validation. The text never states that augmentation is restricted to the training set. If augmentation is performed before the split, augmented copies of the same original image can appear in both the training and test sets, so the accuracies in Table 2 would reflect near-duplicate recognition rather than generalization to unseen images. This is a load-bearing issue because the central claim of high identification accuracy depends entirely on the validity of the held-out test set. The authors must clarify the order of operations and, if augmentation preceded the split, rerun all experiments with augmentation applied only to the training partition.","section":"Section 3.1, Section 3.2"},{"comment":"The model is evaluated on subsets of each dataset: 10 of 50 self-curated classes, 20 of 80 Indian Medicinal Leaves classes, and 10 of 115 MED117 classes. No selection criterion is provided, and the chosen classes are not enumerated. The abstract reports the resulting accuracies (99.5%, 98.4%, 99.7%) without any qualification, implying that the full datasets were tested. This overstates the coverage of the results. The authors should either evaluate on the full datasets or explicitly state in the abstract and conclusion that the results are for subsets, and they must justify that the chosen subsets are representative of the full datasets and of real-world identification difficulty.","section":"Section 3.1, Abstract, Table 2"},{"comment":"The reported accuracies are from a single experimental run with no standard deviations, confidence intervals, or repeated-seed information. The model and optimizer are effectively selected using validation accuracy, and the test accuracy is reported without accounting for this selection. Single-run results of this kind are not sufficient to establish that the proposed model reliably achieves the claimed accuracy. The authors should report the mean and standard deviation over multiple runs (or at least provide the results from several independent runs), and they should describe the model-selection procedure clearly.","section":"Section 4, Table 2"},{"comment":"The text is inconsistent about which optimizer produced the reported results. Section 4 states 'RMSprop showed the best results' and later 'Right after Adam, RMSprop and SGD (Momentum) performed best,' while the conclusion says 'Adam and RMSprop have performed best in all three datasets.' Table 2 lists only one accuracy per dataset, without indicating the optimizer or epoch that produced it. The authors must specify, for each dataset, the optimizer and epoch count corresponding to the numbers in Table 2, and reconcile the claims about optimizer performance.","section":"Section 4, Section 6"},{"comment":"The comparison in Table 3 is misleading as presented. The proposed method is compared with prior models on the Indian Medicinal Leaves dataset, but the proposed method was evaluated on a 20-class subset, whereas the prior works (e.g., MTJNet, ECA-VGG19) report results on the full 80-class dataset. This is not a like-for-like comparison. The table should clearly indicate the number of classes used for each method, or the comparison should be restricted to methods evaluated under the same protocol.","section":"Table 3"}],"minor_comments":[{"comment":"There are numerous typographical and grammatical errors, including 'trainning' (Section 4), 'achived' (Section 4), 'eopch' (Section 4), 'image suze' (Section 4), 'Ensamble' (Table 3), 'mages' (Figure 2 caption), and 'Indian Medicinal Leaves image' (Section 3.1). A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The statement 'we have not set any learning rate, and the default value of 0.001 is considered for all the optimizers' is not accurate for standard frameworks; for example, Keras's SGD default learning rate is 0.01 while Adam and RMSprop default to 0.001. The authors should specify the deep learning framework and the exact hyperparameter values used for each optimizer.","section":"Section 3.3"},{"comment":"The subfigures showing training and validation accuracy versus epoch do not indicate which optimizer they correspond to. Since the paper compares Adam, RMSprop, and SGD with momentum, each curve should be labeled with the optimizer or the figure should be split accordingly.","section":"Figure 5"},{"comment":"The contribution states that the self-curated dataset contains 42,250 images of 50 medicinal plants, but Section 3.1 reports that the class sizes range from 110 to 1800 images. The total of 42,250 is inconsistent with the stated per-class range (50 classes at an average of 845 images each is possible, but the range 110-1800 suggests the total would be between 5,500 and 90,000; please clarify the exact distribution and confirm the total count).","section":"Section 1.3, Section 3.1"},{"comment":"The related-work table contains inconsistencies with the text: for example, reference [16] is described in the text as using 1500 images of 40 plants, but the table says 500 images of 40 plants; reference [19] is called both 'SDAMPI' and 'SDAMPI' in different places. These discrepancies should be corrected.","section":"Table 1"},{"comment":"The self-curated dataset is said to be 'available upon request,' which is not a practical form of data availability for reproducibility. The authors should upload the dataset to a public repository, as they have done for the other two datasets.","section":"Section 7.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a practical problem and reports very high accuracies, but the experimental protocol as written contains a likely train/test leakage via pre-split augmentation, and the abstract overstates the coverage by omitting that only subsets of the datasets were used. These are load-bearing issues that can be fixed by rerunning with a correct split protocol and by reporting results transparently. If the authors address these points, the paper could become acceptable; as it stands, the central claim is not supported by the evidence presented. The paper also lacks code/data release, and the comparison table is not like-for-like. I would encourage the editor to request a major revision with a clear request for corrected experiments and full transparency about the subsets and splits."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper: the headline accuracies (99.55%, 98.49%, 99.74%) are probably a leakage artifact. Section 3.1 describes generating four augmented images per source leaf, and says augmentation is applied to 'the dataset'. The 80/10/10 train/test/validation split is described in Section 3.2, after augmentation is already done. The text never says augmentation was restricted to the training split. So augmented copies of the same original leaf can easily end up in both train and test. That makes the test set measure near-duplicate recognition, not generalization.\n\nWhat's genuinely new here is the self-curated dataset: 42,250 images of 50 Assam medicinal plants, captured with an iPhone 14 Pro. That could be useful to the community. But only 10 of the 50 classes are used for evaluation, and the dataset is 'available upon request' rather than deposited. So the contribution is largely a promise.\n\nWhat the paper does well is a straightforward comparison of three optimizers (Adam, RMSprop, SGD with momentum) on the same CNN, with precision/recall/F1 reported. The related work section is adequate. The architecture is a stock six-conv-layer CNN; no new layer, loss, or training scheme is proposed.\n\nBeyond the leakage ambiguity, the subset selection is unjustified: 10 of 50, 20 of 80, 10 of 115 classes, with no enumeration or criterion. The abstract reports the accuracies without mentioning subsets, which overstates coverage. There is a single run, no error bars, and no code release. Table 3 lists 'Proposed 99%' while Table 2 says 99.55% on the Indian dataset, a small inconsistency.\n\nIf the augmentation was actually done after the split, the authors should say so explicitly and re-run with a clarified protocol. If not, the accuracies are invalid. Either way, the paper as written can't be trusted.\n\nThis is a paper for a workshop or a methods note at best. I would not send it to peer review in its current form; the leakage risk is a load-bearing flaw, and the dataset is not accessible to verify anything. I would, however, encourage the authors to release the dataset and fix the protocol, because a clean evaluation on the full self-curated data would be worth reading.","headline":"The reported accuracies are likely inflated by pre-split data augmentation; the paper's only real asset, the self-curated dataset, is neither released nor properly evaluated.","tokens_in":9647,"tokens_out":3052,"would_cite":false,"duration_ms":37082,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact six-layer CNN identifies medicinal leaves at over 99 percent accuracy across three datasets.","keywords":["medicinal plant identification","leaf classification","convolutional neural network","self-curated dataset","Assam medicinal plants","image augmentation","optimizer comparison","Ayurveda"],"falsifier":"Retrain the same architecture on the full Indian Medicinal Leaves dataset (80 classes), MED117 (115 classes), and the self-curated collection (50 classes) with the same split, augmentation, and optimizers; if any full-scale accuracy falls substantially below the reported 99.55/98.49/99.74, the central claim as stated for the datasets is refuted. A reader can also request the self-curated data (available upon request) and enumerate the chosen subsets to check whether the selected classes are the easiest.","tokens_in":8824,"feed_emoji":"🍃","tokens_out":6765,"duration_ms":61052,"temperature":0.7,"pith_summary":"This paper claims that a comparatively small convolutional neural network, built from scratch with six convolution layers, can identify medicinal plant leaves as accurately as much heavier transfer-learning models. The authors report test accuracies of 99.55% on 20 classes of the Indian Medicinal Leaves dataset, 98.49% on 10 classes of MED117, and 99.74% on 10 classes of their own newly curated dataset of 50 medicinal plants photographed in Assam. They also contribute the self-curated collection, roughly 42,250 images across 50 species, and compare three optimizers (Adam, RMSprop, SGD with momentum) to show the result is not tied to one training rule. If the claim holds, reliable leaf identification no longer requires an expert botanist at the point of collection, which matters for safe harvesting of Ayurvedic material.","feed_headline":"Six-layer CNN hits 99.7% on medicinal leaf identification","feed_subtitle":"A from-scratch model matches transfer-learning baselines across three leaf datasets, including a new 50-species Assam set.","key_machinery":"The load-bearing object is the six-layer CNN itself: six 3x3 convolution layers (32 filters in the first, 64 in the rest) each followed by 2x2 max-pooling, then flatten, a 64-unit dense layer with ReLU, batch normalization, dropout 0.1, and a softmax classifier. It carries the argument by showing that a deliberately simple architecture, trained from scratch rather than fine-tuned, can extract the leaf-texture features needed to separate visually similar medicinal species. The other load-bearing pieces are the three datasets (two public, one newly curated) and the augmentation pipeline (vertical/horizontal flip, zoom to 1.0, 10-degree anticlockwise rotation) that produces four training images from each original.","core_discovery":"The authors' central claim is that a custom CNN with six 3x3 convolution layers, ReLU activations, max pooling after each convolution, a 64-unit dense layer with batch normalization and 10% dropout, and a softmax output can classify medicinal leaves with high accuracy across datasets of different provenance. Trained from scratch on 256x256 RGB images with random flip, zoom, and 10-degree rotation augmentation and an 80/10/10 split, the model reached 99.55% on a 20-class subset of Indian Medicinal Leaves, 98.49% on a 10-class subset of MED117, and 99.74% on a 10-class subset of the self-curated dataset, with the best results obtained using Adam and RMSprop. On the Indian dataset the model's 99% is comparable to the best previously reported values (98.8% for ECA-VGG19 and 99.71% for MTJNet), which the authors take as evidence that a simple from-scratch network can stand in for pretrained architectures. They also observe that accuracy improves with dataset size, pointing to the value of their curated collection.","pith_inferences":["Editorial inference: testing on all classes of each dataset, not just subsets, would likely yield lower accuracy because adding visually similar species increases inter-class confusion; the paper's subset choice is unexplained, so the generalization gap is unknown.","Editorial inference: the self-curated dataset was shot under controlled conditions (leaves on white paper, daylight, a high-end smartphone), so real-world accuracy on cluttered or low-light field images is an open question that a mobile-app follow-up would need to answer.","Editorial inference: a direct comparison on the same full datasets, with the same splits, between this six-layer CNN and the cited transfer-learning baselines would settle whether the accuracy advantage is due to architecture, augmentation, or the particular class subsets used."],"forward_implications":["A lightweight from-scratch CNN can reach accuracy comparable to pretrained transfer-learning models, so computational cost need not be a barrier to deploying leaf identification in the field.","The 42,250-image, 50-species self-curated Assam dataset gives the community, if released, a new resource for training and evaluating medicinal-plant classifiers in a region with high ethnobotanical diversity.","Because the model's accuracy improves with dataset size in the authors' experiments, expanding per-class image counts is a direct route to further gains.","On the Indian Medicinal Leaves dataset, the proposed model at 99% is within about one point of the strongest prior result (99.71%), suggesting the simpler architecture is a competitive baseline for future work."],"supporting_citations":[{"why":"Supplies the Indian Medicinal Leaves dataset; the 20-class subset is one of the three test beds and yields the 99.55% accuracy.","marker":"[10]"},{"why":"Supplies the MED117 dataset; the 10-class subset is the second test bed and yields the 98.49% accuracy.","marker":"[9]"},{"why":"Provides the VGG16/VGG19/ensemble baselines (96.1/97.6/99%) that the proposed model is compared against on the Indian dataset.","marker":"[6]"},{"why":"Provides the ECA-VGG19 baseline (98.8%) on the same Indian dataset, used to situate the proposed model's accuracy.","marker":"[11]"},{"why":"Provides the MTJNet baseline (99.71%), the strongest prior result the paper compares its 99% against.","marker":"[17]"},{"why":"Defines the Adam optimizer, one of the three optimizers whose choice determines the best reported accuracies.","marker":"[5]"}],"fun_headline_variants":["Six-layer CNN hits 99.7% on medicinal leaf ID","From-scratch CNN matches transfer-learning on leaf ID","99.7% accuracy: custom CNN for Ayurvedic leaf ID","Lean CNN identifies medicinal leaves with 99.7% accuracy","Self-curated dataset boosts CNN to 99.7% leaf accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline numbers come from small, unstated subsets (10 of 50, 20 of 80, and 10 of 115 classes), and the paper assumes those subsets are representative of the full identification problem, so if the omitted classes are harder, the reported accuracies will not generalize to the complete datasets.","fun_headline_variants_meta":{"raw":{"variants":["Six-layer CNN hits 99.7% on medicinal leaf ID","From-scratch CNN matches transfer-learning on leaf ID","99.7% accuracy: custom CNN for Ayurvedic leaf ID","Lean CNN identifies medicinal leaves with 99.7% accuracy","Self-curated dataset boosts CNN to 99.7% leaf accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000854,"raw_usage":{"total_tokens":3728,"prompt_tokens":981,"completion_tokens":2747,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":2658}},"tokens_in":597,"tokens_out":2747,"duration_ms":20463,"temperature":1.0,"reasoning_tokens":2658,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:05:38.002814+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same architecture on the full Indian Medicinal Leaves dataset (80 classes), MED117 (115 classes), and the self-curated collection (50 classes) with the same split, augmentation, and optimizers; if any full-scale accuracy falls substantially below the reported 99.55/98.49/99.74, the central claim as stated for the datasets is refuted. A reader can also request the self-curated data (available upon request) and enumerate the chosen subsets to check whether the selected classes are the easiest.","supporting_citations":[{"cited_title":"org/10.17632/748F8JKPHB.3","cited_arxiv_id":null,"evidence_quote":"Supplies the Indian Medicinal Leaves dataset; the 20-class subset is one of the three test beds and yields the 99.55% accuracy."},{"cited_title":"https://doi.org/10.17632/DTVBWRHZNZ.4","cited_arxiv_id":null,"evidence_quote":"Supplies the MED117 dataset; the 10-class subset is the second test bed and yields the 98.49% accuracy."},{"cited_title":"https://doi.org/10.2139/ssrn.4804339","cited_arxiv_id":null,"evidence_quote":"Provides the VGG16/VGG19/ensemble baselines (96.1/97.6/99%) that the proposed model is compared against on the Indian dataset."}],"review_version":1}