{"id":"22fd2945-3426-4587-85a8-483866d0e017","arxiv_id":"2412.13312","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An automated pipeline using tsfresh features and AutoML can separate ozone-exposed from non-exposed ivy plant signals in the lab, but the held-out accuracy is lower than the headline.","lead":"This paper tests whether machine learning can tell when a plant has been exposed to ozone by reading its electrical signals. The authors report high classification accuracy in training data, but lower and uneven results on held-out data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 94.6% 'unseen data' accuracy is a validation-set result, not the held-out test result; the paper's own test-set accuracies are 73-77% for Leaf/Combined and 57-74% for Stem, so the headline claim is overstated.","rationale":"The reader's weakest_assumption was the absence of a sham-exposure control for the pre-exposure baseline. That is a real threat to external validity, and I do not dispute it. However, I see an even more direct, text-internal problem with the central claim: the abstract's '94.6% on unseen data' is not a held-out result. The 94.6% figure is produced by feature selection and AutoML on the analysis dataset, with repeated 80/20 validation splits, and the paper's own held-out test results are considerably lower (Sec. III-C). The authors transparently acknowledge this as meta-overfitting, but the abstract still presents the higher validation number as an 'unseen data' accuracy. This is not a matter of consensus or interpretation; it is a mismatch between the headline and the paper's own reported test-set numbers. The reader's rationale did mention that the headline accuracy is a validation result rather than a held-out result, so there is partial agreement, but the reader chose the sham-control issue as the weakest assumption. My concern is more central because even if a sham control were added, the current headline number would still be unsupported. The correct fix is to report the held-out test accuracy as the headline and to make clear that the 94.6% is a feature-selection/validation result. This does not change the reader's CONDITIONAL verdict: the paper remains conditionally acceptable as a toolchain demonstration, but the conditions should explicitly include correcting the abstract and either adding a sham control or substantially hedging the causal interpretation.","tokens_in":11168,"tokens_out":7306,"duration_ms":73523,"concrete_test":"Recompute the held-out accuracy for the best feature subsets (Leaf 62, Stem 69, Combined 94 features) strictly on the 20% test split defined in Sec. II-A, training only on the 80% analysis split and freezing the feature set before any test-set evaluation. If the Leaf result remains near 94.6%, the abstract stands; if it is approximately 77% as reported in Sec. III-C, then the '94.6% on unseen data' claim must be replaced by the held-out figure. As an additional robustness check, run nested cross-validation with feature selection performed inside each training fold, so that no validation-based selection leaks into the test estimate, and compare that unbiased estimate with the headline claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims that the toolchain classifies plant ozone exposure 'with accuracies of up to 94.6% on unseen data.' The only place 94.60% appears is Sec. III-B, where it is the accuracy of the best forward-selected feature subset (62 leaf features) averaged over 100 random 80/20 splits of the analysis dataset. That is validation performance during model and feature selection, not performance on the held-out test set defined in Sec. II-A. The paper's own held-out evaluation in Sec. III-C reports Leaf accuracy of 76.96% with the selected features and 73.26% with all features, Combined accuracy of 77.30%, and Stem accuracy of 57.21% with the selected features. The paper explicitly attributes the gap to meta-overfitting. Thus the central quantitative claim as stated is contradicted by the paper's own test-set results. The toolchain may still have value at the lower, honestly reported accuracy, but the '94.6% on unseen data' claim is not supported and should not be the headline. A related but secondary concern is that the no-ozone class is taken from the 10 minutes immediately before each exposure, so without a sham-exposure control the model could partly be learning procedural or temporal artifacts; this reinforces the need to restate performance claims conservatively.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an automated toolchain for classifying ozone exposure in ivy (Hedera helix) from plant electrical differential potentials. The pipeline uses tsfresh to extract 787 features per 10-minute signal slice, Naive AutoML to select and optimize classifiers, and a forward feature selection procedure to reduce the feature set. Experiments cover leaf, stem, and combined electrode placements, with an additional claim that the same toolchain transfers to wind detection in a different plant species. The headline result is a classification accuracy of up to 94.6% on unseen data, but the paper's own held-out test results are substantially lower (76.96% Leaf, 77.30% Combined, and 57.21% Stem with selected features).","tokens_in":11503,"tokens_out":6641,"duration_ms":65103,"significance":"If the reported performance were supported, the paper would make a useful practical contribution: an automated, species- and stimulus-generic pipeline for phytosensing that avoids hand-crafted features and manual model selection. Strengths include public release of the code, use of generic time-series features, a reasonable choice of AutoML framework, and an unusually candid discussion of meta-overfitting and learning-curve variance in Sections III-B and III-C. However, two load-bearing problems currently block the stated significance: the '94.6% on unseen data' figure is a validation result, not a held-out test result, and the no-ozone baseline is temporally confounded with the exposure procedure because no sham-exposure control is described. The generalizability claim in Section IV is also reported without sufficient detail to be evaluated. The toolchain may still be valuable at the honestly reported lower accuracy or as a methodological template, but the manuscript as written overstates what is established.","major_comments":[{"comment":"The abstract's central claim of 'accuracies of up to 94.6% on unseen data' is not supported by the paper's own evaluation. The 94.6% value appears in Section III-B as the average validation accuracy of the best forward-selected feature subset (62 leaf features) over 100 random 80/20 splits of the analysis dataset, evaluated during model and feature selection. Section III-C reports the held-out test accuracies with the selected features as 76.96% for Leaf, 77.30% for Combined, and 57.21% for Stem, with 57.21% near chance for a balanced two-class problem. The abstract and the concluding paragraph in Section III-C must be rewritten to report the test-set performance, or the evaluation must be redone with a nested/selection-aware protocol so that the reported number is a true unbiased estimate.","section":"Abstract; Section III-B; Section III-C"},{"comment":"The no-ozone class is defined as the 10 minutes immediately before each scheduled ozone exposure, and no sham-exposure control is described. Because the positive and negative slices are adjacent in a repeating 2-hour cycle, the classifier can learn temporal position within the cycle, recovery from the previous ozone dose, or procedural artifacts from the ozone generator and airflow rather than a physiological response to ozone. This is not a minor detail: it directly undermines the attribution of the classification to ozone exposure. A sham-exposure condition in which the generator/fan is operated without ozone would be the standard control. At minimum, the manuscript must explicitly state this as a limiting condition and soften the causal claim that the model detects 'ozone exposure'.","section":"Section II-A"},{"comment":"The random 80/20 split of samples across plants places the ozone and pre-ozone slices of the same exposure episode into both training and test partitions. Since these slices are 10 minutes apart and come from the same plant, the classifier can exploit strong temporal correlation and plant-specific signal baselines, inflating the reported test accuracy. To support the claim that the method generalizes to unseen data, the authors should report an evaluation using leave-one-exposition-out or leave-one-plant-out splits, or at least analyze how performance changes when adjacent slices are kept together. This is a load-bearing issue for the 'unseen data' wording.","section":"Section II-A; Section III-C"},{"comment":"The claimed transfer result of 95.39% accuracy for wind detection in ZZ plants is presented without the experimental details needed for assessment: no dataset size, no number of exposures, no description of preprocessing or feature extraction, no split protocol, and no statement of whether this is a test-set or validation number. Since 'our approach can be used for other plant species and stimuli' is one of the three bullet contributions in the introduction, this result cannot be left as an unreviewable aside; either full details must be provided or the claim should be removed or explicitly labeled preliminary.","section":"Section IV"}],"minor_comments":[{"comment":"The ±200 mV cutoff is described as removing 'physically illogical values,' but no justification or count of removed samples is given; please report the fraction of removed measurements and the sensitivity of the results to this threshold.","section":"Section II-B"},{"comment":"The word 'accuracy' is used for both validation and test results without consistent labeling; for example, Table I and Figure 3 report analysis-data validation scores while Section III-C reports test scores. Add explicit labels such as 'validation' and 'test' in all captions and in the text.","section":"Table I; Figure 3; Section III-C"},{"comment":"The threshold model's maximum accuracy of 70.83% should be clearly labeled as a validation result on the analysis data, since the paragraph currently reads as if it were a standalone result without indicating the split.","section":"Section III-A; Figure 4"},{"comment":"The sentence 'ROC AUC captures this uncertainty' is imprecise; ROC AUC measures ranking quality across classification thresholds, not predictive uncertainty. Please rephrase.","section":"Section II-C"},{"comment":"The y-axis is labeled 'EDP [normalized]' while the feature extraction is performed in mV; please clarify whether normalization is applied to the data used for feature extraction and, if so, where it enters the pipeline.","section":"Figure 2"},{"comment":"The phrase 'This maybe counter-intuitive finding' contains a typo and should read 'This may be counter-intuitive'.","section":"Section III-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is from an interdisciplinary applied group and the code release is a genuine asset. I would ask the editor to pay particular attention to the abstract/test-set discrepancy: the '94.6% on unseen data' figure is likely to be quoted independently of the paper's own test results and must be corrected before any acceptance. The missing sham-exposure control is a scientific limitation that should be prominently disclosed, ideally in a dedicated limitations paragraph, and the Section IV transfer claim is currently unverifiable. The paper may be salvageable as a toolchain contribution with honest, conservative claims, but it cannot be accepted with the current headline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the headline number doesn't survive contact with the test set. The abstract claims up to 94.6% on unseen data, but that figure is the validation accuracy from forward feature selection on the analysis split (Sec. III-B). Their own Sec. III-C test-set results are 76.96% (leaf, selected features), 77.30% (combined), and 57.21% (stem). They acknowledge this as meta-overfitting, which is honest, but the abstract as written is misleading.\n\nWhat's actually new: instead of hand-picking features and classifiers as in Chatterjee et al. and Dolfi et al., they use tsfresh to extract a large generic feature set, Naive AutoML to pick a pipeline, and forward selection to shrink the feature set. This is a sensible automation layer for phytosensing, and they back it with publicly released code and a decent set of diagnostics: learning curves, ROC analysis, comparison to a threshold model. That part is solid.\n\nThe bigger soft spot is experimental, not methodological. The no-ozone class is defined as the ten minutes immediately before each ozone exposure, with no sham-exposure control. So the model could be learning the ozone generator switching on, airflow changes, or the plant's recovery state rather than the ozone itself. The exposure concentration (~1.4 ppm) is also two orders of magnitude above EU ambient thresholds; they note this in the conclusion but it further limits the real-world claim.\n\nI'd send this to review. The toolchain is usable, the code is out, and the weaknesses are fixable: reword the abstract to report the held-out numbers, add a sham control or at least discuss the confound explicitly, and put the test-set results up front. As it stands, conditional acceptance with major revisions.","headline":"The 94.6% 'unseen data' accuracy is a validation-set number; the paper's own held-out tests land at 57-77%, so the abstract overstates the result, but the toolchain and honest discussion of meta-overfitting make it worth refereeing.","tokens_in":11993,"tokens_out":2312,"would_cite":true,"duration_ms":21833,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A generic automated toolchain classifies plant ozone exposure from electrical signals with up to 94.6% accuracy.","keywords":["phytosensing","ozone detection","plant electrophysiology","automated machine learning","tsfresh feature extraction","air quality monitoring","Hedera helix","electrical differential potential"],"falsifier":"Run a sham-exposure control in which the ozone generator and air circulation are activated for ten minutes without producing ozone, on the same schedule. If a classifier trained on real-ozone versus pre-exposure baselines also separates sham-exposure from pre-exposure windows with comparable accuracy, the method is detecting the procedure rather than ozone.","tokens_in":11003,"feed_emoji":"🌿","tokens_out":6916,"duration_ms":58384,"temperature":0.7,"pith_summary":"This paper sets out to prove that a living plant can be turned into an automated ozone detector without hand-crafted features or manual model selection. The authors record electric differential potentials from ivy plants inside a Faraday cage, expose the plants to ten-minute ozone pulses, and feed the signals into a toolchain that extracts 787 generic time-series features per slice, searches for a classification pipeline with automated machine learning, and prunes the features with forward selection. They report separating ozone-exposed from non-exposed ten-minute windows with accuracies up to 94.6% and ROC AUC values up to 0.9985 on the analysis split, and lower held-out test accuracies (76.96% leaf, 77.30% combined, 57.21% stem) that they attribute to meta-overfitting. If these claims hold, pollutant monitoring in cities could be built from dense networks of low-cost plant-based sensors rather than sparse, expensive stations.","feed_headline":"Plant signals reveal ozone exposure at up to 94.6%","feed_subtitle":"Generic features plus automated model search turn an ivy plant into a pollutant detector without hand-crafted features.","key_machinery":"The load-bearing mechanism is a three-stage toolchain applied to 10-minute slices of electrical differential potential. First, the tsfresh library computes 787 generic time-series features per slice, with background subtraction that removes the features of a slice taken 10 to 20 minutes before the stimulus to reduce inter-plant variance. Second, Naive AutoML greedily searches over preprocessing steps and scikit-learn classifiers, optimizing either ROC AUC or accuracy with five stratified 80/20 validation splits. Third, a semi-greedy forward feature selection iteratively adds the feature that most improves ROC AUC while keeping the $n$ best candidate sets per feature-count to allow temporary suboptimality; the selected 62 to 94 features are what the final models use.","core_discovery":"The central claim, in the paper's own terms, is that a generic automatic toolchain can classify whether a plant is being exposed to ozone from the electric differential potential recorded by two needle electrodes. The pipeline computes 787 tsfresh features per 10-minute signal slice, uses Naive AutoML to select and tune a classifier, and then applies a semi-greedy forward feature selection that keeps several candidate feature sets per size. On the analysis (validation) split, feature selection lifts ROC AUC to 0.9901 for the leaf, 0.9063 for the stem, and 0.9985 for the combined dataset, with accuracies of 94.60%, 82.64%, and 89.34% respectively. On the held-out 20% test split the selected-feature models reach ROC AUC 0.8790 (leaf), 0.6665 (stem), and 0.9111 (combined), with accuracies of 76.96%, 57.21%, and 77.30%; the authors explain the drop as meta-overfitting of the pipeline. They also report 95.39% accuracy for a wind-versus-no-wind classification on ZZ plants measured with a different phytosensor, which they present as evidence of generalizability.","pith_inferences":["The reported 94.6% figure is an analysis-split number; a deployed monitor should be benchmarked against the held-out numbers, roughly 77% accuracy, until the pipeline is re-validated in situ.","Because the pre-exposure 10-minute window is used as the non-ozone class, a sham-exposure control is needed to rule out the alternative that the classifier learns the generator start or the airflow pattern rather than ozone physiology.","The generic tsfresh features are not physiologically interpretable, so the toolchain cannot by itself say which plant electrical response carries the ozone information; targeted experiments would be needed to close that gap.","The peak exposure of about 1,447 ppb in these experiments is far above typical urban ozone levels near health thresholds, so real-world accuracy at 60 to 200 ppb remains an open empirical question that the paper's outlook already identifies."],"forward_implications":["A new plant species or stimulus can be turned into a classifier by rerunning the same three-stage pipeline, without choosing features or models by hand.","Leaf measurements carry more ozone information than stem measurements, so sensor placement near leaves should be prioritized in future phytosensing devices.","Combining leaf and stem signals is at least as accurate as leaf alone, so multi-point measurement is a safe design choice.","The low-cost PhytoNode hardware records enough signal detail for classification, supporting the feasibility of dense urban monitoring networks.","Performance on held-out data is substantially below validation performance, so pipelines trained on small datasets should expect a meta-overfitting penalty when deployed."],"supporting_citations":[{"why":"Supplies the tsfresh library that computes the 787 generic time-series features from each electrical signal slice.","marker":"[34]"},{"why":"Supplies the Naive AutoML framework that automatically selects and optimizes the classification pipeline.","marker":"[33]"},{"why":"Provides the separate ZZ-plant dataset and the background-subtraction preprocessing used to demonstrate generalizability to other species and stimuli.","marker":"[22]"},{"why":"Earlier ozone classification on tomato plants with hand-selected features; the baseline this work extends by automating feature and model selection.","marker":"[24]"},{"why":"Prior ozone-detection method based on change detection and correlation waveform analysis that requires manual thresholds; the approach this work aims to replace.","marker":"[28]"},{"why":"Describes the PhytoNode hardware used to record the electric differential potentials in the ozone experiments.","marker":"[7]"},{"why":"Inspires the preprocessing pipeline of rolling median filtering and downsampling applied before feature extraction.","marker":"[18]"}],"fun_headline_variants":["Ozone exposure from plant electric signals: 94.6% accuracy","Plants as ozone detectors: ML hits 94.6% on signals","Electric plant signals forecast ozone with 94.6% accuracy","Simple plant signals reveal ozone at 94.6% via auto ML","Plant electricity predicts ozone exposure: 94.6% accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The experiment assumes that the ten minutes immediately before each scheduled ozone pulse are a clean, ozone-free resting baseline, meaning two hours of recovery fully clears the previous dose, the plant is not anticipating the procedure, and the ozone generator and airflow themselves do not alter the electrical signal.","fun_headline_variants_meta":{"raw":{"variants":["Ozone exposure from plant electric signals: 94.6% accuracy","Plants as ozone detectors: ML hits 94.6% on signals","Electric plant signals forecast ozone with 94.6% accuracy","Simple plant signals reveal ozone at 94.6% via auto ML","Plant electricity predicts ozone exposure: 94.6% accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000327,"raw_usage":{"total_tokens":1869,"prompt_tokens":1025,"completion_tokens":844,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":752}},"tokens_in":641,"tokens_out":844,"duration_ms":8206,"temperature":1.0,"reasoning_tokens":752,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:14:51.987310+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a sham-exposure control in which the ozone generator and air circulation are activated for ten minutes without producing ozone, on the same schedule. If a classifier trained on real-ozone versus pre-exposure baselines also separates sham-exposure from pre-exposure windows with comparable accuracy, the method is detecting the procedure rather than ozone.","supporting_citations":[{"cited_title":"Time series feature extraction on basis of scalable hypothesis tests (tsfresh – a python package),","cited_arxiv_id":null,"evidence_quote":"Supplies the tsfresh library that computes the 787 generic time-series features from each electrical signal slice."},{"cited_title":"Naive automated machine learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the Naive AutoML framework that automatically selects and optimizes the classification pipeline."},{"cited_title":"Stimulus classification with electrical potential and impedance of living plants: Comparing discriminant analysis and deep-learning methods,","cited_arxiv_id":null,"evidence_quote":"Provides the separate ZZ-plant dataset and the background-subtraction preprocessing used to demonstrate generalizability to other species and stimuli."},{"cited_title":"Exploring strategies for classification of external stimuli using statistical features of the plant electrical response,","cited_arxiv_id":null,"evidence_quote":"Earlier ozone classification on tomato plants with hand-selected features; the baseline this work extends by automating feature and model selection."},{"cited_title":"Plant electrical activity analysis for ozone pollution critical level detection,","cited_arxiv_id":null,"evidence_quote":"Prior ozone-detection method based on change detection and correlation waveform analysis that requires manual thresholds; the approach this work aims to replace."},{"cited_title":"Phyton- ode upgraded: Energy-efficient long-term environmental monitoring using phytosensing,","cited_arxiv_id":null,"evidence_quote":"Describes the PhytoNode hardware used to record the electric differential potentials in the ozone experiments."},{"cited_title":"De- tecting stress caused by nitrogen deficit using deep learning techniques applied on plant electrophysiological data,","cited_arxiv_id":null,"evidence_quote":"Inspires the preprocessing pipeline of rolling median filtering and downsampling applied before feature extraction."}],"review_version":1}