REVIEW 4 major objections 4 minor 40 references
Hydra-Bench: A Benchmark for Multi-Modal Leaf Wetness Sensing
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Hydra-Bench offers a public multi-modal dataset for leaf wetness detection, with depth-aware radar-plus-camera fusion reaching 96% indoor accuracy.
desk verdict A genuinely new dataset for leaf wetness sensing, but the reported benchmark numbers are internally inconsistent and the evaluation protocol is missing, so the paper as written can't serve as a benchmark. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dataset sample: a synchronized trio of raw mmWave chirp data from a 77–81 GHz FMCW radar, a SAR image formed by a range-migration algorithm with phase compensation and a plane-wave decomposition theorem, and an RGB frame from a camera calibrated to the radar's field of view. A filename convention encodes the dry/wet label, collection date, sensor distance, sample index, and SAR cross-section depth, so every capture is self-describing. The benchmark model is Hydra's two-stage fusion pipeline: a CNN extracts features from depth-aligned SAR slices and RGB imagery, then a transformer with depth-aware positional encoding models relationships across SAR depths for the final wet/dry decision.
What would settle it
Take 30 published samples, re-measure each leaf's wetness with an independent method such as pre- and post-wetting weight or a commercial leaf-wetness sensor, and compare against the filename labels; if more than a small handful disagree, the reported accuracy rests on mislabeled ground truth rather than sensor capability.
Extended reading notes
Core claim
The paper's central claim is that the missing piece for reliable leaf wetness detection is a calibrated, multi-modal public dataset, and that Hydra-Bench supplies it. Each sample is a three-way synchronized capture of raw mmWave reflections, a reconstructed SAR image, and an RGB image, with the two imaging modalities aligned to a common field of view. On this data, the depth-aware fusion model reaches 96% ± 2.14% indoor accuracy and about 90% in outdoor farm settings, outperforming camera-only (86.13% ± 2.13%) and SAR-only (83.43% ± 3.05%) baselines as well as early and late fusion variants. The scan-distance results show accuracy staying above 90% when the SAR scan window is 150 mm or wider and degrading at shorter distances, and the release of raw data turns the dataset into a benchmark for SAR imaging algorithm optimization rather than only a classification benchmark.
Load-bearing premise
The benchmark's utility rests on the assumption that the 292 dry/wet labels are correct and that radar-to-camera calibration stayed consistent across six months and five plant species; if label noise crept in or calibration drifted, the reported accuracies and the dataset's value as a benchmark both erode.
Editorial extensions
If this is right
- Researchers can train and compare leaf wetness classifiers on a common resource without needing their own radar-camera rig.
- The reported fusion advantage—96% versus 86% camera-only and 83% SAR-only—gives a concrete target for future multi-modal fusion methods.
- The raw mmWave files let imaging researchers benchmark SAR reconstruction algorithms directly, not just downstream classification.
- The scan-distance curves quantify the resolution and coverage trade-off of SAR scanning, so future systems can choose scan windows against a known degradation curve.
- The small outdoor subset documents what happens when lighting and weather vary, giving a baseline for field-deployment claims.
Reading between the lines
- Beyond the paper: the 24 outdoor samples are too few to certify real-world performance, so the roughly 90% farm accuracy should be read as a pilot result until a larger outdoor split exists.
- Beyond the paper: the binary dry/wet labels ignore partially wet leaves and wetness duration, which are what disease models actually track; the naming convention could be extended to continuous or multi-class wetness labels.
- Beyond the paper: the raw data release makes it possible to learn SAR image formation end-to-end instead of using the fixed range-migration pipeline, an optimization direction the paper names but does not explore.
- Beyond the paper: an obvious stress test is low-light RGB plus SAR fusion, since the paper motivates multi-modality partly by RGB's lighting sensitivity; isolating that subset would show how much SAR contributes when vision fails.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Hydra-Bench, a multi-modal dataset for leaf wetness detection that contains synchronized mmWave raw data, SAR images, and RGB images collected over six months from five plant species, with 268 indoor and 24 outdoor sample pairs. The authors report benchmark results using their own Hydra model, including indoor accuracy of 96%, outdoor accuracy of approximately 90%, single-modality baselines (camera-only 86.13%, SAR-only 83.43%), and comparisons across fusion strategies and scan distances. The stated contribution is that the dataset can serve as a benchmark for future multi-modal fusion and SAR imaging algorithm optimization.
Significance. If the dataset artifact is made available as described, it is a potentially useful resource: synchronized raw mmWave data, SAR images, and RGB images with an accompanying SAR imaging codebase are not common in agricultural sensing, and the six-month, five-species collection adds diversity. However, the evaluation layer that would turn this artifact into a benchmark is currently unreliable. The reported accuracy numbers are internally contradictory, the train/test protocol is absent, and the evaluation is exclusively performed with a model developed by the same authors on the same dataset. These issues do not invalidate the dataset itself, but they do invalidate the quantitative claims as written and prevent third-party comparison against the reported baselines.
major comments (4)
- [Section 4, scan-distance paragraphs] The two consecutive scan-distance analyses report different accuracies for the same distances: the first reports 95.52% ± 1.46% at 200 mm, 91.32% ± 2.65% at 150 mm, and 82.1% ± 4.13% at 100 mm, while the next paragraph reports 95.43% ± 1.47% at 200 mm, 93.38% ± 2.56% at 150 mm, and 84.10% ± 4.53% at 100 mm. These contradictory numbers mean the benchmark results cannot be reproduced or compared against future methods, and the paper does not explain which set of numbers is authoritative.
- [Sections 3.4, 3.5, and 4] No train/test split is defined anywhere in the manuscript. The file structure (Section 3.4) and naming convention (Section 3.5) encode only group, date, distance, and index, and Section 4 never states how the 292 samples are partitioned into training, validation, and test sets. Since Section 1 says the dataset was used for training and testing Hydra, the reported accuracies cannot be distinguished from in-sample fits, and future researchers have no protocol under which to reproduce or compare against the numbers.
- [Section 4, fusion results] The text states that Hydra achieves 96% ± 2.14% accuracy and then says this 'improves to 85.41% ± 2.24% with our data enhancement techniques.' A number lower than the reported baseline cannot be an improvement, and the sentence is irreconcilable with the reported accuracy values. This is not a mere wording issue because the fusion-strategy comparison is one of the paper's central benchmark claims.
- [Section 4, outdoor results] The outdoor accuracy is reported only as 'approximately 90%' with no sample size, confidence interval, or per-condition breakdown. Given that the entire outdoor subset contains only 24 samples across dynamic conditions (rainy, dawn, and low-light night), the stated outdoor robustness claim is not quantitatively evaluable, and the paper should either provide the full protocol and error bars or explicitly mark the outdoor numbers as preliminary.
minor comments (4)
- [Section 3.1] The frequency slope is written as 'k = 70.295M Hz/µ'; the units are malformed and should be MHz/µs.
- [Section 3.5] The example filename '0 0119 200 1 200.jpg' contains spaces where the naming convention specifies underscores; the example should match the declared format exactly.
- [Section 1] The dataset URL appears only in the Introduction; a dedicated data-availability statement with access conditions, license, and expected citation format would be more appropriate for a dataset paper.
- [General] The paper repeatedly says the dataset is 'precisely calibrated' (Sections 2.3 and 3.1) but does not report any calibration error or drift check across the six-month collection period; a sentence describing the calibration validation would strengthen the resource description.
Circularity Check
No significant circularity: the dataset is a standalone released artifact; the self-referential Hydra evaluation weakens the benchmark evidence but does not reduce the central claim to its inputs.
full rationale
The paper's central object is a released multi-modal dataset (raw mmWave data, SAR images, RGB images, and SAR processing source code), and the benchmark claim is that this resource enables future research on multi-modal fusion and SAR imaging optimization. That claim does not reduce to a fit or definition: the dataset can be downloaded and reused independently of Hydra's reported numbers. The evaluation is indeed centered on the authors' own Hydra model (Section 3.6: 'benchmarking process centered on the Hydra framework [1]'), and Section 1 states the dataset is 'used for training and testing Hydra [1]'. If the reported accuracies were computed without a held-out split, they would be in-sample fits rather than predictions of generalization; however, the paper never explicitly states that training and testing occurred on the same exact samples, so this remains a reproducibility concern rather than a demonstrated circular reduction. No equation or definition is shown to be equivalent to its input, and the dataset artifact itself does not depend on the reported accuracy values. The contradictory scan-distance accuracy figures (95.52 vs 95.43 at 200 mm, 91.32 vs 93.38 at 150 mm, 82.1 vs 84.10 at 100 mm) and the absence of any documented train/test split are serious correctness/reproducibility issues that should be fixed, but under the specified circularity definitions they do not constitute circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption mmWave reflectivity distinguishes wet from dry leaf surfaces
- domain assumption Leaf wetness is a binary state (fully dry vs fully saturated) sufficient for LWD benchmarking
- standard math Weyl's representation theorem and range migration algorithm correctly reconstruct SAR images from the collected raw data
- domain assumption The calibration protocol from the authors' prior Hydra work transfers unchanged to this dataset
Cite this review
Pith. "Pith review of Hydra-Bench: A Benchmark for Multi-Modal Leaf Wetness Sensing." pith.science (2026). https://pith.science/paper/V4HQV4N6
@misc{pith2026250722685,
author = {Pith},
title = {Pith review of: Hydra-Bench: A Benchmark for Multi-Modal Leaf Wetness Sensing},
year = {2026},
howpublished = {\url{https://pith.science/paper/V4HQV4N6}},
note = {Machine review of arXiv:2507.22685}
}
read the original abstract
Leaf wetness detection is a crucial task in agricultural monitoring, as it directly impacts the prediction and protection of plant diseases. However, existing sensing systems suffer from limitations in robustness, accuracy, and environmental resilience when applied to natural leaves under dynamic real-world conditions. To address these challenges, we introduce a new multi-modal dataset specifically designed for evaluating and advancing machine learning algorithms in leaf wetness detection. Our dataset comprises synchronized mmWave raw data, Synthetic Aperture Radar (SAR) images, and RGB images collected over six months from five diverse plant species in both controlled and outdoor field environments. We provide detailed benchmarks using the Hydra model, including comparisons against single modality baselines and multiple fusion strategies, as well as performance under varying scan distances. Additionally, our dataset can serve as a benchmark for future SAR imaging algorithm optimization, enabling a systematic evaluation of detection accuracy under diverse conditions.
Reference graph
Works this paper leans on
-
[1]
Hydra-Bench: A Benchmark for Multi-Modal Leaf Wetness Sensing
INTRODUCTION Agriculture is a significant part of the global economy, ac- counting for approximately 4% of global GDP and exceeding 25% in some developing nations [2]. However, the increas- ing frequency and severity of plant diseases show significant threats to agricultural productivity, food security, and biodi- versity [3]. A major factor underlying di...
work page Pith review arXiv 2025
-
[2]
UNDERSTANDING THE PROBLEM 2.1. mmWave Sensing mmWave utilizes electromagnetic waves with wavelengths ranging from 1 to 10 millimeters. This short wavelength en- dows the mmWave radar with high sensitivity to fine surface textures, making it well-suited for detecting subtle changes such as leaf wetness. One of mmWave’s key advantages lies in its responsive...
-
[3]
DATASET 3.1. Implementation As illustrated in Figure 3a, our SAR imaging system is equipped with a two-axis mechanical scanner optimized for high-speed data acquisition. The scanner is meticulously cal- ibrated for plant-scale analysis, featuring a horizontal range of 150 mm and a vertical range of 100 mm. The mmWave radar mounted on this scanning platfor...
-
[4]
Our evaluation focuses on distinguishing between wet and dry leaves
EV ALUATION We conduct a comprehensive evaluation of Hydra to assess its accuracy, robustness, and efficiency in detecting leaf wet- ness under diverse environmental conditions. Our evaluation focuses on distinguishing between wet and dry leaves. As Hydra Camera mmLeaf 60 70 80 90 100Accuracy (%) (a) LWD Accuracy Fusion w Data Enhancement Fusion Early Fus...
-
[5]
RELATED WORK Artificial Intelligence in Agricultural IoT. The conver- gence of artificial intelligence and the Internet of Things (AIoT) has transformed agricultural practices, enabling in- telligent, scalable, and connected farm management [23]. Long-range communication technologies, such as LoRa and satellite-based networking, improve data coverage and ...
-
[6]
CONCLUSION This dataset paper presents a comprehensive and multi-modal dataset that addresses critical gaps in current leaf wetness detection research. By combining mmWave SAR and RGB imaging across a diverse set of plant species and environ- mental conditions, we enable rigorous benchmarking of ma- chine learning models under both controlled and real-wor...
-
[7]
ACKNOWLEDGEMENT This work was partially supported by NSF CAREER Award 2338976
-
[8]
Hydra: Accurate multi-modal leaf wetness sensing with mm-wave and camera fusion,
Yimeng Liu, Maolin Gan, Huaili Zeng, Li Liu, Younsuk Dong, and Zhichao Cao, “Hydra: Accurate multi-modal leaf wetness sensing with mm-wave and camera fusion,” in Proceedings of ACM MobiCom, 2024
work page 2024
Show all 40 references
-
[9]
Agriculture and food,
THE WORLD BANK, “Agriculture and food,” https://www.worldbank.org/en/topic/ agriculture/overview, 2024, Accessed: 2024-03-14
2024
-
[10]
Climate change impacts on plant pathogens, food security and paths forward,
Brajesh K Singh, Manuel Delgado-Baquerizo, Eleonora Egidi, Emilio Guirado, Jan E Leach, Hongwei Liu, and Pankaj Trivedi, “Climate change impacts on plant pathogens, food security and paths forward,”Nature Re- views Microbiology, vol. 21, no. 10, pp. 640–656, 2023
2023
-
[11]
Effects of leaf wetness duration, temperature, and host phenologi- cal stage on infection of walnut by xanthomonas arbori- cola pv. juglandis,
Concepci ´o Moragrega and Isidre Llorente, “Effects of leaf wetness duration, temperature, and host phenologi- cal stage on infection of walnut by xanthomonas arbori- cola pv. juglandis,” MDPI Plants, vol. 12, no. 15, 2023
2023
-
[12]
Modeling leaf wetness in relation to plant disease epidemiology,
L Huber and TJ Gillespie, “Modeling leaf wetness in relation to plant disease epidemiology,” Annual review of phytopathology, vol. 30, no. 1, pp. 553–577, 1992
1992
-
[13]
Reconsidering leaf wetness duration determination for plant disease management,
Tracy Rowlandson, Mark Gleason, Paulo Sentelhas, Terry Gillespie, Carla Thomas, and Brian Hornbuckle, “Reconsidering leaf wetness duration determination for plant disease management,” Plant Disease, vol. 99, no. 3, pp. 310–319, 2015
2015
-
[14]
Use of leaf wetness and temperature to time fungicide applications to control an- thracnose fruit rot of strawberry in florida,
SJ MacKenzie and NA Peres, “Use of leaf wetness and temperature to time fungicide applications to control an- thracnose fruit rot of strawberry in florida,” Plant dis- ease, vol. 96, no. 4, pp. 522–528, 2012
2012
-
[15]
Control of powdery mildew using the uc davis powdery mildew risk index,
WD Gubler, MR Rademacher, SJ Vasquez, and CS Thomas, “Control of powdery mildew using the uc davis powdery mildew risk index,” APSnet Feature. Published online. The American Phytopathological So- ciety, St. Paul, MN, 1999
1999
-
[16]
Validation of weather and leaf wetness forecasts for a lettuce downy mildew warning system,
BM Wu, KV Subbarao, AHC van Bruggen, and GGH Pennings, “Validation of weather and leaf wetness forecasts for a lettuce downy mildew warning system,” Canadian journal of plant pathology , vol. 23, no. 4, pp. 371–383, 2001
2001
-
[17]
An optical method of dew estimation,
S Duvdevani, “An optical method of dew estimation,” Quarterly Journal of the Royal Meteorological Society , vol. 73, no. 317-318, pp. 282–296, 1947
1947
-
[18]
Poster: mmleaf: Versatile leaf wetness detection via mmwave sensing,
Maolin Gan, Yimeng Liu, Li Liu, Chenshu Wu, Youn- suk Dong, Huacheng Zeng, and Zhichao Cao, “Poster: mmleaf: Versatile leaf wetness detection via mmwave sensing,” in Proceedings of ACM MobiSys, 2023
2023
-
[19]
PHYTOS 31 Manual Web,
METER Group, “PHYTOS 31 Manual Web,” http: //library.metergroup.com/Manuals/ 20434_PHYTOS31_Manual_Web.pdf, 2021, Accessed: Nov 21, 2022
2021
-
[20]
A bio-mimetic leaf wetness sensor from replica molding of leaves,
Brian H Nguyen, Gregory S Gilbert, and Marco Rolandi, “A bio-mimetic leaf wetness sensor from replica molding of leaves,” Advanced Sensor Research, vol. 2, no. 6, pp. 2200033, 2023
2023
-
[21]
Adonis: Neural-enhanced fine-grained leaf wetness sensing with efficient mmwave imaging,
Yimeng Liu, Maolin Gan, Gen Li, Younsuk Dong, and Zhichao Cao, “Adonis: Neural-enhanced fine-grained leaf wetness sensing with efficient mmwave imaging,” in Proceedings of IEEE INFOCOM, 2025
2025
-
[22]
Proteus: : Enhanced mmwave leaf wetness detection with cross-modality knowledge transfer,
Yimeng Liu, Maolin Gan, Huaili Zeng, Yidong Ren, Gen Li, Younsuk Dong, Xiaobo Tan, and Zhichao Cao, “Proteus: : Enhanced mmwave leaf wetness detection with cross-modality knowledge transfer,” in Proceed- ings of ACM SenSys, 2025
2025
-
[23]
Iwr1642,
Texas Instruments, “Iwr1642,” https://www.ti. com/product/IWR1642, 2024, Accessed: 2024- Oct-28
2024
-
[24]
Dca1000evm,
Texas Instruments, “Dca1000evm,” https://www. ti.com/tool/DCA1000EVM, 2024, Accessed: 2024-Oct-28
2024
-
[25]
Azure kinect dk,
Microsoft, “Azure kinect dk,” https: //azure.microsoft.com/en-us/products/ kinect-dk, 2024, Accessed: 2024-10-28
2024
-
[26]
Signal processing for fmcw sar,
Adriano Meta, Peter Hoogeboom, and Leo P Ligthart, “Signal processing for fmcw sar,” IEEE Transactions on Geoscience and Remote Sensing , vol. 45, no. 11, pp. 3519–3532, 2007
2007
-
[27]
Near-field mimo-sar millimeter-wave imaging with sparsely sam- pled aperture data,
Muhammet Emin Yanik and Murat Torlak, “Near-field mimo-sar millimeter-wave imaging with sparsely sam- pled aperture data,” IEEE Access , vol. 7, pp. 31801– 31819, 2019
2019
-
[28]
Development and demonstration of mimo-sar mmwave imaging testbeds,
Muhammet Emin Yanik, Dan Wang, and Murat Torlak, “Development and demonstration of mimo-sar mmwave imaging testbeds,” IEEE Access , vol. 8, pp. 126019– 126038, 2020
2020
-
[29]
Ausbreitung elektromagnetischer wellen ¨uber einem ebenen leiter,
H. Weyl, “Ausbreitung elektromagnetischer wellen ¨uber einem ebenen leiter,” Annalen der Physik , vol. 365, no. 21, pp. 481–500, 1919
1919
-
[30]
Arti- ficial intelligence of things: A survey,
Shakhrul Iman Siam, Hyunho Ahn, Li Liu, Samiul Alam, Hui Shen, Zhichao Cao, Ness Shroff, Bhaskar Krishnamachari, Mani Srivastava, and Mi Zhang, “Arti- ficial intelligence of things: A survey,”ACM Trans. Sen. Netw., Aug. 2024, Just Accepted
2024
-
[31]
Demeter: Reliable cross-soil lpwan with low-cost signal polarization alignment,
Yidong Ren, Wei Sun, Jialuo Du, Huaili Zeng, Younsuk Dong, Mi Zhang, Shigang Chen, Yunhao Liu, Tianx- ing Li, and Zhichao Cao, “Demeter: Reliable cross-soil lpwan with low-cost signal polarization alignment,” in Proceedings of ACM MobiCom, 2024
2024
-
[32]
Sate- riot: High-performance ground-space networking for rural iot,
Yidong Ren, Amalinda Gamage, Li Liu, Mo Li, Shi- gang Chen, Younsuk Dong, and Zhichao Cao, “Sate- riot: High-performance ground-space networking for rural iot,” in Proceedings of ACM MobiCom, 2024
2024
-
[33]
Charm: Exploiting geo- graphical diversity through coherent combining in low- power wide-area networks,
Adwait Dongare, Revathy Narayanan, Akshay Gadre, Anh Luong, Artur Balanuta, Swarun Kumar, Bob Ian- nucci, and Anthony Rowe, “Charm: Exploiting geo- graphical diversity through coherent combining in low- power wide-area networks,” in 2018 17th ACM/IEEE International Conference ...
2018
-
[34]
Adapting lora ground stations for low-latency imaging and inference from lora-enabled cubesats,
Akshay Gadre, Zachary Machester, and Swarun Kumar, “Adapting lora ground stations for low-latency imaging and inference from lora-enabled cubesats,” ACM Trans. Sen. Netw., vol. 20, no. 5, July 2024
2024
-
[35]
Aeroecho: Towards agricultural low-power wide-area backscatter with aerial excitation source,
Yidong Ren, Gen Li, Yimeng Liu, Younsuk Dong, and Zhichao Cao, “Aeroecho: Towards agricultural low-power wide-area backscatter with aerial excitation source,” in Proceedings of IEEE INFOCOM, 2025
2025
-
[36]
Soilcares: Towards low-cost soil macronutrients and moisture monitoring using rf- vnir sensing,
Juexing Wang, Yuda Feng, Gouree Kumbhar, Guangjing Wang, Qiben Yan, Qingxu Jin, Robert C. Ferrier, Jie Xiong, and Tianxing Li, “Soilcares: Towards low-cost soil macronutrients and moisture monitoring using rf- vnir sensing,” in Proceedings of ACM MobiSys, 2024
2024
-
[37]
Leafeon: To- wards accurate, robust and low-cost leaf water con- tent sensing using mmwave radar,
Mark Cardamis, Hong Jia, Hao Qian, Wenyao Chen, Yihe Yan, Oula Ghannoum, Aaron Quigley, Chung Tung Chou, and Wen Hu, “Leafeon: To- wards accurate, robust and low-cost leaf water con- tent sensing using mmwave radar,” arXiv preprint arXiv:2410.03680, 2024
-
[38]
Drlic: Deep rein- forcement learning for irrigation control,
Xianzhong Ding and Wan Du, “Drlic: Deep rein- forcement learning for irrigation control,” in 2022 21st ACM/IEEE International Conference on Informa- tion Processing in Sensor Networks (IPSN) , 2022, pp. 41–53
2022
-
[39]
Detec- tion of passageways in natural foliage using biomimetic sonar,
Ruihao Wang, Yimeng Liu, and Rolf M ¨uller, “Detec- tion of passageways in natural foliage using biomimetic sonar,” Bioinspiration & Biomimetics, vol. 17, no. 5, pp. 056009, 2022
2022
-
[40]
Geofl: A frame- work for efficient geo-distributed cross-device federated learning,
Maolin Gan, Lanpeng Li, Samiul Alam, Li Liu, Luyang Liu, Mi Zhang, and Zhichao Cao, “Geofl: A frame- work for efficient geo-distributed cross-device federated learning,” in Proceedings of IEEE INFOCOM, 2025
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.