Pith. sign in

REVIEW 2 major objections 4 minor 33 references

Enhancing Wearable Tap Water Audio Detection through Subclass Annotation in the HD-Epic Dataset

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read A newly annotated tap water subclass in the HD-Epic dataset is easier for lightweight audio classifiers to learn than the generic water class, making on-device running-tap detection more practical.

desk verdict Useful, well-documented tap-water annotations on HD-Epic with an overclaimed learnability result that doesn't survive the paper's own LOPO numbers. read the letter →

arxiv 2505.20788 v2 pith:PFWBYLG2 submitted 2025-05-27 cs.HC cs.LG

classification cs.HCcs.LG
keywords audioactivityrecognitionwearablecomputingtapwaterdetectiondatasetannotationHD-Epicsoundeventclassificationon-devicemachinelearninghandwashing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a precise, newly annotated 'tap water' subclass in the HD-Epic wearable audio dataset is a valuable resource, and that this narrow class is easier for lightweight classifiers to learn than the dataset's broad 'water' class. The authors hand-labeled 717 running-tap audio events, about 157 minutes in total, inside the existing water annotations, and show that tap water covers roughly 62% of water-labeled time while being far more acoustically homogeneous. Why this matters: microphone data from wearables raises privacy concerns, so the useful signal has to be extracted on the device by small models. If tap water really is easier to learn, then on-device audio context for tasks like hand-washing detection, cooking monitoring, and water-use tracking becomes more practical. The paper supports the claim with a CNN and a random forest that both reach higher F1 scores relative to a random baseline on tap water than on water.

What carries the argument

The central object is the new tap water annotation layer on the HD-Epic dataset, built by a single annotator who re-examined all water and pour labels lasting at least three seconds and re-labeled, shortened, or split them into 717 precise running-tap events. The argument that this class is easier to learn is carried by two quantitative instruments: the coverage and intersection-over-union statistics that establish tap water as a subclass of water, and the F1-ratio-to-baseline metric that normalizes classifier performance against a uniform-random dummy classifier to account for class imbalance. The lightweight classifiers themselves—a random forest on 41-dimensional spectral and MFCC features and a five-layer CNN on log-mel spectrograms—are deliberately simple, because the intended use is on-device inference where the audio never leaves the wearable. The comparison of tap water against water under the same models and splits is what turns the annotation effort into a claim about learnability.

What would settle it

Run the same classifiers on a version of the data where water and tap water occur equally often, and compare their raw detection scores; if tap water no longer beats water, the central 'easier to learn' claim is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the generic 'water' label in HD-Epic is a superclass of many acoustically different sounds, whereas 'tap water' is a more uniform subclass, and that this uniformity makes the subclass learnable even though it is rarer. The authors created 717 hand-labeled tap water annotations (9,595 seconds, 98% of them at least three seconds long) by revisiting the 1,098 water and pour annotations longer than three seconds; 97.8% of the new tap water time falls inside the original water labels, and the intersection-over-union between the two classes is 0.616. In a random train-test split, the CNN reaches an F1 of 0.75 on tap water versus 0.71 on water, with ratios to a uniform-random baseline of 668% versus 410%; a random forest shows the same pattern. In leave-one-participant-out evaluation the absolute F1 values are closer (CNN 0.48 for tap water, 0.52 for water), but the ratio to baseline is still higher for tap water (447% versus 315%), and paired tests on these ratios are significant. The paper concludes that the new labels are a useful, precise addition to HD-Epic and that tap water should be treated as its own class.

Load-bearing premise

The conclusion that tap water is easier to learn assumes that comparing each classifier's score against a random-guessing baseline fairly cancels out the fact that tap water is much rarer than water; if that comparison is unfair, the 'easier' claim collapses, because on the held-out-participant evaluation tap water's raw score is actually lower.

Editorial extensions

If this is right

  • If the central claim holds, wearable devices can detect running taps with small models (a CNN or random forest) that run locally, so microphone audio need not be stored or transmitted, addressing the privacy constraint that motivates on-device processing.
  • The 717 precise tap water labels add a new, publicly available class to HD-Epic, letting other researchers train and evaluate tap water detectors on nine real home environments rather than isolated clips.
  • Treating tap water as its own class instead of the broad water class should improve downstream tasks such as hand-washing detection, cooking step monitoring, bathroom-usage tracking, and water-wastage detection, because the sound is more specific to those intentional activities.
  • Because tap water covers about 62% of water-labeled time and is 97.8% contained in water annotations, models trained on water already contain much of the information needed; the new labels mainly sharpen the boundary between running taps and other water sounds.
  • The released pre-trained ExecuTorch model gives a ready starting point for deploying tap water detection on mobile CPUs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'easier to learn' conclusion depends on the ratio-to-baseline normalization; on the leave-one-participant-out split the absolute F1 for tap water (0.48) is lower than for water (0.52), so if the normalization is judged unfair, the learnability claim weakens even though the dataset itself may remain valuable.
  • Because all 717 annotations were made by a single annotator, a natural next step would be a second-annotator reliability study; high agreement would strengthen the resource, while low agreement would suggest the tap/water boundary is harder to draw than the classifier results imply.
  • The three-second cutoff excluded roughly 11% of water-labeled duration, so very short or faint tap events may be underrepresented; real deployments with brief taps could perform worse than these experiments suggest.
  • The paper does not fuse audio with IMU data, but its motivation points to a testable extension: combining tap water audio detection with wearable motion sensing could reduce both false positives and false negatives in hand-washing detection beyond either modality alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper contributes a new set of 717 hand-labeled tap water annotations (about 157 minutes) built on top of the existing water and pour labels in the HD-Epic egocentric dataset. The authors analyze the relationship between tap water and the original water class using IoU and coverage statistics, and they train two lightweight classifiers (a random forest and a CNN) under both a random train-test split and a leave-one-participant-out (LOPO) protocol. The central claim, stated in the abstract and Section 4, is that the tap water class is 'easier to learn' than the original water class, supported mainly by the ratio of classifier F1 to a uniform-random baseline F1.

Significance. If the annotation resource is valid, this is a useful community contribution: precise tap water timestamps in a realistic egocentric audio dataset can support privacy-preserving on-device applications such as hand washing detection and water usage monitoring. The paper is transparent about the labeling process, provides open code, data, and a pretrained ExecuTorch model, and uses lightweight models appropriate for wearable deployment. However, the comparative learnability claim, which is the paper's headline result, is not supported by the evidence as presented, and the statistical procedure used to justify it is flawed. The underlying dataset contribution remains plausible, but the analysis needs to be reworked before the claim can be accepted.

major comments (2)
  1. [Section 3.4.4, Table 4, Section 4] The paper's own LOPO results in Table 4 show lower absolute F1 for tap water than for water (CNN: 0.48 vs 0.52), so the only quantitative support for the 'easier to learn' claim is the ratio-to-baseline artifact described above.
  2. [Section 3.2 and Section 3.3 (Eq. 2)] This is a load-bearing issue for a dataset contribution because the validity of the new labels is the paper's main product, but it is fixable with additional analysis or careful wording.
minor comments (4)
  1. [Section 3.2.1] The sentence 'we hat to many water annotations had to be ignored' appears to be a typo and should read 'we had to ignore many water annotations'.
  2. [Table 4 and Figure 2] The term 'LOSO-CV' is used inconsistently with 'LOPO' in the text and Section 3.4.1; please use a single acronym.
  3. [Section 4] The Shapiro-Wilk test is reported for a sample of nine participants; with such a small sample the test has low power, so the statement 'indicated normality' should be phrased more cautiously.
  4. [Table 4] The abbreviation 'R. to Bln.' is not defined in the table caption; it is only described in Section 3.4.4. Adding a footnote to the table would improve readability.

Circularity Check

1 steps flagged · score 1.0 of 10

Only a minor self-definitional validation; the central dataset and learnability claims are empirical and not circular.

  1. self definitional [Section 3.3, 'Coverage of tap water by water' (Table 3); cf. Section 3.2.1 label creation]
    "The coverage value is high, as we only annotated near the water labels of the original dataset. However, the value also supports the initial assumption that tap water is a subclass of water, as it is mostly covered by water annotations."

    Tap water annotations were created by starting from the existing water and pour labels and relabeling only those intervals (Section 3.2: 'we did not go through all video recordings again at their entire length, but rather focused on improving the existing labels of water and pour'; only labels of at least three seconds were relabeled). Because the new class was therefore defined as a sub-interval of water by construction, the ~97.8% coverage is an input to the annotation procedure, not an independent observation. Presenting it as supporting evidence that 'tap water is a subclass of water' confirms the construction rather than testing a consequence. The paper acknowledges the cause, so this is a transparent sanity check and is not load-bearing for the main dataset or learnability claims.

full rationale

The only identified circular step is the coverage sanity check in Section 3.3: since the tap water labels were produced by relabeling existing water and pour annotations, the finding that about 97.8% of tap water lies inside water annotations is true by construction and cannot independently confirm the subclass relation. The paper itself states why the coverage is high, so this is a minor, transparent validation rather than a load-bearing derivation. The central claims, namely that 717 precise tap water annotations (about 157 minutes) were created and that lightweight classifiers can learn the class, are supported by the annotation process, the IoU statistics, and the classifier experiments in Table 4 and Figure 2 under both a random split and leave-one-participant-out evaluation. The learnability comparison relies on the ratio of classifier F1 to a uniform-random baseline (Section 3.4.4), which may be statistically biased toward the rarer tap water class; however, that is a metric-validity concern, not a reduction of the prediction to its inputs. The self-citations [2] and [3] are background for hand-washing and sensing and are not load-bearing. No imported uniqueness theorem or ansatz-via-citation pattern is present. Overall score 1 reflects a non-load-bearing self-definitional validation.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities. The 'tap water' label is an annotation of existing audio data, not a postulated object. The central claims rest on the external quality of HD-Epic, standard audio features, and the chosen baseline metric.

free parameters (5)
  • n_estimators for random forest = 200
    Chosen by hand; sklearn default is 100, set to 200 without tuning. Not optimized per class.
  • class_weight for random forest = "balanced"
    Set to balanced to handle imbalance; chosen a priori, not fitted.
  • CNN architecture = not fully specified
    Five 2D convolutional layers with 2x2 max-pooling and two fully connected layers. Layer sizes, number of filters, dropout, and training epochs are not stated, so the exact model is not reproducible from the paper alone.
  • Feature window length = 2 seconds, no overlap
    A fixed choice for both MFCC and log-mel spectrogram extraction; not optimized.
  • Minimum label duration for relabeling = 3 seconds
    Water/pour labels shorter than 3 seconds were not relabeled, covering about 90 percent of water duration. This choice may miss short tap water events.
assumptions (3)
  • domain assumption HD-Epic water and pour labels are sufficiently accurate to serve as the starting point for tap water annotation and for training and evaluation.
    The paper relies on the quality of the original HD-Epic audio annotations, and on the recording environment being representative, without independent verification.
  • domain assumption Log-mel spectrograms and MFCC features capture discriminative information for tap water versus other sounds.
    The paper uses standard audio features, citing prior work; no new feature analysis is provided.
  • domain assumption The 'uniform' dummy classifier provides a meaningful baseline to compare learnability across classes.
    The ratio-to-baseline F1 is the key metric supporting the 'easier to learn' claim; a uniform random predictor's F1 is strongly affected by class prevalence, which may not control for imbalance as intended.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Wearable Tap Water Audio Detection through Subclass Annotation in the HD-Epic Dataset." pith.science (2026). https://pith.science/paper/PFWBYLG2

@misc{pith2026250520788,
  author       = {Pith},
  title        = {Pith review of: Enhancing Wearable Tap Water Audio Detection through Subclass Annotation in the HD-Epic Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PFWBYLG2}},
  note         = {Machine review of arXiv:2505.20788}
}
read the original abstract

Wearable human activity recognition has been shown to benefit from the inclusion of acoustic data, as the sounds around a person often contain valuable context. However, due to privacy concerns, it is usually not ethically feasible to record and save microphone data from the device, since the audio could, for instance, also contain private conversations. Rather, the data should be processed locally, which in turn requires processing power and consumes energy on the wearable device. One special use case of contextual information that can be utilized to augment special tasks in human activity recognition is water flow detection, which can, e.g., be used to aid wearable hand washing detection. We created a new label called tap water for the recently released HD-Epic data set, creating 717 hand-labeled annotations of tap water flow, based on existing annotations of the water class. We analyzed the relation of tap water and water in the dataset and additionally trained and evaluated two lightweight classifiers to evaluate the newly added label class, showing that the new class can be learned more easily.

Figures

Figures reproduced from arXiv: 2505.20788 by the authors.

Figure 1
Figure 1. Two examples per class of the log-mel spectrograms, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Results of classifier training and evaluation for Task A and Task B. F1 score results (upper row) are displayed for the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 18 canonical work pages

  1. [1]

    Arnab Barna, Abdul Kadar Muhammad Masum, Mohammad Emdad Hossain, Erfanul Hoque Bahadur, and Mohammed Shamsul Alam. 2019. A Study on Human Activity Recognition Using Gyroscope, Accelerometer, Temperature and Humidity Data. In 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE) (Cox’sBazar, Bangladesh, 2019-02). IEEE, ...

  2. [2]

    Scholl, Roselind Lieb, Kristof Van Laerhoven, and Karina Wahl

    Robin Burchard, Philipp M. Scholl, Roselind Lieb, Kristof Van Laerhoven, and Karina Wahl. 2022. WashSpot: Real-Time Spotting and Detection of Enacted Com- pulsive Hand Washing with Wearable Devices. InProceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Cambridge United Kingdom, 2022-09-11). ACM, 483–487. doi:1...

  3. [3]

    Robin Burchard and Kristof Van Laerhoven. 2025. Multi-Modal Atmospheric Sens- ing to Augment Wearable IMU-Based Hand Washing Detection. In Sensor-Based Activity Recognition and Artificial Intelligence (Cham, 2025), Orhan Konak, Bert Arnrich, Gerald Bieber, Arjan Kuijper, and Sebastian Fudickar (Eds.). Springer Nature Switzerland, 55–68. doi:10.1007/978-3-...

  4. [4]

    Chieh-Feng Cheng, Abbas Rashidi, Mark Davenport, and David Anderson. 2016. Audio Signal Processing for Activity Recognition of Construction Heavy Equip- ment. doi:10.22260/ISARC2016/0078 Enhancing Wearable Tap Water Audio Detection Through Subclass Annotation in the HD-Epic Dataset

  5. [5]

    Selina Chu, Shrikanth Narayanan, and C.-C. Jay Kuo. 2009. Environmental Sound Recognition With Time–Frequency Audio Features. 17, 6 (2009), 1142–

  6. [6]

    Stefania Cristina, Vladimir Despotovic, Rodrigo Pérez-Rodríguez, and Slavisa Aleksic. 2024. Audio- and Video-Based Human Activity Recognition Systems in Healthcare. 12 (2024), 8230–8245. doi:10.1109/ACCESS.2024.3353138

  7. [7]

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Per- rett, Will Price, and Michael Wray. 2022. Rescaling Egocentric Vision: Collec- tion, Pipeline and Challenges for EPIC-KITCHENS-100. 130, 1 (2022), 33–55. doi:10.1007/s11263-021-01531-2

  8. [8]

    Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra

Show all 33 references
  1. [9]

    Biying Fu, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. 2020. Sensing Technology for Human Activity Recognition: A Comprehensive Survey. 8 (2020), 83791–83820. doi:10.1109/ACCESS.2020.2991891

  2. [10]

    Gemmeke, Daniel P

    Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. 2017. Au- dio Set: An Ontology and Human-Labeled Dataset for Audio Events. In 2017 IEEE International Conference on Acoustics, Speech and Signal...

  3. [11]

    Patrice Guyot, Julien Pinquier, and Régine André-Obrecht. 2012. Water Flow Detection from a Wearable Device with a New Feature, the Spectral Cover. In 2012 10th International Workshop on Content-Based Multimedia Indexing (CBMI) (2012-06). 1–6. doi:10.1109/CBMI.2012.6269814

  4. [12]

    Dade Hayes. 2020. Apple Watch Can Now Tell If You’re Washing Your Hands. Dead- line. https://deadline.com/2020/06/apple-watch-can-tell-if-you-are-washing- your-hands-coronavirus-1202966657/

  5. [13]

    Jaesung Huh, Jacob Chalk, Evangelos Kazakos, Dima Damen, and Andrew Zis- serman. 2023. Epic-Sounds: A Large-Scale Dataset of Actions That Sound. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2023-06). 1–5. doi:10.1109/...

  6. [14]

    Jaesung Huh, Jacob Chalk, Evangelos Kazakos, Dima Damen, and Andrew Zis- serman. 2025. EPIC-SOUNDS: A Large-Scale Dataset of Actions that Sound. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2025)

  7. [15]

    Kyuwoong Hwang and Soo-Young Lee. 2012. Environmental Audio Scene and Activity Recognition through Mobile-Based Crowdsourcing. 58, 2 (2012), 700–705. doi:10.1109/TCE.2012.6227479

  8. [16]

    Seung Ho Hyun. 2023. Sound-Event Detection of Water-Usage Activities Using Transfer Learning. 24, 1 (2023), 22. doi:10.3390/s24010022

  9. [17]

    Apple Inc. 2024. Set up Handwashing on Apple Watch . Apple Support. https: //support.apple.com/en-my/guide/watch/apdc9b9f04a8/watchos

  10. [18]

    Misha Karim, Shah Khalid, Aliya Aleryani, Jawad Khan, Irfan Ullah, and Zafar Ali. 2024. Human Action Recognition Systems: A Review of the Trends and State-of-the-Art. 12 (2024), 36372–36390. doi:10.1109/ACCESS.2024.3373199

  11. [19]

    Dawei Liang and Edison Thomaz. 2019. Audio-Based Activities of Daily Living (ADL) Recognition with Large-Scale Acoustic Embeddings from Online Videos. 3, 1 (2019), 1–18. doi:10.1145/3314404 arXiv:1810.08691 [cs]

  12. [20]

    Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, Omar Emara, Sam Pollard, Kranti Parida, Kaiting Liu, Prajwal Gatti, Siddhant Bansal, Kevin Flanagan, Ja- cob Chalk, Zhifan Zhu, Rhodri Guerrier, Fahd Abdelazim, Bin Zhu, Davide Moltisanti, Michael Wray, Hazel Doughty, and Dima Da...

  13. [21]

    Karol J. Piczak. 2015. ESC: Dataset for Environmental Sound Classification. In Proceedings of the 23rd ACM International Conference on Multimedia (Brisbane Australia, 2015-10-13). ACM, 1015–1018. doi:10.1145/2733373.2806390

  14. [22]

    Garcia, Nuno Pombo, and Francisco Flórez-Revuelta

    Ivan Miguel Pires, Nuno M. Garcia, Nuno Pombo, and Francisco Flórez-Revuelta

  15. [23]

    Arjun Prashanth, S. L. Jayalakshmi, and R. Vedhapriyavadhana. 2024. A Review of Deep Learning Techniques in Audio Event Recognition (AER) Applications. 83, 3 (2024), 8129–8143. doi:10.1007/s11042-023-15891-z

  16. [24]

    Soonshin Seo, Changmin Kim, and Ji-Hwan Kim. 2022. Convolutional Neural Networks Using Log Mel-Spectrogram Separation for Audio Event Classification with Unknown Devices. 21, 2 (2022), 497–522. doi:10.13052/jwe1540-9589.21216

  17. [25]

    Babak Taati, Jasper Snoek, David Giesbrecht, and Alex Mihailidis. 2010. Water Flow Detection in a Handwashing Task. In2010 Canadian Conference on Computer and Robot Vision (2010-05). 175–182. doi:10.1109/CRV.2010.30

  18. [26]

    Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2019. Deep Learning for Sensor-Based Activity Recognition: A Survey. 119, C (2019), 3–11. doi:10.1016/j.patrec.2018.02.010

  19. [27]

    Ah Hyun Yuh and Soon Ju Kang. 2021. Real-Time Sound Event Classification for Human Activity of Daily Living Using Deep Neural Network. In 2021 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Comput- ing & Communications (GreenCom) and IEEE Cyber, ...

  20. [28]

    Shibo Zhang, Yaxuan Li, Shen Zhang, Farzad Shahabi, Stephen Xia, Yu Deng, and Nabil Alshurafa. 2022. Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances. 22, 4 (2022), 1476. Issue 4. doi:10.3390/s22041476

  21. [29]

    Yunhua Zhang, Hazel Doughty, Ling Shao, and Cees G. M. Snoek. 2022. Audio- Adaptive Activity Recognition Across Video Domains. In 2022 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) (New Orleans, LA, USA, 2022-06). IEEE, 13781–13790. doi:10.1109/CVPR52...

  22. [30]

    Haoyu Zhuang, Liqiang Xu, Yuuki Nishiyama, and Kaoru Sezaki. 2023. Detecting Hand Hygienic Behaviors In-the-Wild Using a Microphone and Motion Sensor on a Smartwatch. In Distributed, Ambient and Pervasive Interactions (Cham, 2023) (Lecture Notes in Computer Science) , Norbert ...

  23. [1158]

    doi:10.1109/TASL.2009.2017438

  24. [2017]

    doi:10.48550/arXiv.1711

    User Environment Detection with Acoustic Sensors Embedded on Mobile Devices for the Recognition of Activities of Daily Living . doi:10.48550/arXiv.1711. 00124 arXiv:1711.00124 [cs]

  25. [2022]

    30 (2022), 829–852

    FSD50K: An Open Dataset of Human-Labeled Sound Events. 30 (2022), 829–852. doi:10.1109/TASLP.2021.3133208

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.