Pith. sign in

REVIEW 4 major objections 6 minor 2 references

An Explainable AI based approach for Monitoring Animal Health

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Neck sensors plus KNN read cow behavior at 0.99 test AUC

desk verdict A useful applied ML recipe for cow behavior classification, but the reported test AUC of 0.99 is probably inflated by temporal leakage from overlapping windows and pre-split lag features; the data pipeline and model comparison are solid enough to deserve a serious referee, though the numbers need to be re-run. read the letter →

arxiv 2508.10210 v3 pith:Y2LEAXVW submitted 2025-08-13 cs.LG cs.AI

Rahul Jana , Shubham Dixit , Mrityunjay Sharma , Ritesh Kumar This is my paper · ORCID
classification cs.LGcs.AI
keywords cattlebehaviorclassificationaccelerometerwearablesensorsk-nearestneighborsexplainableAISHAPfeaturestabilityprecisionlivestockfarming
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports that a k-nearest-neighbor classifier, trained on features extracted from neck-worn tri-axial accelerometer windows, can separate four merged classes of dairy cow behavior—standing/moving, ruminating, resting while lying, and miscellaneous activities—with a test AUC of 0.99 on a 12-cow farm. The authors combine statistical, wavelet, orientation, and lag-based features from sliding windows and use SHAP to explain which accelerometer axes drive each class. They argue this explainable model gives farmers actionable health and activity insights without needing to watch animals continuously. The central claim is that this specific feature-and-classifier configuration achieves near-perfect discrimination on this dataset.

What carries the argument

The sliding-window feature-extraction pipeline is the central mechanism: raw three-axis accelerometer samples are compressed into minutes, then a window of 156 samples with step 39 yields statistical moments (mean, variance, skewness, kurtosis, quantiles), discrete wavelet transform coefficients, orientation angles (roll, pitch), and lag features up to five previous points. These feature vectors are classified by k-nearest neighbors, and SHAP attributions identify which axis-specific features carry each behavior's signal.

What would settle it

Retrain and retest the same KNN pipeline using a split by contiguous time blocks (e.g., morning versus afternoon) or by cow identity rather than random windows; if the test AUC falls well below 0.99, overlapping windows rather than a general behavior signal drove the score.

Watch

Extended reading notes

Core claim

The paper claims that KNN with 3 neighbors, Manhattan distance, and distance weighting, using a window of 156 samples and a step of 39, achieves a testing AUC of 0.99, accuracy of 0.9389, precision of 0.9404, recall of 0.9322, and F1 of 0.9360 when classifying four behavior classes constructed from ten raw annotated behaviors. The authors find that merging minority classes into an ETC (miscellaneous) class and standing plus moving into STN outperforms both undersampling and SMOTE oversampling. SHAP analysis shows that for standing, X-axis features dominate; for resting while lying, Y-axis features dominate; and for rumination, X- and Y-axis features matter most, aligning with the physical or

Load-bearing premise

The result assumes that randomly splitting overlapping 156-sample windows into train and test keeps the observations independent; because windows share most raw accelerometer samples, this premise is doubtful and the 0.99 test AUC likely overstates generalization to new continuous recordings.

Editorial extensions

If this is right

  • The four-class scheme makes the problem tractable and outperforms the originally imbalanced ten-class setup.
  • KNN with the specified window and step sizes generalizes to unseen test windows with AUC 0.99 in this farm setting.
  • SHAP and KS-stability analysis yield a ranked, axis-specific feature set that can guide practitioners in interpreting accelerometer readings.
  • The pipeline can be extended to other accelerometer-based activity recognition tasks, as the authors state in conclusion.
  • Deviations in daily activity ratios could flag cattle health or welfare anomalies for farmers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the data were split by contiguous time blocks or by cow identity instead of random windows, the test AUC might drop substantially because consecutive windows heavily overlap and share raw samples.
  • The ETC class merges six different behaviors, so the model's ability to distinguish licking from attacking or drinking is not actually tested.
  • The prominence of lag features in SHAP suggests temporal dynamics carry much of the signal; evaluating KNN without lag features would isolate their contribution.
  • The near-perfect result is from one 12-cow farm and should not be read as proof of performance across breeds, seasons, or farm layouts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents an end-to-end pipeline for classifying dairy-cow behavior from neck-mounted tri-axial accelerometer data. Ten annotated behaviors are merged into four classes (STN, RUS, REL, ETC). Features are extracted from sliding windows (window 156 samples, step 39 for the best configuration) and include statistical, wavelet, Savitzky-Golay, and lag features. Several machine-learning classifiers are hyperparameter-tuned and evaluated. The headline claim is that k-nearest neighbors achieves an AUC of 0.98 ± 0.0026 on training and 0.99 on the test set, with test accuracy 0.9389, precision 0.9404, recall 0.9322, and F1 0.9360 (Tables 4–5, Figures 4c–4d). SHAP analysis and KS-based feature stability are used to interpret the model. The paper is an application-oriented study emphasizing pre-processing, model selection, and explainability.

Significance. If the reported performance were obtained under a clean evaluation protocol, the paper would provide a useful, low-cost tool for automated cattle behavior monitoring and a concrete example of explainable AI for agricultural IoT. The authors deserve credit for releasing code, specifying the annotation pipeline with multiple cross-checks, reporting training-set standard deviations, and testing several window lengths and classifiers. However, the central quantitative claim is not yet substantiated because the train/test split is performed after constructing heavily overlapping feature windows with lag features, which can leak information across the split. The SHAP and stability analyses inherit the same problem. The contribution is therefore conditional on a leakage-free re-evaluation.

major comments (4)
  1. [§2.5.2, Step 2; Algorithm 1; Tables 4–5] The evaluation protocol is vulnerable to temporal leakage. The dataset is split randomly into 60/20/20 after feature extraction, but the feature matrix is built by sliding a 156-sample window with step 39 (75% overlap) and lag features up to 5 points are generated on the full time series before the split (Algorithm 1, lines 16–17). Consecutive rows of the feature matrix share most raw accelerometer samples, and lagged features reference earlier rows that may fall into a different partition. A random split can therefore place near-duplicate windows in both training and testing, inflating all test metrics. The reported test AUC of 0.99 exceeding the training AUC of 0.98 ± 0.0026 is a diagnostic warning consistent with this leakage. The claim in Step 3 that leakage was prevented is not sufficient given this protocol. The authors should re-run the evaluation using a split that respects tempo
  2. [§2.5.2, Table 5] Test-set metrics are reported as single point estimates with no standard deviation, confidence interval, or significance testing. Since the best model is selected using these numbers, and the margins between KNN and other models are small (e.g., 0.9389 vs. 0.9327 accuracy), the reader cannot assess whether the reported ordering is meaningful. The authors should report test metrics with uncertainty, e.g., from repeated stratified splits or bootstrapping, or from the cross-validation folds used for hyperparameter selection.
  3. [§2.6, Figures 5a–5d, Table 6] The SHAP feature-importance analysis and the KS-based feature-stability analysis are performed on the same feature matrix and model used in the leaked train/test evaluation. If the feature matrix is contaminated by overlap and lag-derived leakage, the resulting SHAP rankings and stability categories are not trustworthy as descriptions of a model that generalizes to new cows or new time periods. The SHAP analysis also uses only 100 randomly sampled values (25 per class) with no repeated sampling or variance estimate; a stable ranking should be demonstrated with multiple random seeds.
  4. [§2.1, §2.4.2; Discussion] The paper claims general applicability to 'monitoring animal health' but the data come from 12 cows on one farm, and the evaluation does not include a per-animal or per-farm split. If the same cow's windows appear in both training and test sets, the reported performance may reflect individual-level memorization rather than behavior recognition across animals. A leave-one-cow-out evaluation or an explicit analysis of between-cow variation is needed to support the stated generalization. This is a load-bearing limitation for the practical claim in the abstract and Discussion.
minor comments (6)
  1. [Abstract and §3] Minor syntax: '0.99 on testing set)' appears in the abstract and is repeated in the Discussion. Also, the phrase 'AUC of mean 0.98' should be 'mean AUC of 0.98'.
  2. [Figure 4] Figure 4a and 4b have identical captions ('Model Performance Analysis for Training Data'); one likely corresponds to the test set or to a different model comparison. The captions should be corrected.
  3. [§2.4.1] The sentence 'First, we compress a minute’s worth of data into a single data point while extracting base features(converting 80 data points to 1)' is unclear because the sampling rate of the accelerometer is never stated. Without the sampling rate, the reader cannot map window lengths in samples (e.g., 156) to seconds, which is needed to interpret the results and reproduce the pipeline.
  4. [Algorithm 1] Algorithm 1 uses 'AcX, AcY, AcZ' in line 9 while other lines use 'AccX, AccY, AccZ'. Please standardize notation. Also, line 16 says 'Generate LAG Features' but the exact data structure shifted (full time series vs. per-window features) should be made explicit.
  5. [Table 3] Table 3 lists hyperparameters for only five model families, while §2.5.3 mentions Logistic Regression, Decision Tree, and Naive Bayes as also evaluated. The results for these models are not shown. Either add them or clarify that they were excluded from the comparison.
  6. [§9] Data availability says 'on request' with no clear repository or access policy. Given the reproducibility emphasis, a permanent data DOI or a detailed data-sharing plan is preferable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: the model evaluation is a genuine train/test exercise and the SHAP analysis is post-hoc explanation, not a fitted prediction.

full rationale

The paper's central derivation chain is: raw 3-axis accelerometer data -> feature extraction (Algorithm 1, including sliding-window and lag features) -> a 60/20/20 train/validation/test split with the test set explicitly segregated before hyperparameter search (Section 2.5.2, Steps 2-3) -> hyperparameter-tuned model training -> test-set evaluation (Tables 4-5, Figure 4). The reported KNN test metrics are computed on a held-out split and are not obtained by fitting a parameter to the test set and then renaming it a prediction. No step reduces to its inputs by construction: the window/step sizes and hyperparameters are selected by grid search on training/validation data, and the final test metrics are out-of-sample evaluations. The SHAP feature attributions (Section 2.6.1) are computed from the already-fitted model on a random sample of 100 values to explain model predictions; they are explicitly post-hoc ('explain the models performance'), not a first-principles derivation of biological causality. The KS stability analysis (Section 2.7) is descriptive of the same features, not a predictive claim. There are no load-bearing self-citations: references to SHAP (Lundberg and Lee, 2017), scikit-learn, and standard ML methods are independent, machine-checkable or externally reproducible tools. A possible concern is that the random split over 75%-overlapping windows with lag features generated before splitting may cause train/test leakage, which could inflate the reported test AUC of 0.99; however, data leakage is a validity/correctness issue, not circularity under the defined taxonomy. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on four hand-picked pipeline choices (class merging strategy, window/step, DWT level, lag depth), the tuned hyperparameters of the winning model, the correctness of the manual annotations, and the adequacy of the handcrafted feature map. No new physical entities are introduced; the 'ETC' class is an aggregation of rare behaviors, not a hidden mechanism.

free parameters (4)
  • KNN hyperparameters = n_neighbors=3, p=1, weights=distance
    Selected by 5-fold grid search on validation (§2.5.2, Table 3); the headline AUC is reported under these values.
  • Window length and step for feature extraction = window=156, step=39
    Chosen as best among scanned configurations (Table 3); all headline numbers are tied to this windowing.
  • Lag depth = 5
    Ad hoc choice in §2.4.1 ('previous 5 data points were used'); stability of the result to this choice is not tested.
  • DWT decomposition level = 3
    Fixed in Algorithm 1 (line 11) without sensitivity analysis.
assumptions (3)
  • domain assumption Video annotations verified by an on-site observer provide correct ground-truth behavior labels.
    Invoked in §2.3 Steps 6-8 and throughout training and evaluation; any systematic annotation error directly corrupts the reported accuracies.
  • domain assumption The handcrafted feature set (statistical, DWT, orientation, lag) preserves all information needed to separate the four merged classes.
    Invoked by Algorithm 1; if the features discard discriminative signal, no classifier can recover it, and the reported AUC is an upper bound on this feature map, not on the raw signal.
  • domain assumption Random stratified splitting yields statistically independent train and test windows.
    Stated in §2.5.2 Step 2. With windows overlapping 75% and lag features computed pre-split, this assumption is likely violated and is load-bearing for the test numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Explainable AI based approach for Monitoring Animal Health." pith.science (2026). https://pith.science/paper/Y2LEAXVW

@misc{pith2026250810210,
  author       = {Pith},
  title        = {Pith review of: An Explainable AI based approach for Monitoring Animal Health},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y2LEAXVW}},
  note         = {Machine review of arXiv:2508.10210}
}
read the original abstract

Monitoring cattle health and optimizing yield are key challenges faced by dairy farmers due to difficulties in tracking all animals on the farm. This work aims to showcase modern data-driven farming practices based on explainable machine learning(ML) methods that explain the activity and behaviour of dairy cattle (cows). Continuous data collection of 3-axis accelerometer sensors and usage of robust ML methodologies and algorithms, provide farmers and researchers with actionable information on cattle activity, allowing farmers to make informed decisions and incorporate sustainable practices. This study utilizes Bluetooth-based Internet of Things (IoT) devices and 4G networks for seamless data transmission, immediate analysis, inference generation, and explains the models performance with explainability frameworks. Special emphasis is put on the pre-processing of the accelerometers time series data, including the extraction of statistical characteristics, signal processing techniques, and lag-based features using the sliding window technique. Various hyperparameter-optimized ML models are evaluated across varying window lengths for activity classification. The k-nearest neighbour Classifier achieved the best performance, with AUC of mean 0.98 and standard deviation of 0.0026 on the training set and 0.99 on testing set). In order to ensure transparency, Explainable AI based frameworks such as SHAP is used to interpret feature importance that can be understood and used by practitioners. A detailed comparison of the important features, along with the stability analysis of selected features, supports development of explainable and practical ML models for sustainable livestock management.

Figures

Figures reproduced from arXiv: 2508.10210 by the authors.

Figure 1
Figure 1. Farm and Cow’s View: (a) Inside view of the farm, (b) Outside view of the farm, (c) Node on the cow’s neck, (d) Sample annotations. (DRN), Urinating (URI), Moving (MOV), Resting in a standing position (RES), Resting in a lying position (REL) and Ruminating in a standing position(RUS) mentioned in table 1. This carefully compiled dataset comprises a data file containing four columns, where the first three represent a… view at source ↗
Figure 2
Figure 2. Accelerometer Reading for Different Activities: (a) Standing, (b) Resting in Lying, (c) Ruminating, (d) Miscellaneous Activity. Algorithm 1 Feature Extraction for Time-Series Data Require: data f rame, step length, window length Ensure: Extract features from DataFrame 1: Initialize f eatures d f = pandas.DataFrame 2: Set deviceID, labels ← data f rame[deviceId, Label] 3: range end ← length of data f rame − window le… view at source ↗
Figure 3
Figure 3. Data collection and annotation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accelerometer Reading for Different Activities: (a) Model Performance Analysis for Training Data. (b) Model Performance Analysis for Training Data. (c) ROC-AUC Curve(0.98 ± 0.0026) for Training Set for best performing KNN. (d) ROC-AUC Curve(0.99) for Testing Set for be…
Figure 5
Figure 5. Figure 5: (a) SHAP Analysis on Standing class (b) Confusion Matrix (c) Top SHAP Features and Average SHAP Values (d) KS Statistics Results [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [2]

    URL: https://pib.gov.in/PressReleasePage.aspx?PRID=2086052

    Press release. URL: https://pib.gov.in/PressReleasePage.aspx?PRID=2086052. accessed: 2025-03-

    work page 2025

  2. [2025]

    URL: https://scikit-learn.org/stable/modules/grid_ search.html#exhaustive-grid-search

    Grid search — scikit-learn 1.2.2 documentation. URL: https://scikit-learn.org/stable/modules/grid_ search.html#exhaustive-grid-search . accessed: 2025-04-02. Dutta, D., Natta, D., Mandal, S., Ghosh, N.,

    work page 2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.