DeviceScope: An Interactive App to Detect and Localize Appliance Patterns in Electricity Consumption Time Series

Adrien Petralia; Paul Boniol; Philippe Charpentier; Themis Palpanas

REVIEW 3 major objections 5 minor 13 references

DeviceScope: An Interactive App to Detect and Localize Appliance Patterns in Electricity Consumption Time Series

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DeviceScope and its CamAL method claim that individual appliance patterns in aggregate smart meter data can be detected and localized using only weak possession labels, reaching the accuracy of strongly supervised NILM with far fewer…

desk verdict The system demo is coherent, but the paper's central claim—that CamAL localizes appliances from possession labels alone—is not supported by the experiments as described, so the headline numbers should be read with skepticism until the labeling details and a control experiment are clarified. read the letter →

arxiv 2506.05912 v1 pith:KHCIV3SK submitted 2025-06-06 cs.LG eess.SP

Adrien Petralia , Paul Boniol , Philippe Charpentier , Themis Palpanas This is my paper

classification cs.LGeess.SP

keywords non-intrusiveloadmonitoringweaklysupervisedlearningclassactivationmapsappliancedetectionlocalizationsmartmeterdatatimeseriesclassificationinteractivevisualization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DeviceScope is an interactive tool for browsing household electricity consumption, and CamAL is the weakly supervised method that powers it. CamAL is trained only on a per-window or per-house label saying whether an appliance exists or was used, not on per-timestamp appliance power. It localizes when the appliance was active by averaging Class Activation Maps from an ensemble of residual-network classifiers and applying that average as an attention mask on the input. The paper reports that this beats the only other weakly supervised baseline by 2.2x in F1 and matches strong-supervised NILM accuracy with 5200x fewer labels in the reported Dishwasher case. If true, appliance-level insight can be extracted from the flood of smart meter data without instrumenting individual appliances.

What carries the argument

The machinery is the ensemble Class Activation Map (CAM). For a window $x$, each ResNet in the ensemble (kernel sizes $5,7,9,15$, selected per appliance) produces $\mathrm{CAM}_n(t) = \sum_k w^k_c\, f^k(t)$, where $w^k_c$ are the weights for class $c$ and $f^k(t)$ is the activation of the $k$-th feature map at time $t$. Each CAM is normalized to $[0,1]$, the normalized maps are averaged to $\mathrm{CAM}_{\mathrm{avg}}(t)$, that average is multiplied pointwise by the input $x(t)$ as an attention mask, passed through a sigmoid, and thresholded at $0.5$ to produce the predicted binary appliance status $\hat{y}(t)$. This converts a window-level detection decision into per-timestamp attributions, with the ensemble and attention multiplication suppressing the background aggregate consumption.

What would settle it

Take a house with submetered ground truth from UKDALE or REFIT, train CamAL only on window-level labels from other houses, and compare its thresholded binary status against the true appliance on-intervals timestamp by timestamp; if localization F1 is no better than a baseline that marks the whole window as active, or if it collapses on low-power continuous devices, the central localization claim fails.

Watch

Extended reading notes

Core claim

CamAL's central claim is that appliance detection and localization in aggregate electricity series can be solved without strong labels. A binary classifier ensemble is trained only on window-level labels; when the ensemble detects the appliance in a window, the normalized class activation maps of all models are averaged, multiplied point-wise by the input consumption, passed through a sigmoid, and thresholded at 0.5 to produce a binary status per timestamp. The paper presents this as the first explainability-based NILM approach and reports that on the Dishwasher case of the IDEAL dataset it achieves 2.2x higher F1 than the weakly supervised baseline at the same label count, and uses 5200x fewer training labels than strong-supervised seq2seq NILM baselines at equal accuracy.

Load-bearing premise

The method assumes that the averaged class activation maps of a classifier trained only on window-level labels point exactly at the timestamps when the target appliance is actually on, rather than at correlated background consumption or other appliances.

Editorial extensions

If this is right

Appliance localization no longer requires per-timestamp ground truth; possession surveys or window-level presence labels are enough for training.
The label bottleneck that limits NILM deployments is relaxed by orders of magnitude, by a factor of 5200x in the reported Dishwasher case.
Explainability methods become a practical route to NILM, not merely post-hoc interpretation tools.
Tools like DeviceScope can show consumers and suppliers which appliances run and when, directly from aggregate meter data.
Because training and testing use distinct houses, the method is intended to transfer across households without submetering every home.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit is that CamAL's predicted binary status could be used as weak supervision to bootstrap stronger NILM models, or its attention weights could serve as soft targets.
The reported 2.2x and 5200x factors come from one appliance (Dishwasher) on one dataset; the same margins are not demonstrated for all five appliances, so generalizing the exact factors to kettles, microwaves, washing machines, or showers is an extrapolation.
Because localization inherits the classifier's discriminative signal, appliances with low power or long flat consumption profiles may produce fragmented on/off predictions under the 0.5 threshold, which is a testable prediction for future evaluation.
The paper alternates between true possession-only labels (IDEAL survey) and window-presence labels derived from disaggregated data (UKDALE, REFIT); these two regimes may behave differently, and the possession-only case is the hardest and most deployment-relevant.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take for your desk: this is a demo paper for DeviceScope, a Streamlit app that detects and localizes appliances in aggregate electricity data using an ensemble of ResNets plus averaged Class Activation Maps. The novel bit is real: it is the first application of CAM-style explainability to NILM localization, and the ensemble-plus-attention combination is not in the cited literature. The system itself is a credible contribution for a demo track, and the use of three public datasets (UKDALE, REFIT, IDEAL) is a plus.

The soft spot is the one the stress-test flagged, and it is load-bearing. The abstract says CamAL needs only possession labels, but Section II-A tells a different story: for UKDALE and REFIT the training labels are window-level presence labels derived from disaggregated appliance loads, which carry temporal variation. Only IDEAL uses true possession labels. The IDEAL result in Fig. 3 is the sole support for the "possession only" claim, and it is one appliance, one dataset, one run, no error bars. A classifier trained on possession labels alone has no direct temporal signal; an ideal CAM would highlight whatever separates owning from non-owning households, which could be baseline consumption or correlated appliances rather than actual activation intervals. The paper needs either a control experiment showing that possession-trained CAMs localize correctly, or a clear statement that the reported results use the stronger presence labels. Without that, the 2.2x and 5200x numbers are not interpretable.

Other issues are minor in comparison. There is no code or training detail beyond the architecture sketch; thresholds and ensemble size look hand-picked; and the evaluation is a single figure rather than per-appliance tables. For a demo paper, that is acceptable if the claims are dialed back. The self-citations are fine—earlier appliance detection work is the natural baseline.

Bottom line: I would not cite the quantitative claims in their current form, but the demo deserves a serious referee. Send it to review, conditionally, with the requirement that the authors clarify the exact supervision used for each dataset and add at least one control separating possession labels from presence labels. If that means more experiments than a demo paper can carry, they should retitle the claim accordingly.

Referee Report

3 major / 5 minor

Summary. The paper presents DeviceScope, an interactive web application for visualizing electricity consumption time series and for detecting and localizing appliance usage from aggregate smart-meter data. The underlying method, CamAL, trains an ensemble of residual-network classifiers on weak labels (appliance possession for IDEAL, window-level presence for UKDALE/REFIT), then extracts and averages class activation maps, masks the input signal, and thresholds the result to produce a binary per-timestamp appliance status prediction. The authors report that CamAL achieves 2.2x better F1 than a weakly supervised baseline and requires 5200x fewer labels than strongly supervised NILM methods on the dishwasher case of the IDEAL dataset. The paper also describes three demonstration scenarios for users to explore the data and compare CamAL against six baselines.

Significance. If the central claim holds, CamAL would be a genuinely useful step toward appliance localization with minimal annotation cost, which is relevant for smart-meter analytics and NILM research. The paper has clear strengths: it provides a live interactive demo (devicescope.streamlit.app), uses three public datasets, and compares against several recent NILM baselines. However, the empirical evidence currently consists of a single figure for one appliance and one dataset, with no error bars, no per-appliance or per-dataset tables, and no ablation study. The label discrepancy between the possession-only claim and the window-presence labels used for two of the three datasets is a load-bearing issue that needs to be resolved before the main claim is credible.

major comments (3)

[II-A] The paper claims that CamAL 'only requires the knowledge of the existence of an appliance in a household to be trained', but the IDEAL training protocol assigns the same possession label to every subsequence of an owning household. A classifier trained on such labels can only separate owning from non-owning households; its CAMs are not anchored to actual activation intervals. The reported localization results on the IDEAL dishwasher case in Fig. 3 therefore need a control experiment to establish that the localization is driven by appliance activations rather than by baseline consumption or correlated appliances. Since the UKDALE and REFIT protocols use window-level presence labels derived from disaggregated data, the paper must report results separately for possession-only and window-presence training, or explicitly weaken the claim to window-presence supervision.
[II-C, Fig. 3] The quantitative evidence is limited to one figure for one appliance (dishwasher, IDEAL). No per-appliance or per-dataset tables are given, no standard deviations or multiple runs are reported, and the exact evaluation protocol is not specified (e.g., window length, overlap, and how predicted binary status is matched to ground-truth appliance state for F1 computation). The 2.2x and 5200x figures may reflect a single favorable configuration. Please add full results for all five appliances across all three datasets, with error bars, and state the evaluation protocol precisely.
[II-B, Steps 5-6] The localization formula s(t) = Sigmoid(CAM_avg(t) ∘ x(t)), followed by rounding at 0.5, is introduced without justification or comparison to simpler alternatives such as thresholding CAM_avg directly or applying Grad-CAM weights. The localization threshold, detection threshold, and ensemble architecture are hand-picked, so a sensitivity analysis or ablation is needed to show that the reported results are not artifacts of these choices.

minor comments (5)

[II-A] The phrase 'using only one label for an entire series' is imprecise because each subsequence receives its own label; please clarify that the label is constant across all subsequences for the IDEAL possession protocol.
[Fig. 1] It is unclear whether the colored intervals in the illustrative figure are ground truth or CamAL predictions; please add a caption explaining the visual encoding.
[Fig. 3] The markers for the seven methods are not all identified in the caption, and the '2.2x' and '5200x' annotations should be explained in the caption or text.
[II-A] The sentence 'we selected the networks that best detected specific appliances' does not state a selection criterion; please specify how the ensemble members are chosen and whether this selection is done on a validation set.
[Abstract] The statement 'This paper appeared in ICDE 2025' belongs in a footnote or camera-ready notice, not in the abstract of a submission under review.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CamAL's localization is a post-hoc CAM heuristic, not a relabeled copy of its training inputs, and the few self-citations are not load-bearing.

full rationale

The derivation chain in CamAL is not circular. Training uses weak window-level labels (possession for IDEAL; presence/absence for UKDALE/REFIT), and the localization output is produced by thresholding a sigmoid of the averaged class activation map multiplied by the input. The CAM is a function of the trained classifier's parameters, not a direct transcription of the training labels, so the predicted appliance status is not equivalent to the input labels by construction. The paper's self-citations ([2], [5], [8]) justify architectural choices and the CAM formulation, but the CAM concept itself is attributed to external prior work ([9], [10]), and the cited results are empirically falsifiable rather than definitions of the central claim. Figure 3's 2.2x/5200x numbers are experimental comparisons, not fitted-input predictions. One genuine concern, noted by the skeptic, is that for IDEAL each subsequence receives the same possession label, so a classifier trained on those labels may not contain temporal information about appliance activation; but this is an overclaim or validity gap about whether the weak supervision is sufficient, not a circular reduction. Similarly, the abstract's 'only requires knowledge of existence' is stronger than the window-level presence labels actually used for UKDALE/REFIT, but again this is a scope mismatch rather than a self-referential derivation. No step in the paper reduces, by equation or by self-citation, to its own input, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the CAM-localization assumption, hand-chosen thresholds and ensemble architecture, and on the use of derived presence labels as a proxy for possession labels. No new physical or ontological entities are introduced; DeviceScope and CamAL are software artifacts.

free parameters (3)

Localization threshold = 0.5
Used to convert the attention signal s(t) into binary appliance status via rounding; no calibration or sensitivity analysis is reported in Section II-B.
Detection threshold = 0.5
Used to decide whether the ensemble probability indicates the appliance is detected; no tuning is reported in Section II-B.
Ensemble architecture choice = 5 models, kernel sizes {5,7,9,15}
The number of ResNet models and kernel sizes are chosen by hand, with selection based on detection performance rather than localization performance, as described in Section II-A.

assumptions (3)

domain assumption CAM relevance from weak-label classifiers localizes actual appliance activation intervals
The whole localization pipeline depends on this; no proof or ablation is provided, and it is exactly what the demo is intended to show.
ad hoc to paper A sigmoid of the pointwise product of normalized average CAM and the raw signal is a valid probability of appliance status
No statistical or calibration justification is given for this mapping in Section II-B; the threshold of 0.5 is arbitrary.
domain assumption Window-level presence labels carry enough information to learn appliance-specific discriminative features
This is the premise of weak supervision; for UKDALE and REFIT these labels are derived from measured disaggregated load curves in Section II-A, which is stronger than pure possession labels.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DeviceScope: An Interactive App to Detect and Localize Appliance Patterns in Electricity Consumption Time Series." pith.science (2026). https://pith.science/paper/KHCIV3SK

@misc{pith2026250605912,
  author       = {Pith},
  title        = {Pith review of: DeviceScope: An Interactive App to Detect and Localize Appliance Patterns in Electricity Consumption Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KHCIV3SK}},
  note         = {Machine review of arXiv:2506.05912}
}

read the original abstract

In recent years, electricity suppliers have installed millions of smart meters worldwide to improve the management of the smart grid system. These meters collect a large amount of electrical consumption data to produce valuable information to help consumers reduce their electricity footprint. However, having non-expert users (e.g., consumers or sales advisors) understand these data and derive usage patterns for different appliances has become a significant challenge for electricity suppliers because these data record the aggregated behavior of all appliances. At the same time, ground-truth labels (which could train appliance detection and localization models) are expensive to collect and extremely scarce in practice. This paper introduces DeviceScope, an interactive tool designed to facilitate understanding smart meter data by detecting and localizing individual appliance patterns within a given time period. Our system is based on CamAL (Class Activation Map-based Appliance Localization), a novel weakly supervised approach for appliance localization that only requires the knowledge of the existence of an appliance in a household to be trained. This paper appeared in ICDE 2025.

Figures

Figures reproduced from arXiv: 2506.05912 by the authors.

**Figure 2.** Illustration of the proposed CamAL approach. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 4.** illustrates the inputs and features of DeviceScope. The (publicly available) datasets considered in our system are UKDALE, REFIT, and IDEAL [11]–[13] (though, users could upload other datasets, as well). Each dataset comprises several houses monitored by sensors that record the total main and appliance-level power for a period of time (used only during evaluation). We are interested in detecting five common applian… view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Illustration of the main frames of DeviceScope. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

[1]

Petralia, P

A. Petralia, P. Boniol, P. Charpentier, and T. Palpanas. DeviceScope. [Online]. Available: https://devicescope.streamlit.app/

work page
[2]

Appliance detection using very low-frequency smart meter time series,

A. Petralia, P. Charpentier, P. Boniol, and T. Palpanas, “Appliance detection using very low-frequency smart meter time series,” ine-Energy ’23, 2023

work page 2023
[3]

A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (nilm) in residential context,

H. Rafiq, P. Manandhar, E. Rodriguez-Ubinas, O. Ahmed Qureshi, and T. Palpanas, “A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (nilm) in residential context,”Energy and Buildings, 2024

work page 2024
[4]

Electricity demand ac- tivation extraction: From known to unknown signatures, using similarity search,

P. Laviron, X. Dai, B. Huquet, and T. Palpanas, “Electricity demand ac- tivation extraction: From known to unknown signatures, using similarity search,” ine-Energy ’21, 2021

work page 2021
[5]

Adf & transapp: A transformer-based framework for appliance detection using smart meter consumption series,

A. Petralia, P. Charpentier, and T. Palpanas, “Adf & transapp: A transformer-based framework for appliance detection using smart meter consumption series,”PVLDB, vol. 17, no. 3, 2023

work page 2023
[6]

Bake off redux: a review and experimental evaluation of recent time series classification algo- rithms,

M. Middlehurst, P. Sch ¨afer, and A. Bagnall, “Bake off redux: a review and experimental evaluation of recent time series classification algo- rithms,”Data Min. Knowl. Discov., vol. 38, 2024

work page 2024
[7]

Time series classification from scratch with deep neural networks: A strong baseline,

Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,”IJCNN, 2016

work page 2016
[8]

Dcam: Dimension- wise class activation map for explaining multivariate data series classi- fication,

P. Boniol, M. Meftah, E. Remy, and T. Palpanas, “Dcam: Dimension- wise class activation map for explaining multivariate data series classi- fication,” inSIGMOD ’22, 2022

work page 2022

Show all 13 references

[9]

Learning deep features for discriminative localization,

B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” inCVPR, 2016

2016
[10]

Grad-cam: Visual explanations from deep networks via gradient-based localization,

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in2017 ICCV, 2017

2017
[11]

The uk-dale dataset, domestic appliance- level electricity demand and whole-house demand from five uk homes,

J. Kelly and W. Knottenbelt, “The uk-dale dataset, domestic appliance- level electricity demand and whole-house demand from five uk homes,” Scientific Data, vol. 2, 2015

2015
[12]

REFIT Smart Home dataset,

S. Firth, T. Kane, V . Dimitriou, T. Hassan, F. Fouchal, M. Coleman et al., “REFIT Smart Home dataset,” 2017

2017
[13]

The ideal household energy dataset, electricity, gas, contextual sensor data and survey data for 255 uk homes,

M. Pullinger, J. Kilgour, N. Goddard, N. Berliner, L. Webb, M. Dzikovska, H. Lovell, J. Mann, C. Sutton, J. Webb, and M. Zhong, “The ideal household energy dataset, electricity, gas, contextual sensor data and survey data for 255 uk homes,”Scientific Data, 2021

2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Petralia, P

A. Petralia, P. Boniol, P. Charpentier, and T. Palpanas. DeviceScope. [Online]. Available: https://devicescope.streamlit.app/

work page

[2] [2]

Appliance detection using very low-frequency smart meter time series,

A. Petralia, P. Charpentier, P. Boniol, and T. Palpanas, “Appliance detection using very low-frequency smart meter time series,” ine-Energy ’23, 2023

work page 2023

[3] [3]

A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (nilm) in residential context,

H. Rafiq, P. Manandhar, E. Rodriguez-Ubinas, O. Ahmed Qureshi, and T. Palpanas, “A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (nilm) in residential context,”Energy and Buildings, 2024

work page 2024

[4] [4]

Electricity demand ac- tivation extraction: From known to unknown signatures, using similarity search,

P. Laviron, X. Dai, B. Huquet, and T. Palpanas, “Electricity demand ac- tivation extraction: From known to unknown signatures, using similarity search,” ine-Energy ’21, 2021

work page 2021

[5] [5]

Adf & transapp: A transformer-based framework for appliance detection using smart meter consumption series,

A. Petralia, P. Charpentier, and T. Palpanas, “Adf & transapp: A transformer-based framework for appliance detection using smart meter consumption series,”PVLDB, vol. 17, no. 3, 2023

work page 2023

[6] [6]

Bake off redux: a review and experimental evaluation of recent time series classification algo- rithms,

M. Middlehurst, P. Sch ¨afer, and A. Bagnall, “Bake off redux: a review and experimental evaluation of recent time series classification algo- rithms,”Data Min. Knowl. Discov., vol. 38, 2024

work page 2024

[7] [7]

Time series classification from scratch with deep neural networks: A strong baseline,

Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,”IJCNN, 2016

work page 2016

[8] [8]

Dcam: Dimension- wise class activation map for explaining multivariate data series classi- fication,

P. Boniol, M. Meftah, E. Remy, and T. Palpanas, “Dcam: Dimension- wise class activation map for explaining multivariate data series classi- fication,” inSIGMOD ’22, 2022

work page 2022