Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation

Serra Aksoy

REVIEW 3 major objections 4 minor 1 cited by

Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Swin Transformer attains 94% binary and 73% multiclass accuracy on thermal PV fault detection, and XRAI saliency maps align with thermal physics.

desk verdict A usable empirical benchmark with a transparent but confounded comparison; the XRAI 'physics validation' claim is not supported by the evidence. read the letter →

arxiv 2509.07039 v1 pith:PDB4MMNT submitted 2025-09-08 cs.LG cs.CV

Serra Aksoy This is my paper · ORCID

classification cs.LGcs.CV

keywords thermalphotovoltaicfaultdetectioninfraredthermographyvisiontransformersSwinTransformerconvolutionalneuralnetworksXRAIsaliencyanalysisexplainableAIsolarpanelanomaly

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether deep-learning models that detect faults in photovoltaic modules from thermal infrared images base their decisions on the physics of the faults rather than on superficial image cues. It benchmarks two CNNs and two vision transformers on 20,000 labelled infrared images covering normal operation and eleven fault types, and reports that Swin Transformer performs best: 94% binary anomaly-detection accuracy and 73% multiclass fault classification, against 69% for the convolutional models. The non-accuracy claim is interpretability: XRAI saliency maps of correctly classified images show attention to localized hotspots for cell defects, linear thermal paths for diode failures, and thermal boundaries for vegetation shading, which the paper reads as validation that model decisions align with thermal physics. If this holds, operators have a reason to trust automated thermal inspection for high-value electrical faults, while soiling and multi-fault categories mark the limits imposed by thermal image quality.

What carries the argument

The central instrument is XRAI (eXplainable AI for Region-based Analysis), a saliency method that partitions an input image into regions and assigns each region an attribution score, producing heat-map explanations of the model's decision. The comparison itself is carried by four ImageNet-pretrained architectures, with Swin-Tiny's shifted-window hierarchical attention identified as the best-performing mechanism for capturing both localized thermal anomalies and panel-level patterns. The interpretive step that carries the paper's main claim is matching the shapes visible in XRAI maps, such as hotspots, linear paths, and thermal boundaries, to the textbook thermal signatures of each fault clas

What would settle it

Take a held-out set of correctly classified thermal images with expert-annotated fault regions, such as diode locations or hotspot cells. Compare the top-attribution XRAI region with the annotated fault region using intersection-over-union. If mean IoU is near chance across classes, or if removing the top-saliency region leaves the model's prediction unchanged, the claimed physics alignment is not supported. A complementary test would run the same procedure on misclassified or slightly perturbed images and check whether saliency still tracks the known thermal signature.

Watch

Extended reading notes

Core claim

The paper reports that on 20,000 infrared thermal images of photovoltaic modules, spanning one normal class and eleven fault classes, Swin-Tiny achieves the best performance among the four tested architectures: 94% binary anomaly-detection accuracy and 73% multiclass fault-classification accuracy with macro F1 0.70, slightly ahead of ViT-Tiny (94%/72%) and ahead of ResNet-18 and EfficientNet-B0 (93%/69%). It further claims that XRAI saliency maps for correctly classified instances reveal attention patterns consistent with thermal physics: localized round hotspots for cell faults, linear thermal paths at bypass-diode locations, boundary-following attention for vegetation shading, and crack-fo

Load-bearing premise

The load-bearing premise is that a saliency map computed on correctly classified images faithfully reveals which image regions actually drove the model's decision; if saliency is a rough or misleading proxy, the paper's evidence that decisions follow thermal physics collapses.

Editorial extensions

If this is right

If the benchmark holds, centralized utility-scale monitoring can adopt Swin-Tiny for multiclass fault discrimination, while resource-constrained edge deployments can still rely on CNNs for binary anomaly screening.
Electrical faults such as diode, offline-module, and shadowing are reliably detectable, meaning automated inspection can target the failure modes with the highest per-incident energy loss.
Soiling and multi-fault categories will likely require complementary sensing or higher-resolution thermal imaging rather than further architectural tuning on the present data.
The physics-validated saliency procedure offers a transferable template for validating AI decisions in other thermal diagnostics, such as grid equipment, wind turbines, and battery systems.
The study provides a common benchmark for future work comparing convolutional and attention-based models on thermal PV fault data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the physics-alignment claim would be stronger if saliency were examined for misclassified images too; restricting XRAI to correctly classified examples leaves open the possibility that the maps rationalize a decision after the fact.
Editorial inference: a quantitative test is available — compare XRAI's top-attribution regions against independent expert annotations of fault locations (for example, diode bounding boxes) using intersection-over-union; visual inspection alone is not needed.
Editorial inference: if the resolution-bottleneck explanation is right, then upgrading thermal camera resolution or fusing optical imagery should lift soiling F1-scores above the reported 0.20–0.33 range, while changing only the classifier architecture should not.
Editorial inference: the training budgets are not fully matched across architectures (CNNs trained for 10–15 epochs at fixed learning rate; Swin used a cosine warm-restart scheduler), so part of the transformer advantage may be attributable to training configuration rather than architecture alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a usable but flawed empirical study. The paper compares four standard architectures on a public thermal PV dataset and adds an XRAI interpretability analysis. The useful results are the per-class F1 breakdown and the clean negative result on soiling detection. The headline claims — a fair architecture ranking and physics-validated interpretability — do not hold up, because the comparison is confounded and the XRAI analysis is qualitative and selection-biased.

What's new: the specific four-way comparison with XRAI on Raptor-Maps hasn't been published before, and the paper is transparent about its training configurations (Table 3) and class imbalance. That transparency is exactly what exposes the problems.

Soft spots, in order of severity. First, the architecture comparison is not controlled: ResNets run at 128×128 for 10–15 epochs with batch 64 and fixed LR; transformers run at 224×224 for 15 epochs with batch 32, and Swin alone gets CosineAnnealingWarmRestarts. So the 1–4 point accuracy differences can't be attributed to architecture. Second, the XRAI 'physics validation' rests on visual inspection of saliency maps for correctly classified examples only. There are no ground-truth masks, no quantitative alignment metric, no baseline against a random or edge-detector model. The paper itself concedes that multi-fault and soiling attention maps are ambiguous, which undercuts the blanket conclusion that decisions align with thermal physics. Third, the 'first' claim is softened by Le et al.'s higher accuracy on the same dataset.

What's genuinely useful: the soiling F1 of 0.20–0.33 across all models is a worthwhile negative result, and the per-class analysis is a solid reference for practitioners. The work is honest in reporting limitations, just not honest enough to support the abstract's claims.

Who should read it: someone doing applied thermal PV monitoring who wants a baseline and a list of which classes are hard. Not someone looking for reliable architecture guidance or a validated interpretability method.

Recommendation: I'd send this to peer review rather than desk reject — it's a concrete empirical study on a public dataset with a reproducible setup, and the negative result on soiling is useful. But I'd expect the referee to flag the confounded comparison and the overclaimed interpretability, and I'd advise rejection unless the authors add a controlled experiment and a quantitative validation of the saliency claims.

Referee Report

3 major / 4 minor

Summary. The paper benchmarks four deep learning architectures (ResNet-18, EfficientNet-B0, ViT-Tiny, Swin-Tiny) on a 20,000-image thermal infrared dataset for photovoltaic fault detection, using binary anomaly detection and 11-class fault classification. The authors report that Swin Transformer achieves the highest accuracy (94% binary, 73% multiclass) and use XRAI saliency maps to argue that the models learn physically meaningful features such as localized hotspots, linear diode paths, and vegetation boundaries, thereby 'validating' alignment with thermal physics principles. The paper also discusses per-class performance, confusion patterns, and deployment implications.

Significance. If the central claims held, the paper would provide a useful benchmark and an interpretability-validated approach for thermal PV fault detection, a domain where deployment is often limited by lack of trust. The paper has strengths: it uses a public dataset, compares four architectures, reports per-class metrics, and explicitly discusses failure modes (e.g., soiling). However, the contribution as stated—'physics-validated interpretability'—is not supported by the evidence presented. The XRAI analysis is qualitative, selection-biased, and lacks any quantitative or baseline comparison. The architecture comparison is also confounded by unequal training protocols. These issues are load-bearing for the paper's main claims.

major comments (3)

[Section 2.2 / Table 3] The claimed architectural superiority of Swin-Tiny is confounded by the training protocol. Table 3 shows that ResNet-18 and EfficientNet-B0 use 10–15 epochs, batch size 64, input resolution 128×128, and no scheduler, while ViT-Tiny and Swin-Tiny use 15 epochs, batch size 32, input resolution 224×224, and Swin additionally uses CosineAnnealingWarmRestarts. The 1–4 percentage point accuracy differences (e.g., 73% vs. 69% multiclass) cannot be attributed to architecture alone; they may reflect resolution, regularization via scheduler, or optimization differences. The text states these choices were made 'for the sake of comparison consistency,' but they are not consistent. The authors should either train all models under a matched protocol or perform a sensitivity analysis (e.g., varying epochs/resolution per architecture) and report confidence intervals from repeated runs.
[Section 3.5.1 / Section 3.5.4 / Abstract] The XRAI analysis does not constitute 'physics-validated interpretability.' Saliency maps are generated only for correctly classified instances (Section 3.5.1), which prevents any statement about model behavior on errors. No ground-truth masks or quantitative alignment metric are defined for hotspots, diode paths, vegetation boundaries, etc.; the interpretation is purely visual. There is no baseline comparison, so it is unknown whether random or edge-detector models would produce similarly 'physically meaningful' maps. The paper itself concedes in Section 3.5.4 that maps for Diode-Multi, Hot-Spot-Multi, and Cell-Multi are ambiguous, and that Soiling attention 'may be learning to detect soiling by way of secondary thermal responses.' These concessions undercut the blanket conclusion in the Abstract and Section 4.1 that model decisions align with thermal physics. A load-bearing fix require
[Section 2.1 / Section 2.2 / Section 4.3] The manuscript contains contradictory statements about input resolution. Section 2.1 states all images were 'resized to 128×128 pixels'; Section 2.2 says convolutional models 'natively processed images at their original resolution without explicit resizing'; Table 3 lists input resolution 128×128 for CNNs and 224×224 for transformers; Section 4.3 refers to '40×24 pixel resolution with substantial pixelation.' This inconsistency makes the experimental setup irreproducible and affects interpretation of the resolution confound. The authors should clarify the native dataset resolution, the exact preprocessing pipeline for each model family, and the source of the 40×24 figure.

minor comments (4)

[Section 3.5 / Table 8] The XRAI saliency images in Table 8 are not legible in the provided manuscript. Higher-resolution crops or additional zoomed panels are needed for the reader to assess the 'localized hotspots' and 'linear thermal paths' claims.
[Section 2.2 / Table 3] The text says 'Most models employed fixed learning rates for the sake of comparison consistency,' yet Swin-Tiny uses CosineAnnealingWarmRestarts. Please explain why this asymmetry is appropriate and discuss its potential impact on the comparison.
[Section 2.2] The fine-tuning strategy is not fully specified: are all layers fine-tuned or only the classification head? This is important for understanding the effect of ImageNet pretraining on thermal data and for reproducibility.
[References] Some references are incomplete or malformed (e.g., reference 20 lists 'V, G. R. N.' as an author; reference 6 is a preprint with odd spacing). Please normalize all references to the journal style.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; XRAI validation is qualitative but not circular by construction.

full rationale

The paper's claimed derivation chain is an empirical benchmark: train four architectures on a fixed dataset, report test accuracies, and compute XRAI saliency maps for correctly classified examples. There is no equation by which a fitted parameter is renamed as a prediction, and no result is defined in terms of the conclusion it is said to support. The central interpretability claim—that XRAI maps 'validate' alignment with thermal physics—is supported only by qualitative visual interpretation of the model's own saliency maps, without quantitative alignment metrics, ground-truth thermal masks, or baseline comparisons. That is a genuine limitation of evidence quality, and the paper itself concedes ambiguity for Diode-Multi, Hot-Spot-Multi, Cell-Multi, and soiling (Section 3.5.4). However, this is not circularity under the specified patterns: the saliency maps are not constructed from the physics claim, no self-citation is load-bearing, and no uniqueness theorem or ansatz is imported from prior author work. The accuracy comparison is a straightforward held-out evaluation. Therefore, despite the soft epistemic weakness of the XRAI validation, there is no specific circular reduction to exhibit, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claims rely on the assumptions listed; none are introduced as mathematical axioms, but they are domain assumptions that are plausible yet untested in this paper. The main hand-chosen factors are the per-model training protocols, which vary systematically and confound the comparison.

free parameters (2)

Per-architecture training protocol (epochs, batch size, resolution, LR schedule) = ResNet-18/EfficientNet: 10-15 epochs, batch 64, 128x128, fixed LR; ViT/Swin: 15 epochs, batch 32, 224x224; Swin uses Cos
These choices are not controlled across architectures, so any performance difference may be caused by the protocol rather than the model; this is a set of hand-chosen hyperparameters the central ranking depends on.
Input normalization = Channel-wise mean/std = 0.5 to [-1,1] (later text also mentions [0,1])
Unclear and inconsistent, affects training and is a modeling choice that could change results.

assumptions (3)

domain assumption ImageNet-pretrained weights transfer to single-channel thermal images replicated to three channels.
Assumed in Section 2.2; no experiments test whether pretraining helps or whether channel replication preserves thermal information.
domain assumption XRAI saliency maps reliably indicate which image regions drive the model's decision.
Standard tool, but the paper cites Chung et al. showing XAI methods can be misleading; no validation on this dataset.
domain assumption The upsampled 128x128 and 224x224 images preserve the thermal signatures of all fault types.
The paper later claims 40x24 native resolution limits detection, which partially contradicts this assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation." pith.science (2026). https://pith.science/paper/PDB4MMNT

@misc{pith2026250907039,
  author       = {Pith},
  title        = {Pith review of: Benchmarking Vision Transformers and CNNs for Thermal Photovoltaic Fault Detection with Explainable AI Validation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PDB4MMNT}},
  note         = {Machine review of arXiv:2509.07039}
}

read the original abstract

Artificial intelligence deployment for automated photovoltaic (PV) monitoring faces interpretability barriers that limit adoption in energy infrastructure applications. While deep learning achieves high accuracy in thermal fault detection, validation that model decisions align with thermal physics principles remains lacking, creating deployment hesitancy where understanding model reasoning is critical. This study provides a systematic comparison of convolutional neural networks (ResNet-18, EfficientNet-B0) and vision transformers (ViT-Tiny, Swin-Tiny) for thermal PV fault detection, using XRAI saliency analysis to assess alignment with thermal physics principles. This represents the first systematic comparison of CNNs and vision transformers for thermal PV fault detection with physics-validated interpretability. Evaluation on 20,000 infrared images spanning normal operation and 11 fault categories shows that Swin Transformer achieves the highest performance (94% binary accuracy; 73% multiclass accuracy) compared to CNN approaches. XRAI analysis reveals that models learn physically meaningful features, such as localized hotspots for cell defects, linear thermal paths for diode failures, and thermal boundaries for vegetation shading, consistent with expected thermal signatures. However, performance varies significantly across fault types: electrical faults achieve strong detection (F1-scores >0.90) while environmental factors like soiling remain challenging (F1-scores 0.20-0.33), indicating limitations imposed by thermal imaging resolution. The thermal physics-guided interpretability approach provides methodology for validating AI decision-making in energy monitoring applications, addressing deployment barriers in renewable energy infrastructure.

Figures

Figures reproduced from arXiv: 2509.07039 by the authors.

**Figure 1.** Class-wise Visualization of Infrared Module Types All the thermal images were subjected to standardization procedures to have uniform input dimensions and maintain data integrity. The images were transformed into grayscale mode and resized to 128×128 pixels through cubic interpolation to retain thermal gradient details. The pixel values were normalized between [0,1] through division by 255 to facilitate stable gradi… view at source ↗

**Figure 2.** Binary Class Distribution Across Train, Validation, and Test Sets The multiclass classification splits the fault-only dataset of 10,000 samples into 7,000 training, 1,500 validation, and 1,500 test samples with the natural class distribution reflecting actual fault occurrence patterns replicated ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Multiclass Distribution Across Train, Validation, and Test Splits Figures 2 and 3 illustrate the class distribution of the training, validation, and test sets for binary and multiclass classifications, respectively, reflecting the balanced nature of the binary classification system and the complex issue of class imbalance in the multiclass classification system. 2.2 Experimental Setup Four of the contemporary deep l… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Experimental Setup Flowchart ResNet-18 served as the base convolutional network with its final fully connected layer changed to produce 2 units for binary or 11 units for multiclass fault classification. The model used its [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Joint-Embedding Predictive Architecture for Solar PV Panel Fault Classification
eess.IV 2026-07 accept novelty 5.5 of 10

JEFFNet fuses StoP-JEPA semantic embeddings with EfficientNetV2-S features for thermal IR PV fault classification, beating GEPFNet on F1 for multiclass and binary tasks with 47% fewer parameters.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · cited by 1 Pith paper

[7]

Bommes, L. et al. Anomaly Detection in IR Images of PV Modules using Supervised Contrastive Learning. Preprint at https://doi.org/10.48550/ARXIV .2112.02922 (2021). 8. Vlaminck, M., Heidbuchel, R., Philips, W. & Luong, H. Region-Based CNN for Anomaly Detection in PV Power Plants Using Aerial Imagery. Sensors 22, 1244 (2022). 9. Li, B. et al. Trustworthy A...

work page Pith review arXiv doi:10.48550/arxiv.2112.02922 2021
[17]

& Vadhavkar, N

Millendorf, M., Obropta, E. & Vadhavkar, N. Infrared Solar Module Dataset for Anomaly Detection. in (2020). 18. Le, M., Le, D. & Ha Thi Vu, H. Thermal inspection of photovoltaic modules with deep convolutional neural networks on edge devices in AUV . Measurement 218, 113135 (2023). 19. Daher, D. H. et al. Photovoltaic failure diagnosis using imaging techn...

work page 2020

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [7]

Bommes, L. et al. Anomaly Detection in IR Images of PV Modules using Supervised Contrastive Learning. Preprint at https://doi.org/10.48550/ARXIV .2112.02922 (2021). 8. Vlaminck, M., Heidbuchel, R., Philips, W. & Luong, H. Region-Based CNN for Anomaly Detection in PV Power Plants Using Aerial Imagery. Sensors 22, 1244 (2022). 9. Li, B. et al. Trustworthy A...

work page Pith review arXiv doi:10.48550/arxiv.2112.02922 2021

[2] [17]

& Vadhavkar, N

Millendorf, M., Obropta, E. & Vadhavkar, N. Infrared Solar Module Dataset for Anomaly Detection. in (2020). 18. Le, M., Le, D. & Ha Thi Vu, H. Thermal inspection of photovoltaic modules with deep convolutional neural networks on edge devices in AUV . Measurement 218, 113135 (2023). 19. Daher, D. H. et al. Photovoltaic failure diagnosis using imaging techn...

work page 2020