Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hybrid knowledge distillation transfers both class probabilities and attention maps from a Swin Transformer to a MobileNetV3 student, and the paper reports that the student reaches near-teacher accuracy on tomato disease classification…

desk verdict Standard distillation recipe with useful device benchmarks, but the headline accuracy claims are undermined by dataset switching and inconsistent numbers—worth a careful referee, not acceptance. read the letter →

arxiv 2504.16128 v1 pith:PGPMUYFG submitted 2025-04-21 cs.CV cs.AIcs.LG

Stanley Mugisha , Rashid Kisitu , Florence Tushabe This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords knowledgedistillationSwinTransformerMobileNetV3plantdiseaseclassificationagriculturalIoTedgeinferenceattentiontransfermodelcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a lightweight MobileNetV3 classifier can be taught to match a heavy Swin Transformer on plant-disease diagnosis by distilling two kinds of knowledge at once: the teacher's softened class probabilities and its spatial attention maps. The authors' core problem is that these two architectures are structurally incompatible, so they add an alignment stage that projects channel counts and resizes attention maps before the losses are compared. On tomato leaf datasets, the distilled student is reported at 94.58% accuracy, within about one point of the Swin teacher, while using 0.22 GFLOPs instead of 34.1 GFLOPs and running at 22-88 ms per image on Raspberry Pi and smartphone hardware. If these measurements hold, the practical payoff is that farmers could run transformer-level diagnostic accuracy on devices in the field, without cloud connectivity.

What carries the argument

The load-bearing mechanism is the hybrid loss $\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{CE}} + \alpha \mathcal{L}_{\text{logit}} + \beta \mathcal{L}_{\text{attn}}$ combined with adaptive attention alignment. Two $1\times 1$ convolution adapters $g_T$ and $g_S$ project the teacher's and student's attention maps to a common channel dimension, and bilinear interpolation resizes the teacher's maps to the student's spatial dimensions; the aligned maps are flattened, passed through a softmax, and compared with KL divergence. The logit term is a temperature-scaled KL divergence between class distributions, with the paper's configuration at temperature $\tau = 6$ and weights $\alpha = 0.7$, $\beta = 0.3$. This alignment machinery is what lets a hierarchical window-based transformer teach a depthwise-convolution network despite mismatched resolutions and channel counts.

What would settle it

Run the same Swin-to-MobileNetV3 distillation on a fixed split of the full 18,160-image PlantVillage-Tomato dataset with early stopping applied only to validation, then measure test accuracy and phone latency in the same way; if the student fails to stay within a few points of the teacher or the 82% latency reduction disappears on identical hardware, the paper's central claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that hybrid knowledge distillation can shrink the accuracy gap between a 195-million-parameter Swin-Large teacher and a 4.2-million-parameter MobileNetV3 student on fine-grained tomato disease classification. The authors hold that neither softened logits nor attention maps alone carry enough information, and that the missing piece is an alignment module that reconciles the teacher's $7\times 7$ attention maps and 768 channels with the student's $14\times 14$ feature maps and 160 channels before a joint loss supervises both class probabilities and spatial focus. They report 94.58% test accuracy for the hybrid-distilled student on the tomato-village dataset, above attention-only (92.41%) and logit-only (92.62%) variants and close to the Swin teacher's 94.57% test accuracy on that set, while the larger PlantVillage comparison in the abstract gives 92.4% versus the teacher's 95.9%. The same student is measured at 0.22 GFLOPs, 13-24 MB after quantization, 34 ms per image on a smartphone GPU, and about 22 ms on a Raspberry Pi 5, which the authors read as ViT-level diagnostic precision on edge devices.

Load-bearing premise

The main accuracy and latency claims are established on the smaller 8-class, 4,526-image tomato-village dataset rather than the full PlantVillage set named in the abstract, so the central claim assumes that this smaller test split is representative and was not indirectly selected by early stopping; the paper offers no evidence for that assumption.

Editorial extensions

If this is right

  • If the reported numbers hold, a phone or Raspberry Pi can run tomato disease classification at 92-95% accuracy in tens of milliseconds, removing the need for cloud round-trips in the field.
  • The ablation results imply that combining logit and attention transfer is better than either alone for this task, with the hybrid model outperforming both single-channel distillations by roughly two percentage points.
  • Post-training integer quantization cuts memory to as little as 13.4 MB on a smartphone and lowers latency to about 86-88 ms, with a modest accuracy drop to 91.1%, which the paper says makes the model compatible with low-bandwidth IoT updates.
  • The comparison against TinyML baselines suggests that tiny models under 1 MB pay a large accuracy penalty (34.3%), so the distilled MobileNetV3 occupies a more usable point in the accuracy-efficiency trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the alignment recipe generalizes, the same channel-projection plus spatial-resizing trick could be applied to other ViT-to-CNN distillation pairs, but the paper only demonstrates it on tomato leaves, so that is an untested extension.
  • The abstract's 92.4% and the main text's 94.58% are measured on different datasets, so the two numbers should not be compared head-to-head; a reader would need a single benchmark to see the true accuracy gain.
  • A testable extension is to measure energy consumption in millijoules per inference, which the paper names as future work; that quantity would settle whether solar-powered nodes can sustain this workload.
  • A natural stress test is to run the same distillation on cassava or rice datasets, where the authors cite larger CNN-to-ViT gaps; if the gap does not close there, the method may be specific to tomato-like fine-grained tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a hybrid knowledge distillation framework that transfers both logit and attention knowledge from a Swin Transformer teacher to a MobileNetV3 student for tomato leaf disease classification on resource-constrained devices. The method uses a dual loss combining cross-entropy, temperature-scaled logit KL divergence, and KL divergence between normalized attention maps, with learnable channel projection and spatial interpolation to handle cross-architecture mismatches. The authors report that the distilled MobileNetV3 reaches 92.4% accuracy versus 95.9% for Swin-Large on PlantVillage-Tomato, with large latency and memory reductions on smartphones and Raspberry Pi, and claim 'ViT-level diagnostic precision on edge devices.' The manuscript includes teacher/student selection experiments, temperature sweeps, distillation-method ablations, comparisons to state-of-the-art models, and deployment benchmarks on three edge devices.

Significance. If the reported results were consistent and reproducible, the paper would be a useful applied contribution: it combines two well-known distillation signals (logits and attention) for a realistic IoT deployment scenario and provides detailed deployment measurements (latency, memory, model size) that are often missing from distillation papers. The explicit training configuration, algorithm pseudocode, and deployment tables are strengths. However, the central claim is currently undermined by internal inconsistencies in the headline accuracy numbers, an unexplained switch of the evaluation dataset after Section V-C, ambiguous model-selection/test-split isolation, and contradictory efficiency figures. Because these issues affect exactly the claim advertised in the abstract, the manuscript needs substantial revision before the results can be relied upon.

major comments (5)
  1. [Abstract and Section IV-A] The abstract and introduction attribute the headline results to the PlantVillage-Tomato dataset (18,160 images, 10 classes), but Section IV-A states that 'after student model selection, for most of the experiments starting from sub-section V-C onwards, we restricted ourselves to the use of the tomato village dataset because it was peer-reviewed and therefore more reliable.' Tables VI and VII do not state which dataset they use. Since Tomato-Village has 4,526 images and 8 classes rather than 18,160 images and 10 classes, the claim of 'ViT-level diagnostic precision on edge devices' is only supported, at best, on the smaller and different benchmark. Please identify the dataset for every results table and report the main results separately on both datasets, and revise the abstract accordingly.
  2. [Abstract, Table VI, Table VII, Section V-E, Section VI] The reported student accuracy is inconsistent across the paper: the abstract reports 92.4% for the distilled MobileNetV3 and 95.9% for Swin-Large; Table VI reports 94.58% for Hybrid Dist; Table VII reports 94.6% for Hybrid Distilled; Table III reports Swin-Large test accuracy as 94.57% (Dataset1) and 99.80% (Dataset2); Section V-E text says '92.4% vs. 93.1% in preliminary results'; and Section VI repeats 92.4%. Please reconcile these numbers and state precisely which model, dataset, and experimental condition each number corresponds to.
  3. [Section IV-A and Section IV-B] The paper describes model selection and early stopping, including teacher pretraining with k-fold cross-validation and a two-phase training procedure, but the fixed 70/20/10 train/validation/test split is only introduced in Section IV-A. There is no explicit statement that the test split was never used for model selection or early stopping. If selection decisions were made using the test set, the reported test accuracies would be optimistically biased. Please clarify the protocol: selection should be performed using the validation split only, and the test split evaluated exactly once after all selection is complete.
  4. [Abstract, Table VIII, Table IX, Section VI] The efficiency claims are not mutually consistent. The abstract states '<82% inference latency on IoT devices (23ms on PC CPU and 86ms/image on smartphone CPUs)', while Table VIII reports MobileNetV3 CPU latency as 22.96 ms and Table IX reports quantized latencies of 88.4 ms (Pixel 9 Pro), 86.0 ms (Galaxy J7), and 21.72 ms (Raspberry Pi 5). Section VI claims '34 ms/inference latency and 24 MB memory', while the abstract claims 13 MB memory and Table IX reports quantized memory of 13.4 MB (Pixel), 23.0 MB (Galaxy), and 58.1 MB (Raspberry Pi). Also, comparing Swin-L CPU latency 492.55 ms with MobileNetV3 CPU 22.96 ms gives roughly a 95% reduction, not the stated 82%. Please provide a single consistent set of deployment metrics with clear baselines.
  5. [Section V-C and Section III-E] The paper's second contribution is the 'adaptive attention alignment module' with resolution matching and channel projection, but the ablation study in Section V-C2 only compares Student-Only, Attention-Only, Logit-Only, and Hybrid distillation; it never isolates the effect of the proposed alignment mechanism against a simple resize or bilinear interpolation without learnable channel adapters. Without such an ablation, the central methodological novelty is unsupported. Please add an ablation that removes or simplifies the adaptive alignment module, or temper the novelty claim accordingly.
minor comments (5)
  1. [Abstract] The abstract contains typos: 'lantVillage-Tomato' should be 'PlantVillage-Tomato', 'at an 95% reduction' should be 'a 95% reduction', and the phrase '< 82% in inference latency' is ambiguous as to whether the reduction is greater or less than 82%.
  2. [Table I and Section II] Table I labels the teacher as 'Swin-T' but reports 34.1 GFLOPs, while Section II states that Swin-T has 15.4 GFLOPs and Table VIII attributes 34.1 GFLOPs to Swin-Large. Please correct the model name/FLOPs inconsistency.
  3. [Section V-E] The sentence 'its accuracy lags behind Swin Large (92.4% vs. 93.1% in preliminary results)' uses different numbers from the abstract and from Tables III, VI, and VII; this needs to be harmonized with the corrected accuracy table.
  4. [Section V-G4] The comparison with IoT baselines states 'the quantized model performed slightly lower at 91%' but Table X lists MobileNetV3 Quantized accuracy as 91.1%; these should be made consistent.
  5. [References and Notation] Minor textual issues include 'outperfoming' (Section I), 'Simillar' (Section V-D), 'LoRaW AN' (Sections II and V-G4), inconsistent capitalization of 'MobileNet'/'Mobilenet', and unattributed table captions in Tables VIII and X that appear to be missing surrounding text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported accuracies and latencies are empirical measurements of a standard distillation training recipe, not quantities derived from the method's own definitions or from load-bearing self-citation.

full rationale

The paper proposes a hybrid knowledge-distillation training procedure: a frozen Swin Transformer teacher provides softened logits and attention maps, a MobileNetV3 student is trained with L_total = L_CE + alpha*L_logit + beta*L_attn, and the reported outcomes (92.4% accuracy, 82% latency reduction, 13-24 MB memory, 22-88 ms inference) are measured on held-out test splits and deployed hardware. None of these outputs appears as an input to the method. The loss weights alpha/beta and temperature tau are selected hyperparameters, not quantities solved for from the target accuracy, so the central result is not a fitted parameter renamed as a prediction. The adaptive attention alignment is implemented with 1x1 convolutions, averaging over attention heads, and bilinear interpolation; these are explicit operations, not an ansatz smuggled in to force the desired result. The only self-citation in the paper is reference [15], a prior paper by one of the authors on cobot haptic interfaces; it is cited in a general list of works on vision transformers and plays no role in justifying the distillation framework or the reported numbers. There is no invoked uniqueness theorem and no self-definitional equation in which the claimed prediction is defined in terms of the quantity it supposedly predicts. The separate concern that the abstract attributes results to the 18,160-image 10-class PlantVillage-Tomato dataset while Section IV-A restricts most experiments to the 4,526-image 8-class Tomato-Village dataset is a correctness and benchmark-consistency issue, not circularity: the accuracy numbers are still empirical measurements rather than consequences of the method's definitions. Accordingly, the paper is self-contained as an empirical evaluation and receives a circularity score of 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper adds no new physical or mathematical entities. Its free parameters are standard distillation hyperparameters. The key assumptions are that attention-map KL is an effective cross-architecture objective and that the chosen dataset is a valid evaluation basis; neither is demonstrated beyond the reported experiments.

free parameters (3)
  • Logit distillation loss weight α = 0.7
    Set by hand in Table II, not tuned with a reported validation search.
  • Attention distillation loss weight β = 0.3
    Set by hand in Table II alongside α.
  • Temperature τ = 6.0
    Table V shows τ=2 and τ=6 tied at 95.66%; the main config uses 6.0 without clear justification.
assumptions (3)
  • domain assumption KL divergence between softmax-normalized attention maps transfers useful spatial knowledge
    Section III-C3 defines L_attn as KL(σ(A_T)∥σ(A_S)); no evidence is given that this specific objective benefits this cross-architecture pair.
  • ad hoc to paper Teacher attention from the first window attention block is the right supervision
    Algorithm 1, line 5: 'first window attention block for our case'; no ablation over attention layers.
  • ad hoc to paper The tomato-village dataset is a more reliable evaluation basis because it is peer-reviewed
    Section IV-A states the restriction; peer-review status of a dataset does not establish benchmark reliability for the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT." pith.science (2026). https://pith.science/paper/PGPMUYFG

@misc{pith2026250416128,
  author       = {Pith},
  title        = {Pith review of: Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGPMUYFG}},
  note         = {Machine review of arXiv:2504.16128}
}
read the original abstract

Integrating deep learning applications into agricultural IoT systems faces a serious challenge of balancing the high accuracy of Vision Transformers (ViTs) with the efficiency demands of resource-constrained edge devices. Large transformer models like the Swin Transformers excel in plant disease classification by capturing global-local dependencies. However, their computational complexity (34.1 GFLOPs) limits applications and renders them impractical for real-time on-device inference. Lightweight models such as MobileNetV3 and TinyML would be suitable for on-device inference but lack the required spatial reasoning for fine-grained disease detection. To bridge this gap, we propose a hybrid knowledge distillation framework that synergistically transfers logit and attention knowledge from a Swin Transformer teacher to a MobileNetV3 student model. Our method includes the introduction of adaptive attention alignment to resolve cross-architecture mismatch (resolution, channels) and a dual-loss function optimizing both class probabilities and spatial focus. On the lantVillage-Tomato dataset (18,160 images), the distilled MobileNetV3 attains 92.4% accuracy relative to 95.9% for Swin-L but at an 95% reduction on PC and < 82% in inference latency on IoT devices. (23ms on PC CPU and 86ms/image on smartphone CPUs). Key innovations include IoT-centric validation metrics (13 MB memory, 0.22 GFLOPs) and dynamic resolution-matching attention maps. Comparative experiments show significant improvements over standalone CNNs and prior distillation methods, with a 3.5% accuracy gain over MobileNetV3 baselines. Significantly, this work advances real-time, energy-efficient crop monitoring in precision agriculture and demonstrates how we can attain ViT-level diagnostic precision on edge devices. Code and models will be made available for replication after acceptance.

Figures

Figures reproduced from arXiv: 2504.16128 by the authors.

Figure 1
Figure 1. Hybrid distillation framework addressing cross-architecture challenges [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mobile application for IoT-driven tomato disease detection. The [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Raspberry Pi testbed setup for evaluation of the models. The proposed [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Deployment pipeline for the distilled MobileNetV3 model in IoT [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Vision-Based Embedded System for Noncontact Monitoring of Preterm Infant Behavior in Low-Resource Care Settings

    cs.CV 2025-09 conditional novelty 2.0 of 10

    A quantized MobileNet on a Raspberry Pi achieves 91.8% sleep/awake and 97.7% crying/normal accuracy on a public neonatal dataset, but the state-of-the-art claim is not supported by direct comparison to prior vision methods.

Reference graph

Works this paper leans on

52 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [15]

    Motion strategies for a cobot in a context of intermittent haptic interface,

    V . Guda, S. Mugisha, C. Chevallereau, M. Zoppi, R. Molfino, and D. Chablat, “Motion strategies for a cobot in a context of intermittent haptic interface,” Journal of Mechanisms and Robotics, vol. 14, no. 4, p. 041012, 06 2022. [Online]. Available: https://doi.org/10.1115/1.4054509

    work page doi:10.1115/1.4054509 2022

  2. [1]

    A comparative study of fine-tuning deep learning models for plant disease identification,

    E. C. Too, L. Yujian, S. Njuki, and L. Yingchun, “A comparative study of fine-tuning deep learning models for plant disease identification,” Computers and Electronics in Agriculture , vol. 161, pp. 272– 279, 2019, bigData and DSS in Agriculture. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0168169917313303

    work page 2019

  3. [2]

    Deep learning in agriculture: A survey,

    A. Kamilaris and F. X. Prenafeta-Bold ´u, “Deep learning in agriculture: A survey,” Computers and Electronics in Agriculture , vol. 147, pp. 70–90, 2018. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0168169917308803

    work page 2018

  4. [3]

    Brahimi, M

    M. Brahimi, M. Arsenovic, S. Laraba, S. Sladojevic, K. Boukhalfa, and A. Moussaoui, Deep Learning for Plant Diseases: Detection and Saliency Map Visualisation . Cham: Springer International Publishing, 2018, pp. 93–117

    work page 2018

  5. [4]

    Deep learning models for plant disease detection and diagnosis,

    K. P. Ferentinos, “Deep learning models for plant disease detection and diagnosis,” Computers and Electronics in Agriculture , vol. 145, pp. 311–318, 2018. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0168169917311742

    work page 2018

  6. [5]

    A survey on different plant diseases detection using machine learning techniques,

    S. M. Hassan, K. Amitab, M. Jasinski, Z. Leonowicz, E. Jasinska, T. Novak, and A. K. Maji, “A survey on different plant diseases detection using machine learning techniques,” Electronics, vol. 11, no. 17, 2022. [Online]. Available: https://www.mdpi.com/2079-9292/11/17/2641

    work page 2022

  7. [6]

    A review on the main challenges in automatic plant disease identification based on visible range images,

    J. G. A. Barbedo, “A review on the main challenges in automatic plant disease identification based on visible range images,” Biosystems Engineering, vol. 144, pp. 52–60, 2016. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S1537511015302476

    work page 2016

  8. [7]

    Towards sustainable agriculture: A transformer-based hybrid model for advanced leaf disease classification,

    S. Sidana, “Towards sustainable agriculture: A transformer-based hybrid model for advanced leaf disease classification,” in 2024 15th Inter- national Conference on Computing Communication and Networking Technologies (ICCCNT), 2024, pp. 1–6

    work page 2024

Show all 52 references
  1. [8]

    Comparing vision transformers and convolutional neural networks for image classification: A literature review,

    J. Maur ´ıcio, I. Domingues, and J. Bernardino, “Comparing vision transformers and convolutional neural networks for image classification: A literature review,” Applied Sciences , vol. 13, no. 9, 2023. [Online]. Available: https://www.mdpi.com/2076-3417/13/9/5521

    2023

  2. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021. [Online]. Available: https://arxi...

    2021 arXiv

  3. [10]

    Transformers in vision: A survey,

    S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in vision: A survey,” ACM Comput. Surv. , vol. 54, no. 10s, 2022. [Online]. Available: https://doi.org/10.1145/3505244

    2022 doi

  4. [11]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Com- puter Vision – ECCV 2020 . Cham: Springer International Publishing, 2020, pp. 213–229

    2020

  5. [12]

    Image transformer,

    N. Parmar, A. Vaswani, J. Uszkoreit, Łukasz Kaiser, N. Shazeer, A. Ku, and D. Tran, “Image transformer,” 2018. [Online]. Available: https://arxiv.org/abs/1802.05751

    2018 arXiv

  6. [13]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distillation through attention,” 2021. [Online]. Available: https://arxiv.org/abs/2012. 12877

    2021

  7. [14]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Los Alamitos, CA, USA: IEEE Computer Society, Oct 2021, pp. ...

    2021

  8. [16]

    Transformer in transformer,

    K. Han, A. Xiao, E. Wu, J. Guo, C. Xu, and Y . Wang, “Transformer in transformer,” 2021. [Online]. Available: https://arxiv.org/abs/2103.00112

    2021 arXiv

  9. [17]

    Searching for MobileNetV3 ,

    A. Howard, M. Sandler, B. Chen, W. Wang, L.-C. Chen, M. Tan, G. Chu, V . Vasudevan, Y . Zhu, R. Pang, H. Adam, and Q. Le, “ Searching for MobileNetV3 ,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Los Alamitos, CA, USA: IEEE Computer Society, Nov. 201...

    2019

  10. [18]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” 2015. [Online]. Available: https://arxiv.org/abs/1503.02531

    2015 arXiv

  11. [19]

    Leveraging logit uncertainty for better knowledge distillation,

    Z. Guo, D. Wang, Q. He, and P. Zhang, “Leveraging logit uncertainty for better knowledge distillation,” Scientific Reports , vol. 14, no. 1, p. 31249, Dec 2024. [Online]. Available: https: //doi.org/10.1038/s41598-024-82647-6

    2024 doi

  12. [20]

    Tomato leaf dis- ease recognition based on multi-task distillation learning,

    B. Liu, S. Wei, F. Zhang, N. Guo, H. Fan, and W. Yao, “Tomato leaf dis- ease recognition based on multi-task distillation learning,” FRONTIERS IN PLANT SCIENCE , 2024

    2024

  13. [21]

    An improved deep learning approach for localization and recognition of plant leaf diseases,

    Y . Alqahtani, M. Nawaz, T. Nazir, A. Javed, F. Jeribi, and A. Tahir, “An improved deep learning approach for localization and recognition of plant leaf diseases,” Expert Systems with Applications, vol. 230, p. 120717, 2023. [Online]. Available: https: //www.sciencedirect.com/...

    2023

  14. [22]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023. [Online]. Available: https://arxiv.org/abs/1706.03762

    2023 arXiv

  15. [23]

    Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,

    S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” 2017. [Online]. Available: https://arxiv.org/abs/1612. 03928

    2017

  16. [24]

    Revisiting knowledge distillation via label smoothing regularization,

    L. Yuan, F. E. Tay, G. Li, T. Wang, and J. Feng, “Revisiting knowledge distillation via label smoothing regularization,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 3902–3910

    2020

  17. [25]

    Farmbeats: an iot platform for data-driven agriculture,

    D. Vasisht, Z. Kapetanovic, J.-h. Won, X. Jin, R. Chandra, A. Kapoor, S. N. Sinha, M. Sudarshan, and S. Stratman, “Farmbeats: an iot platform for data-driven agriculture,” in Proceedings of the 14th USENIX Confer- ence on Networked Systems Design and Implementation , ser. NSDI...

    2017

  18. [26]

    An open access repository of images on plant health to enable the development of mobile disease diagnostics,

    D. P. Hughes and M. Salathe, “An open access repository of images on plant health to enable the development of mobile disease diagnostics,”

  19. [27]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Commun. ACM , vol. 60, no. 6, p. 84–90, May 2017. [Online]. Available: https://doi.org/10.1145/ 3065386

    2017

  20. [28]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015. [Online]. Available: https: //arxiv.org/abs/1409.1556

    2015 arXiv

  21. [29]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” 2017. [Online]. Available: https://arxiv.org/abs/1704.04861

    2017 arXiv

  22. [30]

    Visual transformers: Token- based image representation and processing for computer vision,

    B. Wu, C. Xu, X. Dai, A. Wan, P. Zhang, Z. Yan, M. Tomizuka, J. Gonzalez, K. Keutzer, and P. Vajda, “Visual transformers: Token- based image representation and processing for computer vision,” 2020. [Online]. Available: https://arxiv.org/abs/2006.03677

    2020 arXiv

  23. [31]

    Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer,

    S. Mehta and M. Rastegari, “Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer,” 2022. [Online]. Available: https://arxiv.org/abs/2110.02178

    2022 arXiv

  24. [32]

    Low-power deep learning model for plant disease detection for smart-hydroponics using knowledge distillation techniques,

    A. Musa, M. Hassan, M. Hamada, and F. Aliyu, “Low-power deep learning model for plant disease detection for smart-hydroponics using knowledge distillation techniques,” Journal of Low Power Electronics and Applications , vol. 12, no. 2, 2022. [Online]. Available: https://www.md...

    2022

  25. [33]

    Efficientnet: Rethinking model scaling for convolutional neural networks,

    M. Tan and Q. V . Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” 2020. [Online]. Available: https: //arxiv.org/abs/1905.11946

    2020 arXiv

  26. [34]

    Knowledge distillation from bert transformer to speech transformer for intent classification,

    Y . Jiang, B. Sharma, M. Madhavi, and H. Li, “Knowledge distillation from bert transformer to speech transformer for intent classification,” ARXIV-CS.CL, 2021

    2021

  27. [35]

    Pmvt: a lightweight vision transformer for plant disease identification on mobile devices,

    G. Li, Y . Wang, Q. Zhao, P. Yuan, and B. Chang, “Pmvt: a lightweight vision transformer for plant disease identification on mobile devices,” Frontiers in Plant Science , vol. 14, 2023. [Online]. Available: https://www.frontiersin.org/journals/plant-science/articles/10. 3389/f...

    2023

  28. [36]

    Show, attend and distill:knowledge distillation via attention-based feature matching,

    M. Ji, B. Heo, and S. Park, “Show, attend and distill:knowledge distillation via attention-based feature matching,” 2021. [Online]. Available: https://arxiv.org/abs/2102.02973

    2021 arXiv

  29. [37]

    Frequency attention for knowledge distillation,

    C. Pham, V .-A. Nguyen, T. Le, D. Phung, G. Carneiro, and T.-T. Do, “Frequency attention for knowledge distillation,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2024, pp. 2266–2275

    2024

  30. [38]

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,

    J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 7130–7138

    2017

  31. [39]

    Knowledge distillation in vision transformers: A critical review,

    G. Habib, T. J. Saleem, and B. Lall, “Knowledge distillation in vision transformers: A critical review,” 2024. [Online]. Available: https://arxiv.org/abs/2302.02108

    2024 arXiv

  32. [40]

    Cross-architecture knowledge distillation,

    Y . Liu, J. Cao, B. Li, W. Hu, J. Ding, L. Li, and S. Maybank, “Cross-architecture knowledge distillation,” Int. J. Comput. Vision , vol. 132, no. 8, p. 2798–2824, Feb. 2024. [Online]. Available: https://doi.org/10.1007/s11263-024-02002-0

    2024 doi

  33. [41]

    Vitkd: Feature- based knowledge distillation for vision transformers,

    Z. Yang, Z. Li, A. Zeng, Z. Li, C. Yuan, and Y . Li, “Vitkd: Feature- based knowledge distillation for vision transformers,” in2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024, pp. 1379–1388

    2024

  34. [42]

    Internet of things in agriculture, recent advances and future challenges,

    A. Tzounis, N. Katsoulas, T. Bartzanas, and C. Kittas, “Internet of things in agriculture, recent advances and future challenges,” Biosystems Engineering, vol. 164, pp. 31–48, 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1537511017302544

    2017

  35. [43]

    Role of iot technology in agriculture: A systematic literature review,

    M. S. Farooq, S. Riaz, A. Abid, T. Umer, and Y . B. Zikria, “Role of iot technology in agriculture: A systematic literature review,” Electronics, vol. 9, no. 2, 2020. [Online]. Available: https://www.mdpi.com/2079-9292/9/2/319

    2020

  36. [44]

    H-at: Hybrid attention transfer for knowledge distillation,

    Y . Qu, W. Deng, and J. Hu, “H-at: Hybrid attention transfer for knowledge distillation,” in Pattern Recognition and Computer Vision , Y . Peng, Q. Liu, H. Lu, Z. Sun, C. Liu, X. Chen, H. Zha, and J. Yang, Eds. Cham: Springer International Publishing, 2020, pp. 249–260

    2020

  37. [45]

    “tomato-village

    M. Gehlot, R. K. Saxena, and G. C. Gandhi, ““tomato-village”: a dataset for end-to-end tomato disease detection in a real-world environment,” Multimedia Syst. , vol. 29, no. 6, p. 3305–3328, Aug

  38. [46]

    Dataset of tomato leaves,

    Y .-H. Huang, Mei-Ling; Chang, “Dataset of tomato leaves,” Mendeley Data, vol. V1, 2020. [Online]. Available: https://data.mendeley.com/ datasets/ngdgg79rzb/1

    2020

  39. [47]

    EfficientNet: Rethinking model scaling for convolutional neural networks,

    M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 ...

    2019

  40. [49]

    Tiny machine learning: Progress and futures [feature],

    J. Lin, L. Zhu, W.-M. Chen, W.-C. Wang, and S. Han, “Tiny machine learning: Progress and futures [feature],” IEEE Circuits and Systems Magazine , vol. 23, no. 3, p. 8–34, 2023. [Online]. Available: http://dx.doi.org/10.1109/MCAS.2023.3302182 Stanley Mugisha received the Bsc. a...

    2023

  41. [2016]

    Available: https://arxiv.org/abs/1511.08060

    [Online]. Available: https://arxiv.org/abs/1511.08060

    arXiv

  42. [2018]

    Available: http://arxiv.org/abs/1801.04381

    [Online]. Available: http://arxiv.org/abs/1801.04381

    arXiv

  43. [2022]

    Since 2024, he works at Soroti university as a lecturer

    In 2023, He worked as a postdoc research fellow in human robot collaboration at the university of Tartu in Estonia. Since 2024, he works at Soroti university as a lecturer. His research interest include: Computer vision, human robot interaction, IoT, and embedded Machine Learn...

    2023

  44. [2023]

    Available: https://doi.org/10.1007/s00530-023-01158-y

    [Online]. Available: https://doi.org/10.1007/s00530-023-01158-y

    doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.