REVIEW 5 major objections 5 minor 39 references
EdgeMLBalancer: A Self-Adaptive Approach for Dynamic Model Switching on Resource-Constrained Edge Devices
T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read A CPU-aware epsilon-greedy switcher balances accuracy, CPU use, and fairness for on-device object detection.
desk verdict Section III-A2's score function rewards declining confidence, so the reported accuracy gains cannot be explained by the published algorithm. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The MAPE-K feedback loop with the epsilon-greedy model-selector is the load-bearing mechanism. The Monitor collects $(m_i, C_i, U_i)$ per frame; the Analyzer computes windowed averages and the score $S_{m_i}$; the Planner applies epsilon-greedy to pick the next model; the Executor swaps the active model and logs metrics; the Knowledge component stores the model repository, score table, and log registry. The epsilon-greedy parameter $\epsilon$ controls the exploration/exploitation trade-off, and the score formula is intended to combine CPU efficiency ($\min(U_i, U_{\text{avg}})$) with accuracy relative to history ($1 - C_{\text{avg}}/C_i$).
What would settle it
Recompute the score from the paper's own example: with $U_i = 13\%$, $U_{\text{avg}} = 18\%$, $C_i = 54.42\%$, $C_{\text{avg}} = 55.94\%$, the score $S_{m_i} = 13 \times (1 - 55.94/54.42) \approx -0.36$; if $C_i$ is raised to $60\%$ while all else stays equal, the score becomes positive, so the planner would prefer the lower-confidence model, contradicting the paper's stated goal of favoring accuracy.
Extended reading notes
Core claim
EdgeMLBalancer's central claim is that a self-adaptation loop built on the MAPE-K pattern can dynamically switch among four on-device object-detection models — EfficientDet Lite0, Lite1, Lite2, and SSD MobileNet V1 — to balance computational efficiency and detection accuracy on an Android device. For each frame the monitor records CPU usage $U_i$ and a confidence score $C_i$; the analyzer aggregates these over a window to historical averages $U_{\text{avg}}$ and $C_{\text{avg}}$, then assigns each model a score $S_{m_i} = \min(U_i, U_{\text{avg}}) \times (1 - C_{\text{avg}}/C_i)$. The planner selects the lowest-scoring model with probability $1-\epsilon$, and with probability $\epsilon$ randomly explores another model to avoid starvation. The evaluation on real-time traffic video reports a 43.62% improvement in model-usage fairness over the Naive baseline, higher average accuracy than both Naive and Round Robin with Boosting, and only a modest 0.85 s average switching time.
Load-bearing premise
The decision engine assumes that the score $S_{m_i} = \min(U_i, U_{\text{avg}}) \times (1 - C_{\text{avg}}/C_i)$ correctly ranks models with the lowest score as the best trade-off, yet its confidence term makes a drop in current confidence relative to the historical average produce a lower (more favorable) score, which runs opposite to the stated goal of favoring accuracy.
Editorial extensions
If this is right
- A threshold-free, self-adaptive switcher can run entirely on a smartphone, removing the need for cloud offloading in real-time monitoring.
- The same MAPE-K loop can be repurposed for other resource metrics (memory, battery, temperature) and other model families, not just object detectors.
- Fairness through epsilon-greedy prevents a single model from dominating, which the paper argues makes the system more robust to workload shifts.
- The reported switching overhead of 0.85 seconds is small enough for near-real-time operation, though it is higher than the Naive approach's 0.50 seconds.
Reading between the lines
- If the score formula is read literally, lowering the current confidence $C_i$ below the historical average $C_{\text{avg}}$ makes the score more negative and hence more attractive to the planner; the reported accuracy gains may therefore owe more to epsilon-greedy exploration than to the score's accuracy term.
- The fairness metric measures how evenly frames are distributed across models, but it does not weight models by their intrinsic quality; a perfectly fair allocation could select a weak model too often on easy frames.
- A natural testable extension is to vary $\epsilon$ over time, annealing it from exploration-heavy to exploitation-heavy, to see whether the accuracy/fairness balance improves further.
- The authors note future plans for LLM-based vision and hybrid edge-cloud offloading; the current MAPE-K loop would need additional monitors (e.g., network latency) to make offloading decisions safely.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EdgeMLBalancer, a self-adaptive model-switching approach for object detection on resource-constrained Android devices. The system monitors per-frame CPU usage and detection confidence, computes a performance score per model, and uses an epsilon-greedy policy to select the next model, with the aim of balancing computational efficiency, accuracy, and fairness of model usage. The evaluation compares EdgeMLBalancer against a Naive threshold-based approach and a Round-Robin-with-Boosting approach on a 30-minute Indian-traffic video, reporting in Table I that EdgeMLBalancer achieves the highest average accuracy (17.36%), competitive CPU usage, and improved model-usage fairness.
Significance. If the results held, the contribution would be of practical interest: a lightweight, on-device, MAPE-K-based model switcher with public code, real smartphone deployment, and a fairness-oriented exploration mechanism. The authors should be credited for prototyping on a real device, releasing code and models, and addressing threats to validity explicitly. However, the central decision-making formula is internally inconsistent with its stated goal, and the empirical evidence consists of one short run per approach with no statistical support. These issues place the main claims of significant accuracy and fairness improvements on an unsupported basis.
major comments (5)
- [III-A2, III-A3] The score function S_mi = min(U_i, U_avg) * (1 - C_avg/C_i) is sign-inverted with respect to its stated purpose. Because the planner selects the model with the lowest score, a model whose current confidence C_i is below the historical average C_avg makes the factor (1 - C_avg/C_i) negative and is rewarded, so the only accuracy-aware signal actively favors confidence-declining models. The paper's own example (S_mi = -0.3627 described as 'a better score' due to 'slight decline in confidence') confirms the inversion. This is load-bearing: with exploitation probability 1 - epsilon, the published algorithm cannot explain the reported 17.36% average accuracy in Table I unless the formula, the selection direction, or the reported evaluation is corrected.
- [IV-A, IV-B, Table I] The central comparative claim rests on a single 30-minute run per approach. No error bars, confidence intervals, or statistical tests are reported, and the manuscript's conclusion-validity paragraph concedes the limited duration. The term 'significant improvements' in the abstract and RQ1 is therefore unsupported. Please report multiple independent runs, variance, and a defined ground-truth protocol for the accuracy metric; the current statement that accuracy is 'the percentage of correctly detected objects compared to the ground truth' omits how ground truth was obtained for a 30-minute Indian-traffic video.
- [IV-A, Table I] The frame counts in Table I (1952, 2482, and 2458) over 30 minutes correspond to roughly 1.1–1.4 frames per second, yet Section IV-A states that the input is a 30-minute recording at 60 frames per second. Either frames are dropped or sampled, or the input was not processed continuously; this must be clarified because it directly affects the accuracy and CPU measurements and the real-time claim.
- [IV-B, RQ2, Figure 5] The reported 43.62% fairness improvement is largely by construction. Epsilon-greedy explicitly randomizes model choice with probability epsilon, so it mechanically reduces selection disparity regardless of model quality or runtime context. To support the fairness claim, the authors should compare against a pure random-selection baseline with the same average switch rate, vary epsilon, and show that the resulting distribution reflects runtime conditions rather than only forced exploration.
- [IV-B, RQ3] The switching-time comparison in Figure 6 and Table I is not controlled: the approaches perform different numbers of switches and process different numbers of frames, so average switching time alone does not support the conclusion that EdgeMLBalancer has 'superior decision-making'. Please report per-switch distributions, switch frequency, and the context (e.g., idle versus active inference) in which switching times were measured.
minor comments (5)
- [III-A2] The notation 'arg min(U_i, U_avg)' in the score formula should be 'min(U_i, U_avg)', since a minimum value, not an argument, is intended.
- [III-A2] The citation '[5]' attached to the score formula is unclear; please specify whether the score is adopted from EcoMLS or newly introduced, and clarify the relationship.
- [IV-A] The text contains typos such as 'we measured different metrics metrics' and 'optimzal performance'; a proofreading pass is needed.
- [Abstract, I] There are grammatical errors in phrases such as 'the existing system lack robust mechanisms' and 'leaving to inefficiencies'; these should be corrected.
- [IV-B, Figures 4 and 6] If the box plots and averages are based on a single run per approach, the captions should state this explicitly; if multiple runs were performed, the number of runs should be given.
Circularity Check
Fairness improvement is partly by construction; the score function's sign makes the 'accuracy-aware' term reward declining confidence, though the device-level CPU/accuracy comparison against baselines remains empirical.
-
self definitional
[Abstract; Section III-A3 (Epsilon-Greedy Strategy, Algorithm 1 lines 5-6); Section IV-B RQ2]
"The approach leverages epsilon-greedy strategy which promotes fairness and prevents resource starvation... Exploration: Randomly select model excluding the best model then ... Mnext ← random(M) ... we can say that the Epsilon-Greedy (EdgeMLBalancer) represents an improvement in fairness of 43.62% over Naive and 41.47% over Round Robin with Boosting in terms of reducing the selection disparity between the models."
Fairness is operationalized in RQ2 as 'reducing the selection disparity between the models.' Epsilon-greedy is defined with a uniform random exploration arm, so random model choice mechanically spreads selections across models. The reported fairness improvement is therefore a direct consequence of injecting random choice into the policy, not an independent empirical discovery about the models or workload. The 43.62%/41.47% figures quantify the effect of the algorithm's own exploration relative to deterministic baselines, but the qualitative claim that epsilon-greedy 'promotes fairness' is true by definition of the exploration arm.
-
other
[Section III-A2 (Score Generator) and Section III-A3 (Planner, Algorithm 1 lines 7-11)]
"The performance score Smi for a model mi is calculated as: Smi = arg min(Ui, Uavg) × (1 − Cavg/Ci) ... The negative score reflects slight decline in the confidence, but the low CPU usage reduces the penalty, making it a better score."
The Planner exploits by picking arg min S (Algorithm 1: Mnext ← arg min(S)). For a fixed CPU term, the factor (1 − C_avg/C_i) decreases as C_i decreases, so a model whose current confidence falls below its historical average receives a lower, better score. The paper's own example confirms that a 'slight decline in confidence' yields a better score.
full rationale
The core contribution is an epsilon-greedy model switcher that selects by a defined score and compares itself to two baselines. The fairness result is substantially self-definitional because the exploration arm is a uniform random selection over models, so reduced selection disparity follows directly from the algorithm's own randomness. The score function used in exploitation contains a sign inversion that makes declining confidence score better, and the paper's own worked example confirms this. This makes the claimed accuracy-aware mechanism internally inconsistent, and since the 17.36% accuracy figure is attributed to that mechanism, that result cannot be certified by the published equations. However, the CPU usage and accuracy measurements on the device, and the head-to-head comparison against naive and round-robin baselines, are empirical observations that are not themselves circular. The paper does not rely on any self-citation chain for its central claims, and there is no importation of a uniqueness theorem. The circularity is partial: fairness is by construction, and the score-based selection is self-referential relative to its own stated objective, but the measured CPU/accuracy trade-off has independent empirical content. Hence a score of 4 is appropriate.
Assumptions & free parameters
free parameters (3)
- epsilon (exploration probability) in epsilon-greedy =
not reported
- window size n for averaging confidence and CPU =
not reported
- score function weights =
not fitted; functional form chosen by authors
assumptions (6)
- domain assumption CPU usage percentage is a valid real-time proxy for computational load and energy consumption of model inference.
- domain assumption Frame-level average confidence score is a valid proxy for detection accuracy.
- ad hoc to paper Lower values of Smi = min(Ui, Uavg) * (1 - Cavg/Ci) indicate better model performance.
- domain assumption The 30-minute Indian-traffic video and the Samsung Galaxy M21 are representative of real-time edge object detection workloads.
- domain assumption A ground-truth annotation of the traffic video exists and is correct, against which accuracy percentages are computed.
- domain assumption Fairness is measured as equal model-usage counts, assuming equal utility for all models.
Cite this review
Pith. "Pith review of EdgeMLBalancer: A Self-Adaptive Approach for Dynamic Model Switching on Resource-Constrained Edge Devices." pith.science (2026). https://pith.science/paper/C5LZG7WV
@misc{pith2026250206493,
author = {Pith},
title = {Pith review of: EdgeMLBalancer: A Self-Adaptive Approach for Dynamic Model Switching on Resource-Constrained Edge Devices},
year = {2026},
howpublished = {\url{https://pith.science/paper/C5LZG7WV}},
note = {Machine review of arXiv:2502.06493}
}
read the original abstract
The widespread adoption of machine learning on edge devices, such as mobile phones, laptops, IoT devices, etc., has enabled real-time AI applications in resource-constrained environments. Existing solutions for managing computational resources often focus narrowly on accuracy or energy efficiency, failing to adapt dynamically to varying workloads. Furthermore, the existing system lack robust mechanisms to adaptively balance CPU utilization, leading to inefficiencies in resource-constrained scenarios like real-time traffic monitoring. To address these limitations, we propose a self-adaptive approach that optimizes CPU utilization and resource management on edge devices. Our approach, EdgeMLBalancer balances between models through dynamic switching, guided by real-time CPU usage monitoring across processor cores. Tested on real-time traffic data, the approach adapts object detection models based on CPU usage, ensuring efficient resource utilization. The approach leverages epsilon-greedy strategy which promotes fairness and prevents resource starvation, maintaining system robustness. The results of our evaluation demonstrate significant improvements by balancing computational efficiency and accuracy, highlighting the approach's ability to adapt seamlessly to varying workloads. This work lays the groundwork for further advancements in self-adaptation for resource-constrained environments.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
- [1]
-
[3]
SWITCH: An Exemplar for Evaluating Self-Adaptive ML-Enabled Systems
Arya Marda, Shubham Shantanu Kulkarni, and Karthik Vaidhyanathan, ”SWITCH: An Exemplar for Evaluating Self-Adaptive ML- Enabled Systems,” arXiv.org, abs/2402.06351, 2024. DOI: 10.48550/arxiv.2402.06351
work page Pith review arXiv doi:10.48550/arxiv.2402.06351 2024
-
[4]
Theresia Ratih Dewi Saputri and Seok-Won Lee, ”The Application of Machine Learning in Self-Adaptive Systems: A Systematic Litera- ture Review,” IEEE Access , vol. 8, pp. 205948–205967, 2020. DOI: 10.1109/ACCESS.2020.3036037
arXiv 2020
-
[5]
Meghana Tedla, Shubham Shantanu Kulkarni, and Karthik Vaid- hyanathan, ”EcoMLS: A Self-Adaptation Approach for Architecting Green ML-Enabled Systems,” arXiv.org, abs/2404.11411, 2024
work page Pith review arXiv 2024
-
[7]
James Adu Ansere, Dung T. Tran, Octavia A. Dobre, Hyundong Shin, George K. Karagiannidis, Trung Q. Duong, ”Energy-Efficient Optimiza- tion for Mobile Edge Computing With Quantum Machine Learning,” IEEE Wireless Communications Letters ( Volume: 13, Issue: 3) , March
- [8]
-
[9]
Available: https://www.coral.ai/models/object-detection/?utm source=chatgpt.com, Accessed: Dec
Coral, ”An Object Detection Model Can Identify Multiple Ob- jects and Their Location in an Image,” Coral AI Website , [Online]. Available: https://www.coral.ai/models/object-detection/?utm source=chatgpt.com, Accessed: Dec. 27, 2024
work page 2024
-
[11]
Dagoberto Castellanos–Nieves and Luis Garc ´ıa-Forte, ”Strategies of Au- tomated Machine Learning for Energy Sustainability in Green Artificial Intelligence,” Applied Sciences , vol. 14, no. 14, pp. 6196–6196, 2024. DOI: 10.3390/app14146196
Show all 39 references
-
[12]
Dagoberto Castellanos-Nieves and Luis Garc ´ıa-Forte, ”Improving Auto- mated Machine-Learning Systems through Green AI,” Applied Sciences,
-
[13]
Eryilmaz, F
E. Eryilmaz, F. Trollmann, and S. Albayrak, ”Conceptual application of the MAPE-K feedback loop to opportunistic sensing,” in 2015 IEEE Symposium on Service-Oriented System Engineering (SOSE) , 2015, pp. 1–6, doi: https://doi.org/10.1109/SDF.2015.7347697
2015
-
[14]
X. Chen, H. Wang, Y . Zheng, and L. Guo, ”Self-adaptive resource allocation for cloud-based software services based on iterative QoS prediction model,” Future Generation Computer Systems , vol. 105, pp. 287–296, 2020. DOI: 10.1016/j.future.2019.12.005
2020 doi
- [15]
-
[16]
M ¨uhlbauer, F
S. M ¨uhlbauer, F. Sattler, C. Kaltenecker, J. Dorn, S. Apel, and N. Siegmund, ”Analysing the Impact of Workloads on Modeling the Perfor- mance of Configurable Software Systems,” in Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), ...
2023
-
[17]
T. Chen, Y . Liu, and J. Chen, ”An integrated approach to active model adaptation and on-line dynamic optimisation of batch processes,” Journal of Process Control , vol. 23, pp. 1350–1359, 2013. DOI: 10.1016/J.JPROCONT.2013.09.010
2013 doi
- [18]
-
[19]
Zhang, N
Z. Zhang, N. Wang, H. Wu, C. Tang, and R. Li, ”MR-DRO: A Fast and Efficient Task Offloading Algorithm in Heterogeneous Edge/Cloud Computing Environments,” IEEE Internet of Things Journal , vol. 10, pp. 3165–3178, 2023. DOI: 10.1109/JIOT.2021.3126101
2023
-
[20]
T. Dinh, Q. La, T. Quek, and H. Shin, ”Learning for Com- putation Offloading in Mobile Edge Computing,” IEEE Transac- tions on Communications , vol. 66, pp. 6353–6367, 2018. DOI: 10.1109/TCOMM.2018.2866572
2018
-
[21]
Y . Li, X. Wang, X. Gan, H. Jin, L. Fu, and X. Wang, ”Learning- Aided Computation Offloading for Trusted Collaborative Mobile Edge Computing,” IEEE Transactions on Mobile Computing , vol. 19, pp. 2833–2849, 2020. DOI: 10.1109/TMC.2019.2934103
2020
-
[22]
Gheibi, D
O. Gheibi, D. Weyns, and F. Quin, ”Applying Machine Learning in Self- Adaptive Systems,” ACM Transactions on Autonomous and Adaptive Systems (TAAS), vol. 15, pp. 1–37, 2020. DOI: 10.1145/3469440
2020 doi
-
[23]
Casimiro, D
M. Casimiro, D. Soares, D. Garlan, L. Rodrigues, and P. Romano, ”Self- Adapting Machine Learning-based Systems via a Probabilistic Model Checking Framework,” ACM Transactions on Autonomous and Adaptive Systems, 2024. DOI: 10.1145/3648682
2024 doi
-
[24]
Available: https://developer.android.com/studio
Google, ”Android Studio: The Official IDE for Android.” [Online]. Available: https://developer.android.com/studio
-
[25]
Strubell, A
E. Strubell, A. Ganesh, and A. McCallum, ”Energy and Policy Consid- erations for Deep Learning in NLP,” arXiv preprint arXiv:1906.02243 , 2019
1906 arXiv
-
[26]
Self-adaptive systems: A system- atic literature review across categories and domains,
T. Wong, M. Wagner, and C. Treude, “Self-adaptive systems: A system- atic literature review across categories and domains,” Information and Software Technology, vol. 148, 2022
2022
-
[27]
A synthesis of green architectural tactics for ml-enabled systems,
Heli J ¨arvenp¨a¨a and Patricia Lago and Justus Bogner and Grace Lewis and Henry Muccini and Ipek Ozkaya, “A synthesis of green architectural tactics for ml-enabled systems,” arXiv preprint arXiv:2312.09610, 2023
2023 arXiv
-
[28]
An energy-aware approach to design self-adaptive ai-based applications on the edge,
A. Tundo, M. Mobilio, S. Ilager, I. Brandic, E. Bartocci, and L. Mariani, “An energy-aware approach to design self-adaptive ai-based applications on the edge,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE Computer Society, 2023
2023
-
[29]
Huang, V
J. Huang, V . Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y . Song, S. Guadarrama, and K. Murphy, ”Speed/Accuracy Trade-offs for Modern Convolutional Object Detec- tors,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Reco...
2017 doi
-
[30]
Kotar and R
K. Kotar and R. Mottaghi, ”Interactron: Embodied Adaptive Object De- tection,” in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2022, pp. 14840–14849. [Online]. Available: https...
2022
-
[31]
J.-B. Hou, X. Zhu, and X.-C. Yin, ”Self-Adaptive Aspect Ra- tio Anchor for Oriented Object Detection in Remote Sensing Im- ages,” Remote Sensing , vol. 13, no. 7, 2021. [Online]. Available: https://www.mdpi.com/2072-4292/13/7/1318
2021
-
[32]
Zhang, Z
C. Zhang, Z. Li, J. Liu, P. Peng, Q. Ye, S. Lu, T. Huang, and Y . Tian, ”Self-Guided Adaptation: Progressive Representation Alignment for Domain Adaptive Object Detection,” IEEE Transactions on Multimedia, vol. PP, pp. 1–1, May 2021
2021
-
[33]
Goyal and J
K. Goyal and J. Singhai, ”Texture-Based Self-Adaptive Moving Ob- ject Detection Technique for Complex Scenes,” Computers Electri- cal Engineering , vol. 70, pp. 275–283, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0045790616301501
2018
-
[34]
S. Deng, S. Li, K. Xie, W. Song, X. Liao, A. Hao, and H. Qin, ”A Global-Local Self-Adaptive Network for Drone-View Object Detection,” IEEE Transactions on Image Processing, vol. 30, pp. 1556–1569, 2021
2021
-
[35]
D. Weyns, ”Engineering Self-Adaptive Software Systems – An Or- ganized Tour,” in Proceedings of the 2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W) , Sep. 2018. DOI: 10.1109/FAS-W.2018.00012
2018
-
[36]
M. Tan, R. Pang, and Q. Le, ”EfficientDet: Scalable and Efficient Object Detection,” in Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10778–10787,
2020
- [37]
-
[38]
S. A. Bashir, F. Khursheed, and I. Abdoulahi, ”Adaptive Epsilon-Greedy Exploration for Finite Systems,” Global Operations Research , vol. 34, no. 4, Nov. 2021. DOI: 10.37896/GOR34.04/044
2021 doi
-
[39]
Thadikamalla and P
S. Thadikamalla and P. Joshi, ”Exploration Strategies in Adaptive Traffic Signal Control: A Comparative Analysis of Epsilon-Greedy, UCB, Softmax, and Thomson Sampling,” in Proceedings of the 2023 7th International Symposium on Innovative Approaches in Smart Tech- nologies (ISA...
2023
-
[2020]
DOI: 10.1109/cvpr42600.2020.01079
2020
-
[2023]
DOI: 10.3390/app132011583
-
[2024]
DOI: 10.1109/LWC.2023.3338571
2023
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.