LA-IMR: Latency-Aware, Predictive In-Memory Routing and Proactive Autoscaling for Tail-Latency-Sensitive Cloud Robotics

Chanh Nguyen; Erik Elmroth; Eunil Seo

REVIEW 5 major objections 6 minor 41 references

LA-IMR: Latency-Aware, Predictive In-Memory Routing and Proactive Autoscaling for Tail-Latency-Sensitive Cloud Robotics

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Predictive latency model cuts cloud–edge P99 tail latency by up to 20.7%.

desk verdict A plausible systems result with a real measured P99 gain, but the latency model is not derived or validated as claimed and the attribution to the model is circular; worth a serious referee, but only with major revisions. read the letter →

arxiv 2505.07417 v3 pith:2FZKYAYL submitted 2025-05-12 cs.DC

Eunil Seo , Chanh Nguyen , Erik Elmroth This is my paper · ORCID

classification cs.DC

keywords taillatencyP99proactiveautoscalingedge-cloudinferencemodelingKubernetespredictiveroutingcloudrobotics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single closed-form latency model, calibrated once per hardware tier, can predict when an inference service is about to breach its latency service-level objective, and that acting on that prediction at millisecond timescales suppresses tail-latency spikes. The proposed control layer, LA-IMR, uses the model for three coupled decisions: routing each request to a replica that meets its latency budget, offloading traffic upstream when a burst is imminent, and scaling replica pools before queues build. On a Kubernetes edge–cloud testbed running YOLOv5m and EfficientDet under bursty arrivals, this proactive scheme lowers 99th-percentile response latency by up to 20.7% and cuts P99 variance by more than half relative to a latency-reactive autoscaler. If these gains hold, tail latency in hybrid edge–cloud inference can be controlled by model prediction rather than by reacting to overload after it appears.

What carries the argument

The load-bearing object is the calibrated affine power-law latency model, $L^{\text{infer}}_{m,i} = (L_m / S_{m,i})(1 + U_i^{\gamma})$, expanded to the per-replica form $L = \alpha + \beta \tilde{\lambda}^{\gamma}$, where $\tilde{\lambda}$ is the per-replica arrival rate, $\alpha$ is the idle baseline latency, $\beta$ is the load slope, and $\gamma$ is an empirically fitted super-linear exponent. This model is combined with an Erlang-C $M/M/c$ queueing delay term, and the two instantiations $g_{m,i}(\lambda)$ and $g_{m,i}(N_{m,i})$ together translate real-time arrival-rate measurements into two control actions: immediate routing and offloading decisions, and proactive replica-count targets exported as a custom Kubernetes metric. The machinery works by turning queue build-up from a lagging symptom into a predicted quantity, so capacity changes can start before latency actually degrades.

What would settle it

Calibrate Eq. (8) on one workload and hardware mix, then run LA-IMR on a different bursty trace or a different tier composition without recalibrating the model's parameters. If the P99 reductions vanish or turn negative, or if the model's predicted latencies deviate from measured latencies by more than a few percent, the claim that proactive model-predicted control is what suppresses tail latency would be called into question.

Watch

Extended reading notes

Core claim

The core claim is that end-to-end inference latency can be written as a closed-form affine power law of per-replica utilization, $L = \alpha + \beta \tilde{\lambda}^{\gamma}$, plus a network round-trip term and an $M/M/c$ queueing delay, and that this one expression can drive both fast per-request routing and slower capacity planning. The latency model has two complementary instantiations — fixed-replica latency as a function of arrival rate, $g_{m,i}(\lambda)$, and fixed-traffic latency as a function of replica count, $g_{m,i}(N_{m,i})$ — so the router can precompute feasible replicas from the first and the autoscaler can export a desired-replica-count metric to Kubernetes from the second. Acting on these predictions before queues form, LA-IMR keeps P99 latency inside the configured SLO envelope and, in the reported bursty-load experiments, reduces P99 latency by up to 20.7% at peak arrival rate, narrows the inter-quartile range by 27%, reduces the maximum outlier by 41%, and lowers P99 standard deviation by more than 60%.

Load-bearing premise

The whole control loop depends on the calibrated affine power-law latency model, Eq. (8), staying accurate for the actual bursty arrival traces and heterogeneous tiers, even though its parameters and the EWMA weight, utilization floor, and latency-budget multiplier are fitted and tuned offline on the same testbed.

Editorial extensions

If this is right

Under bursty arrivals, LA-IMR reduces P99 latency by up to 20.7% at peak load and by roughly 9% on average compared with latency-reactive autoscaling.
P99 latency variance drops by more than half, with the inter-quartile range shrinking by 27% and the worst outlier by 41%.
The custom desired-replica metric removes the 60–120 s reaction lag typical of CPU-driven Kubernetes autoscaling, enabling just-in-time scale-out before queues build.
The same closed-form model serves both millisecond-scale routing and slower capacity planning, so the control loop has a single consistent latency prediction to act on.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the affine power-law form transfers to other inference services, the same controller could be applied to generic microservice SLO management without retraining a model: the calibration cost is only three parameters per hardware tier.
The paper's own suggested extension to fast- and slow-window arrival estimators is a natural test: it would show whether the model's predictive gains survive the correlated, multi-service spikes that the bounded-Pareto emulation does not capture.
A stricter SLO multiplier $x$ would likely expose how sensitive the gains are to the offline-tuned EWMA weight, utilization floor, and latency-budget multiplier, motivating the online self-tuner the paper lists as future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: read this as a systems paper, not a theory paper. The measured 20.7% P99 reduction over a reactive autoscaler is concrete and probably real. But the paper does not show that the analytic model is what produces the gain.

What's actually new: an event-driven control loop that combines a fitted power-law latency model, Erlang-C queueing, in-memory per-request routing, and a Kubernetes custom-metric autoscaler into one system. That integration is not in the literature, and the evaluation on a real CloudGripper testbed with two vision workloads is a step up from simulation. The authors also admit the offline tuning and the shared-cluster noise, which is honest.

The soft spots are real. Eq. (8) is presented as a derivation from Eq. (5), but it isn't: expanding U^γ with U = B/Rmax + (λR/Rmax) gives cross terms unless γ=1. So the affine power law is an assumed form, not a derived one. The validation in Fig. 2 uses γ=1.49, but the experiments use γ=0.90, with no reconciliation. Figure 2 fits the model to the same measurements it calls validation—that's not a prediction. And Table VI shows LA-IMR P99 exceeding the stated 1.8s SLO at every load, so the claim that the system keeps P99 inside the envelope is false. The baseline is only described as 'latency-focused', and there is no ablation that replaces the model with a simple capacity rule, so the gain could come from proactive scaling and offline tuning rather than the analytic model. No code or data artifacts are provided, which doesn't help.

Who's it for: people working on cloud-edge autoscaling might get a useful data point and a reminder that proactive scaling helps. But the paper's central modeling claim doesn't hold up as written, and the lack of a no-model ablation means the mechanism is unproven.

My recommendation: send it to peer review—the system and the empirical result deserve referee time—but the authors should be told to fix the derivation, reconcile gamma, add an ablation with a non-model-based proactive scaler, and release artifacts. Without those changes, the 'predictive' component will remain circular.

Referee Report

5 major / 6 minor

Summary. The paper presents LA-IMR, a control layer for hybrid cloud–edge inference that combines a closed-form latency model, event-driven routing, replica autoscaling, and edge-to-cloud offloading. The latency model decomposes end-to-end delay into an affine power-law processing term, a network RTT term, and an Erlang-C queueing term. LA-IMR uses this model to make per-request routing decisions and to export a desired-replica-count custom metric to Kubernetes HPA. The evaluation on a CloudGripper testbed with YOLOv5m and EfficientDet workloads reports that LA-IMR reduces P99 latency by up to 20.7% and cuts P99 variance by more than half compared with a latency-focused reactive autoscaler under bursty arrival rates.

Significance. If the results hold, the system would be a valuable practical contribution: it unifies predictive routing, proactive autoscaling, and offloading in one event-driven loop, and it demonstrates a working Kubernetes-based implementation on a public testbed. The paper also provides a detailed algorithm (Algorithm 1) and reports concrete measurements. However, the central analytic derivation contains a mathematical error, the calibration parameters are inconsistent across sections, the reported P99 latency violates the paper's own SLO at every load level, and the baseline/ablation design does not isolate the contribution of the latency model. These issues are load-bearing for the claim that model-predicted routing and scaling produce the observed gains.

major comments (5)

[§III-C(c), Eqs. (7)–(9)] The derivation of the affine power-law form is algebraically incorrect. Starting from Eq. (5), L = (L_m/S_m,i)[1 + U_i^γ], and substituting U_i = (λ R_m + B_i)/R_max^i, one obtains (L_m/S_m,i)[1 + ((λ R_m + B_i)/R_max^i)^γ], which does not equal Eq. (7)'s α_i + β_m,i λ̃^γ unless γ = 1 or one of the terms in the sum vanishes. For γ ≠ 1, (a + b)^γ ≠ a^γ + b^γ. The affine power-law form should therefore be presented as an empirical approximation, not as a derivation from Eq. (5). This does not by itself invalidate the model, but the paper's claim of a 'derived' closed-form law is overstated.
[§III-C(d) vs §V-A(4)] The calibration in §III-C(d) reports γ = 1.49 for the YOLOv5m model (Fig. 2), while §V-A(4) states that the experiments use γ = 0.90. These values are inconsistent, and no reconciliation or sensitivity analysis is provided. Because γ controls the super-linear growth of predicted latency and directly triggers scaling/offloading decisions in Algorithm 1, the evaluation does not use the same model that was validated. This is a load-bearing inconsistency that casts doubt on whether the validated model is actually the one being evaluated.
[Table VI and §V-A(4)] Table VI shows that LA-IMR's P99 latency exceeds the stated SLO τ = x·L_m = 1.8 s at every arrival rate: 1.989 s at λ = 1, rising to 5.435 s at λ = 6. This directly contradicts statements in the Introduction and §IV-D that LA-IMR 'maintains task-level P99 within SLOs' and 'keeps the p99 latency inside the xL_infer envelope.' A tail-latency controller that misses its own SLO in every tested condition is not meeting its primary objective, even if it improves on the baseline. This needs to be addressed explicitly, either by revising the SLO definition or by acknowledging that the system does not meet the target.
[§V-B, §V-C, §V-D] The baseline is described only as 'latency-focused' or 'reactive' autoscaling with Prometheus-measured latency, and no concrete scaling rule, threshold, or tuning procedure is given. Moreover, there is no ablation that replaces the analytic latency model with a simple capacity rule (e.g., scale on arrival-rate threshold or queue-length threshold). Section V-D admits that the EWMA weight α, utilisation floor ρ_low, and latency-budget multiplier x are tuned offline for the tested SLOs. Without such an ablation, the reported P99 reduction cannot be attributed specifically to the predictive latency model rather than to proactive scaling, offloading, or offline parameter tuning.
[§III-C(d), Fig. 2] The 'empirical validation' in §III-C(d) fits α, β, and γ to the same measurements shown in Table IV and then presents the fitted curve in Fig. 2 as a 'prediction.' No train/test split or holdout validation is described, so the claim that the model tracks observed latencies 'within a few percent' is not evidence of predictive accuracy. This circularity is especially problematic because the same calibrated model is then used to drive the proactive mechanisms in the evaluation.

minor comments (6)

[§III-D] The queueing model is presented inconsistently: the text says 'each replica behaves as an M/M/c queue with c = N servers,' but with round-robin per-replica arrival rate λ/N, the correct model is N independent M/M/1 queues, not one M/M/N queue. Please clarify which model is actually used in the implementation and how the queueing delay in Eq. (12) is computed.
[§IV-A] There are typographical errors: 'early-latency spiks detection' should be 'spike detection,' and the sentence starting 'Because the scaling trigger is' in §IV-D would be clearer if split into two sentences.
[Algorithm 1] The EWMA state λ_accum is used in line 15 but its initial value is never specified; please state how it is initialized at system start.
[§V-A(4)] The paper claims 'millisecond-scale' decisions, but the HPA reconciliation loop runs every 5 s and pod startup takes ~1.8 s; please clarify that the millisecond claim refers to routing per request, not to autoscaling reaction time.
[Figures] Fig. 2 caption does not state that the plotted model is for YOLOv5m; please make the workload explicit. Fig. 4 caption says 'when the arrival rate λ=4 is given' but the figure likely compares across replica counts; consider rewording for clarity.
[References] Some references are to arXiv preprints or non-archival sources (e.g., [16], [30], [34]); consider citing peer-reviewed versions where available, and ensure all citations in the text have a corresponding entry.

Circularity Check

1 steps flagged · score 6.0 of 10

The latency model's 'predictive' validation is an in-sample fit; the P99 gain itself is measured, so circularity is partial.

fitted input called prediction [Section III-C(d), Fig. 2 and Eq. (8)]
"Fig. 2 shows that Eq. (8), with calibrated parameters α_i=0.73, β_m,i=1.29, and γ=1.49, closely matches the measurements. Because the three parameters are re-estimated whenever the hardware mix (S_m,i, R_max_i) or co-tenant load (B_i) changes, the model remains accurate under a wide range of deployment conditions. Such predictive capability is valuable for proactive resource provisioning and request routing during workload fluctuations."

The α, β, γ parameters are calibrated from the same Table IV measurements that Fig. 2 displays, so the close match is a least-squares fit to the plotted data, not an out-of-sample prediction. The paper nevertheless presents this agreement as evidence of 'predictive capability' and uses Eq. (8) in Algorithm 1 to trigger offloading and scaling. The model's claimed accuracy is therefore established only by construction; no holdout data or parameter-free test verifies that the fitted curve predicts the evaluated traces.

full rationale

The headline P99 reduction (up to 20.7%) is a measured system comparison against a reactive baseline, so it is not itself the output of the latency model. Circularity is partial: the load-bearing validation of the model that drives LA-IMR's 'predictive' decisions is an in-sample fit. Additional non-circular weaknesses corroborate the concern: Eq. (8) is presented as an expansion of Eq. (5), but for γ≠1 the algebra does not hold, making the form an assumed ansatz; the validation uses γ=1.49 while §V-A(4) uses γ=0.90; Table VI shows LA-IMR's P99 exceeds the stated SLO τ=1.8s at every arrival rate; and §V-D admits that the EWMA weight, utilization floor, and SLO multiplier x are tuned offline for the tested SLOs. These issues undermine attribution of the gain to the model but do not make the measured P99 comparison itself circular.

Assumptions & free parameters 8 free parameters · 5 assumptions · 2 invented entities

The model is an empirical fit with three hardware-dependent parameters per tier, several hand-tuned control knobs, and assumptions about exponential service times, even load distribution, and stationarity over 1-second windows. The algebraic step from utilization to affine power law is invalid as written, so the power-law form is best treated as an assumed model rather than a derived one.

free parameters (8)

alpha_i (baseline latency offset) = 0.73 (Fig. 2 calibration)
Baseline latency at idle utilization; estimated from the same Table IV data shown in Fig. 2.
beta_m,i (power-law slope) = 1.29 (Fig. 2 calibration)
Scaling coefficient for per-replica arrival rate; fitted to the calibration data.
gamma (latency exponent) = 1.49 (Fig. 2); 0.90 (Section V experiments)
Exponent in the affine power-law model. The paper uses two different values for validation and for the system evaluation, so it is effectively a per-experiment fitted constant.
S_m,i (hardware speed-up) = CPU=1, GPU=2-20, TPU=30-100+ (Table III)
Approximated from prior literature, not measured per instance; enters Eq. (5)-(9).
EWMA smoothing weight alpha = 0.8
Hand-tuned control parameter for the smoothed arrival rate; Section V-A-4.
cost-latency trade-off beta = 2.5
Hand-tuned weight in Eq. (23); Section V-A-4.
latency-budget multiplier x = 2.25
Sets the SLO tau = x*L_infer; tuned offline for the test scenario as admitted in Section V-D.
utilisation floor rho_low = not reported
Scale-in threshold in Algorithm 1; tuned offline per Section V-D.

assumptions (5)

domain assumption Inference service times and inter-arrival times are exponentially distributed
Needed for the M/M/c Erlang-C queueing delay in Eq. (11)-(12). Real inference latencies are not memoryless; this is unvalidated.
ad hoc to paper The affine power-law form L = alpha + beta * lambda^gamma is a valid model of per-replica inference latency
Eq. (8) is not derived from Eq. (5); the 'expansion' of U^gamma is algebraically invalid. The form is assumed and fitted.
domain assumption Replica load is distributed evenly via round-robin, making per-replica utilization U_i = (lambda/N * R + B)/R_max
Eq. (10) and (7) assume even distribution; the M/M/c model then treats the N replicas as one shared queue, which is inconsistent with round-robin.
domain assumption A 1-second sliding-window arrival rate captures the instantaneous workload for SLO prediction
Used in Algorithm 1 SLIDINGRATE; bursty arrivals shorter than 1 s may not be detected.
domain assumption The calibrated parameters remain valid across the heterogeneous edge and cloud tiers and the bounded-Pareto burst traces
Calibration done on Raspberry Pi 4 with 3 CPUs; system evaluation uses the same cluster but the paper does not show transfer to cloud tier.

invented entities (2)

LA-IMR control layer independent evidence
purpose: SLO-aware routing and autoscaling controller
A software system, not a physical postulate; implemented and evaluated experimentally.
PM-HPA (Predictive-Metric Horizontal Pod Autoscaler) independent evidence
purpose: Proactive Kubernetes autoscaler exporting desired replicas custom metric
Implemented and benchmarked; not an unobserved entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LA-IMR: Latency-Aware, Predictive In-Memory Routing and Proactive Autoscaling for Tail-Latency-Sensitive Cloud Robotics." pith.science (2026). https://pith.science/paper/2FZKYAYL

@misc{pith2026250507417,
  author       = {Pith},
  title        = {Pith review of: LA-IMR: Latency-Aware, Predictive In-Memory Routing and Proactive Autoscaling for Tail-Latency-Sensitive Cloud Robotics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2FZKYAYL}},
  note         = {Machine review of arXiv:2505.07417}
}

read the original abstract

Hybrid cloud-edge infrastructures now support latency-critical workloads ranging from autonomous vehicles and surgical robotics to immersive AR/VR. However, they continue to experience crippling long-tail latency spikes whenever bursty request streams exceed the capacity of heterogeneous edge and cloud tiers. To address these long-tail latency issues, we present Latency-Aware, Predictive In-Memory Routing and Proactive Autoscaling (LA-IMR). This control layer integrates a closed-form, utilization-driven latency model with event-driven scheduling, replica autoscaling, and edge-to-cloud offloading to mitigate 99th-percentile (P99) delays. Our analytic model decomposes end-to-end latency into processing, network, and queuing components, expressing inference latency as an affine power-law function of instance utilization. Once calibrated, it produces two complementary functions that drive: (i) millisecond-scale routing decisions for traffic offloading, and (ii) capacity planning that jointly determines replica pool sizes. LA-IMR enacts these decisions through a quality-differentiated, multi-queue scheduler and a custom-metric Kubernetes autoscaler that scales replicas proactively -- before queues build up -- rather than reactively based on lagging CPU metrics. Across representative vision workloads (YOLOv5m and EfficientDet) and bursty arrival traces, LA-IMR reduces P99 latency by up to 20.7 percent compared to traditional latency-only autoscaling, laying a principled foundation for next-generation, tail-tolerant cloud-edge inference services.

Figures

Figures reproduced from arXiv: 2505.07417 by the authors.

**Figure 2.** The inference latency measured in the real operations [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Latency metrics for user robot19 under varying arrival rates, showing super-linear growth in average, P95, and P99 latencies. where ρm,i Nm,i = λm Nm,iµm,i < 1 (stability constraint). Processing and network delays are unaffected by the replica count once λ is fixed. Queueing delay shrinks as Nm,i grows because both the service-pool capacity Nm,iµm,i increases linearly and the utilisation ρm,i(Nm,i) falls hyperbolic… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Inference latency comparison between the microservice and monolithic service architecture as the number of replica [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Real-time latency prediction uses the arrival rate [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: CloudGripper work cells performing object manipula [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Latency comparison of LA-IMR and the baseline latency-based method across varying arrival rates [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Box-plots of P99 latencies (Prometheus measurements) [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 35 canonical work pages

[1]

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection,

L. Suresh, M. Canini, S. Schmid, and A. Feldmann, “C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection,” in12th USENIX Symposium on Networked Systems Design and Implementation (NSDI). USENIX, 2015, pp. 513–528

work page 2015
[2]

The Tail at Scale,

J. Dean and L. A. Barroso, “The Tail at Scale,”Communications of the ACM, vol. 56, no. 2, pp. 74–80, 2013

work page 2013
[3]

GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks,

R. S. Kannan, L. Subramanian, A. Raju, J. Ahn, J. Mars, and L. Tang, “GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks,” inProceedings of the Fourteenth EuroSys Conference

work page
[4]

An edge-cloud integrated framework for flexible and dynamic stream analytics,

X. Wang, A. Khan, J. Wang, A. Gangopadhyay, C. E. Busart, and J. Freeman, “An edge-cloud integrated framework for flexible and dynamic stream analytics,”Future Generation Computer Systems, vol. 135, pp. 158–171, 2022

work page 2022
[5]

Resource Provisioning in Edge Computing for Latency Sensitive Applications

A. Abouaomar, S. Cherkaoui, Z. Mlika, and A. Kobbane, “Resource provisioning in edge computing for latency sensitive applications,”arXiv preprint arXiv:2201.11837, 2022

work page Pith review arXiv 2022
[6]

Latency minimization for intelligent reflecting surface aided mobile edge computing,

T. Bai, C. Pan, Y . Deng, M. Elkashlan, A. Nallanathan, and L. Hanzo, “Latency minimization for intelligent reflecting surface aided mobile edge computing,”arXiv preprint arXiv:1910.07990, 2019

arXiv 1910
[7]

Dependency-aware task offloading based on application hit ratio,

J. Zhang, X. Wang, P. Yuan, H. Dong, P. Zhang, and Z. Tari, “Dependency-aware task offloading based on application hit ratio,”IEEE Transactions on Services Computing, vol. 17, no. 6, pp. 3373–3387, 2024

work page 2024
[8]

Redis: Remote Dictionary Server,

S. Sanfilippo, “Redis: Remote Dictionary Server,” https://redis.io/, 2025, accessed: 2025-04-05

work page 2025

Show all 41 references

[9]

Ultra-low latency multi-task offloading in mobile edge computing,

H. Zhang, Y . Yang, X. Huang, C. Fang, and P. Zhang, “Ultra-low latency multi-task offloading in mobile edge computing,”IEEE Access, vol. 9, pp. 32 569–32 580, 2021

2021
[10]

Offloading schemes in mobile edge computing for ultra-reliable low latency communications,

J. Liu and Q. Zhang, “Offloading schemes in mobile edge computing for ultra-reliable low latency communications,”IEEE Access, vol. 6, pp. 12 825–12 837, 2018

2018
[11]

Model-driven cluster resource management for ai workloads in edge clouds,

Q. Liang, W. A. Hanafy, A. Ali-Eldin, and P. Shenoy, “Model-driven cluster resource management for ai workloads in edge clouds,”ACM Transactions on Autonomous and Adaptive Systems, vol. 18, no. 1, pp. 1–26, 2023

2023
[12]

ECO: Edge- Cloud Optimization of 5G Applications,

K. Rao, G. Coviello, W.-P. Hsiung, and S. Chakradhar, “ECO: Edge- Cloud Optimization of 5G Applications,” in2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid). IEEE, 2021, pp. 649–658

2021
[13]

Queue-length-based offloading for delay sensitive applications in federated cloud-edge-fog systems,

R.-H. Hwang, Y .-C. Lai, and Y .-D. Lin, “Queue-length-based offloading for delay sensitive applications in federated cloud-edge-fog systems,” in IEEE Consumer Communications and Networking Conference (CCNC). IEEE, 2021, pp. 1–6

2021
[14]

Partial offloading schemes for latency and computation sensitive tasks,

J. Ahmad, M. S. Hossain, F. Al Awsaf, A. M. Islam, and S. M. Hasan, “Partial offloading schemes for latency and computation sensitive tasks,” in2022 IEEE Region 10 Symposium (TENSYMP). IEEE, 2022, pp. 1–6

2022
[15]

Autoscaling for Handling Peak Loads in Cloud Applications,

A. Gandhi, S. Koelbl, M. Harchol-Balter, and A. Wolman, “Autoscaling for Handling Peak Loads in Cloud Applications,”IEEE Transactions on Services Computing, vol. 7, no. 4, pp. 707–719, 2014

2014
[16]

Deep Reinforcement Learning for Autoscaling Virtual Machines in Cloud Environments,

L.-Y . Chen, S. Katsikas, and Q. Zhang, “Deep Reinforcement Learning for Autoscaling Virtual Machines in Cloud Environments,” inProceed- ings of the International Conference on Service-Oriented Computing (ICSOC), 2020, pp. 105–122

2020
[17]

Large-Scale Cluster Management at Google with Borg,

A. Verma, L. Pedrosa, M. Korupolu, D. Oppenheimer, E. Tune, and J. Wilkes, “Large-Scale Cluster Management at Google with Borg,” in Proceedings of the 10th European Conference on Computer Systems (EuroSys), 2015, pp. 1–17

2015
[18]

Predictive Autoscaling for Cloud HPC: Combining Performance Models and Observations,

E. Caron, F. Desprez, and A. Muresan, “Predictive Autoscaling for Cloud HPC: Combining Performance Models and Observations,”Future Generation Computer Systems, vol. 86, pp. 427–438, 2018

2018
[19]

Machine Learning-Driven Autoscal- ing of Microservices in Containerized Environments,

N. Sharma, R. Kaur, and I. Singh, “Machine Learning-Driven Autoscal- ing of Microservices in Containerized Environments,” inProceedings of the IEEE International Conference on Cloud Engineering (IC2E), 2020, pp. 100–110

2020
[20]

Adaptive Cloud Resource Scaling for Real-Time Data Streaming,

A. Ali, X. Chen, and G. De Luca, “Adaptive Cloud Resource Scaling for Real-Time Data Streaming,”IEEE Transactions on Cloud Computing, vol. 9, no. 1, pp. 30–42, 2021

2021
[21]

AutoScale: Toward Automatic Resource Scaling in Microservices Using Machine Learning,

R. Kojima, Y . Sato, S. Kuroda, and T. Watanabe, “AutoScale: Toward Automatic Resource Scaling in Microservices Using Machine Learning,” inProceedings of the 22nd IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2022, pp. 428–438

2022
[22]

Heracles: Improving resource efficiency at scale with flexible fine-grained resource control,

D. Lo, L. Eyraud-Dubois, C. Kozyrakis, and P. Ranganathan, “Heracles: Improving resource efficiency at scale with flexible fine-grained resource control,” inProceedings of the 42nd Annual International Symposium on Computer Architecture (ISCA). IEEE, 2015, pp. 650–662

2015
[23]

Farm: Fast remote memory,

A. Dragojevi ´c, D. Narayanan, O. Hodson, and M. Castro, “Farm: Fast remote memory,” inProceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI). USENIX, 2014, pp. 401–414

2014
[24]

Faster: A concurrent key-value store with in-place updates,

B. Chandramouli, G. Graefe, and D. Zhukov, “Faster: A concurrent key-value store with in-place updates,” inProceedings of the 2018 International Conference on Management of Data (SIGMOD). ACM, 2018, pp. 275–290

2018
[25]

An analysis of in-memory system performance for real-time analytics,

Y . Qin, L. Zhang, and M. Sanchez, “An analysis of in-memory system performance for real-time analytics,”IEEE Transactions on Parallel and Distributed Systems, vol. 30, no. 8, pp. 1624–1637, 2019

2019
[26]

Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,

Y . Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” inProceedings of the 22nd International Conference on Architectural Support for Programming Languages and Operating Systems...

2017
[27]

Edgekv: An in-memory key-value store for dynamic offloading in edge environments,

H. Jeong, S. Lee, and M. Yoo, “Edgekv: An in-memory key-value store for dynamic offloading in edge environments,” inProceedings of the IEEE International Conference on Edge Computing. IEEE, 2021, pp. 85–94

2021
[28]

Adaptive in-memory offloading for real-time iot analytics,

D. Li, W. Xue, and S. Wu, “Adaptive in-memory offloading for real-time iot analytics,”Journal of Parallel and Distributed Computing, vol. 162, pp. 45–57, 2022

2022
[29]

Faster r-cnn: Towards real-time object detection with region proposal networks,

S. Ren, “Faster r-cnn: Towards real-time object detection with region proposal networks,”arXiv preprint arXiv:1506.01497, 2015

2015 arXiv
[30]

Yolov3: An incremental improvement,

A. Farhadi and J. Redmon, “Yolov3: An incremental improvement,” inComputer vision and pattern recognition, vol. 1804. Springer Berlin/Heidelberg, Germany, 2018, pp. 1–6

2018
[31]

Deep neural networks for object detection,

C. Szegedy, A. Toshev, and D. Erhan, “Deep neural networks for object detection,”Advances in neural information processing systems, vol. 26, 2013

2013
[32]

The pascal visual object classes (voc) challenge,

M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisser- man, “The pascal visual object classes (voc) challenge,”International journal of computer vision, vol. 88, pp. 303–338, 2010

2010
[33]

Speed/accuracy trade-offs for modern convolutional object detectors,

J. Huang, V . Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y . Song, S. Guadarramaet al., “Speed/accuracy trade-offs for modern convolutional object detectors,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp...

2017
[34]

Edge ai: On-demand accelerating deep neural network inference via edge computing,

E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge ai: On-demand accelerating deep neural network inference via edge computing,”IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 447–457, 2019

2019
[35]

A survey on mobile edge networks: Convergence of computing, caching and communications,

S. e. a. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,”IEEE Access, vol. 9, pp. 67 502–67 528, 2021

2021
[36]

Jellyfish: Timely inference serving for dynamic edge networks,

V . Nigade, P. Bauszat, H. Bal, and L. Wang, “Jellyfish: Timely inference serving for dynamic edge networks,” in2022 IEEE Real-Time Systems Symposium (RTSS), 2022, pp. 277–290

2022
[37]

In-datacenter performance analysis of a tensor processing unit,

N. P. Jouppi, C. Young, N. Patil, D. Patterson, and et al., “In-datacenter performance analysis of a tensor processing unit,”SIGARCH Comput. Archit. News, vol. 45, no. 2, p. 1–12, Jun. 2017. [Online]. Available: https://doi.org/10.1145/3140659.3080246

2017
[38]

Gpu implementation of neural networks,

K.-S. Oh and K. Jung, “Gpu implementation of neural networks,”Pattern Recognition, vol. 37, no. 6, pp. 1311–1314, 2004. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320304000524

2004
[39]

Kleinrock,Theory, Volume 1, Queueing Systems

L. Kleinrock,Theory, Volume 1, Queueing Systems. USA: Wiley- Interscience, 1975

1975
[40]

ultralytics/yolov5: YOLOv5 by Ultralytics,

G. Jocheret al., “ultralytics/yolov5: YOLOv5 by Ultralytics,” 2020. [Online]. Available: https://github.com/ultralytics/yolov5

2020
[41]

Efficientdet: Scalable and efficient object detection,

M. Tan, R. Pang, and Q. V . Le, “Efficientdet: Scalable and efficient object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 10 781–10 790. [Online]. Available: https://arxiv.org/abs/1911.09070

2020 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection,

L. Suresh, M. Canini, S. Schmid, and A. Feldmann, “C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection,” in12th USENIX Symposium on Networked Systems Design and Implementation (NSDI). USENIX, 2015, pp. 513–528

work page 2015

[2] [2]

The Tail at Scale,

J. Dean and L. A. Barroso, “The Tail at Scale,”Communications of the ACM, vol. 56, no. 2, pp. 74–80, 2013

work page 2013

[3] [3]

GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks,

R. S. Kannan, L. Subramanian, A. Raju, J. Ahn, J. Mars, and L. Tang, “GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks,” inProceedings of the Fourteenth EuroSys Conference

work page

[4] [4]

An edge-cloud integrated framework for flexible and dynamic stream analytics,

X. Wang, A. Khan, J. Wang, A. Gangopadhyay, C. E. Busart, and J. Freeman, “An edge-cloud integrated framework for flexible and dynamic stream analytics,”Future Generation Computer Systems, vol. 135, pp. 158–171, 2022

work page 2022

[5] [5]

Resource Provisioning in Edge Computing for Latency Sensitive Applications

A. Abouaomar, S. Cherkaoui, Z. Mlika, and A. Kobbane, “Resource provisioning in edge computing for latency sensitive applications,”arXiv preprint arXiv:2201.11837, 2022

work page Pith review arXiv 2022

[6] [6]

Latency minimization for intelligent reflecting surface aided mobile edge computing,

T. Bai, C. Pan, Y . Deng, M. Elkashlan, A. Nallanathan, and L. Hanzo, “Latency minimization for intelligent reflecting surface aided mobile edge computing,”arXiv preprint arXiv:1910.07990, 2019

arXiv 1910

[7] [7]

Dependency-aware task offloading based on application hit ratio,

J. Zhang, X. Wang, P. Yuan, H. Dong, P. Zhang, and Z. Tari, “Dependency-aware task offloading based on application hit ratio,”IEEE Transactions on Services Computing, vol. 17, no. 6, pp. 3373–3387, 2024

work page 2024

[8] [8]

Redis: Remote Dictionary Server,

S. Sanfilippo, “Redis: Remote Dictionary Server,” https://redis.io/, 2025, accessed: 2025-04-05

work page 2025