Pith. sign in

REVIEW 4 major objections 5 minor 69 references

Towards VM Rescheduling Optimization Through Deep Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VM rescheduling: RL matches optimal within 3% in 1.1 seconds

desk verdict Solid RL-for-scheduling paper with a useful latency-aware formulation, but the '2.86% from optimal' headline is not backed by a certified MIP gap. read the letter →

arxiv 2505.17359 v1 pith:2FTN35LC submitted 2025-05-23 cs.LG

Xianzhong Ding , Yunkai Zhang , Binbin Chen , Donghao Ying , Tieying Zhang , Jianjun Chen , Lei Zhang , Alberto Cerpa
show 1 more author
Wan Du
This is my paper
classification cs.LG
keywords virtualmachinereschedulingdeepreinforcementlearningfragmentratecloudresourcemanagementtwo-stageactiondecompositionsparseattentionrisk-seekingevaluationlatency-awareoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that VM rescheduling has a property most scheduling optimizations lack: while a solver is thinking, VMs keep arriving and leaving, so a slow answer is a worse answer even if it is mathematically better. It proposes VMR2L, a deep reinforcement learning system that treats each migration as one step in an episode, trains offline on logged VM-PM mappings in a deterministic simulator, and returns a rescheduling plan in about one second. On an industry-scale dataset, VMR2L lands within 2.86% of the fragment rate achieved by an exact mathematical optimizer at a 50-migration limit, where the optimizer needs 50 minutes. The paper also shows the system can absorb service constraints, different objectives, and workload distributions outside its training set.

What carries the argument

The load-bearing machinery is a deterministic-transition reinforcement learning setup plus three customizations: a two-stage actor decomposition in which a VM actor first picks which VM to move and a PM actor then picks a destination from the machines that remain feasible; sparse local attention that lets each VM attend only to the physical machine and sibling VMs in its own placement tree; and risk-seeking evaluation, which samples multiple full migration trajectories from the trained policy and deploys the single trajectory the simulator predicts to be best. The paper also uses a dense reward based on the change in fragment size on the source and destination physical machines rather than a single sparse final fragment-rate reward.

What would settle it

Deploy a VMR2L policy in a live cluster, record the actual VM arrivals and exits during its 1.1-second planning window, then execute the plan the simulator ranks best and measure the realized fragment rate; if the realized fragment rate is no better than a simple greedy heuristic run under the same five-second budget, the deterministic-simulator assumption that carries the argument fails.

Watch

Extended reading notes

Core claim

The central claim is that VM rescheduling should be modeled as a sequential decision problem, not a one-shot optimization, and that a reinforcement learning agent trained on a perfect simulator can make the problem tractable at data-center scale. The fragment rate is the share of CPU capacity too fragmented to host a 16-core VM, and the agent's policy decomposes the action 'migrate this VM to that PM' into two choices, uses sparse local attention so each VM sees its sibling VMs on the same physical machine, and, at deployment, samples several candidate migration trajectories and deploys only the one the simulator ranks best. The measured consequence is that VMR2L achieves a fragment rate of 0.2941 versus 0.2859 for the near-optimal mixed-integer programming reference at a migration limit of 50, while keeping inference at 1.1 seconds and scaling to a cluster with 4,546 VMs and 1,176 PMs.

Load-bearing premise

The whole approach assumes the simulator is perfect: given the current state and a migration action, the next state and the objective change are known exactly, so the best sampled trajectory in simulation is also the best plan in the real cluster.

Editorial extensions

If this is right

  • A planner that returns within seconds can be used during off-peak rescheduling windows, because its plan remains feasible while VM state churns.
  • The same trained agent can serve different migration-number limits, saving the operational cost of maintaining one model per limit.
  • Hard constraints such as anti-affinity can be enforced by masking candidate physical machines in the second stage, with only mild fragment-rate degradation even when the conflict ratio is high.
  • Risk-seeking evaluation converts extra GPU parallelism directly into solution quality, because more sampled trajectories mean a better best trajectory without retraining.
  • A policy trained on high and low workloads generalizes to a middle workload it never saw, suggesting that extreme-workload training data can cover the distribution gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the latency-quality coupling holds generally, then any online reoptimization problem should report time-to-solution alongside final objective, because slow solvers quietly degrade the value of their own answers.
  • The same risk-seeking recipe could transfer to other deterministic combinatorial problems with expensive planning and cheap simulation, such as bin packing, compiler phase ordering, or network traffic engineering.
  • A direct production test of VMR2L would measure the realized fragment rate after the chosen plan is executed under real VM churn; if the realized rate systematically trails the simulator-predicted rate, the deterministic-environment assumption would need to be relaxed with prediction or replanning.
  • The success of training on extreme workloads suggests that resource-management RL policies may need less data variety than expected, as long as the training distribution brackets the test distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies the problem of virtual machine rescheduling (VMR) in large data centers, where the objective is to minimize the fragment rate by migrating a limited number of VMs under strict latency constraints. The authors formulate VMR as a mixed-integer program, argue that the inference time of the optimizer matters because VM states change dynamically, and propose VMR2L, a deep reinforcement learning system with three main components: a two-stage action decomposition that separates VM selection from PM selection, a sparse-attention feature extractor that captures VM-PM tree-level relations, and a risk-seeking evaluation procedure that samples multiple trajectories and deploys the best one. The system is evaluated on two real-world datasets of cluster mappings, comparing against heuristic, MIP, search-based, and learning baselines, and is reported to achieve fragment rates close to the MIP solution while taking only seconds per mapping. The authors also open-source their code and datasets.

Significance. If the results hold, this is a practically important contribution: VM rescheduling is indeed understudied relative to initial VM placement, and the paper provides a complete system with a strong experimental setup, including two real datasets, seven baselines across method families, ablations, constraint handling, and generalization tests. The open-sourced code and datasets are valuable to the community. The risk-seeking evaluation idea is interesting and clearly motivated by the deterministic simulator. However, the central quantitative claim—that VMR2L is only 2.86% behind the optimal solution—is not currently supported because the MIP baseline is not certified optimal, and the performance numbers are all produced inside a perfect deterministic simulator rather than under dynamic conditions. These issues affect the headline claims and must be addressed before the results can be considered established.

major comments (4)
  1. [Section 5.2, Fig. 9] The central claim that VMR2L is 2.86% behind the 'optimal MIP solution' (0.2941 vs. 0.2859 at MNL=50) is unsupported because the Gurobi baseline is never certified optimal. The paper refers to the same result as 'near-optimal' (Sections 2.1, 5.3) and does not report termination status, MIPGap, or best-bound values. With 2089 VMs, 280 PMs, and 2 NUMAs per PM, the MILP contains on the order of 1.17 million binary variables plus integer variables, so proving optimality within a 50.55-minute time limit is not automatic. Please report the solver termination status, relative/absolute MIP gap, or use a certified optimum (or a lower bound) in the comparison. If the true optimum is lower, the gap from VMR2L to the optimal solution is larger than 2.86% and the headline claim would need to be revised.
  2. [Section 3.1, Section 2.2] All reported FR values are computed using the deterministic simulator described in Section 3.1, which assumes that 'given the current state and action, we can exactly know the next state and the change in objective.' This simulator does not model VM arrivals or exits during the 1.1s/2.2s/3.8s inference windows, yet the motivating experiment in Section 2.2 (Fig. 5) explicitly shows that inference time degrades the achieved performance because VM states change dynamically. The production claim that VMR2L achieves near-optimal performance needs a dynamic replay evaluation in which the computed plan is replayed against a stream of VM changes; without it, the current evaluation only establishes performance in a static, simulated setting. Please either add such an evaluation or clearly and prominently state that the claims are limited to the static problem.
  3. [Section 5.2, Tables 2-5] The main quantitative comparisons are presented as point estimates without confidence intervals or statistical significance. Section 4 states that the authors report averages over 3-5 runs with different random seeds and show confidence intervals in the convergence plots, but Fig. 9 and Tables 2-5 do not include any variance information. Given that the key difference between VMR2L (0.2941) and MIP (0.2859) is small, error bars or a significance test are essential to assess whether VMR2L's gap to MIP and its advantages over baselines are meaningful. Please add confidence intervals or standard deviations to the main result tables and figures.
  4. [Section 4, Datasets] The dataset anonymization procedure randomly removes some VMs and redeploys the remaining VMs to any PMs that can fit them. This may alter the natural fragmentation patterns, VM size distributions, and workload correlations that make the real VMR problem hard, potentially biasing results in favor of the learned policy. The paper should quantify how the anonymized datasets differ from the raw traces (e.g., FR distribution, VM type distribution, workload distribution) and, if possible, evaluate on at least one held-out set that is closer to the original traces. This would strengthen the external validity of the 'industry-scale data center' claims.
minor comments (5)
  1. [Abstract] The system name is inconsistent: the abstract calls it 'VM2RL' while the rest of the paper uses 'VMR2L' (e.g., Section 3.1, Section 5.2). Please unify the name.
  2. [Section 5.2, first paragraph] The sentence 'VMR2L achieves a lower FR compared to all baselines' is contradicted by the same paragraph, which reports MIP FR 0.2859 versus VMR2L FR 0.2941 at MNL=50. Please rephrase to 'lower FR than all non-MIP baselines' or otherwise clarify.
  3. [Section 5.3, first paragraph] The phrase 'FR performance reduces 16.46% without the two-stage framework' is ambiguous: does FR worsen or improve? Since FR is a cost metric, 'reduces' likely means 'degrades by 16.46%', but the wording should be made explicit.
  4. [Section 2.2, Fig. 5] The description of the experiment in Section 2.2 is under-specified: it says 'we use Gurobi to compute a near-optimal solution... which takes 50.55 minutes' and then 'compute the final performance that could be achieved as if the near-optimal solution was instead returned in a shorter period of time,' but does not explain how the 'as if' simulation handles infeasible actions over time. Please provide a precise description of the simulation procedure.
  5. [Section 5.4, Table 2] The affinity constraint table reports results for 'Aff. Level 0' which is not defined in the text (it appears in Fig. 13 and Section 5.4 but the relation between 'Aff. Ratio' and 'Aff. Level' is not stated). Please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: VMR2L's performance is benchmarked against independent MIP and heuristic baselines, and its inference-time trajectory selection is not a fitted prediction.

full rationale

I traced the paper's claimed derivation chain: the MIP formulation (Eqs. 1-7), the RL state/action/reward design (Eqs. 8-9), the two-stage action decomposition, the sparse-attention architecture, and the risk-seeking evaluation. The MIP formulation defines the fragment-rate objective that is later used as the evaluation metric, but VMR2L is not constructed from that objective by definition; it is trained with a surrogate dense reward and then compared against the same objective as an external benchmark. Risk-seeking evaluation samples multiple trajectories from a fixed policy and uses the deterministic simulator to keep the best one; this is inference-time search, not fitting a parameter and then reporting a forced prediction. The only self-citation ([20], for 'each PM can host multiple VMs that run independently') is co-located with an independent citation [28] and is not load-bearing for any central claim. The paper repeatedly calls the Gurobi result 'near-optimal' (Sections 2.1 and 5.3) without reporting an optimality gap, so the headline 'optimal MIP solution' in Section 5.2 is an evidentiary concern about certification, not circular reasoning. No equation or fitted parameter reduces to the claimed result by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The system has no invented physical entities. Its central claim rests on the determinism of the rescheduling environment, the representativeness of the anonymized datasets, and the near-optimality of the MIP baseline.

free parameters (3)
  • reward normalization constant c = 64
    Chosen by hand in Section 3.1 to scale fragment-size rewards to the range [-15*4/64, 15*4/64]; the central results are likely insensitive to it, but it is a tunable constant.
  • action threshold quantiles for risk-seeking = grid-searched over {0.95, 0.98, 0.99, 0.995}; best combination on validation set
    Section 5.3: the quantiles for masking low-probability VM and PM actions are tuned on the validation set and then applied to the test set.
  • number of sampled trajectories = 16 (with 8 GPUs, 2.2s)
    Section 5.3: more trajectories lower FR but increase latency; 16 is a practical choice within the 5-second limit.
assumptions (5)
  • domain assumption Given the current state and action, the next state and objective change are exactly known.
    Section 3.1: this justifies the deterministic simulator used for training and risk-seeking evaluation.
  • domain assumption Live migration of VMs incurs low overhead and requires transferring only memory due to compute-storage separation.
    Section 1: this motivates the feasibility of frequent rescheduling.
  • domain assumption 16-core FR is the operational metric at ByteDance; other metrics are extensions.
    Section 1: the objective is defined on 16-core CPU fragments; the paper notes extensions in Section 5.5.
  • domain assumption Gurobi returns a near-optimal solution within 50 minutes for a single mapping.
    Section 2.2: used as the optimal baseline; no MIP optimality gap is reported.
  • domain assumption Anonymized mappings remain representative of real production workloads.
    Section 4: random removal and redeployment may change the workload distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards VM Rescheduling Optimization Through Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/2FTN35LC

@misc{pith2026250517359,
  author       = {Pith},
  title        = {Pith review of: Towards VM Rescheduling Optimization Through Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2FTN35LC}},
  note         = {Machine review of arXiv:2505.17359}
}
read the original abstract

Modern industry-scale data centers need to manage a large number of virtual machines (VMs). Due to the continual creation and release of VMs, many small resource fragments are scattered across physical machines (PMs). To handle these fragments, data centers periodically reschedule some VMs to alternative PMs, a practice commonly referred to as VM rescheduling. Despite the increasing importance of VM rescheduling as data centers grow in size, the problem remains understudied. We first show that, unlike most combinatorial optimization tasks, the inference time of VM rescheduling algorithms significantly influences their performance, due to dynamic VM state changes during this period. This causes existing methods to scale poorly. Therefore, we develop a reinforcement learning system for VM rescheduling, VM2RL, which incorporates a set of customized techniques, such as a two-stage framework that accommodates diverse constraints and workload conditions, a feature extraction module that captures relational information specific to rescheduling, as well as a risk-seeking evaluation enabling users to optimize the trade-off between latency and accuracy. We conduct extensive experiments with data from an industry-scale data center. Our results show that VM2RL can achieve a performance comparable to the optimal solution but with a running time of seconds. Code and datasets are open-sourced: https://github.com/zhykoties/VMR2L_eurosys, https://drive.google.com/drive/folders/1PfRo1cVwuhH30XhsE2Np3xqJn2GpX5qy.

Figures

Figures reproduced from arXiv: 2505.17359 by the authors.

Figure 1
Figure 1. The number of VM arrivals and exits per minute. The green line indicates a continuous VMS process over 24 hours. … PM1 PM2 PMn Free-20 … VMn+1 VMn 1 FR = 50% VM3-24 Free-12 VM2-28 … PM VM Free VM1-4 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 4
Figure 4. FR and inference time at different MNLs. Elbow Point [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Effect of inference time on achieved performance [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figures from the paper (10 more)
Figure 6
Figure 6. Figure 6: The first stage of VMR2L processes all VMs and PMs via shared embedding networks, based on which the VM actor selects a VM to be rescheduled. State PMk PM Actor Stage 2 PM Mask # PMs All PMs Embeddings Action Embedding Network Shared Selected VMi Embedding Selected VMi…
Figure 7
Figure 7. Figure 7: Once a candidate VM is selected by the VM actor, VMR2L masks out all the PMs that cannot host the candidate VM. The PM actor only accesses the selected VM, and then selects a destination PM from the unmasked PMs. service stability. For example, an application may requi…
Figure 8
Figure 8. Figure 8: VM actor architecture with sparse local-attention to capture the tree-level features. 2. Each PM attends to other PMs’ updated embeddings and each VM attends to other VMs’ updated embed￾dings with self-attention. 3. The new VM embeddings attend to the new PM em￾bedding…
Figure 9
Figure 9. Figure 9: Fragment rate (left) and inference time (right) of VMR2L compared with baselines at different MNLs. 0 500 1000 1500 2000 Training Steps 0.3 0.4 0.5 Test Fragment Rate Sparse Attention Vanilla Attention w/o Attention [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Ablation on Sparse Attention. 10 5 10 3 10 1 VM Selection Probability 0 2 4 Count 1e6 [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 13
Figure 13. Figure 13: Constraints on Medium (left) and Multi-Resource (right). 0 20 40 60 Used MNL HA VMR2L MIP 0.55 0.50 0.45 0.4 0.35 0.3 0.25 Fragment Rate Goal 0.2 0.4 0.6 Fragment Rate HA VMR 2L MIP GOAL [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: MNL performance under different FR goals. 5.4 Different Constraints with Two-Stage Framework More Resource Constraints. To analyze how the two-stage framework supports different constraints, we compare it with two baselines: i) Penalty: a penalty of −5 is given if the…
Figure 15
Figure 15. Figure 15: CPU usage on PMs under different workloads. 0 10 20 30 40 50 AVG Migration Number Limit 0.0 0.2 0.4 Fragment Rate VMR2L_SEP VMR2L [PITH_FULL_IMAGE:figures/full_fig_p011_15.png]
Figure 18
Figure 18. Figure 18: FR and time performance on the Large dataset. to the larger MNL used on L and M, we choose POP as the standard baseline since it is easy to tune and exhibits strong FR performances. First, we see that VMR2L outperforms the two baselines when trained on the same worklo…
Figure 21
Figure 21. Figure 21: VM-PM Migration Details. Each equal-sized rec￾tangle represents a NUMA node within a PM. Different colors indicate the total allocated size of a VM type on each NUMA. At step 38, VMR2L removes a 4-core VM (from the orange section) from the top PM, which eliminates fra…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 59 canonical work pages

  1. [1]

    Cluster design in data center.https://core.vmware.com/resource/vsan- cluster-design-large-clusters-versus-small-clusters#section1

    work page

  2. [2]

    Cplex optimizer.https://www.ibm.com/analytics/cplex-optimizer

    work page

  3. [3]

    Gurobi solver.https://www.gurobi.com/

    work page

  4. [4]

    Kubernetes scheduler.https://kubernetes.io/docs/concepts/ scheduling-eviction/kube-scheduler/

    work page

  5. [5]

    Accessed: 2024-10-05

    Numa architecture platforms.https://uefi.org/htmlspecs/ACPI_Spec_ 6_4_html/17_NUMA_Architecture_Platforms/NUMA_Architecture_ Platforms.html. Accessed: 2024-10-05

    work page 2024

  6. [6]

    W., Gani, A., Hamid, S

    Ahmad, R. W., Gani, A., Hamid, S. H. A., Shiraz, M., Yousafzai, A., and Xia, F.A survey on virtual machine migration and server consolidation frameworks for cloud data centers.Journal of network and computer applications 52(2015), 11–25

    work page 2015

  7. [7]

    InProceedings of the 61st ACM/IEEE Design Automation Conference (2024), pp

    An, Z., Ding, X., and Du, W.Go beyond black-box policies: Rethinking the design of learning agent for interpretable and verifiable hvac con- trol. InProceedings of the 61st ACM/IEEE Design Automation Conference (2024), pp. 1–6. [8]Ba, J. L., Kiros, J. R., and Hinton, G. E.Layer normalization, 2016

    work page 2024

  8. [9]

    InProceed- ings of the AAAI Conference on Artificial Intelligence(2020), vol

    Barrett, T., Clements, W., Foerster, J., and Lvovsky, A.Exploratory combinatorial optimization with reinforcement learning. InProceed- ings of the AAAI Conference on Artificial Intelligence(2020), vol. 34, pp. 3243–3250

    work page 2020

Show all 69 references
  1. [10]

    M.Pattern Recognition and Machine Learning (Information Science and Statistics), 1 ed

    Bishop, C. M.Pattern Recognition and Machine Learning (Information Science and Statistics), 1 ed. Springer, 2007

    2007

  2. [11]

    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W.Openai gym, 2016

    2016

  3. [12]

    Cai, Q., Hang, W., Mirhoseini, A., Tucker, G., W ang, J., and Wei, W.Reinforcement learning driven heuristic optimization.Workshop on Deep Reinforcement Learning for Knowledge Discovery (DRL4KDD) abs/1906.06639(2019)

    2019 arXiv

  4. [13]

    M., and Du, W.Marlp: Time-series forecasting control for agricultural managed aquifer recharge

    Chen, Y., Y ang, K., An, Z., Holder, B., Paloutzian, L., Bali, K. M., and Du, W.Marlp: Time-series forecasting control for agricultural managed aquifer recharge. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(2024)

    2024

  5. [14]

    G., Jul, E., Limpach, C., Pratt, I., and W arfield, A.Live migration of virtual machines

    Clark, C., Fraser, K., Hand, S., Hansen, J. G., Jul, E., Limpach, C., Pratt, I., and W arfield, A.Live migration of virtual machines. In Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation - Volume 2(Berkeley, CA, USA, 2005), USENIX Asso...

    2005

  6. [15]

    arXiv preprint arXiv:1810.04805v2(2018)

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805v2(2018)

    2018 arXiv

  7. [16]

    Ding, X., An, Z., Rathee, A., and Du, W.A safe and data-efficient model-based reinforcement learning system for hvac control.IEEE Internet of Things Journal(2025)

    2025

  8. [17]

    Ding, X., Cerpa, A., and Du, W.Exploring deep reinforcement learn- ing for holistic smart building control.ACM Transactions on Sensor Networks 20, 3 (2024), 1–28

    2024

  9. [18]

    Ding, X., Cerpa, A., and Du, W.Multi-zone hvac control with model- based deep reinforcement learning.IEEE Transactions on Automation Science and Engineering(2024)

    2024

  10. [19]

    Ding, X., and Du, W.Optimizing irrigation efficiency using deep rein- forcement learning in the field.ACM Transactions on Sensor Networks 20, 4 (2024), 1–34

    2024

  11. [20]

    Ding, X., Zhang, Y., Chen, B., Ying, D., Zhang, T., Chen, J., Zhang, L., Cerpa, A., and Du, W.Vmr2l: Virtual machines rescheduling using reinforcement learning in data centers, 2023

    2023

  12. [21]

    InInternational Conference on Learning Representations(2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N.An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on L...

    2021

  13. [22]

    A multi-task selected learning approach for solving 3d flexible bin packing problem

    Duan, L., Hu, H., Qian, Y., Gong, Y., Zhang, X., Wei, J., and Xu, Y. A multi-task selected learning approach for solving 3d flexible bin packing problem. InProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems(Richland, SC, 2019), AAMAS ’...

    2019

  14. [23]

    Reinforcement learning for variable selection in a branch and bound algorithm

    Etheve, M., Alès, Z., Bissuel, C., Juan, O., and Kedad-Sidhoum, S. Reinforcement learning for variable selection in a branch and bound algorithm. InInternational Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research(2020), Spring...

    2020

  15. [24]

    Pytorch: An open source machine learning framework.https://pytorch.org/, 2019

    Facebook AI Research. Pytorch: An open source machine learning framework.https://pytorch.org/, 2019. Accessed: April 23, 2023

    2019

  16. [25]

    Advances in Neural Information Processing Systems 32(2019)

    Gasse, M., Chételat, D., Ferroni, N., Charlin, L., and Lodi, A.Exact combinatorial optimization with graph convolutional neural networks. Advances in Neural Information Processing Systems 32(2019)

    2019

  17. [26]

    Gupta, P., Gasse, M., Khalil, E., Mudigonda, P., Lodi, A., and Ben- gio, Y.Hybrid models for learning to branch.Advances in neural information processing systems 33(2020), 18087–18097

    2020

  18. [27]

    T., Nguyen, T

    Ha, C. T., Nguyen, T. T., Bui, L. T., and W ang, R.An online pack- ing heuristic for the three-dimensional container loading problem in dynamic environments and the physical internet. InApplications of Evolutionary Computation: 20th European Conference, EvoApplications 2017, A...

    2017

  19. [28]

    E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M., et al.Protean: Vm allocation service at scale

    Hadary, O., Marshall, L., Menache, I., Pan, A., Greeff, E. E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M., et al.Protean: Vm allocation service at scale. InProceedings of the 14th USENIX Conference on Operating Systems Design and Implementation(2020)

    2020

  20. [29]

    Haj-Ali, A., Huang, Q. J., Xiang, J., Moses, W., Asanovic, K., W awrzynek, J., and Stoica, I.Autophase: Juggling hls phase order- ings in random forests with deep reinforcement learning.Proceedings of Machine Learning and Systems 2(2020), 70–81

    2020

  21. [30]

    InProceedings of the AAAI conference on artificial intelligence(2018), vol

    Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D.Deep reinforcement learning that matters. InProceedings of the AAAI conference on artificial intelligence(2018), vol. 32

    2018

  22. [31]

    Hong, Z., Yuan, Z., Zhang, Q., Chen, H., Dong, J., Huang, F., and Huang, X.Next-generation database interfaces: A survey of llm-based text-to-sql.arXiv preprint arXiv:2406.08426(2024)

    2024

  23. [32]

    InProceedings of the 36th Interna- tional Conference on Machine Learning(2019)

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Larous- silhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S.Parameter- efficient transfer learning for NLP. InProceedings of the 36th Interna- tional Conference on Machine Learning(2019)

    2019

  24. [33]

    InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Melbourne, Australia, July 2018), I

    Howard, J., and Ruder, S.Universal language model fine-tuning for text classification. InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Melbourne, Australia, July 2018), I. Gurevych and Y. Miyao, Eds., Associati...

    2018

  25. [34]

    J., Shen, Y., W allis, P., Allen-Zhu, Z., Li, Y., W ang, S., W ang, L., and Chen, W.Lora: Low-rank adaptation of large language models

    Hu, E. J., Shen, Y., W allis, P., Allen-Zhu, Z., Li, Y., W ang, S., W ang, L., and Chen, W.Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685(2021)

    2021 arXiv

  26. [35]

    Hu, H., Zhang, X., Y an, X., W ang, L., and Xu, Y.Solving a new 3d bin packing problem with deep reinforcement learning method, 2017

    2017

  27. [36]

    Huang, S., Dossa, R. F. J., Ye, C., Braga, J., Chakraborty, D., Mehta, K., and Araújo, J. G.Cleanrl: High-quality single-file implementa- tions of deep reinforcement learning algorithms.Journal of Machine Learning Research 23, 274 (2022), 1–18

    2022

  28. [37]

    L., Likitha, V., Suneetha, B., and Vignesh, T.Analysis of ci/cd application in kubernetes architec- ture.Mathematical Statistician and Engineering Applications 71, 4 (Mar

    Janani, K., Anuhya, K., Manaswini, V. L., Likitha, V., Suneetha, B., and Vignesh, T.Analysis of ci/cd application in kubernetes architec- ture.Mathematical Statistician and Engineering Applications 71, 4 (Mar. 2023), 11091–11097

    2023

  29. [38]

    P., Littman, M

    Kaelbling, L. P., Littman, M. L., and Moore, A. W.Reinforcement learning: A survey.Journal of Artificial Intelligence Research 4(1996). 15

    1996

  30. [39]

    In2022 IEEE 9th International Confer- ence on Data Science and Advanced Analytics (DSAA)(2022), pp

    Kumar, D., and Li, S.Separating storage and compute with the databricks lakehouse platform. In2022 IEEE 9th International Confer- ence on Data Science and Advanced Analytics (DSAA)(2022), pp. 1–2

    2022

  31. [40]

    Levine, S., Kumar, A., Tucker, G., and Fu, J.Offline reinforcement learning: Tutorial, review, and perspectives on open problems.ArXiv abs/2005.01643(2020)

    2020 arXiv

  32. [41]

    Li, D., Ren, C., Gu, Z., W ang, Y., and Lau, F.Solving packing problems by conditional query learning, 2020

    2020

  33. [42]

    InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(New York, NY, USA, 2018), KDD ’18, Association for Computing Machinery, p

    Li, X., Yuan, M., Chen, D., Y ao, J., and Zeng, J.A data-driven three- layer algorithm for split delivery vehicle routing problem with 3d container loading constraint. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(New York, ...

    2018

  34. [43]

    InProceedings of the 5th Workshop on Machine Learning and Systems(New York, NY, USA, 2025), EuroMLSys ’25, Association for Computing Machinery

    Lin, M., and Jeon, H.Understanding oversubscribed memory manage- ment for deep learning training. InProceedings of the 5th Workshop on Machine Learning and Systems(New York, NY, USA, 2025), EuroMLSys ’25, Association for Computing Machinery

    2025

  35. [44]

    Lin, M., Zhou, K., and Su, P.Drgpum: Guiding memory optimiza- tion for gpu-accelerated applications. InProceedings of the 28th ACM International Conference on Architectural Support for Programming Lan- guages and Operating Systems, Volume 3(New York, NY, USA, 2023), Associatio...

    2023

  36. [45]

    B., Meng, Z., and Alizadeh, M.Learning scheduling algorithms for data processing clusters

    Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., and Alizadeh, M.Learning scheduling algorithms for data processing clusters. InProceedings of the ACM special interest group on data communication. 2019, pp. 270–288

    2019

  37. [46]

    Mazyavkina, N., Sviridov, S., Ivanov, S., and Burnaev, E.Reinforce- ment learning for combinatorial optimization: A survey, 2020

    2020

  38. [47]

    F., Uhlig, V., Krieger, O., and Xenidis, J.Virtualization for high-performance computing.SIGOPS Oper

    Mergen, M. F., Uhlig, V., Krieger, O., and Xenidis, J.Virtualization for high-performance computing.SIGOPS Oper. Syst. Rev. 40, 2 (apr 2006), 8–11

    2006

  39. [48]

    InProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles(2021)

    Narayanan, D., Kazhamiaka, F., Abuzaid, F., Kraft, P., Agrawal, A., Kandula, S., Boyd, S., and Zaharia, M.Solving large-scale granular resource allocation problems efficiently with pop. InProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles(2021)

    2021

  40. [49]

    Padberg, M., and Rinaldi, G.A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems.SIAM review 33, 1 (1991), 60–100

    1991

  41. [50]

    microsoft

    Panigrahy, R., Talwar, K., Uyeda, L., and Wieder, U.Heuristics for vector bin packing.research. microsoft. com(2011)

    2011

  42. [51]

    M., and Oliveira, J

    Parreño, F., Alvarez-V aldés, R., Tamarit, J. M., and Oliveira, J. F.A maximal-space algorithm for the container loading problem.INFORMS Journal on Computing 20, 3 (2008), 412–422

    2008

  43. [52]

    K., Larma, M

    Petersen, B. K., Larma, M. L., Mundhenk, T. N., Santiago, C. P., Kim, S. K., and Kim, J. T.Deep symbolic regression: Recovering math- ematical expressions from data via risk-seeking policy gradients. In International Conference on Learning Representations(2021)

    2021

  44. [53]

    E., Perescu-Popescu, L., and Mastorakis, N.Multilayer perceptron and neural networks.WSEAS Transactions on Circuits and Systems 8, 7 (2009), 579–588

    Popescu, M.-C., Balas, V. E., Perescu-Popescu, L., and Mastorakis, N.Multilayer perceptron and neural networks.WSEAS Transactions on Circuits and Systems 8, 7 (2009), 579–588

    2009

  45. [54]

    Rengarajan, D., V aidya, G., Sarvesh, A., Kalathil, D., and Shakkot- tai, S.Reinforcement learning with sparse rewards using guidance from offline demonstration.arXiv preprint arXiv:2202.04628(2022)

    2022 arXiv

  46. [55]

    T.Learn- ing by playing solving sparse reward tasks from scratch

    Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., van de Wiele, T., Mnih, V., Heess, N., and Springenberg, J. T.Learn- ing by playing solving sparse reward tasks from scratch. InProceedings of the 35th International Conference on Machine Learning(10–15 Jul 2018)...

    2018

  47. [56]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O.Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347(2017)

    2017 arXiv

  48. [57]

    H., Rahmani, A

    Shirvani, M. H., Rahmani, A. M., and Sahafi, A.A survey study on virtual machine migration and server consolidation techniques in dvfs-enabled cloud datacenter: taxonomy and challenges.Journal of King Saud University-Computer and Information Sciences 32, 3 (2020)

    2020

  49. [58]

    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al.Mas- tering the game of go without human knowledge.nature(2017)

    2017

  50. [59]

    Song, J., Yue, Y., Dilkina, B., et al.A general large neighborhood search framework for solving integer linear programs.Advances in Neural Information Processing Systems 33(2020), 20012–20023

    2020

  51. [60]

    A., Yousafzai, A., V asilakos, A

    Talebian, H., Gani, A., Sookhak, M., Abdelatif, A. A., Yousafzai, A., V asilakos, A. V., and Yu, F. R.Optimizing virtual machine place- ment in iaas data centers: taxonomy, review and open issues.Cluster Computing 23(2020), 837–878

    2020

  52. [61]

    InPro- ceedings of the Seventeenth European Conference on Computer Systems (2022), pp

    Thalheim, J., Okelmann, P., Unnibhavi, H., Gouicem, R., and Bha- totia, P.Vmsh: hypervisor-agnostic guest overlays for vms. InPro- ceedings of the Seventeenth European Conference on Computer Systems (2022), pp. 678–696

    2022

  53. [62]

    N., Kaiser, Ł., and Polosukhin, I.Attention is all you need.Ad- vances in neural information processing systems 30(2017)

    V aswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I.Attention is all you need.Ad- vances in neural information processing systems 30(2017)

    2017

  54. [63]

    K., Shenoy, P., V an Der Merwe, J., Hwang, J., Liu, G., and Chaufournier, L.Cloudnet: dynamic pool- ing of cloud resources by live wan migration of virtual machines

    Wood, T., Ramakrishnan, K. K., Shenoy, P., V an Der Merwe, J., Hwang, J., Liu, G., and Chaufournier, L.Cloudnet: dynamic pool- ing of cloud resources by live wan migration of virtual machines. IEEE/ACM Trans. Netw. 23, 5 (Oct. 2015), 1568–1583

    2015

  55. [64]

    Xia, Y., Tsugawa, M., Fortes, J. A., and Chen, S.Large-scale vm placement with disk anti-colocation constraints using hierarchical decomposition and mixed integer programming.IEEE Transactions on Parallel and Distributed Systems 28, 5 (2016), 1361–1374

    2016

  56. [65]

    InProceedings of the 22nd International Conference on Information Processing in Sensor Networks(2023)

    Y ang, K., Chen, Y., Chen, X., and Du, W.Link quality modeling for lora networks in orchards. InProceedings of the 22nd International Conference on Information Processing in Sensor Networks(2023)

    2023

  57. [66]

    InACM MobiSys(2024)

    Y ang, K., Chen, Y., and Du, W.OrchLoc: In-Orchard Localization via a Single LoRa Gateway and Generative Diffusion Model-based Fingerprinting. InACM MobiSys(2024)

    2024

  58. [67]

    Zhang, J., Zi, B., and Ge, X.Attend2pack: Bin packing through deep reinforcement learning with attention.ArXiv abs/2107.04333(2021)

    2021 arXiv

  59. [68]

    Zhang, Q., Chen, S., Bei, Y., Yuan, Z., Zhou, H., Hong, Z., Dong, J., Chen, H., Chang, Y., and Huang, X.A survey of graph retrieval- augmented generation for customized large language models.arXiv preprint arXiv:2501.13958(2025)

    2025

  60. [69]

    S., Tian, Y., Zhang, Y., and Jin, X

    Zhu, H., Gupta, V., Ahuja, S. S., Tian, Y., Zhang, Y., and Jin, X. Network planning with deep reinforcement learning. InProceedings of the 2021 ACM SIGCOMM 2021 Conference(2021), pp. 258–271

    2021

  61. [70]

    Learning to pack: A data-driven tree search algorithm for large-scale 3d bin packing problem

    Zhu, Q., Li, X., Zhang, Z., Luo, Z., Tong, X., Yuan, M., and Zeng, J. Learning to pack: A data-driven tree search algorithm for large-scale 3d bin packing problem. InProceedings of the 30th ACM International Con- ference on Information & Knowledge Management(New York, NY, USA,...

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.