Abstract

Unlike VMs, containerization is a modern method for packaging and deploying software in distributed environments like the cloud. Containers are widely used due to their efficient software packaging and deployment. Efficient management of containers is crucial in dynamic cloud environments with heterogeneous infrastructure. Deep learning techniques are being applied to optimize resource utilization in cloud environments, including mapping containers to suitable nodes for energy conservation. However, the existing works on container scheduling have limitations like inability to cope with dynamic runtime scenarios. To overcome this problem, the aim of this paper is to design and implement a framework using deep reinforcement learning techniques to improve container scheduling and load balancing. The proposed algorithm, Reinforcement Learning based Container Scheduling (RLbCS), uses an action-reward iterative approach to optimize container scheduling. Experimental results showed that RLbCS outperformed existing methods, achieving a 92% success rate in placing containers and optimizing resource utilization. The proposed method can be integrated with cloud-based systems to automatically schedule containers for resource optimization and load balancing.

Keywords

Container Services, Container Scheduling, Load Balancing, Cloud Computing, Deep Reinforcement Learning,

Downloads

Download data is not yet available.

References

  1. S. Swarup, E.M. Shakshuki, A. Yasar, Task scheduling in cloud using deep reinforcement learning. Procedia Computer Science, 184, (2021) 42-51. https://doi.org/10.1016/j.procs.2021.03.016
  2. L. Wang, Q. Weng, W. Wang, C. Chen, B. Li, (2020) Metis: Learning to schedule long-running applications in shared container clusters at scale. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE, USA. https://doi.org/10.1109/SC41405.2020.00072
  3. F. Rossi, M. Nardelli, V. Cardellini, (2019) Horizontal and vertical scaling of container-based applications using reinforcement learning. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), IEEE, Italy. https://doi.org/10.1109/CLOUD.2019.00061
  4. A. Singh, G.S. Aujla, R.S. Bali, Container-based load balancing for energy efficiency in software-defined edge computing environment. Sustainable Computing: Informatics and Systems, 30, (2021) 100463. https://doi.org/10.1016/j.suscom.2020.100463
  5. I. Ahmad, M.G. AlFailakawi, A. AlMutawa, L. Alsalman, Container scheduling techniques: A survey and assessment. Journal of King Saud University-Computer and Information Sciences, 34(7), (2022) 3934-3947. https://doi.org/10.1016/j.jksuci.2021.03.002
  6. S. Swarup, E.M. Shakshuki, A.Yasar, Energy efficient task scheduling in fog environment using deep reinforcement learning approach. Procedia Computer Science, 191, (2021) 65-75. https://doi.org/10.1016/j.procs.2021.07.012
  7. L. Deng, Z. Wang, H. Sun, B. Li, X. Yang, A deep reinforcement learning-based optimization method for long-running applications container deployment. International Journal of Computers Communications & Control, 18(4), (2023) 1-17. https://doi.org/10.15837/ijccc.2023.4.5013
  8. Z. Wang, M. Goudarzi, M. Gong, R. Buyya, Deep reinforcement learning-based scheduling for optimizing system load and response time in edge and fog computing environments. Future Generation Computer Systems, 152, (2024) 55-69. https://doi.org/10.1016/j.future.2023.10.012
  9. G.P. Mattia, R. Beraldi, P2PFaaS: A framework for FaaS peer-to-peer scheduling and load balancing in Fog and Edge computing. SoftwareX, 21, (2023) 101290. https://doi.org/10.1016/j.softx.2022.101290
  10. L. Wang, S. Guo, P. Zhang, H. Yue, Y. Li, C. Wang, Z. Cao D. Cui, An Efficient Load Prediction‐Driven Scheduling Strategy Model in Container Cloud. International Journal of Intelligent Systems, 2023(1), (2023) 5959223. https://doi.org/10.1155/2023/5959223
  11. Y. Li, X. Guo, Z. Meng, J. Qin, X. Li, X. Ma, J. Yang, (A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning. Electronics, 12(19), (2023) 3991. https://doi.org/10.3390/electronics12193991
  12. S. Luo, Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Applied Soft Computing, 91, (2020) 106208. https://doi.org/10.1016/j.asoc.2020.106208
  13. O. Oleghe, Container placement and migration in edge computing: Concept and scheduling models. IEEE Access, 9, (2021) 68028-68043. https://doi.org/10.1109/ACCESS.2021.3077550
  14. J. Zhang, T. Wang, L. Cheng, Time-Sensitive and Resource-Aware Concurrent Workflow Scheduling for Edge Computing Platforms Based on Deep Reinforcement Learning. Applied Sciences, 13(19), (2023) 10689. https://doi.org/10.3390/app131910689
  15. Z. Ma, S. Shao, S. Guo, Z. Wang, F. Qi, A. Xiong, Container migration mechanism for load balancing in edge network under power Internet of Things. IEEE Access, 8, (2020) 118405-118416. https://doi.org/10.1109/ACCESS.2020.3004615
  16. A.G. Ramos, E. Silva, J.F. Oliveira, A new load balance methodology for container loading problem in road transportation. European Journal of Operational Research, 266(3), (2018) 1140-1152. https://doi.org/10.1016/j.ejor.2017.10.050
  17. D. Baburao, T. Pavankumar, C.S.R. Prabhu, (2019) Survey on service migration, load optimization and load balancing in fog computing environment. IEEE 5th International Conference for Convergence in Technology (I2CT), IEEE, India. https://doi.org/10.1109/I2CT45611.2019.9033579
  18. A. Talukder, S.F. Abedin, M.S. Munir, C.S. Hong, (2018) Dual threshold load balancing in SDN environment using process migration. In 2018 International Conference on Information Networking (ICOIN), IEEE, Thailand. https://doi.org/10.1109/ICOIN.2018.8343226
  19. K. Kaur, S. Garg, G. Kaddoum, F. Gagnon, D.N.K. Jayakody, (2019) Enlob: Energy and load balancing-driven container placement strategy for data centers. IEEE Globecom Workshops (GC Wkshps), IEEE, USA. https://doi.org/10.1109/GCWkshps45667.2019.9024592
  20. K. Aruna, G. Pradeep, Ant Colony Optimization-based Light Weight Container (ACO-LWC) Algorithm for Efficient Load Balancing. Intelligent Automation & Soft Computing, 34(1), (2022) 1-15. https://doi.org/10.32604/iasc.2022.024317
  21. M.K. Patra, S. Misra, B. Sahoo, A.K. Turuk, GWO-based simulated annealing approach for load balancing in cloud for hosting container as a service. Applied Sciences, 12(21), (2022) 11115. https://doi.org/10.3390/app122111115
  22. S. Rabiu, C.H. Yong, S.M.S. Mohamad, A cloud-based container micro services: a review on load-balancing and auto-scaling issues. International Journal of Data Science, 3(2), (2022) 80-92. https://doi.org/10.18517/ijods.3.2.80-92.2022
  23. O. Smimite, K. Afdel, Hybrid solution for container placement and load balancing based on aco and bin packing. International Journal of Advanced Computer Science and Applications, 11(11), (2020) 1-10. https://doi.org/10.14569/IJACSA.2020.0111174
  24. K. Li, C. Chang, K. Yun, J. Zhang, (2021) Research on container migration mechanism of power edge computing on load balancing. IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), IEEE, China. https://doi.org/10.1109/ICCCBDA51879.2021.9442546
  25. X.I.E. Xiaojing, S.S. Govardhan, (2020) A service mesh-based load balancing and task scheduling system for deep learning applications. 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), IEEE, Australia. https://doi.org/10.1109/CCGrid49817.2020.00009
  26. D. Zhao, M. Mohamed, H. Ludwig, Locality-aware scheduling for containers in cloud computing. IEEE Transactions on cloud computing, 8(2), (2018) 635-646. https://doi.org/10.1109/TCC.2018.2794344
  27. W.A. Hanafy, A.E. Mohamed, S.A. Salem, (2017) A load balancing with power optimization algorithm for container-based infrastructure management. 12th International Conference on Computer Engineering and Systems (ICCES), IEEE, Egypt. https://doi.org/10.1109/ICCES.2017.8275296
  28. M.K. Patra, D. Patel, B. Sahoo, A.K. Turuk, (2020) A randomized algorithm for load balancing in containerized cloud. In 2020 10th International conference on cloud computing, data science & engineering (confluence), IEEE, India. https://doi.org/10.1109/Confluence47617.2020.9058147
  29. W. Thongthavorn, P. Rattanatamrong, (2019) Multi-container application migration with load balanced and adaptive parallel TCP. International Conference on High Performance Computing & Simulation (HPCS), IEEE, Ireland. https://doi.org/10.1109/HPCS48598.2019.9188218
  30. K.D. Naik, R.R. Sahoo, S.K. Kuana, A Bio inspired Approach for Load Balancing in Container as a Service Cloud Computing Model. International Research Journal on Advanced Science Hub, 5(05), (2023) 426-433. https://doi.org/10.47392/irjash.2023.S058
  31. T. Degris, P.M. Pilarski, R.S. Sutton, (2012) Model-free reinforcement learning with continuous action in practice. In 2012 American control conference (ACC), IEEE, Canada. https://doi.org/10.1109/ACC.2012.6315022
  32. A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. De Maria, V. Panneershelvam, M. Suleyman, C. Beattie, S. Petersen, S. Legg, V. Mnih, K. Kavukcuoglu, D. Silver, (2015) Massively parallel methods for deep reinforcement learning. arXiv. https://doi.org/10.48550/arXiv.1507.04296
  33. B. Recht, C. Re, S. Wright, F. Niu, Hogwild!: A lock-free approach to parallelizing stochastic gradient descent. Advances in neural information processing systems, 24, (2011) 693–701.
  34. T. Tieleman, Lecture 6.5‐rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), (2012) 26.
  35. Q. Zhang, M. Lin, L.T. Yang, Z. Chen, S.U. Khan, P. Li, A double deep Q-learning model for energy-efficient edge scheduling. IEEE Transactions on Services Computing, 12(5), (2018) 739-749. https://doi.org/10.1109/TSC.2018.2867482
  36. W. Funika, P. Koperek, J. Kitowski, (2020) Automatic management of cloud applications with use of proximal policy optimization. In International Conference on Computational Science, Springer International Publishing.
  37. Z. Wang, C. Gwon, T. Oates, A. Iezzi, (2017) Automated cloud provisioning on aws using deep reinforcement learning. arXiv. https://doi.org/10.48550/arXiv.1709.04305