Recent Trends in Parallel Computing Review Article
Adaptive Task Scheduling and Resource Optimization Using AI Middleware
Abstract
Modern distributed and heterogeneous computing systems face significant challenges in dealing with dynamically changing workloads, resource fragmentation, and changing latencies; existing traditional, or rule-based, specialized schedulers are no longer useful in achieving the best system performance. Such limitations highlight the importance of the adaptive scheduling paradigms that can learn, forecast, and react to the actual real-world conditions in the system. Artificial intelligence middleware is also an attractive solution to this issue, as it allows incorporating machine learning and reinforcement learning algorithms in the orchestration layer and, therefore, promotes intelligent decision-making and improves resource distribution and task assignment progressively. This paper suggests a flexible scheduling system, which runs on the AI middleware, incorporates workload forecasting models, reinforcement learning based scheduling policies, reactive scaling, and energy-sensitive decision-making. In the approach, there would be a multi-layer middleware design, which would incorporate real-time monitoring, state-action-reward learning, and a feedback-based scheduler. The major contributions of this paper include the creation of an RL-supported adaptive scheduling algorithm, a single middleware architecture that can be applied in distributed settings, and an in-depth performance analysis covering cloud and edge computing contexts. The experimental results show that the schedulers achieve considerable benefits in delay, throughput, resource exploitation, and energy efficiency when compared with the baseline schedulers. This can apply to cloud platform environments, Internet of Things (IoT)-edge environments, and enterprise distributed systems, hence providing a scalable and smart solution to modern workload management issues.
Keywords
References (26)
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need [preprint]. 2017. arXiv:1706.03762. doi:10.48550/arXiv.1706.03762.
- Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language models are few-shot learners [preprint]. 2020. arXiv:2005.14165. doi:10.48550/arXiv.2005.14165.
- Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, de Laroussilhe Q, Gesmundo A, et al. Parameter-efficient transfer learning for NLP [preprint]. 2019. arXiv:1902.00751. doi:10.48550/arXiv.1902.00751.
- Lester B, Al-Rfou R, Constant N. The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP); Online and Punta Cana, Dominican Republic. 2021. p. 3045–3059. doi:10.18653/v1/2021.emnlp-main.243.
- Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W. LoRA: low-rank adaptation of large language models [preprint]. 2021. arXiv:2106.09685. doi:10.48550/arXiv.2106.09685.
- Dettmers T, Holtzman A, Pagnoni A, Zettlemoyer L. QLoRA: efficient finetuning of quantized large language models. Advances in Neural Information Processing Systems 36 (NeurIPS 2023); New Orleans, LA, USA. 2023. p. 10088–10115. doi:https://doi.org/10.52202/075280-0441.
- Hasan MM, Islam MM. High-performance computing architectures for training large-scale transformer models in cyber-resilient applications. ASRC Procedia Glob Perspect Sci Scholarsh. 2022;2:193–226. doi:10.63125/6zt59y89.
- Zhao L, Gao W, Fang J. Optimizing large language models on multi-core CPUs: a case study of the BERT model. Appl Sci. 2024;14:2364. doi:10.3390/app14062364.
- Han Z, Gao C, Liu J, Zhang J, Zhang SQ. Parameter-efficient fine-tuning for large models: a comprehensive survey [preprint]. 2024. arXiv:2403.14608. doi:10.48550/arXiv.2403.14608.
- Mao H, Schwarzkopf M, Venkatakrishnan SB, Meng Z, Alizadeh M. Learning scheduling algorithms for data processing clusters. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM ’19); Beijing, China. 2019. p. 270–doi:10.1145/3341302.3342080.
- Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–1780. doi:10.1162/neco.1997.9.8.1735. PubMed: 9377276.
- Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 2nd ed. Cambridge (MA): MIT Press; 1998.
- Chen X, Jiao L, Li W, Fu X. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans Netw. 2016;24:2795–2808. doi:10.1109/TNET.2015.2487344.
- Burns B, Grant B, Oppenheimer D, Brewer E, Wilkes J. Borg, Omega, and Kubernetes. Commun ACM. 2016;59(5):50–57. doi:10.1145/2890784.
- Turnbull J. The Docker Book: Containerization is the New Virtualization. Melbourne: James Turnbull; 2014.
- Brewer EA. Kubernetes and the path to cloud native. In: Proceedings of the Sixth ACM Symposium on Cloud Computing. New York, NY, USA: Association for Computing Machinery; 2015:167. doi:10.1145/2806777.2809955.
- Lakhan A, Mohammed MA, Obaid OI, Chakraborty C, Abdulkareem KH, Kadry S. Efficient deep-reinforcement learning aware resource allocation in SDN-enabled fog paradigm. Autom Softw Eng. 2022;29(20). doi:10.1007/s10515-021-00318-6.
- Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R. CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp. 2011;41:23–50. doi:10.1002/spe.995.
- Zhang Q, Lin X, Hao Y, Cao J. Energy-aware scheduling in edge computing based on energy Internet. IEEE Access. 2020;8:229052–229065. doi:10.1109/ACCESS.2020.3044932.
- Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. Spark: cluster computing with working sets. In: 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud ’10). Boston, MA: USENIX Association; 2010.
- Harchol-Balter M. Performance Modeling and Design of Computer Systems: Queueing Theory in Action. Cambridge: Cambridge University Press; 2013.
- Lakshminarayanan A, Sharma S, Ravindran B. Dynamic Action Repetition for Deep Reinforcement Learning. Proc AAAI Conf Artif Intell. 2017;31(1):2133–2139. doi:10.1609/aaai.v31i1.10918.
- Mnih V, Badia AP, Mirza M, Graves A, Lillicrap TP, Harley T, Silver D, Kavukcuoglu K. Asynchronous methods for deep reinforcement learning. In: Balcan MF, Weinberger KQ, editors. Proceedings of the 33rd International Conference on Machine Learning (ICML 2016). New York, USA: PMLR; 2016. p. 1928–1937.
- Gonçalves GD, Drago I, Vieira AB, Couto da Silva AP, Almeida JM, Mellia M. Workload models and performance evaluation of cloud storage services. Comput Netw. 2016;109:183–199. doi:10.1016/j.comnet.2016.03.024.
- Chen T, Li M, Li Y, Lin M, Wang N, Wang M, Xiao T, Xu B, Zhang C, Zhang Z. MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems [preprint]. 2015. arXiv:1512.01274. doi:10.48550/arXiv.1512.01274.
- Xu Z, Tang J, Yin C, Wang Y, Xue G, Wang J, Gursoy MC. ReCARL: Resource allocation in cloud RANs with deep reinforcement learning. IEEE Trans Mob Comput. 2022;21(7):2533–2545. doi:10.1109/TMC.2020.3044282.