Skip to main content
SpringerLink
Log in

Genetic Programming Based Hyper Heuristic Approach for Dynamic Workflow Scheduling in the Cloud

  • Conference paper
  • First Online:
Database and Expert Systems Applications (DEXA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12392))

Included in the following conference series:

Abstract

Workflow scheduling in the cloud is the process of allocating tasks to limited cloud resources to maximise resource utilization and minimise makespan. This is often achieved by adopting an effective scheduling heuristic. Most existing heuristics rely on a small number of features when making scheduling decisions, ignoring many impacting factors that are important to workflow scheduling. For example, the MINMIN algorithm only considers the size of the tasks when making scheduling decisions. Meanwhile, many existing works focused on scheduling a static set of workflow tasks, neglecting the dynamic nature of cloud computing. In this paper, we introduce a new and more realistic workflow scheduling problem that considers different kinds of workflows, cloud resources, and impacting features. We propose a Dynamic Workflow Scheduling Genetic Programming (DSGP) algorithm to automatically design scheduling heuristics for workflow scheduling to minimise the overall makespan of executing a long sequence of dynamically arriving workflows. Our proposed DSGP algorithm can work consistently well regardless of the size of workflows, the number of available resources, or the pattern of workflows. It is evaluated on a well-known benchmark dataset by using the popular WorkflowSim simulator. Our experiments show that scheduling heuristics designed by DSGP can significantly outperform several manually designed and widely used workflow scheduling heuristics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abdelkader, D.M., Omara, F.: Dynamic task scheduling algorithm with load balancing for heterogeneous computing system. Egypt. Inform. J. 13(2), 135–145 (2012)

    Article  Google Scholar 

  2. Arabnejad, V., Bubendorfer, K., Ng, B.: Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1), 29–44 (2018)

    Article  Google Scholar 

  3. Blythe, J., et al.: Task scheduling strategies for workflow-based applications in grids. In: 2005 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2005, vol. 2, pp. 759–767. IEEE (2005)

    Google Scholar 

  4. Braun, T.D., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 61(6), 810–837 (2001)

    Article  Google Scholar 

  5. Buyya, R., Murshed, M.: GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. Concur. Comput.: Pract. Exp. 14(13–15), 1175–1220 (2002)

    Article  Google Scholar 

  6. Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A., Buyya, R.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw.: Pract. Exp. 41(1), 23–50 (2011)

    Google Scholar 

  7. Chen, W., Deelman, E.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)

    Google Scholar 

  8. Mahmood, A., Khan, S.A., Bahlool, R.A.: Hard real-time task scheduling in cloud computing using an adaptive genetic algorithm. Computers 6(2), 15 (2017)

    Article  Google Scholar 

  9. Masood, A., Mei, Y., Chen, G., Zhang, M.: Many-objective genetic programming for job-shop scheduling. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 209–216. IEEE (2016)

    Google Scholar 

  10. Nguyen, S., Zhang, M., Johnston, M., Tan, K.C.: A computational study of representations in genetic programming to evolve dispatching rules for the job shop scheduling problem. IEEE Trans. Evol. Comput. 17(5), 621–639 (2012)

    Article  Google Scholar 

  11. Raghavan, S., Sarwesh, P., Marimuthu, C., Chandrasekaran, K.: Bat algorithm for scheduling workflow applications in cloud. In: 2015 International Conference on Electronic Design, Computer Networks & Automated Verification (EDCAV), pp. 139–144. IEEE (2015)

    Google Scholar 

  12. Rahman, M., Hassan, R., Ranjan, R., Buyya, R.: Adaptive workflow scheduling for dynamic grid and cloud computing environment. Concurr. Comput.: Pract. Exp. 25(13), 1816–1842 (2013)

    Article  Google Scholar 

  13. Sahni, J., Vidyarthi, D.P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2015)

    Article  Google Scholar 

  14. Sonmez, O., Yigitbasi, N., Abrishami, S., Iosup, A., Epema, D.: Performance analysis of dynamic workflow scheduling in multicluster grids. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp. 49–60 (2010)

    Google Scholar 

  15. Tay, J.C., Ho, N.B.: Evolving dispatching rules using genetic programming for solving multi-objective flexible job-shop problems. Comput. Ind. Eng. 54(3), 453–473 (2008)

    Article  Google Scholar 

  16. Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M., et al.: Workflows for e-Science: Scientific Workflows for Grids, vol. 1. Springer, London (2007). https://doi.org/10.1007/978-1-84628-757-2

    Book  Google Scholar 

  17. Xie, J., Mei, Y., Ernst, A.T., Li, X., Song, A.: A genetic programming-based hyper-heuristic approach for storage location assignment problem. In: 2014 IEEE congress on evolutionary computation (CEC), pp. 3000–3007. IEEE (2014)

    Google Scholar 

  18. Yu, Y., Feng, Y., Ma, H., Chen, A., Wang, C.: Achieving flexible scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 3102–3109. IEEE (2019)

    Google Scholar 

  19. Zhan, Z.H., Liu, X.F., Gong, Y.J., Zhang, J., Chung, H.S.H., Li, Y.: Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Comput. Surv. (CSUR) 47(4), 63 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Victoria University of Wellington, Wellington, New Zealand

    Kirita-Rose Escott, Hui Ma & Gang Chen

Authors
  1. Kirita-Rose Escott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kirita-Rose Escott .

Editor information

Editors and Affiliations

  1. Clausthal University of Technology, Clausthal-Zellerfeld, Germany

    Sven Hartmann

  2. Johannes Kepler University of Linz, Linz, Austria

    Josef Küng

  3. Johannes Kepler University of Linz, Linz, Austria

    Gabriele Kotsis

  4. IFS, Vienna University of Technology, Vienna, Wien, Austria

    A Min Tjoa

  5. Johannes Kepler University of Linz, Linz, Austria

    Ismail Khalil

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Escott, KR., Ma, H., Chen, G. (2020). Genetic Programming Based Hyper Heuristic Approach for Dynamic Workflow Scheduling in the Cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2020. Lecture Notes in Computer Science(), vol 12392. Springer, Cham. https://doi.org/10.1007/978-3-030-59051-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59051-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59050-5

  • Online ISBN: 978-3-030-59051-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation