Skip to main content
SpringerLink
Log in
Book cover

International Conference on Service-Oriented Computing

ICSOC 2022: Service-Oriented Computing pp 433–448Cite as

Dual-Tree Genetic Programming for Deadline-Constrained Dynamic Workflow Scheduling in Cloud

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13740))

Abstract

Dynamic workflow scheduling (DWS) aims to allocate abundant cloud resources to process a large number of heterogeneous workflows in order to minimize total operation cost and the penalty for violating deadline constraints. Instead of using manually designed heuristics that cannot work effectively across different problem instances, we develop a new Genetic Programming Hyper-Heuristic (GPHH) algorithm to automatically design scheduling heuristics for a newly formulated deadline-constrained dynamic workflow scheduling in cloud (DCDWSC) problem. Different from previous works, our GPHH algorithm can design a pair of rules for Virtual Machine selection and task selection. A new dual-tree representation is proposed to jointly evolve the rule pair, enabling the algorithm to effectively control the inter-dependencies of the two rules. Experimental results show that our new algorithm can significantly outperform three baseline algorithms on a wide range of testing scenarios.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://aws.amazon.com/ec2/pricing/on-demand/.

  2. 2.

    https://confluence.pegasus.isi.edu/display/pegasus/Deprecated+Workflow+Generator.

References

  1. Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Futur. Gener. Comput. Syst. 100, 98–108 (2019)

    Article  Google Scholar 

  2. Armbrust, M., et al.: Above the clouds: a Berkeley view of cloud computing. Technical report (2009)

    Google Scholar 

  3. Chen, W., Deelman, E.: Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)

    Google Scholar 

  4. Djigal, H., Feng, J., Lu, J., Ge, J.: IPPTS: an efficient algorithm for scientific workflow scheduling in heterogeneous computing systems. IEEE Trans. Parallel Distrib. Syst. 32(5), 1057–1071 (2020)

    Article  Google Scholar 

  5. Escott, K.-R., Ma, H., Chen, G.: Genetic programming based hyper heuristic approach for dynamic workflow scheduling in the cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2020. LNCS, vol. 12392, pp. 76–90. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59051-2_6

    Chapter  Google Scholar 

  6. Escott, K.R., Ma, H., Chen, G.: A genetic programming hyper-heuristic approach to design high-level heuristics for dynamic workflow scheduling in cloud. In: 2020 IEEE Symposium Series on Computational Intelligence, pp. 3141–3148. IEEE (2020)

    Google Scholar 

  7. Faragardi, H.R., Saleh Sedghpour, M.R., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: a budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2020)

    Article  Google Scholar 

  8. Ismayilov, G., Topcuoglu, H.R.: Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing. Futur. Gener. Comput. Syst. 102, 307–322 (2020)

    Article  Google Scholar 

  9. Rasouli Kenari, A., Shamsi, M.: A hyper-heuristic selector algorithm for cloud computing scheduling based on workflow features. Opsearch 58(4), 852–868 (2021). https://doi.org/10.1007/s12597-021-00508-6

    Article  MathSciNet  MATH  Google Scholar 

  10. Liu, J., et al.: Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans. Cloud Comput. 9(3), 1180–1194 (2019)

    Article  Google Scholar 

  11. Liu, Y., Mei, Y., Zhang, M., Zhang, Z.: A predictive-reactive approach with genetic programming and cooperative coevolution for the uncertain capacitated arc routing problem. Evol. Comput. 28(2), 289–316 (2020)

    Article  Google Scholar 

  12. O’Neill, M.: Riccardo Poli, William B. Langdon, Nicholas F. Mcphee: a field guide to genetic programming (2009)

    Google Scholar 

  13. Rizvi, N., Dharavath, R., Wang, L., Basava, A.: A workflow scheduling approach with modified fuzzy adaptive genetic algorithm in IaaS clouds. IEEE Trans. Serv. Comput. (2022). https://doi.org/10.1109/TSC.2022.3174112

  14. Tan, B., Ma, H., Mei, Y., Zhang, M.: A cooperative coevolution genetic programming hyper-heuristics approach for on-line resource allocation in container-based clouds. IEEE Trans. Cloud Comput. 10(3), 1500–1514 (2022). https://doi.org/10.1109/TCC.2020.3026338

    Article  Google Scholar 

  15. Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)

    Article  Google Scholar 

  16. Versluis, L., Iosup, A.: A survey of domains in workflow scheduling in computing infrastructures: community and keyword analysis, emerging trends, and taxonomies. Futur. Gener. Comput. Syst. 123, 156–177 (2021)

    Article  Google Scholar 

  17. Wang, Z.J., et al.: Dynamic group learning distributed particle swarm optimization for large-scale optimization and its application in cloud workflow scheduling. IEEE Trans. Cybern. 50(6), 2715–2729 (2020)

    Article  Google Scholar 

  18. Wu, Q., Ishikawa, F., Zhu, Q., Xia, Y., Wen, J.: Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 28(12), 3401–3412 (2017)

    Article  Google Scholar 

  19. Xiao, J.-P., Hu, X.-M., Chen, W.-N.: Dynamic cloud workflow scheduling with a heuristic-based encoding genetic algorithm. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. LNCS, vol. 12533, pp. 38–49. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63833-7_4

    Chapter  Google Scholar 

  20. Xiao, Q.Z., Zhong, J., Feng, L., Luo, L., Lv, J.: A cooperative coevolution hyper-heuristic framework for workflow scheduling problem. IEEE Trans. Serv. Comput. 15(1), 150–163 (2022)

    Google Scholar 

  21. Xie, Y., Gui, F.X., Wang, W.J., Chien, C.F.: A two-stage multi-population genetic algorithm with heuristics for workflow scheduling in heterogeneous distributed computing environments. IEEE Trans. Cloud Comput. (2021). https://doi.org/10.1109/TCC.2021.3137881

  22. Yang, Y., Chen, G., Ma, H., Zhang, M., Huang, V.: Budget and SLA aware dynamic workflow scheduling in cloud computing with heterogeneous resources. In: 2021 IEEE Congress on Evolutionary Computation, pp. 2141–2148 (2021)

    Google Scholar 

  23. Youn, C.H., Chen, M., Dazzi, P.: Cloud Broker and Cloudlet for Workflow Scheduling. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-5071-8

    Book  Google Scholar 

  24. Yu, Y., Feng, Y., Ma, H., Chen, A., Wang, C.: Achieving flexible scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2019 IEEE Congress on Evolutionary Computation, pp. 3102–3109. IEEE (2019)

    Google Scholar 

  25. Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Correlation coefficient-based recombinative guidance for genetic programming hyperheuristics in dynamic flexible job shop scheduling. IEEE Trans. Evol. Comput. 25(3), 552–566 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Computer Science, Victoria University of Wellington, Wellington, 6140, New Zealand

    Yifan Yang, Gang Chen, Hui Ma & Mengjie Zhang

Authors
  1. Yifan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yifan Yang .

Editor information

Editors and Affiliations

  1. University of Malaga, Málaga, Spain

    Javier Troya

  2. University of Michigan-Dearborn, Dearborn, MI, USA

    Brahim Medjahed

  3. University of Castilla-La Mancha, Ciudad Real, Spain

    Mario Piattini

  4. The University of New South Wales, Sydney, NSW, Australia

    Lina Yao

  5. University of Seville, Seville, Spain

    Pablo Fernández

  6. University of Seville, Seville, Spain

    Antonio Ruiz-Cortés

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yang, Y., Chen, G., Ma, H., Zhang, M. (2022). Dual-Tree Genetic Programming for Deadline-Constrained Dynamic Workflow Scheduling in Cloud. In: Troya, J., Medjahed, B., Piattini, M., Yao, L., Fernández, P., Ruiz-Cortés, A. (eds) Service-Oriented Computing. ICSOC 2022. Lecture Notes in Computer Science, vol 13740. Springer, Cham. https://doi.org/10.1007/978-3-031-20984-0_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20984-0_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20983-3

  • Online ISBN: 978-3-031-20984-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation