Skip to main content
SpringerLink
Log in

Metaheuristic optimization frameworks: a survey and benchmarking

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper performs an unprecedented comparative study of Metaheuristic optimization frameworks. As criteria for comparison a set of 271 features grouped in 30 characteristics and 6 areas has been selected. These features include the different metaheuristic techniques covered, mechanisms for solution encoding, constraint handling, neighborhood specification, hybridization, parallel and distributed computation, software engineering best practices, documentation and user interface, etc. A metric has been defined for each feature so that the scores obtained by a framework are averaged within each group of features, leading to a final average score for each framework. Out of 33 frameworks ten have been selected from the literature using well-defined filtering criteria, and the results of the comparison are analyzed with the aim of identifying improvement areas and gaps in specific frameworks and the whole set. Generally speaking, a significant lack of support has been found for hyper-heuristics, and parallel and distributed computing capabilities. It is also desirable to have a wider implementation of some Software Engineering best practices. Finally, a wider support for some metaheuristics and hybridization capabilities is needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. The strategy pattern is a particular software design pattern, whereby algorithms can be selected at runtime. This pattern is useful for situations where it is necessary to dynamically swap the algorithms used in an application. The strategy pattern is intended to provide a means to define a family of algorithms, encapsulate each one as an object and make them interchangeable Gamma et al. (1994).

  2. Various techniques to adapt metaheuristics to constrained problems have been proposed in literature (c.f. Michalewicz and Fogel (2004) for instance). However, most of these approaches require ad hoc implementation of the techniques depending on the problem and type of constraints to handle; consequently, it is difficult to integrate those proposals into a MOF. Those ad hoc techniques have been omitted in our comparison.

References

  • Aarts E, Lenstra J (1997) Local search in combinatorial optimization. Wiley

  • Ackley DH (1987) A connectionist machine for genetic hillclimbing. Kluwer Academic Publishers, Norwell, MA, USA

    Book  Google Scholar 

  • Alba E, Luque G, García-Nieto J, Ordonez G, Leguizamon G (2007) Mallba: a software library to design efficient optimisation algorithms. Int J Innov Comput Appl 1:74–85. doi:10.1504/IJICA.2007.013403, http://portal.acm.org/citation.cfm?id=1359342.1359349

    Google Scholar 

  • Andresen B, Gordon JM (1994) Constant thermodynamic speed for minimizing entropy production in thermodynamic processes and simulated annealing. Phys Rev E 50(6):4346–4351. doi:10.1103/PhysRevE.50.4346

    Article  Google Scholar 

  • Angeline PJ, Fogel DB, Fogel LJ (1996) A comparison of self-adaptation methods for finite state machines in a dynamic environment. In: Proc. 5th Ann. Conf. on Evolutionary Programming, pp 441–450

  • Arabas J, Michalewicz Z, Mulawka J (1994) Gavaps-a genetic algorithm with varying population size. Evolutionary Computation. In: Proceedings of the First IEEE Conference on Computational Intelligence, vol 1, pp 73–78. doi:10.1109/ICEC.1994.350039

  • Back T, Fogel DB, Michalewicz Z (eds) (1997) Handbook of evolutionary computation. IOP Publishing Ltd., Bristol. http://portal.acm.org/citation.cfm?id=548530

  • Baker J (1987) Reducing bias and inefficiency in the selection algorithm. In: Proceedings of the Second International Conference on Genetic Algorithms, pp 14–21

  • Benavides D, Segura S, Ruiz-Cortès A (2009) Automated analysis of feature models after 20 years: a literature review. (Information Systems Accepted for publication)

  • Birgmeier M (1996) Evolutionary programming for the optimization of trellis-coded modulation schemes. In: Proc. 5th Ann. Conf. on Evolutionary Programming

  • Blanton JL Jr., Wainwright RL (1993) Multiple vehicle routing with time and capacity constraints using genetic algorithms. In: Proceedings of the 5th International Conference on Genetic Algorithms, Morgan Kaufmann Publishers Inc., San Francisco, pp 452–459

  • Brindle A (1981) Genetic algorithms for function optimization. Ph.D. thesis, University of Alberta, Edmonton

  • Brown AW, Wallnau KC (1996) A framework for evaluating software technology. IEEE Softw 13(5):39–49. doi:10.1109/52.536457

    Google Scholar 

  • Brownlee J (2007) Oat: the optimization algorithm toolkit. Tech. rep., Complex Intelligent Systems Laboratory, Swinburne University of Technology

  • Cahon S, Melab N, Talbi EG (2004) Paradiseo: a framework for the reusable design of parallel and distributed metaheuristics. J Heuristics 10(3):357–380. doi:10.1023/B:HEUR.0000026900.92269.ec

    Google Scholar 

  • Chakhlevitch K, Cowling P (2008) Hyperheuristics: recent developments. In: Adaptive and multilevel metaheuristics, pp 3–29

  • Chatterjee A, Siarry P (2006) Nonlinear inertia weight variation for dynamic adaptation in particle swarm optimization. Comput Oper Res 33(3):859–871. doi:10.1016/j.cor.2004.08.012, http://www.sciencedirect.com/science/article/B6VC5-4DBJG28-2/2/57210fee165fec156db017ff5e59aa5f

  • Clerc M (2006) Particle swarm optimization. ISTE Publishing Company

  • Corne D, Knowles JD, Oates MJ (2000) The pareto envelope-based selection algorithm for multi-objective optimisation. In: Proceedings of the 6th International Conference on Parallel Problem Solving from Nature (PPSN VI), Springer, London, pp 839–848

  • Cowling PI, Kendall G, Soubeiga E (2002) Hyperheuristics: a tool for rapid prototyping in scheduling and optimisation. In: Proceedings of the Applications of Evolutionary Computing on EvoWorkshops 2002, Springer, London, pp 1–10

  • Cramer NL (1985) A representation for the adaptive generation of simple sequential programs. In: Proceedings of the 1st International Conference on Genetic Algorithms, L. Erlbaum Associates Inc., Hillsdale, pp 183–187

  • Davis L (1985) Applying adaptive algorithms to epistatic domains. In: Proceedings of the 9th international joint conference on Artificial intelligence (IJCAI’85). Morgan Kaufmann Publishers Inc., San Francisco, pp 162–164

  • Davis L (1989) Adapting operator probabilities in genetic algorithms. In: Proceedings of the third international conference on Genetic algorithms. Morgan Kaufmann Publishers Inc., San Francisco pp 61–69

  • Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6:182–197

    Article  Google Scholar 

  • de Castro L, Von Zuben F (2002) Learning and optimization using the clonal selection principle. IEEE Trans Evol Comput 6(3):239–251

    Google Scholar 

  • Di Gaspero L, Schaerf A (2003) Easylocal++: An object-oriented framework for flexible design of local search algorithms. Softw Pract Exp 33(8):733–765. doi:10.1002/spe.524

    Article  Google Scholar 

  • Dorigo M, Gambardella L (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1(1):53–66. doi:10.1109/4235.585892

    Article  Google Scholar 

  • Dreo, Pétrowski A, Siarry P, Taillard E (2005) Metaheuristics for hard optimization: methods and case studies. Springer

  • Eiben AE, Raué PE, Ruttkay Z (1994) Genetic algorithms with multi-parent recombination. In: Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature (PPSN III), Springer, London, pp 78–87

  • Eshelman LJ (1991) The chc adaptive search algorithm: how to have safe search when engaging in nontraditional genetic recombination. Foundations of Genetic Algorithms pp 265–283. http://ci.nii.ac.jp/naid/10000024547/en/

  • Eshelman LJ, Schaffer JD (1993) Real-coded genetic algorithms and interval-schemata. In: Whitley DL (ed) Foundation of Genetic Algorithms 2, Morgan Kaufmann., San Mateo, pp 187–202

  • Eshelman LJ, Caruana RA, Schaffer JD (1989) Biases in the crossover landscape. In: Proceedings of the third international conference on Genetic algorithms, Morgan Kaufmann Publishers Inc., San Francisco, pp 10–19

  • Feo T, Resende M (1989) A probabilistic heuristic for a computationally difficult set covering problem. Oper Res Lett 8:67–71

    Article  MathSciNet  MATH  Google Scholar 

  • Feo TA, Resende MG (1995) Greedy randomized adaptive search procedures. J Glob Optim 6:109–133

    Article  MathSciNet  MATH  Google Scholar 

  • Fogarty TC (1989) Varying the probability of mutation in the genetic algorithm. In: Proceedings of the 3rd International Conference on Genetic Algorithms, Morgan Kaufmann Publishers Inc., San Francisco, pp 104–109

  • Fogel D, Fogel L, Atmar J (1991) Meta-evolutionary programming, vol 1. In: Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems and Computers, 1991. pp 540–545. doi:10.1109/ACSSC.1991.186507

  • Fogel LJ (1964) On the organization of intellect. Ph.D. thesis, UCLA

  • Fogel LJ, Fogel DB (1986) Artificial intelligence through evolutionary programming. Tech. rep., Final Report for US Army Research Institute, contract no PO-9-X56-1102C-1

  • Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial intelligence through simulated evolution. Wiley

  • Fonseca CM, Fleming PJ (1993) Genetic algorithms for multiobjective optimization: Formulationdiscussion and generalization. In: Proceedings of the 5th International Conference on Genetic Algorithms, Morgan Kaufmann Publishers Inc., San Francisco, pp 416–423

  • Fontoura M, Lucena C, Andreatta A, Carvalho S, Ribeiro C (2001) Using uml-f to enhance framework development: a case study in the local search heuristics domain. J Syst Softw 57(3):201–206

    Google Scholar 

  • Fowler M (2004) Inversion of control containers and the dependency injection pattern. http://www.martinfowler.com/articles/injection.html

  • Gagnè C, Parizeau M (2006) Genericity in evolutionary computation software tools: principles and case-study. Int J Artif Intell Tools 15(2):173–194

    Article  Google Scholar 

  • Gamma E, Helm R, Johnson R, Vlissides J (1994) Design patterns: elements of reusable object-oriented software, illustrated edition. Addison-Wesley Professional

  • García-Nieto J, Alba E, Chicano F (2007) Using metaheuristic algorithms remotely via ros. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation (GECCO ’07), ACM, New York, pp 1510–1510. doi: 10.1145/1276958.1277239

  • Geman S, Geman D (1987) Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. Readings in computer vision: issues, problems, principles, and paradigms, pp 564–584

  • Glover F (1977) Heuristics for integer programming using surrogate constraints. Decis Sci 8(1):156–166. doi:10.1111/j.1540-5915.1977.tb01074.x

    Google Scholar 

  • Glover F (1989) Tabu search—part i. ORSA J Comput 1:190–206

    MathSciNet  MATH  Google Scholar 

  • Glover F, Kochenberger GA (2002) Handbook of metaheuristic. Kluwer Academic Publishers

  • Goldberg D, Lingle R (1985) Alleles loci and the traveling salesman problem. In: Proc. 1st Int. Conf. on Genetic Algorithms and their Applications, pp 154–159

  • Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley.

  • Goldberg DE (1990) A note on boltzmann tournament selection for genetic algorithms and population-oriented simulated annealing. Complex Syst 4(4):445–460

    MATH  Google Scholar 

  • Goldberg DE, Smith RE (1987) Nonstationary function optimization using genetic algorithm with dominance and diploidy. In: Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, L. Erlbaum Associates Inc., Hillsdale, pp 59–68

  • Hansen P, Mladenović N, Perez-Britos D (2001) Variable neighborhood decomposition search. J Heuristics 7(4):335–350. doi:10.1023/A:1011336210885

    Google Scholar 

  • Ho YC, Pepyne DL (2002) Simple explanation of the no-free-lunch theorem and its implications. J Optim Theory Appl 115(3):549–570. http://www.ingentaconnect.com/content/klu/jota/2002/00000115/00000003/00450394

  • Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press

  • Holland JH (1992) Adaptation in natural and artificial systems. MIT Press, Cambridge

    Google Scholar 

  • Horn J, Nafpliotis N, Goldberg D (1994) A niched pareto genetic algorithm for multiobjective optimization. In: Proceedings of the First IEEE Conference on Evolutionary Computation, 1994, vol 1, pp 82–87. doi:10.1109/ICEC.1994.350037

  • Iredi S, Merkle D, Middendorf M (2001) Bi-criterion optimization with multi colony ant algorithms. In: Proceedings of the First International Conference on Evolutionary Multi-Criterion Optimization (EMO ’01), Springer, London, pp 359–372

  • Jong KAD (1975) An analysis of the behavior of a class of genetic adaptive systems. Ph.D. thesis, University of Michigan

  • Kennedy J, Eberhart R (1995) Particle swarm optimization, vol 4. In: Proceedings., IEEE International Conference on Neural Networks, 1995. pp 1942–1948. doi:10.1109/ICNN.1995.488968.

  • Kennedy J, Mendes R (2002) Population structure and particle swarm performance. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC)

  • Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680. http://www.jstor.org/stable/1690046

    Google Scholar 

  • Kitchenham BA (2004) Procedures for undertaking systematic reviews. Tech. rep., Computer Science Department, Keele University

  • Knowles JD, Corne DW (2000) Approximating the nondominated front using the pareto archived evolution strategy. Evol Comput 8(2):149–172. doi:10.1162/106365600568167

    Google Scholar 

  • Koza JR (1992) Genetic programming: on the programming of computers by natural selection. MIT Press

  • Kronfeld M, Planatscher H, Zell A (2010) The EvA2 optimization framework. In: Blum C, Battiti R (eds) Learning and Intelligent Optimization Conference, Special Session on Software for Optimization (LION-SWOP), Springer, Venice, no. 6073 in Lecture Notes in Computer Science, LNCS, pp 247–250. http://www.ra.cs.uni-tuebingen.de/publikationen/2010/Kron10EvA2Short.pdf

  • Luke S, Panait L, Balan G, Paus S, Skolicki Z, Popovici E, Sullivan K, Harrison J, Bassett J, Hubley R, Chircop A, Compton J, Haddon W, Donnelly S, Jamil B, O’Beirne J (2009) Ecj: A java-based evolutionary computation research system. http://cs.gmu.edu/eclab/projects/ecj/

  • Martin Lukasiewycz FR Michael Glaβ, Helwig S (2009) Opt4j—the optimization framework for java. http://www.opt4j.org

  • Meffert K (2006) JUnit Profi-Tips. Entwickler Press

  • Michalewicz Z (1994) Genetic algorithms plus data structures equals evolution programs. Springer, New York

    Google Scholar 

  • Michalewicz Z, Fogel DB (2004) How to solve it: modern heuristics. Springer

  • Mladenović N (1995) A variable neighborhood search algorithm - a new metaheuristic for combinatorial optimization. Abstracts of papers published at Optimization Week, p 112

  • Monmarchè N, Venturini G, Slimane M (2000) On how pachycondyla apicalis ants suggest a new search algorithm. Futur Gener Comput Syst 16(9):937–946

    Article  Google Scholar 

  • Montana DJ (1995) Strongly typed genetic programming. Evol Comput 3(2):199–230. doi:10.1162/evco.1995.3.2.199

    Google Scholar 

  • Montana DJ, Davis L (1989) Training feedforward neural networks using genetic algorithms. In: Proceedings of the 11th international joint conference on Artificial intelligence (IJCAI’ 89). Morgan Kaufmann Publishers Inc., San Francisco, pp 762–767

  • Muhlenbein H (1991) Evolution on time and space-the parallel genetic glgorithm. Foundations of Genetic Algorithms. http://ci.nii.ac.jp/naid/10016718767/en/

  • Nossal GJV, Lederberg J (1958) Antibody production by single cells. Nature 181(4620):1419–1420. doi:10.1038/1811419a0

    Google Scholar 

  • Nulton JD, Salamon P (1988) Statistical mechanics of combinatorial optimization. Phys Rev A 37(4):1351–1356. doi:10.1103/PhysRevA.37.1351

    Article  MathSciNet  Google Scholar 

  • Oliver IM, Smith DJ, Holland JRC (1987) A study of permutation crossover operators on the traveling salesman problem. In: Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, L. Erlbaum Associates Inc., Hillsdale, pp 224–230

  • Parejo JA, Racero J, Guerrero F, Kwok T, Smith K (2003) Fom: A framework for metaheuristic optimization. Computational Science ICCS 2003 Lecture Notes in Computer Science 2660:886–895, no-indexada

  • Parsopoulos K, Vrahatis M (2002a) Recent approaches to global optimization problems through particle swarm optimization. Nat Comput 1(2):235–306

    Article  MathSciNet  MATH  Google Scholar 

  • Parsopoulos KE, Vrahatis MN (2002b) Particle swarm optimization method in multiobjective problems. In: Proceedings of the 2002 ACM symposium on Applied computing (SAC ’02), ACM, New York, pp 603–607. doi:10.1145/508791.508907

  • Price KV, Storn RM, Lampinen JA (2005) Differential evolution: a practical approach to global optimization. Natural Computing Series, Springer, Berlin. http://www.springer.com/west/home/computer/foundations?SGWID=4-156-22-32104365-0&teaserId=68063&CENTER_ID=69103

  • Radcliffe NJ (1991) Forma analysis and random respectful recombination. In: In Foundations of Genetic Algorithms, pp 222–229

  • Rahman I, Das AK, Mankar RB, Kulkarni BD (2009) Evaluation of repulsive particle swarm method for phase equilibrium and phase stability problems. Fluid Phase Equilibria. doi:10.1016/j.fluid.2009.04.014

  • Raidl GR (2006) Hybrid metaheuristics. Springer, chap a unified view on hybrid metaheuristics, pp 1–12

  • Rechenberg I (1965) Cybernetic solution path of an experimental problem. Royal Aircraft Establishment Library Translation 1122, Farnborough

  • Renders JM, Bersini H (1994) Hybridizing genetic algorithms with hill-climbing methods for global optimization: two possible ways. In: Proceedings of the First IEEE Conference on Computational Intelligence, vol 1, pp 312–317. doi:10.1109/ICEC.1994.349948

  • Roli A, Blum C (2008) Hybrid metaheuristics: an introduction. In: Hybrid metaheuristics, Springer

  • Rothlauf F (2006) Representations for genetic and evolutionary algorithms, 2nd edn. Springer

  • Sasaki D (2005) Armoga: an efficient multi-objective genetic algorithm. Tech. rep.

  • Schaffer JD, Morishima A (1987) An adaptive crossover distribution mechanism for genetic algorithms. In: Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, L. Erlbaum Associates Inc., Hillsdale, pp 36–40

  • Schwefel HP (1981) Numerical optimization of computer models. Wiley, New York

    MATH  Google Scholar 

  • Sloane NJA, Hardin RH (1991–2003) Gosset: a general-purpose program for designing experiments. http://www.research.att.com/njas/gosset/index.html

  • de Souza MC, Martins P (2008) Skewed vns enclosing second order algorithm for the degree constrained minimum spanning tree problem. Eur J Oper Res 191(3):677–690. doi:10.1016/j.ejor.2006.12.061, http://www.sciencedirect.com/science/article/B6VCT-4N2KTC4-7/2/7799160d76fbba32ad42f719ee72bbf9

    Google Scholar 

  • Stutzle T, Hoos H (1997) Max–min ant system and local search for the traveling salesman problem. In: EEE International Conference on Evolutionary Computation, 1997, pp 309–314. doi:10.1109/ICEC.1997.592327.

  • Suganthan PN (1999) Particle swarm optimiser with neighbourhood operator. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 1958–1962

  • Suresh R, Mohanasundaram K (2004) Pareto archived simulated annealing for permutation flow shop scheduling with multiple objectives, vol 2. In: IEEE Conference on Cybernetics and Intelligent Systems, 2004, pp 712–717. doi:10.1109/ICCIS.2004.1460675

  • Syswerda G (1991) Foundations of genetic algorithms. Morgan Kaufmann, chap a study of reproduction in generational and steady-state genetic algorithms

  • Talbi EG (2002) A taxonomy of hybrid metaheuristics. J Heuristics 8(5):541–564

    Article  Google Scholar 

  • Ulungu E, Teghem J, Fortemps P, Tuyttens D (1999) Mosa method: a tool for solving multiobjective combinatorial optimization problems. J Multi Criter Decis Anal 8(4):221–236

    Article  MATH  Google Scholar 

  • Van Veldhuizen DA, Lamont GB (2000) Multiobjective optimization with messy genetic algorithms. In: Proceedings of the 2000 ACM symposium on Applied computing (SAC ’00), ACM, New York, pp 470–476. doi:10.1145/335603.335914

  • Ventura S, Romero C, Zafra A, Delgado J, Hervás C (2008) Jclec: a java framework for evolutionary computation. Soft computing—a fusion of foundations, methodologies and applications 12(4):381–392. 10.1007/s00500-007-0172-0

  • Vesterstrm JS, Riget J (2002) Particle swarms: extensions for improved local, multi-modal, and dynamic search in numerical optimization. Ph.D. thesis, Dept. of Computer Science, University of Aarhus

  • Voß S (2001) Meta-heuristics: the state of the art. pp 1–23

  • Voß S (2002) Optimization software class libraries. Kluwer Academic Publishers

  • Wagner S (2009) Heuristic optimization software systemsm odeling of heuristic optimization algorithms in the heuristic lab software environment. Ph.D. thesis, Johannes Kepler University, Linz

  • Whitley D (1989) The genitor algorithim and selection pressure: Why rank-based allocation of reproductive trials is best. In: Proceedings of the Third International Conference on Genetic Algorithms, pp 116–121

  • Whitley D, Rana S, Heckendorn RB (1999) The island model genetic algorithm: on separability, population size and convergence. CIT J Comput Inf Technol 7(1):33–47

    Google Scholar 

  • Wilke DN, Kok S, Groenwold AA (2007) Comparison of linear and classical velocity update rules in particle swarm optimization: notes on diversity. International. Int J Numer Methods Eng 70(8):962–984

    Article  MathSciNet  MATH  Google Scholar 

  • Wilson GC, Mc Intyre A, Heywood MI (2004) Resource review: Three open source systems for evolving programs–lilgp, ecj and grammatical evolution. Genet Program Evolv Mach 5(1):103–105. doi:10.1023/B:GENP.0000017053.10351.dc

  • Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82. doi:10.1109/4235.585893.

    Google Scholar 

  • Wright AH (1994) Genetic algorithms for real parameter optimization. In: Foundations of genetic algorithms, Morgan Kaufmann, pp 205–218

  • Yao X, Liu Y (1996) Fast evolutionary programming. In: Proc. 5th Ann. Conf. on Evolutionary Programming

  • Zhou H, Grefenstette JJ (1986) Induction of finite automata by genetic algorithms. In: Proceedings of the 1986 IEEE International Conference on Systems, Man, and Cybernetics, pp 170–174

  • Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans Evol Computation 3(4):257–271

    Article  Google Scholar 

  • Zitzler E, Laumanns M, Thiele L (2001) Spea2: Improving the strength pareto evolutionary algorithm. Tech. rep., Computer Engineering and Networks Laboratory (TIK). Department of Electrical Engineering. Swiss Federal Institute of Technology (ETH)

Download references

Acknowledgments

We would like to thank Stefan Wagner, Andreas Schaerf, Sebastián Ventura, Sean Luke, Marcel Kronfeld and David L. Woodruff for their helpful comments in earlier versions of this article. We are thankful to David Benavides and Sergio Segura for providing us their inspirational work Benavides et al. (2009), and Ana Galan for her linguistic support. This work has been partially funded by the European Commission (FEDER) and Spanish Government under CICYT project SETI (TIN2009-07366) and the Andalusian Government projects ISABEL (TIC-2533) and THEOS (TIC-5906).

Author information

Authors and Affiliations

  1. University of Sevilla, Seville, Spain

    José Antonio Parejo, Antonio Ruiz-Cortés, Sebastián Lozano & Pablo Fernandez

Authors
  1. José Antonio Parejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Ruiz-Cortés
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastián Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pablo Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Antonio Parejo.

Appendix: Data tables

Appendix: Data tables

In this section, we provide detailed information about the scores obtained in each characteristic by each framework. Interested readers can obtain more detailed information about assessment on characteristics and features (including comments on problems found on the assessment, penalizations on some features and its underlying reasons and informations sources used to assess it) in http://www.isa.us.es/MOFComparison. Moreover, this spreadsheet can be downloaded and exported to various formats, and it is provided in such a way that user can customize weights of each characteristic, feature and area, allowing the creation of tailored benchmarks more adapted to its specific needs (see Tables 9, 10, 11).

Table 9 Scores for C1–C4 and C6
Full size table
Table 10 Scores for C5 design. Implementation and licensing
Full size table
Table 11 Global scores
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Parejo, J.A., Ruiz-Cortés, A., Lozano, S. et al. Metaheuristic optimization frameworks: a survey and benchmarking. Soft Comput 16, 527–561 (2012). https://doi.org/10.1007/s00500-011-0754-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-011-0754-8

Keywords

Search

Navigation