Hybridizing exact methods and metaheuristics: A taxonomy
Introduction
NP-hard problems are difficult to solve and no polynomial time algorithm are known for solving them. Unfortunately, most combinatorial optimization problems, such as the Travelling Salesman, N-Queens, Bin Packing, 0/1 Knapsack, Graph Partitioning, are NP-hard. Two approaches can be considered to solve this kind of problems depending on their size.
For small instances, researchers usually use exact methods. Exact methods find the optimal solution and assess its optimality. There exist numerous exact methods such as the family of Branch and X (Branch and Bound algorithm [58], Branch and Cut algorithm [42], Branch and Price algorithm [12]), Linear Programming, Dynamic Programing, etc. A branch and X algorithm uses a divide and conquer strategy to partition the solution space into subproblems and then optimizes individually each subproblem. Exact methods are known to be time expensive, so they can not be applied to large NP-hard problems or difficult ones.
When instances become too large for exact methods, heuristics and in particular metaheuristics are often used. There are two main categories of metaheuristics: single solution algorithms and population based algorithms. The first category gathers local search (LS) [54], greedy heuristic (GH) [70], simulated annealing (SA) [50], tabu search (TS) [40], Iterated Local Search (ILS) [56] etc. The second category, which is more and more studied, regroups evolutionary algorithms such as genetic algorithms [44], evolution strategies [74], genetic programming [52], and also ant colonies (AC) [31], scatter search (SS) [39], immune systems [48] etc. However, in general, metaheuristics are not able to solve the problems to optimality and some convergence problems can be encountered.
During the last years, many works have been realized on cooperative (or hybrid) optimization approaches. In many cases, best results are obtained with this kind of approaches, especially on real-life problems. At the beginning, cooperations were mainly realized between several metaheuristics. But nowadays, more and more cooperation schemes between metaheuristics and exact approaches are proposed. These strategies usually give good results because they are able to exploit simultaneously the advantages of both types of methods. For example, it may allow to give quality guarantees to the identified solutions.
In this article, we propose to survey the different cooperation between these two types of method. The fact that more and more papers deal with this kind of approaches (see Fig. 1) clearly indicates that it is an important issue for the operational research community. So it seems interesting to classify these works. A state of the art of this type of cooperation has been proposed recently by Stützle and Dumitrescu [34]. They distinguish five classes of approaches for cooperation between exact and local search methods; they also provide an example for each type. The five classes proposed are:
- •
Use exact algorithms to explore large neighborhoods in local search algorithms.
- •
Perform several runs of a local search and exploit information in high quality solutions to define smaller problems that are amenable for solution with exact algorithms.
- •
Exploit bounds in constructive heuristics.
- •
Use information from relaxations of integer programing problems to guide local search or constructive algorithms.
- •
Use exact algorithms for specific procedures in hybrid metaheuristics.
The survey proposed by Stützle and Dumitrescu presents several cooperative approaches to explain the different classes but would have more interest if it were generalized to every optimization methods. Their paper also excludes some combination such as preprocessing.
In [68], Puchinger and Raidl, propose a survey of the state-of-the-art approaches that combine exact methods and metaheuristics. Their survey provides a classification of methods thanks to different classes. The first one deals with collaborative combinations where no algorithm is contained in any other. This class is divided into subclasses:
- •
Sequential execution.
- •
Parallel and intertwined execution.
The second class regroups integrative combinations and is subdivided into two subclasses:
- •
Incorporating exact algorithms in metaheuristics.
- •
Incorporating metaheuristics in exact algorithms.
Puchinger and Raidl illustrate each subclass with examples issued from the literature.
In this article, we propose also to classify different articles issued from the literature but we will also propose a taxonomy of methods that combines exact and heuristic approaches. Important contributions of this article are the formal grammar proposed to classify the methods, integration of conceptual and hierarchical aspects. A separation between design and implementation is also taken into account. Our survey is far from providing an exhaustive list but it will constitute a good way for authors of new cooperation papers to classify their approach or for developers to find ideas on how to combine methods efficiently. In this article, cooperation and hybridization will be used in the same way. These terms will indicate algorithms which combine different optimization methods. The present paper uses the taxonomy proposed by Talbi [80] as we observe that it is valuable for cooperative methods between metaheuristics and exact approaches.
The remainder of the article is organized as follows. In Sections 2 Cooperation method design, 3 Approach design, 4 Implementation issues the taxonomy used in [80] is recalled and illustrated with examples of cooperation between exact methods and metaheuristics, as in [80] the author only considers cooperation between metaheuristics. The taxonomy is divided into three general aspects: cooperation method design (Section 2), approach design (Section 3), and implementation issues (Section 4). In Section 5, the grammar for hybrid metaheuristics is extended, and an annotated review of different references is presented according to the taxonomy. Conclusions are drawn in Section 6.
Section snippets
Cooperation method design
Cooperation involves two main components: the design and the implementation. The former category concerns the cooperative algorithm itself, involving issues such as the functionality and the architecture. The implementation takes into account the hardware platform, programming model and the environment.
In this section, we will focus on the design of the cooperative mecanisms, i.e. how the methods will cooperate. For each type of classification, the derived classes are presented and some
Approach design
Three criteria have been selected for the flat classification of cooperation between exact and heuristic methods: the complete resolution (exact or approximated), the resolution space (global or partial), and the nature of the cooperation (general or specialist).
Implementation issues
The implementation choices of a cooperative algorithm can be a good way to improve the scalability of a method. Hence, an interesting point is to classify algorithms through the kind of implementation. In an optimization point of view, it is interesting to investigate the use of parallelism.
Global overview
As we have extended the taxonomy [80] to the case of cooperation between metaheuristics and exact methods, we recall the grammar for cooperation schemes. In our case, we consider only heterogeneous models, since metaheuristics and exact methods are definitely different!
For the heuristics methods, we use these following abbreviations:
- •
SA: Simulated Annealing
- •
GA: Genetic Algorithm
- •
MA: Memetic Algorithm
- •
ES: Evolution Strategy
- •
GP: Genetic Programming
- •
NN: Neural Network
- •
DW: Descent Walk
- •
LS: Local Search
- •
ILS:
Conclusion
The interest for hybrid metaheuristics is still growing as we can observe that several workshops and conferences are dealing with it: Hybrid Metaheuristics (HM 2005, 2006) workshop of ECAI, Conference on Hybrid Intelligent Systems (HIS 2005, 2006), Application of Hybrid Evolutionary Algorithms to NP-complete problems workshop of GECCO, etc.
In this paper, we have presented an extension of the work of Talbi [80] to cooperation between exact methods and metaheuristics. The objective was to offer a
References (87)
- et al.
Separating capacity constraints in the CVRP using tabu search
European Journal of Operational Research
(1998) - et al.
A survey of very large-scale neighborhood search techniques
Discrete Applied Mathematics
(2002) Beam-aco – hybridizing ant colony optimization with beam search: An application to open shop scheduling
Computers and OR
(2005)- et al.
Upper and lower bounds for the single source capacitated location problem
EJOR
(2003) - et al.
Embedded local search approaches for routing optimization
Computers and OR
(2005) - et al.
A hybrid method combining continuous tabu search and nelder-mead simplex algorithms for the global optimization of multiminima functions
European Journal of Operational Research
(2005) - et al.
Pareto ant colony optimization with ilp preprocessing in multiobjective project portfolio selection
European Journal of Operational Research
(2006) Future paths for integer programming and links to artificial intelligence
Computers and Operations Research
(1986)- et al.
Solving irregular strip packing problems by hybridising simulated annealing and linear programming
European Journal of Operational Research
(2006) - et al.
Hybrid global optimization algorithms for protein structure prediction: Alternating hybrids
Biophysical Journal
(2003)