Hybridizing exact methods and metaheuristics: A taxonomy

https://doi.org/10.1016/j.ejor.2007.07.035Get rights and content

Abstract

The interest about hybrid optimization methods has grown for the last few years. Indeed, more and more papers about cooperation between heuristics and exact techniques are published. In this paper, we propose to extend an existing taxonomy for hybrid methods involving heuristic approaches in order to consider cooperative schemes between exact methods and metaheuristics. First, we propose some natural approaches for the different schemes of cooperation encountered, and we analyse, for each model, some examples taken from the literature. Then we recall and complement the proposed grammar and provide an annotated bibliography.

Introduction

NP-hard problems are difficult to solve and no polynomial time algorithm are known for solving them. Unfortunately, most combinatorial optimization problems, such as the Travelling Salesman, N-Queens, Bin Packing, 0/1 Knapsack, Graph Partitioning, are NP-hard. Two approaches can be considered to solve this kind of problems depending on their size.

For small instances, researchers usually use exact methods. Exact methods find the optimal solution and assess its optimality. There exist numerous exact methods such as the family of Branch and X (Branch and Bound algorithm [58], Branch and Cut algorithm [42], Branch and Price algorithm [12]), Linear Programming, Dynamic Programing, etc. A branch and X algorithm uses a divide and conquer strategy to partition the solution space into subproblems and then optimizes individually each subproblem. Exact methods are known to be time expensive, so they can not be applied to large NP-hard problems or difficult ones.

When instances become too large for exact methods, heuristics and in particular metaheuristics are often used. There are two main categories of metaheuristics: single solution algorithms and population based algorithms. The first category gathers local search (LS) [54], greedy heuristic (GH) [70], simulated annealing (SA) [50], tabu search (TS) [40], Iterated Local Search (ILS) [56] etc. The second category, which is more and more studied, regroups evolutionary algorithms such as genetic algorithms [44], evolution strategies [74], genetic programming [52], and also ant colonies (AC) [31], scatter search (SS) [39], immune systems [48] etc. However, in general, metaheuristics are not able to solve the problems to optimality and some convergence problems can be encountered.

During the last years, many works have been realized on cooperative (or hybrid) optimization approaches. In many cases, best results are obtained with this kind of approaches, especially on real-life problems. At the beginning, cooperations were mainly realized between several metaheuristics. But nowadays, more and more cooperation schemes between metaheuristics and exact approaches are proposed. These strategies usually give good results because they are able to exploit simultaneously the advantages of both types of methods. For example, it may allow to give quality guarantees to the identified solutions.

In this article, we propose to survey the different cooperation between these two types of method. The fact that more and more papers deal with this kind of approaches (see Fig. 1) clearly indicates that it is an important issue for the operational research community. So it seems interesting to classify these works. A state of the art of this type of cooperation has been proposed recently by Stützle and Dumitrescu [34]. They distinguish five classes of approaches for cooperation between exact and local search methods; they also provide an example for each type. The five classes proposed are:

  • Use exact algorithms to explore large neighborhoods in local search algorithms.

  • Perform several runs of a local search and exploit information in high quality solutions to define smaller problems that are amenable for solution with exact algorithms.

  • Exploit bounds in constructive heuristics.

  • Use information from relaxations of integer programing problems to guide local search or constructive algorithms.

  • Use exact algorithms for specific procedures in hybrid metaheuristics.

The survey proposed by Stützle and Dumitrescu presents several cooperative approaches to explain the different classes but would have more interest if it were generalized to every optimization methods. Their paper also excludes some combination such as preprocessing.

In [68], Puchinger and Raidl, propose a survey of the state-of-the-art approaches that combine exact methods and metaheuristics. Their survey provides a classification of methods thanks to different classes. The first one deals with collaborative combinations where no algorithm is contained in any other. This class is divided into subclasses:

  • Sequential execution.

  • Parallel and intertwined execution.

The second class regroups integrative combinations and is subdivided into two subclasses:

  • Incorporating exact algorithms in metaheuristics.

  • Incorporating metaheuristics in exact algorithms.

Puchinger and Raidl illustrate each subclass with examples issued from the literature.

In this article, we propose also to classify different articles issued from the literature but we will also propose a taxonomy of methods that combines exact and heuristic approaches. Important contributions of this article are the formal grammar proposed to classify the methods, integration of conceptual and hierarchical aspects. A separation between design and implementation is also taken into account. Our survey is far from providing an exhaustive list but it will constitute a good way for authors of new cooperation papers to classify their approach or for developers to find ideas on how to combine methods efficiently. In this article, cooperation and hybridization will be used in the same way. These terms will indicate algorithms which combine different optimization methods. The present paper uses the taxonomy proposed by Talbi [80] as we observe that it is valuable for cooperative methods between metaheuristics and exact approaches.

The remainder of the article is organized as follows. In Sections 2 Cooperation method design, 3 Approach design, 4 Implementation issues the taxonomy used in [80] is recalled and illustrated with examples of cooperation between exact methods and metaheuristics, as in [80] the author only considers cooperation between metaheuristics. The taxonomy is divided into three general aspects: cooperation method design (Section 2), approach design (Section 3), and implementation issues (Section 4). In Section 5, the grammar for hybrid metaheuristics is extended, and an annotated review of different references is presented according to the taxonomy. Conclusions are drawn in Section 6.

Section snippets

Cooperation method design

Cooperation involves two main components: the design and the implementation. The former category concerns the cooperative algorithm itself, involving issues such as the functionality and the architecture. The implementation takes into account the hardware platform, programming model and the environment.

In this section, we will focus on the design of the cooperative mecanisms, i.e. how the methods will cooperate. For each type of classification, the derived classes are presented and some

Approach design

Three criteria have been selected for the flat classification of cooperation between exact and heuristic methods: the complete resolution (exact or approximated), the resolution space (global or partial), and the nature of the cooperation (general or specialist).

Implementation issues

The implementation choices of a cooperative algorithm can be a good way to improve the scalability of a method. Hence, an interesting point is to classify algorithms through the kind of implementation. In an optimization point of view, it is interesting to investigate the use of parallelism.

Global overview

As we have extended the taxonomy [80] to the case of cooperation between metaheuristics and exact methods, we recall the grammar for cooperation schemes. In our case, we consider only heterogeneous models, since metaheuristics and exact methods are definitely different!

For the heuristics methods, we use these following abbreviations:

  • SA: Simulated Annealing

  • GA: Genetic Algorithm

  • MA: Memetic Algorithm

  • ES: Evolution Strategy

  • GP: Genetic Programming

  • NN: Neural Network

  • DW: Descent Walk

  • LS: Local Search

  • ILS:

Conclusion

The interest for hybrid metaheuristics is still growing as we can observe that several workshops and conferences are dealing with it: Hybrid Metaheuristics (HM 2005, 2006) workshop of ECAI, Conference on Hybrid Intelligent Systems (HIS 2005, 2006), Application of Hybrid Evolutionary Algorithms to NP-complete problems workshop of GECCO, etc.

In this paper, we have presented an extension of the work of Talbi [80] to cooperation between exact methods and metaheuristics. The objective was to offer a

References (87)

  • V. Nwana et al.

    A co-operative parallel heuristic for mixed zero-one linear programming: Combining simulated annealing with branch and bound

    European Journal of Operational Research

    (2005)
  • M.C. Portmann et al.

    Branch and bound crossed with ga to solve hybrid flowshops

    European Journal of Operational Research

    (1998)
  • M.C. Portmann et al.

    Branch and bound crossed with ga to solve hybrid flowshops

    European Journal of Operational Research

    (1998)
  • R. Spina et al.

    A hybrid approach to the single line scheduling problem with multiple products and sequence-dependent time

    Computers and Industrial Engineering

    (2003)
  • V. T’kindt et al.

    An ant colony optimization algorithm to solve a 2-machine bicriteria flowshop scheduling problem

    European Journal of Operational Research

    (2002)
  • E. Alba, F. Almeida, M. Blesa, C. Cotta, M. Dfaz, I. Dorta, J. Abarr, J. Gonzlez, C. Len, L. Moreno, J. Petit, J. Roda,...
  • D. Applegate, R. Bixby, V. Chvátal, W. Cook, Finding tours in the TSP. Technical Report 99885, Forschungsinstitut fnr...
  • I. Androulakis et al.

    ffbb: A global optimization method for general constrained nonconvex problems

    Journal of Global Optimization

    (1995)
  • A. Atamturk et al.

    A combined lagrangian, linear programming and implication heuristic for large-scale set partitioning problems

    Journal of Heuristics

    (1996)
  • A.C.F. Alvim et al.

    A hybrid improvement heuristic for the one-dimensional bin packing problem

    Journal of Heuristics

    (2004)
  • E.K. Burke et al.

    Effective local and guided variable neighbourhood search methods for the asymmetric travelling salesman problem

  • M. Barake et al.

    Metaheuristics: Computer decision-making

    The Probe Metaheuristic and its Application to the Multiconstraint Knapsack Problem

    (2004)
  • J.A. Bennell et al.

    Hybridising tabu search with optimisation techniques for irregular stock cutting

    Management Science

    (2001)
  • K. Büdenbender et al.

    A hybrid tabu search/branch-and-bound algorithm for the direct flight network design problem

    Transportation Science

    (2000)
  • C. Barnhart et al.

    Branch-and-price: Column generation for solving huge integer programs

    Operations Research

    (1998)
  • J.F. Bard et al.

    A branch-and-cut procedure for the vehicle routing problem with time windows

    Transportation Science

    (2002)
  • M. Basseur et al.

    Cooperation between branch and bound and evolutionary approaches to solve a biobjective flow shop problem

  • R. Bent, P. Van Hentenryck, A two-stage hybrid local search for the vehicle routing problem with time windows....
  • C. Cotta et al.

    Hybridizing genetic algorithms with branch and bound techniques for the resolution of the tsp

  • D. Clements, J. Crawford, D. Joslin, G. Nemhauser, M. Puttlitz, M. Savelsbergh, Heuristic optimization: a hybrid AI/OR...
  • A. Chabrier, E. Danna, C. Le Pape, Coopération entre génération de colonnes sans cycle et recherche locale appliquée au...
  • T.G. Crainic et al.

    A simplex-based tabu search method for capacitated network design

    INFORMS Journal on Computing

    (2000)
  • Y. Caseau, F. Laburthe, Effective forget-and-extend heuristics for scheduling problems, in: CP-AI-OR’02 – Fourth...
  • R.K. Congram et al.

    An iterated dynasearch algorithm for the single-machine total weighted tardiness scheduling problem

    INFORMS Journal on Computing

    (2002)
  • C. Cotta et al.

    Embedding branch and bound within evolutionary algorithms

    Applied Intelligence

    (2003)
  • K.P. Dahal et al.

    Evolutionary hybrid approaches for generation scheduling in power systems

    European Journal of Operational Research

    (2006)
  • B. De Backer et al.

    Solving vehicle routing problems using constraint programming and metaheuristics

    Journal of Heuristics

    (2000)
  • F. Della Croce et al.

    Recovering beam search: Enhancing the beam search approach for combinatorial optimization problems

    Journal of Heuristics

    (2004)
  • M. Dorigo, V. Maniezzo, A. Colorni, Positive feedback as a search strategy. Technical Report No. 91016, Dipartimento di...
  • J. Denzinger, T. Offermann, On cooperation between evolutionary algorithms and other search paradigms, in: CEC-99,...
  • C. Duhamel, A. Quilliot, Coupling a metaheuristic with an exact method for the local access network design problem, In...
  • I. Dumitrescu, T. Stützle, Combinations of local search and exact algorithms, in: EvoWorkshops 2003, 2003, pp....
  • H. Feltl et al.

    An improved hybrid genetic algorithm for the generalized assignment problem

  • Cited by (0)

    View full text