Optimizing hadoop parameter settings with gene expression programming guided PSO
Created by W.Langdon from
gp-bibliography.bib Revision:1.8129
- @Article{journals/concurrency/KhanHLTK17,
-
author = "Mukhtaj Khan and Zhengwen Huang and Maozhen Li and
Gareth A. Taylor and Mushtaq Khan",
-
title = "Optimizing hadoop parameter settings with gene
expression programming guided {PSO}",
-
journal = "Concurrency and Computation: Practice and Experience",
-
year = "2017",
-
volume = "29",
-
number = "3",
-
keywords = "genetic algorithms, genetic programming, gene
expression programming, PSO",
-
bibdate = "2017-05-20",
-
bibsource = "DBLP,
http://dblp.uni-trier.de/db/journals/concurrency/concurrency29.html#KhanHLTK17",
-
DOI = "doi:10.1002/cpe.3786",
-
abstract = "Hadoop MapReduce has become a major computing
technology in support of big data analytics. The Hadoop
framework has over 190 configuration parameters, and
some of them can have a significant effect on the
performance of a Hadoop job. Manually tuning the
optimum or near optimum values of these parameters is a
challenging task and also a time consuming process.
This paper optimizes the performance of Hadoop by
automatically tuning its configuration parameter
settings. The proposed work first employs gene
expression programming technique to build an objective
function based on historical job running records, which
represents a correlation among the Hadoop configuration
parameters. It then employs particle swarm optimization
technique, which makes use of the objective function to
search for optimal or near optimal parameter settings.
Experimental results show that the proposed work
enhances the performance of Hadoop significantly
compared with the default settings. Moreover, it
outperforms both rule-of-thumb settings and the
Starfish model in Hadoop performance optimization",
- }
Genetic Programming entries for
Mukhtaj Khan
Zhengwen Huang
Maozhen Li
Gareth A Taylor
Mushtaq Khan
Citations