Using Decomposed Error for Reproducing Implicit Understanding of Algorithms
Created by W.Langdon from
gp-bibliography.bib Revision:1.7954
- @Article{Owen:EC,
-
author = "Caitlin A. Owen and Grant Dick and Peter A. Whigham",
-
title = "Using Decomposed Error for Reproducing Implicit
Understanding of Algorithms",
-
journal = "Evolutionary Computation",
-
note = "Forthcoming",
-
keywords = "genetic algorithms, genetic programming, evolutionary
machine learning, error decomposition, bias variance
trade-off, stochastic algorithms, geometric semantic
genetic programming, ensemble learning, symbolic
regression",
-
ISSN = "1063-6560",
-
DOI = "doi:10.1162/evco_a_00321",
-
size = "20 pages",
-
abstract = "Reproducibility is important for having confidence in
evolutionary machine learning algorithms. Although the
focus of reproducibility is usually to recreate an
aggregate prediction error score using fixed random
seeds, this is not sufficient. Firstly, multiple runs
of an algorithm, without a fixed random seed, should
ideally return statistically equivalent results.
Secondly, it should be confirmed whether the expected
behaviour of an algorithm matches its actual behaviour,
in terms of how an algorithm targets a reduction in
prediction error. Confirming the behaviour of an
algorithm is not possible when using a total error
aggregate score. Using an error decomposition framework
as a methodology for improving the reproducibility of
results in evolutionary computation addresses both of
these factors. By estimating decomposed error using
multiple runs of an algorithm and multiple training
sets, the framework provides a greater degree of
certainty about the prediction error. Also, decomposing
error into bias, variance due to the algorithm
(internal variance) and variance due to the training
data (external variance) more fully characterises
evolutionary algorithms. This allows the behaviour of
an algorithm to be confirmed. Applying the framework to
a number of evolutionary algorithms shows that their
expected behaviour can be different to their actual
behaviour. Identifying a behaviour mismatch is
important in terms of understanding how to further
refine an algorithm as well as how to effectively apply
an algorithm to a problem.",
- }
Genetic Programming entries for
Caitlin A Owen
Grant Dick
Peter Alexander Whigham
Citations