Discovering Gated Recurrent Neural Network Architectures
Created by W.Langdon from
gp-bibliography.bib Revision:1.8098
- @InCollection{Rawal:2020:DNE,
-
author = "Aditya Rawal and Jason Liang and Risto Miikkulainen",
-
title = "Discovering Gated Recurrent Neural Network
Architectures",
-
booktitle = "Deep Neural Evolution: Deep Learning with Evolutionary
Computation",
-
publisher = "Springer, Singapore",
-
year = "2020",
-
editor = "Hitoshi Iba and Nasimul Noman",
-
series = "Natural Computing Series",
-
chapter = "9",
-
pages = "233--251",
-
keywords = "genetic algorithms, genetic programming, ANN, Deep
Neuroevolution, LSTM",
-
isbn13 = "978-981-15-3684-7",
-
DOI = "doi:10.1007/978-981-15-3685-4_9",
-
abstract = "Gated recurrent networks such as those composed of
Long Short-Term Memory (LSTM) nodes have recently been
used to improve state of the art in many sequential
processing tasks such as speech recognition and machine
translation. However, the basic structure of the LSTM
node is essentially the same as when it was first
conceived 25 years ago. Recently, evolutionary and
reinforcement-learning mechanisms have been employed to
create new variations of this structure. This chapter
proposes a new method, evolution of a tree-based
encoding of the gated memory nodes, and shows that it
makes it possible to explore new variations more
effectively than other methods. The method discovers
nodes with multiple recurrent paths and multiple memory
cells, which lead to significant improvement in the
standard language modeling benchmark task. The chapter
also shows how the search process can be sped up by
training an LSTM network to estimate performance of
candidate structures, and by encouraging exploration of
novel solutions. Thus, evolutionary design of complex
neural network structures promises to improve
performance of deep learning architectures beyond human
ability to do so.",
-
notes = "Chapter 9, the structure of long short-term memory
(LSTM) nodes is optimized using a genetic
programming-based approach. It is shown that
evolutionary optimisation of LSTM can result in design
which is more complex and powerful compared to human
design",
- }
Genetic Programming entries for
Aditya Rawal
Jason Liang
Risto Miikkulainen
Citations