Evolutionary Optimization of Hyperparameters in Deep Learning Models

Created by W.Langdon from gp-bibliography.bib Revision:1.8923

@InProceedings{Kim:2019:CEC,

• author = "Jin-Young Kim and Sung-Bae Cho",
• title = "Evolutionary Optimization of Hyperparameters in Deep Learning Models",
• booktitle = "2019 IEEE Congress on Evolutionary Computation (CEC)",
• year = "2019",
• editor = "Carlos A. {Coello Coello} and Mengjie Zhang and Kay Chen Tan",
• pages = "831--837",
• month = "10-13 " # jun,
• address = "Wellington, New Zealand",
• keywords = "genetic algorithms, genetic programming",
• isbn13 = "978-1-7281-2154-",
• DOI = "10.1109/CEC.2019.8790354",
• abstract = "Recently, deep learning is one of the most popular techniques in artificial intelligence. However, to construct a deep learning model, various components must be set up, including activation functions, optimization methods, a configuration of model structure called hyperparameters. As they affect the performance of deep learning, researchers are working hard to find optimal hyperparameters when solving problems with deep learning. Activation function and optimization technique play a crucial role in the forward and backward processes of model learning, but they are set up in a heuristic way. The previous studies have been conducted to optimize either activation function or optimization technique, while the relationship between them is neglected to search them at the same time. we propose a novel method based on genetic programming to simultaneously find the optimal activation functions and optimization techniques. In genetic programming, each individual is composed of two chromosomes, one for the activation function and the other for the optimization technique. To calculate the fitness of one individual, we construct a neural network with the activation function and optimization technique that the individual represents. The deep learning model found through our method has 82.5percent and 53.0percent of accuracies for the CIFAR-10 and CIFAR-100 datasets, which outperforms the conventional methods. Moreover, we analyze the activation function found and confirm the usefulness of the proposed method.",
• notes = "Also known as \cite{8790354}",

}

Genetic Programming entries for Jin-Young Kim Sung Bae Cho

Citations