Optimal Control via Reinforcement Learning with Symbolic Policy Approximation

Created by W.Langdon from gp-bibliography.bib Revision:1.8194

@Article{KUBALIK:2017:IFAC-PapersOnLine,

• author = "Jiri Kubalik and Eduard Alibekov and Robert Babuska",
• title = "Optimal Control via Reinforcement Learning with Symbolic Policy Approximation",
• journal = "IFAC-PapersOnLine",
• volume = "50",
• number = "1",
• pages = "4162--4167",
• year = "2017",
• note = "20th IFAC World Congress",
• keywords = "genetic algorithms, genetic programming, reinforcement learning, value iteration, symbolic regression, nonlinear model-based control, optimal control",
• ISSN = "2405-8963",
• DOI = "doi:10.1016/j.ifacol.2017.08.805",
• URL = "http://www.sciencedirect.com/science/article/pii/S2405896317312594",
• abstract = "Model-based reinforcement learning (RL) algorithms can be used to derive optimal control laws for nonlinear dynamic systems. With continuous-valued state and input variables, RL algorithms have to rely on function approximators to represent the value function and policy mappings. This paper addresses the problem of finding a smooth policy based on the value function represented by means of a basis-function approximator. We first show that policies derived directly from the value function or represented explicitly by the same type of approximator lead to inferior control performance, manifested by non-smooth control signals and steady-state errors. We then propose a novel method to construct a smooth policy represented by an analytic equation, obtained by means of symbolic regression. The proposed method is illustrated on a reference-tracking problem of a 1-DOF robot arm operating under the influence of gravity. The results show that the analytic control law performs at least equally well as the original numerically approximated policy, while it leads to much smoother control signals. In addition, the analytic function is readable (as opposed to black-box approximators) and can be used in further analysis and synthesis of the closed loop",

}

Genetic Programming entries for Jiri Kubalik Eduard Alibekov Robert Babuska

Citations