Evolving behaviors for bounded-flow tracking control of second-order dynamical systems

https://doi.org/10.1016/j.engappai.2018.10.001Get rights and content

Abstract

A two-stage methodology for the development of nonlinear analytical controllers for tracking control in second-order dynamical systems subject to flow variable constraints is proposed. It extends the concepts of behavior-based control to describe the system as the summation of its unforced, forced, and learned behaviors. While the unforced behavior is characterized by its analytical dynamical model, the forced and learned behaviors are introduced in the system by means of a Control-Theory-based controller and an evolutionary learning process based in the Genetic Programming paradigm. The integration of both approaches in a unified framework allows the system to exhibit a good tracking performance while keeping the flow variable bounded to a desired value, parametrized as a boundary interval. A set of 180993 learned behaviors, which preserves asymptotic convergence to the desired behavior while achieving a bounded flow variable, were discovered by the evolutionary process. Simulation results show the effectiveness of the found nonlinear tracking controllers with the highest fitness value, as well as the one with the lower structural complexity. A performance comparison between numerical simulations and real-time experiments for a mechatronic prototype is also provided to illustrate the feasibility of the proposed method in real-world applications.

Introduction

Second-Order Dynamical Systems (SODS) are widely used in engineering to describe the dynamic behavior of mechanical, electrical, pneumatic, and hydraulic systems, among others (see e.g. Ogata, 2003, Fabien, 2009, Ogata, 2010, Khalil, 2002). The general form of these systems is described by the ordinary differential equation aq̈(t)+bq̇(t)+cq(t)=du(t),where q(t),u(t)R is the output and the input, respectively, while q̇(t) and q̈(t) denote the first and second time-derivative of the output. The coefficients a, b, c, and d denote the system parameters. In systems dynamics theory, q̇(t) is the so-called flow variable while u(t) is the effort variable (Ogata, 2003, Fabien, 2009).

The significance of (1) can be extended to a broad range of applications. For instance, the Double-Integrator System (DIS) is described from (1) by taking b and c equal to zero while a and d equal to one. DIS have been commonly used in robotics to describe the motion of a rigid body of a single Degree-Of-Freedom (DOF) with unit moment of inertia (see e.g. Knoll and Röbenack, 2011, Serpelloni et al., 2016), and to model nodes in networked multi-agent systems (Pettersen et al., 2006, Olfati-Saber et al., 2007, Garcia et al., 2016). Other applications of (1) include, among others, modeling of armature-controlled DC motors (neglecting the armature inductance), connectivity in wireless communication and cooperative motion in swarm robotics, in the form of a virtual spring–damper system (Naruse, 2015, Urcola et al., 2008, Tardioli et al., 2010).

The dynamics (1) is indeed an Euler–Lagrange system for which many control strategies can be applied with the aim to make the system output to reach a given reference. If the reference is constant, the control problem is known as set-point regulation while if it is variable is known as (trajectory) tracking. Despite different successful control strategies for these problems have been proposed in Khalil (2002), Kelly et al. (2005) and Spong et al. (2006), their designs do not consider explicitly the physical capabilities of the system, like the maximal efforts that can be demanded from the actuators or the maximal flow that can be developed by the system. In practice, the different variables and signals are physically limited, so these controllers must carefully be implemented since they could easily cause the system to operate under saturated conditions when the limits are overtaken. It has been proved that, under saturation, the system performance is severely deteriorated and could fastly lead to instability (Alvarez-Ramírez et al., 2008). The design of control strategies that take into account these physical limitations is not an easy task since the more constraints are considered, the more complex the controller design will result.

In mechanical systems, the flow variable of a SODS corresponds to the linear or angular velocity. Control of mechanical systems subject to velocity constraints has been an interesting topic of research in robotics and some Control Theory (CT) based solution have been proposed (see e.g. Ngo and Mahony, 2006, Omrcen et al., 2007, Garelli et al., 2011, Hu et al., 2013, Salinas et al., 2016). In Ngo and Mahony (2006), a modified proportional–derivative (PD) controller is proposed for solving the regulation problem in robot manipulators subject to joint velocity constraints. Since the controller includes some damping terms, asymptotic convergence to the reference is slow. In Omrcen et al. (2007), a method for compensating velocity saturation is designed for redundant manipulators. It is based on the robot kinematics, the null-space approach, and the pseudo-inverse Jacobian. The method is restrictive as it is no longer valid in singular configurations. In this sense, Garelli et al. (2011) handles robot singularities and speed constraints in its control solution based on sliding modes; however, the proposal allows the control signals to exhibit chattering which is not suitable for practical implementations in mechanical systems. In Hu et al. (2013), the velocity constraints problem is considered for the attitude control of a spacecraft. Among these results, a common characteristic is that the system is subject to different operating conditions and the performance is deteriorated when the signals are saturated, although stability is preserved. In consequence, the developed controllers are not valid if any initial velocity is out of the specified bound (the results remain limited as the velocity bound cannot be changed online to a lower value to modify the system performance). Furthermore, these contributions mainly deal with the set-point regulation problem (constant reference). Different from these works, in Peñaloza-Mejía et al. (2015), a novel solution has been proposed for trajectory tracking in an omnidirectional mobile robot subject to velocity constraints. The proposal consists of an inverse-dynamics control with an additive excess-of-energy dissipation term that allows the system to fastly converge to the trajectory while keeping the velocities bounded to desired values set by the user.

Some other proposals introduce the use of Soft Computing techniques as an optimization tuning tool to modify the performance of CT-based controllers (Kim and Park, 2005, Chiang and Chen, 2017, Sedghizadeh and Beheshti, 2018). For this scheme, the controller is derived by applying the CT approach and the aim is to find a better performance by testing constant values (either for the system parameters or the controller gains) using Soft Computing techniques, such as Fuzzy Logic, NNs (Neural Networks), PSO (Particle Swarm Optimization), GA (Genetic Algorithms) and GP (Genetic Programming), among others. The optimized parameters aim to increase the convergence rate to the desired behavior as well as fulfill desired features in the response of the system, such as bounded variables or signals demanded by the controller. Nevertheless, most Soft Computing techniques are unable to provide a rigorous analysis of the system response as CT approach does.

The design of the different solutions, either for set-point regulation or trajectory tracking, has been a real challenge. These have led to different fixed control strategies which strongly impose the closed-loop system behavior. Considering (Peñaloza-Mejía et al., 2015), the controller is more flexible as it consists of two terms: one for the tracking task and one for the bounded velocity task. This is one of the keys which allows the system to fastly track the reference while keeping the velocities within their limits. Just to mention, in Peñaloza-Mejía et al. (2015), the bounding-velocity term solely acts to slow down the motion when the velocity has reached the specified limits (the velocity no longer increases), while it has no effect while the velocities are below these limits. Since the design of the bounding-velocity term in Peñaloza-Mejía et al. (2015) was done by hand and it really took its time, an interesting question that arises here is if it is possible to automatically synthesize another (better) controller, but by using computational methods, such that the system achieves asymptotic tracking of the reference with bounded flow variable. Some state-of-the-art proposals in evolutionary robotics have shown that it is possible to automatically synthesize controllers for some problems in robotics (see e.g. Lee and Hallam, 1999, Abdessemed and Benmahammed, 2001, Ng and Johansson, 2002, Nelson et al., 2009, Song et al., 2011, Fukunaga et al., 2012, Lamini et al., 2018).

In this work, an analytic behavior-based framework has been applied for the automatic synthesis of controllers to fulfill physical constraints while achieving the desired task. It takes the flexibility of designing nonlinear controllers for a SODS (1) as the sum of partial control laws to simultaneously accomplish the tracking of the desired reference while keeping bounded the flow variable of the controlled system. Taking advantage of the GP paradigm, the search for behaviors fulfilling both objectives is implemented, where a set of 180993 fittest solutions is obtained.

Mataric (1994) and Matarić and Michaud (2008) developed the idea of behavior-based systems by introducing basis behaviors that lead the system to achieve a goal. Each basis behavior is defined as a minimal set of actions used for decision-making and action–execution processes. Behavior-based control was brought in by Arkin (1998) to endow intelligence into robots through the analysis of behaviors of biological systems and by taking advantage of the interaction of the system with the environment. This approach was developed given a situatedness property of the system to adapt its dynamics to real-world environments via mutual interaction.

In contrast to the traditional approach, our framework extends the concepts of behavior-based control to introduce two essential properties: an analytical representation of the system–environment interaction and the inclusion of the internal dynamics. Both properties are generated by the introduction of CT approach to provide an structure to represent behaviors analytically, and it allows to formulate the feedback in the automation of a learning process for the system. On the other hand, Genetic Programming (GP) has been highly successful as a technique for getting computers to automatically solve problems without having to tell them explicitly (Langdon and Poli, 2010). GP technique suggests a syntactical tree form to represent a solution, where their elements are program instructions or mathematical operators and operands. The novelty of the applied framework in this work is the integration of CT and GP techniques, to derive a scheme for the construction of analytic solutions, in the automation of the synthesis of nonlinear controllers. Specifically, it has been developed to address the tracking problem in a SODS where a bounded flow variable is required. It takes advantage of the analytic representation of the control problem and stability properties of the controlled system, provided by the CT approach, to guarantee a behavior that converge to the desired reference. Then, a GP-based process is implemented for the automatic search of behavior modifiers to keep a bounded flow variable without compromising the achievement of the previous desired behavior.

An overview of the key aspects of the proposed framework is shown in Fig. 1. The entire behavior of the system is characterized by its natural behavior, which is composed of three basis behaviors plus the unknown and unmodeled dynamics, the parametric uncertainties due to normal wear and tear, and the external disturbances from the environment. The basis behaviors are composed of analytic functions and the behavior synthesis is performed by a learning process given a previous CT-based design. The first basis behavior denoted as the unforced behavior is the dynamic model of the system which depends on initial conditions without applying any input or excitation signal. As for the aforementioned control problem addressed in this work, the unforced behavior is the SODS given in (1) with the effort variable u(t) equal to zero. Such definition can be related to the free response of (1) within the CT approach. Whereas the unforced behavior is given by its modeled internal dynamics, the second basis behavior (called forced behavior) is induced by the action of a CT-based controller denoted as uCT. The advantage of applying a CT-based controller is its property of guaranteeing the performance of the system through the concept of stability of equilibrium points. In this case, the equilibrium points of the SODS are those initial conditions that, whenever the system starts at them, it will remain in that equilibrium points for all future time. The forced behavior proposed for the SODS is the fulfilling of control objective given in (2); that is, it must guarantee the convergence of the system to a desired reference qd(t). Then, the consistency of this induced behavior is analyzed by applying one of the most important criteria within CT denominated the Lyapunov stability.

The forced behavior of the system is an innate behavior; i.e., once the stability of the system applying this controller is verified, fulfillment of the specified objective is guaranteed. This behavior is a general solution considering changes in the environment and the initial conditions of the system since the problem setup is based in the CT approach. Finally, the third basis behavior is the learned behavior generated by applying nonlinear controllers derived from a learning process. The learned behaviors act as an adaptive mechanism of the system either to change the environment or to modify its conduct to exhibit desired features. In our approach, the CT-based representation of the control problem is useful to describe the behavior of the system through a multivariable mathematical function, where each part of this function accomplishes an specific goal. Specifically, GP allows the implementation of a learning process where the system acquires the ability to perform under flow variable restrictions while exhibiting the desired response. This new acquired ability is generated by each GP fitted discovered solution denoted as uBF plus the CT-based controller uCT. Here, subscript BF stands for bounded flow since the generated learned behavior aims to limit the flow variable of the SODS within a pre-specified interval. Thus, the natural behavior of the system, defined as the closed-loop dynamics within the CT terminology, is computed as the solution of the SODS (1) while applying the full control input u(t)=uCT+uBF.

In this work, an analytic behavior-based control framework is applied, for the automatic development of a set of output tracking controllers with constrained flow variable for a SODS. To the best of the author’s knowledge, the trajectory tracking problem with desired bounded flow variable had not been studied under this framework, only recently in the work of Peñaloza Mejía et al. (2017), where the problem was particularly addressed for the double-integrator system. In the current work, the synthesis of nonlinear controllers is extended to a wider class of systems modeled by SODS described by (1).

A summary of the main contributions of this paper is listed as follows.

  • The formulation of a learning process for the synthesis of nonlinear controllers in a SODS to solve the tracking problem while exhibiting a bounded flow variable. The implementation of an analytic behavior-based framework allows the definition of the behavior of the system as the sum of three basis behaviors, and the discovery of desired behaviors of the system where a constrained variable is considered.

  • CT approach is applied to provide an analytical representation of the behaviors, and an automation structure to generate new behaviors in the SODS. It has been also used as feedback for the learning process of the system aiming to acquire new features.

  • A traditional CT-based controller is introduced to generate a forced behavior in the SODS which is characterized by the property of guaranteeing the performance of the system through the concept of Lyapunov stability.

  • The GP paradigm allows to implement a learning process in the SODS to find solutions for the flow variable restriction without an explicit programming. This work takes advantage of the properties of the GP to build analytical solutions, and to automate the synthesis of nonlinear controllers for the stated control problems.

  • For the SODS given by the dynamics (1), the learning process must fulfill two main requirements: (a) it must not affect the forced behavior of the system to converge to a desired (twice continuously differentiable) output qd(t)C2, and (b) its entire performance (i.e., its natural behavior) must be measurable within an index of suitability.

  • The developed methodology based on the framework described in this work, can be redesigned where the system is able to learn new features, and embed them as part of its natural behavior. More terms can be added to the effort variable of the system where each one of them aims to fulfill a desirable trait in the behavior of the system without inhibiting each other.

  • The learning process was implemented in a SODS, where all the parameters of the system are the unit. Nevertheless, the discovered nonlinear controllers are flexible since an experimental study was carried out in a mechatronic prototype (modeled by a SODS) where velocity constraints must be met.

The rest of the paper is organized as follows. In Section 2, the synthesis of a tracking CT-based controller is presented, and the description of the layout for the search of evolved behaviors is introduced. The proposed methodology, applied to the design of analytical nonlinear controllers using the GP paradigm, is described in Section 3. In addition, the set of fittest learned behaviors which solve the tracking control problem of a SODS while imposing a bounded flow variable, is also presented. Section 4 provides the results of numerical simulations and real-time experiments in a mechatronic prototype, modeled by a SODS, applying a selected controller from the discovered learned behaviors. Finally, the conclusions are given in Section 5.

Section snippets

Synthesis of tracking controllers

In the following, the formulation upon which the proposed framework for the evolutionary learning process of a SODS towards the development of a set of nonlinear controllers enforcing constrained variables, is presented.

Evolved behavior modifiers for bounded flow variable in a SODS

In this section, a methodology towards the development of nonlinear controllers, joining the CT-based tracking controller built up previously and a GP-based set of learned behaviors, to achieve simultaneously control objectives stated in (2), (3), is presented.

As proposed, let us set the effort variable (4) as the sum of two control inputs given as uCT and uBF. Then, let uBF (known as the bounding flow variable controller) be defined as a set of GP-based control laws generically denoted as uGP.

Application to a mechatronic plant

The aim of this section is to illustrate the effectiveness of applying the learned behavior in a physical system: the Quanser IP02 linear motion plant (Manual, 2012). To this end, numerical simulations and real-time experiments were carried out in this mechatronic plant, considering the trajectory tracking problem with desired bounded velocity, and under the action of the evolved controller. Fig. 10 shows the mechatronic plant, which consists of an aluminum cart sliding along a stainless steel

Conclusions

A new methodology devoted to the design of nonlinear analytic controllers, for tracking control in a SODS subject to flow-variable constraints, has been presented. A conceptual model to extend the Behavior-based Control approach through a framework merging Control Theory with Genetic Programming paradigm has been proposed. The conceptual model let us define the natural behavior of the system as the summation of its basis behaviors given by analytic functions representing its properties, actions

Acknowledgments

The authors acknowledge the unknown reviewers for their valuable comments that helped improve the quality of the paper. Eddie Clemente thanks TecNM the support given through the project 6474.18-P.

References (38)

  • ArkinR.C.

    Behavior-Based Robotics (Intelligent Robotics and Autonomous Agents)

    (1998)
  • FabienB.

    Analytical System Dynamics: Modeling and Simulation

    (2009)
  • FukunagaA. et al.

    Evolving controllers for high-level applications on a service robot: a case study with exhibition visitor flow control

    Genet. Program. Evol. Mach.

    (2012)
  • GarciaE. et al.

    Decentralised event-triggered consensus of double integrator multi-agent systems with packet losses and communication delays

    IET Control Theory Appl.

    (2016)
  • GarelliG. et al.

    Sliding mode speed auto-regulation technique for robotic tracking

    Robot. Auton. Syst.

    (2011)
  • KellyR. et al.

    Control of Robot Manipulators in Joint Space

    (2005)
  • KhalilH.

    Nonlinear systems

    (2002)
  • KimD. et al.

    Intelligent PID Controller Tuning of AVR System Using GA and PSO

    (2005)
  • KozaJ. et al.

    Automatic creation of human-competitive programs and controllers by means of genetic programming

    Genet. Program. Evol. Mach.

    (2000)
  • Cited by (9)

    • Evolved Extended Kalman Filter for first-order dynamical systems with unknown measurements noise covariance

      2022, Applied Soft Computing
      Citation Excerpt :

      For later use, the theoretical basis of EKF, an overview of the analytic behaviors framework, and the description of the logistic map employed to exemplify the proposed approach, are detailed. The analytic behaviors framework is a recent control design scheme for the automated synthesis of analytic nonlinear controllers proposed by [30], and extended in [31–34]. This approach, developed to address control problems, relates the behavior of artificial systems with the principles that govern the behaviors and the learning process in humans.

    • Synthetic-analytic behavior-based control framework: Constraining velocity in tracking for nonholonomic wheeled mobile robots

      2019, Information Sciences
      Citation Excerpt :

      A traditional PD with inverse dynamics is proposed to generate the forced behavior, while the learned responses are searched by an evolutionary approach for a bounded flow variable. The present work extends and develops the ideas introduced by the authors in [6,22,23]. It aims to control a nonholonomic wheeled mobile robot under velocity constraints using a PI-type controller.

    • Analytic Behaviors applied to backlash compensation

      2022, Proceedings of the 24th Robotics Mexican Congress, COMRob 2022
    • Tuning of a SNF tracking controller using genetic algorithms

      2021, Proceedings of the 2021 23rd Robotics Mexican Congress, COMRob 2021
    View all citing articles on Scopus
    View full text