Grammar-Based Vectorial Genetic Programming for Symbolic Regression

Abstract

Vectorial Genetic Programming (GP) is a young branch of GP, where the training data for symbolic models not only include regular, scalar variables, but also allow vector variables. Also, the model’s abilities are extended to allow operations on vectors, where most vector operations are simply performed component-wise. Additionally, new aggregation functions are introduced that reduce vectors into scalars, allowing the model to extract information from vectors by itself, thus eliminating the need of prior feature engineering that is otherwise necessary for traditional GP to utilize vector data. And due to the white-box nature of symbolic models, the operations on vectors can be as easily interpreted as regular operations on scalars. In this paper, we extend the ideas of vectorial GP of previous authors, and propose a grammar-based approach for vectorial GP that can deal with various challenges noted. To evaluate grammar-based vectorial GP, we have designed new benchmark functions that contain both scalar and vector variables, and show that traditional GP falls short very quickly for certain scenarios. Grammar-based vectorial GP, however, is able to solve all presented benchmarks.

Appendix

Below, the equations to generate the target variable for each benchmark are listed. Each scalar variable $x_i$ is defined by a uniform distribution with lower and upper bound, denoted by $\mathcal {U}(\mathrm {lower}, \mathrm {upper})$. Each vector variable $\boldsymbol{v_i}$ is defined by two hidden, scalar variables for defining the mean and standard deviation of the vector. These hidden variables are also defined and obtained via uniform distributions, like a regular scalar variable. Then, based on the hidden mean and standard deviation for each vector variable, we define the final uniform distribution where the values for each individual vector is sampled from, where the upper and lower bound are $\mu \pm \sqrt{12}\sigma / 2$. In this case, we denote the vector variable by $\mathcal {U}(\mu = [\text {lower mean}, \text {uper mean}], \sigma = [\text {lower std dev}, \text {upper std dev}]; \mathrm {length})$. This step ensures that the mean and standard deviation of each vector is also randomized.

$$\begin{aligned} \mathrm {test\_A\_01} \quad y&= 2.5 \cdot \mathrm {mean}(\boldsymbol{v_1}) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \mathrm {test\_A\_02} \quad y&= 2.5 \cdot \mathrm {mean}(\boldsymbol{v_1}) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \mathrm {test\_A\_03} \quad y&= 2.5 \cdot x_1 + \mathrm {mean}(\boldsymbol{v_1}) + 2.0 \\ x_1&\sim \mathcal {U}(1, 4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \mathrm {test\_A\_04} \quad y&= x_1 \cdot \mathrm {var}(\boldsymbol{v_1}) / 3.0 - 3.0 \cdot \mathrm {mean}(\boldsymbol{v_2}) / x_2 \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,8]; 20) \\ \mathrm {test\_B\_01} \quad y&= x_1 \cdot \mathrm {mean}(\boldsymbol{v_1} + \boldsymbol{v_2}) \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \mathrm {test\_B\_02} \quad y&= x_1 \cdot \mathrm {mean}(\boldsymbol{v_1} \cdot \boldsymbol{v_2}) \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \mathrm {test\_B\_03} \quad y&= x_1 \cdot \mathrm {std}(\boldsymbol{v_1} + \boldsymbol{v_2})\\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \mathrm {test\_B\_04} \quad y&= x_1 \cdot \mathrm {std}(\boldsymbol{v_1} \cdot \boldsymbol{v_2}) \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \mathrm {test\_B\_05} \quad y&= x_1 \cdot \mathrm {mean}((\boldsymbol{v_1} + 2\boldsymbol{v_3}) / (0.5\boldsymbol{v_2})) \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \boldsymbol{v_3}&\sim \mathcal {U}(\mu =[2,4], \sigma =[0.05,0.15]; 20) \\ \mathrm {test\_B\_06} \quad y&= x_1 \cdot \mathrm {std}((\boldsymbol{v_1} + 2\boldsymbol{v_3}) / (0.5\boldsymbol{v_2})) \\ x_1&\sim \mathcal {U}(1, 4) \\ x_2&\sim \mathcal {U}(-8, -4) \\ \boldsymbol{v_1}&\sim \mathcal {U}(\mu =[4,8], \sigma =[2,4]; 20) \\ \boldsymbol{v_2}&\sim \mathcal {U}(\mu =[10,20], \sigma =[4,6]; 20) \\ \boldsymbol{v_3}&\sim \mathcal {U}(\mu =[2,4], \sigma =[0.05,0.15]; 20) \\ \end{aligned}$$