Abstract
We focus on the evolutionary design of programs capable of capturing more randomness and outliers in the input data set than the standard genetic programming (GP)-based methods typically allow. We propose Genetic Programming with Associative Memory (GPAM) – a GP-based system for symbolic regression which can utilize a small associative memory to store various data points to better approximate the original data set. The method is evaluated on five standard benchmarks in which a certain number of data points is replaced by randomly generated values. In another case study, GPAM is used as an on-chip generator capable of approximating the weights for a convolutional neural network (CNN) to reduce the access to an external weight memory. Using Cartesian genetic programming (CGP), we evolved expression-memory pairs that can generate weights of a single CNN layer. If the associative memory contains 10% of the original weights, the weight generator evolved for a convolutional layer can approximate the original weights such that the CNN utilizing the generated weights shows less than a 1% drop in the classification accuracy on the MNIST data set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Data sets are available at https://doi.org/10.5281/zenodo.7583555.
References
Capra, M., Bussolino, B., Marchisio, A., Shafique, M., Masera, G., Martina, M.: An updated survey of efficient hardware architectures for accelerating deep convolutional neural networks. Future Internet 12(7), 113 (2020)
Dupuis, E., Novo, D., O’Connor, I., Bosio, A.: A heuristic exploration of retraining-free weight-sharing for CNN compression. In: 27th Asia and South Pacific Design Automation Conference, ASP-DAC, pp. 134–139. IEEE (2022)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and Huffman coding. In: 4th International Conference on Learning Representations, ICLR (2016)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
La Cava, W., et al.: Contemporary symbolic regression methods and their relative performance. In: Vanschoren, J., Yeung, S. (eds.) Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. vol. 1 (2021)
La Cava, W.G., Helmuth, T., Spector, L., Moore, J.H.: A probabilistic and multi-objective analysis of lexicase selection and \(\epsilon \)-lexicase selection. Evol. Comput. 27(3), 377–402 (2019)
Langdon, W.B.: Genetic Programming and Data Structures: Genetic Programming \(+\) Data Structures = Automatic Programming! Springer, Cham (1998)
LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. ATT Labs. https://yann.lecun.com/exdb/mnist (2010)
McDermott, J., et al.: Genetic programming needs better benchmarks. In: Proceedings of the 14th International Conference on Genetic and Evolutionary Computation, pp. 791–798. ACM (2012)
Miller, J.F.: Cartesian Genetic Programming. Springer, Berlin (2011)
Schmidt, M.D., Lipson, H.: Coevolution of fitness predictors. IEEE Trans. Evol. Comput. 12(6), 736–749 (2008)
Stanley, K.O.: Compositional pattern producing networks: a novel abstraction of development. Genet. Program Evolvable Mach. 8(2), 131–162 (2007)
Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009)
Sze, V., Chen, Y., Yang, T., Emer, J.S.: Efficient processing of deep neural networks. Synth. Lect. Comput. Archit. 15(2), 1–341 (2020)
Teller, A.: The evolution of mental models. In: Kinnear, K.E., Jr. (ed.) Advances in Genetic Programming, pp. 199–219. MIT Press, Cambridge (1994)
Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming. Genet. Program Evolvable Mach. 15(2), 195–214 (2014). https://doi.org/10.1007/s10710-013-9210-0
Villegas-Cortez, J., Olague, G., Aviles, C., Sossa, H., Ferreyra, A.: Automatic synthesis of associative memories through genetic programming: a first co-evolutionary approach. In: Di Chio, C., et al. (eds.) EvoApplications 2010. LNCS, vol. 6024, pp. 344–351. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12239-2_36
Acknowledgements
This work was supported by the Czech science foundation project 21-13001S, and it was partly carried out under the COST Action CA19135 (CERCIRAS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Juza, T., Sekanina, L. (2023). GPAM: Genetic Programming with Associative Memory. In: Pappa, G., Giacobini, M., Vasicek, Z. (eds) Genetic Programming. EuroGP 2023. Lecture Notes in Computer Science, vol 13986. Springer, Cham. https://doi.org/10.1007/978-3-031-29573-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-29573-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29572-0
Online ISBN: 978-3-031-29573-7
eBook Packages: Computer ScienceComputer Science (R0)