Abstract
A top end graphics card (GPU) plus a suitable SIMD interpreter can deliver a several hundred fold speed up, yet cost less than the computer holding it. We give highlights of AI and computational intelligence applications in the new field of general purpose computing on graphics hardware (GPGPU). In particular, we surveyed genetic programming (GP) use with GPU. We gave several applications from Bioinformatics and showed that how the fastest GP is based on an interpreter rather than compilation. Finally using GP to generate GPU CUDA kernel C++ code is sketched.
Similar content being viewed by others
Notes
Tesla are high end NVIDIA GPUs dedicated to computation rather than generating graphical images.
General-purpose computation on graphics hardware http://www.gpgpu.org/.
Genetic programming on general purpose graphics processing units http://www.gpgpgpu.com/.
References
Anderson DT, Luke RH III, Keller JM (2008) Speedup of fuzzy clustering through stream processing on graphics processing units. IEEE Trans Fuzzy Syst 16:1101–1106
Arabnia HR, Oliver MA (1987) A transputer network for the arbitrary rotation of digitised images. Comput J 30:425–432
Arora R, Tulshyan R, Deb K (2010) Parallelization of binary and real-coded genetic algorithms on GPU using CUDA. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3680–3687
Bai H, OuYang D, Li X, He L, Yu H (2009) MAX-MIN ant system on GPU with CUDA. Fourth international conference on innovative computing, information and control (ICICIC), 2009. IEEE, Kaohsiung, 7–9 Dec., pp 801–804
Baskaran MM, Ramanujam J, Sadayappan P (2010) Automatic C-to-CUDA code generation for affine programs. In: Gupta R (ed) 9th International Conference Compiler Construction, CC 2010, Paphos, Cyprus, March 20–28, Lecture Notes in Computer Science, 6011, pp 244–263. Springer, Berlin. (Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010)
Bernhard F, Keriven R (2006) In: Alexandrov VN, van Albada GD, Sloot PMA, Dongarra J (eds) Proceedings of the 6th international conference on computational science, ICCS 2006, Part IV, Reading, May 28–31, Lecture Notes in Computer Science, 3994, pp 236–243. Springer, Berlin
Bhuiyan MA, Pallipuram VK, Smith MC (2010) Acceleration of spiking neural networks in emerging multi-core and GPU architectures. In: Karypis G (ed) Ninth IEEE international Wwrkshop on high performance computational biology. Atlanta, 19 April
Chitty DM (2007) A data parallel approach to genetic programming using programmable graphics hardware. In: Thierens D, Beyer HG, Bongard J, Branke J, Clark JA, Cliff D, Congdon CB, Deb K, Doerr B, Kovacs T, Kumar S, Miller JF, Moore J, Neumann F, Pelikan M, Poli R, Sastry K, Stanley KO, Stutzle T, Watson RA, Wegener I (eds) GECCO ’07: Proceedings of the 9th annual conference on genetic and evolutionary computation. London, 7–11 July. ACM Press, pp 1566–1573
Chwatal AM, Raidl GR, Zöch M (2010) Fitting multi-planet transit models to photometric time-data series by evolution strategies. In: Pelikan M, Branke J (eds) Genetic and evolutionary computation conference, GECCO 2010. ACM, Portland, Oregon, July 7–11, pp 377–384
Chong FS, Langdon WB (1999) Java based distributed genetic programming on the internet. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference, Orlando, 13–17 July 1229. Morgan Kaufmann (Full text in technical report CSRP-99-7)
Clayton TF, Patel LN, Leng G, Murray AF, Lindsay IA (2008) Rapid evaluation and evolution of neural models using graphics card hardware. In: Keijzer M, Antoniol G, Congdon CB, Deb K, Doerr B, Hansen N, Holmes JH, Hornby GS, Howard D, Kennedy J, Kumar S, Lobo FG, Miller JF, Moore J, Neumann F, Pelikan M, Pollack J, Sastry K, Stanley K, Stoica A, Talbi EG, Wegener I (eds) GECCO ’08: Proceedings of the 10th annual conference on Genetic and evolutionary computation. ACM, Atlanta, 12–16 July, pp 299–306
Comte P (2009) Design & implementation of parallel linear GP for the IBM cell processor. In: Raidl G, Rothlauf F, Squillero G, Drechsler R, Stuetzle T, Birattari M, Congdon CB, Middendorf M, Blum C, Cotta C, Bosman P, Grahl J, Knowles J, Corne D, Beyer HG, Stanley K, Miller JF, van Hemert J, Lenaerts T, Ebner M, Bacardit J, O’Neill M, Di Penta M, Doerr B, Jansen T, Poli R, Alba E (eds) GECCO ’09: Proceedings of the 11th annual conference on genetic and evolutionary computation, Montreal, 8–12 July. ACM
Cole N, Desell T, Lombrana Gonzalez D, Fernandez de Vega F, Magdon-Ismail M, Newberg H, Szymanski B, Varela C (2010) Evolutionary algorithms on volunteer computing platforms: The milkyway@home project. In: Fernandez de Vega F, Cantu-Paz E (eds) Parallel and distributed computational intelligence, chapter 4, pp. 63–90. Springer, Berlin
de P Veronese L, Krohling RA (2010) Differential Evolution algorithm on the GPU with C-CUDA. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 1878–1884
Del Rizzo B (2008). Dice puts faith in nvidia PhysX technology for Mirror’s Edge. NVIDIA Corporation press release, Santa Clara
Desell T, Anderson DP, Magdon-Ismail M, Newberg H, Szymanski B, Varela CA (2010) An analysis of massively distributed evolutionary algorithms. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence, Barcelona. 18–23 July, pp 873–880
Ebner M, Reinhardt M, Albert J (2005) Evolution of vertex and pixel shaders. In: Keijzer M, Tettamanzi A, Collet P, van Hemert JI, Tomassini M (eds) Proceedings of the 8th European conference on genetic programming. Lausanne, 30 March–1 April, Lecture Notes in Computer Science, 3447. Springer, Berlin, pp 261–270
Flynn MJ (1972) Some computer organizations and their effectiveness. IEEE Trans Comput C-21:948–960
Fidjeland AK, Roesch EB, Shanahan MP, Luk W (2009) NeMo: A platform for neural modelling of spiking neurons using GPUs. In: 20th IEEE international conference on application-specific systems, architectures and processors, ASAP 2009. Rennes, 7–9 July, pp 137–144
Fok K-L, Wong T-T, Wong M-L (2007) Evolutionary computing on consumer graphics hardware. IEEE Intell Syst 22:69–78
Franco MA, Krasnogor N, Bacardit J (2010) Speeding up the evaluation of evolutionary learning systems using GPGPUs. In: Pelikan M, Branke J (eds) Genetic and evolutionary computation conference, GECCO 2010. ACM, Portland, Oregon, July 7–11, pp 1039–1046
Fidjeland AK, Shanahan MP (2010) Accelerated simulation of spiking neural networks using GPUs. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 536–543
Gobron S, Devillard F, Heit B (2007) Retina simulation using cellular automata and GPU programming. Mach Vision Appl 18:331–342
Han B, Taha TM (2010a) Acceleration of spiking neural network based pattern recognition on NVIDIA graphics processors. Appl Optics 49 49:B83–B91
Han B, Taha TM (2010b) Neuromorphic models on a GPGPU cluster. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3050–3057
Harding S, Banzhaf W (2007) Fast genetic programming on GPUs. In: Ebner M, O’Neill M, Ekárt A, Vanneschi L, Esparcia-Alcázar AI (eds) Proceedings of the 10th European conference on genetic programming. Valencia, Spain, 11–13 April, Lecture Notes in Computer Science, 4445. Springer, Berlin, pp 90–101
Harding SL, Banzhaf W (2009) Distributed genetic programming on GPUs using CUDA. In: Hidalgo I, Fernandez F, Lanchares J (eds) Workshop on parallel architectures and bioinspired algorithms. Raleigh, Sept 13
Harris C (1997) An investigation into the application of genetic programming techniques to signal analysis and feature detection. PhD thesis, University College, London
Harris C, Haines K (2005) Iterative solutions using programmable graphics processing units. In: The 14th IEEE international conference on fuzzy systems, FUZZ ’05. Reno, Nevada, 22–25 May, pp 12–18 (IEEE)
Harvey N, Luke R, Keller JM, Anderson D (2008) Speedup of fuzzy logic through stream processing on graphics processing units. In: Wang J (ed) 2008 IEEE world congress on computational intelligence. Hong Kong, 1–6 June, pp 3809–3815
Huang C-H, Racoceanu D, Roux L, Putt TC (2010) Bio-inspired computer visual system using GPU and visual pattern assessment language (ViPAL): application on breast cancer prognosis. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 1103–1110
Izydorczyk J, Izydorczyk M (2010) Microprocessor scaling: what limits will hold. IEEE Comput 43:20–26
Juille H, Pollack JB (1996) Massively parallel genetic programming. In: Angeline PJ, Kinnear Jr, KE (eds) Advances in genetic programming 2, Chap. 17. MIT Press, Cambridge, pp 339–358
Kannan S, Ganji R (2010) Porting Autodock to CUDA. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3815–3822
Klein J, Spector L (2007) Unwitting distributed genetic programming via asynchronous JavaScript and XML. In Thierens D, Beyer H-G, Bongard J, Branke J, Clark JA, Cliff D, Congdon CB, Deb K, Doerr B, Kovacs T, Kumar S, Miller JF, Moore J, Neumann F, Pelikan M, Poli R, Sastry K, Stanley KO, Stutzle T, Watson RA, Wegener I (eds) GECCO ’07: Proceedings of the 9th annual conference on genetic and evolutionary computation, London, 7–11 July, pp 1628–1635. ACM Press
Keith MJ, Martin MC (1994) Genetic programming in C++: Implementation issues. In: Kinnear Jr KE (ed) Advances in genetic programming, Chap. 13. pp 285–310. MIT Press, Cambridge
Langdon WB (2008) Evolving GeneChip correlation predictors on parallel graphics hardware. In: Wang J (ed) 2008 IEEE world congress on computational intelligence. Hong Kong, 1–6 June, pp 4152–4157
Langdon WB, Banzhaf W (2008) A SIMD interpreter for genetic programming on GPU graphics cards. In: O’Neill M, Vanneschi L, Gustafson S, Esparcia Alcazar AI, De Falco I, Della Cioppa A, Tarantino E (eds) Proceedings of the 11th European conference on genetic programming, EuroGP 2008, Naples, 26–28 March, Lecture Notes in Computer Science, 4971, pp 73–85. Springer, Berlin
Langdon WB, Harrison AP (2008) GP on SPMD parallel graphics hardware for mega bioinformatics data mining. Soft Comput 12:1169–1183 (Special Issue on Distributed Bioinspired Algorithms)
Langdon WB (2009) A CUDA SIMT interpreter for genetic programming. Technical report TR-09-05. Department of Computer Science, King’s College London, Strand, 18 June 2009
Langdon WB, Harrison AP (2009) Evolving DNA motifs to predict GeneChip probe performance. Algorithms in molecular biology, vol 4
Langdon WB (2010a) Large scale bioinformatics data mining with parallel genetic programming on graphics processing units. In: Fernandez de Vega F, Cantu-Paz E (eds) Parallel and distributed computational intelligence, Chap 5. Studies in computational intelligence, vol 279, pp 113–141. Springer, Berlin
Langdon WB (2010b) A many threaded CUDA interpreter for genetic programming. In: Esparcia-Alcazar AI, Ekart A, Silva S, Dignum S, Uyar AS (eds) Proceedings of the 13th European conference on genetic programming, EuroGP 2010, Istanbul, 7–9 April, LNCS, vol 6021, pp 146–158. Springer, Berlin
Langdon WB, Harman M (2010) Evolving a CUDA kernel from an nVidia template. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 2376–2383
Lewis TE, Magoulas GD (2009) Strategies to minimise the total run time of cyclic graph based genetic programming with GPUs. In: Raidl G, Rothlauf F, Squillero G, Drechsler R, Stuetzle T, Birattari M, Congdon CB, Middendorf M, Blum C, Cotta C, Bosman P, Grahl J, Knowles J, Corne D, Beyer HG, Stanley K, Miller JF, van Hemert J, Lenaerts T, Ebner M, Bacardit J, O’Neill M, Di Penta, M, Doerr B, Jansen T, Poli R, Alba E (eds) GECCO ’09: roceedings of the 11th annual conference on genetic and evolutionary computation. ACM, Montreal, 8–12 July, pp 1379–1386
Luong, The Van, Melab N, Talbi E-G (2010) Parallel hybrid evolutionary algorithms on GPU. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 2734–2741
Luo Z, Liu H, Wu X (2005) Artificial neural network computation on graphic process unit. In: International joint conference on neural networks, IJCNN ’05, Montreal, 31 July–4 August, pp 622–626
Lindblad F, Nordin P, Wolff K (2002) Evolving 3D model interpretation of images using graphics hardware. In: Fogel DB, El-Sharkawi MA, Yao X, Greenwood G, Iba H, Marrow P, Shackleton M (eds) Proceedings of the 2002 congress on evolutionary computation CEC2002. IEEE Press, 12–17 May, pp 225–230
Liu, Weiguo, Schmidt B, Voss G, Schroder A, Muller-Wittig W (2006) Bio-sequence database scanning on a GPU. In: 20th international parallel and distributed processing symposium, IPDPS 2006. IEEE Press, Rhodes, 25–29 April
Maitre O, Baumes LA, Lachiche N, Corma A, Collet P (2009) Coarse grain parallelization of evolutionary algorithms on GPGPU cards with EASEA. In: Raidl G, Rothlauf F, Squillero G, Drechsler R, Stuetzle T, Birattari M, Congdon CB, Middendorf M, Blum C, Cotta C, Bosman P, Grahl J, Knowles J, Corne D, Beyer HG, Stanley K, Miller JF, van Hemert J, Lenaerts T, Ebner M, Bacardit J, O’Neill M, Di Penta M, Doerr B, Jansen T, Poli R, Alba E (eds) GECCO ’09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, Montreal, Québec, Canada, 8-12 July, pp. 1403–1410. ACM
Maitre O, Querry S, Lachiche N, Collet P (2010) EASEA parallelization of tree-based genetic programming. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. IEEE, Barcelona, 18–23 July, pp 1997–2004
Manavski S, Valle G (2008) CUDA compatible GPU cards as efficient hardware accelerators for Smith-Waterman sequence alignment. BMC Bioinform 9:S10
Meyer-Spradow J, Loviscach J (2003) Evolutionary design of BRDFs. In: Chover M, Hagen H, Tost D (eds) Eurographics 2003 short paper proceedings, pp 301–306
Moore GE (1965) Cramming more components onto integrated circuits. Electronics 38:114–117
Mussi L, Cagnoni S, Daolio F (2009) GPU-based road sign detection using particle swarm optimization. In: Ninth international conference on intelligent systems design and applications, ISDA 2009, Pisa. IEEE Computer Society, November 30–December 2, pp 152–157
Munawar A, Wahib M, Munawar A, Wahib M (2009) Theoretical and empirical analysis of a GPU based parallel Bayesian optimization algorithm. In: International conference on parallel and distributed computing, applications and Technologies, 2009. IEEE, Higashi, 8–11 Dec, pp 457–462
Nageswaran JM, Dutt N, Krichmar JL, Nicolau A, Veidenbaum AV (2009) A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw 22:791–800
Nowotny T (2010) Parallel implementation of a spiking neuronal network model of unsupervised olfactory learning on NVidia CUDA. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3238–3245
Oh K-S, Jung K (2004) GPU implementation of neural networks. Pattern Recognit 37:1311–1314
Owens JD, Houston M, Luebke D, Green S, Stone JE, Phillips JC (2008) GPU computing. In: Proceedings of the IEEE, vol 96, pp 879–899 (Invited paper)
Openshaw S, Turton I (1994) Building new spatial interaction models using genetic programming. In: Fogarty TC (ed) Evolutionary Computing, AISB workshop, Leeds, UK, 11–13 April (unpublished)
Poli R, Langdon WB (1999) Sub-machine-code genetic programming. In: Spector L, Langdon WB, O’Reilly UM, Angeline PJ (eds) Advances in genetic programming 3, Chap 13, pp 301–323. MIT Press, Cambridge
Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming. Published via http://www.lulu.com and freely. Available at http://www.gp-field-guide.org.uk (with contributions by Koza JR)
Prabhu RD (2008) SOMGPU: an unsupervised pattern classifier on graphical processing unit. In: Wang J (ed) 2008 IEEE world congress on computational intelligence, Hong Kong, 1–6 June, pp 1011–1018
Ribeiro B, Lopes N, Silva C (2010) High-performance bankruptcy prediction model using graphics processing units. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence, Barcelona, 18–23 July, pp 2210–2216
Rieffel J, Saunders F, Nadimpalli S, Zhou, Harvey, Hassoun S, Rife J, Trimmer B (2009) Evolving soft robotic locomotion in PhysX. GECCO ’09: Proceedings of the 11th annual conference companion on genetic and evolutionary computation conference. ACM, Montreal, 8–12 July, pp 2499–2504
Robilliard D, Marion-Poty V, Fonlupt C (2008) Population parallel GP on the G80 GPU. In O’Neill M, Vanneschi L, Gustafson S, Esparcia Alcazar AI, De Falco I, Della Cioppa A, Tarantino E (eds) Proceedings of the 11th European conference on genetic programming, EuroGP 2008. Naples, 26–28 March, Lecture Notes in Computer Science, vol 4971, pp 98–109. Springer, Berlin
Robilliard D, Marion-Poty V, Fonlupt C (2009a) High performance genetic programming on GPU. In: Folino G, Krasnogor N, Mastroianni C, Zambonelli F (eds) Proceedings of the 2009 workshop on bio-inspired algorithms for distributed systems. ACM, Barcelona, June 15–19, pp 85–94 (paper invited for the FGCS special issue)
Robilliard D, Marion-Poty V, Fonlupt C (2009b) Genetic programming on graphics processing units. Genet Program Evol Mach 10:447–471 (Special issue on parallel and distributed evolutionary algorithms, part I)
Sinnott-Armstrong NA, Greene CS, Cancare F, Moore JH (2009) Accelerating epistasis analysis in human genetics with consumer graphics hardware. BMC Research Notes 2:147
Sinnott-Armstrong NA, Greene CS, Moore JH (2010) Fast genome-wide epistasis analysis using ant colony optimization for multifactor dimensionality reduction analysis on graphics processing units. In: Pelikan M and Branke J (eds) Genetic and evolutionary computation conference, GECCO 2010. ACM, Portland July 7–11, pp 215–216
Soca N, Blengio JL, Pedemonte M, Ezzatti P (2010) PUGACE, a cellular evolutionary algorithm framework on GPUs. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3891–3898
Taha TM, Yalamanchili P, Bhuiyan M, Jalasutram R, Chen C, Linderman R (2010) Neuromorphic algorithms on clusters of PlayStation 3s. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3040–3049
Tran J, Jordan D, Luebke D (2004) New challenges for cellular automata simulation on the GPU. ACM. Poster, Los Angeles
Tsutsui S, Fujimoto N (2010) An analytical study of GPU computation for solving QAPs by parallel evolutionary computation with independent run. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 889–889
Wirawan A, Kwoh C, Hieu N, Schmidt B (2008) CBESW: sequence alignment on the playstation 3. BMC Bioinform 9:377
Wilson G, Banzhaf W (2010) Deployment of parallel linear genetic programming using GPUs on PC and video game console platforms. Genet Program Evol Mach 11:147–184
Wong M-L, Wong T-T, and Fok K-L (2005) Parallel evolutionary algorithms on graphics processing unit. In: Corne D, Michalewicz Z, McKay B, Eiben G, Fogel D, Fonseca C, Greenwood G, Raidl G, Tan KC, Zalzala A (eds) Proceedings of the 2005 IEEE congress on evolutionary computation. Edinburgh, 2–5 Sept, pp 2286–2293
Wong ML and Cui G (2010) Data mining using parallel multi-objective evolutionary algorithms on graphics hardware. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 3815–3822
Yudanov D, Shaaban M, Melton R, Reznik L (2010) GPU-based implementation of real-time system for spiking neural networks. In: Sobrevilla P (ed) 2010 IEEE world congress on computational intelligence. Barcelona, 18–23 July, pp 2143–2150
Acknowledgments
Work started in Memorial University, Newfoundland (with Wolfgang Banzhaf) and continued at Essex University and King’s College, London prior to UCL. Some of it using pre-release hardware donated by NVIDIA. I would like to thank Gernot Ziegler of NVIDIA and Lidia Yamamoto. Funded by EPSRC grant (http://gow.epsrc.ac.uk/ViewGrant.aspx?GrantRef=EP/G060525/2) EP/G060525/2
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Langdon, W.B. Graphics processing units and genetic programming: an overview. Soft Comput 15, 1657–1669 (2011). https://doi.org/10.1007/s00500-011-0695-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-011-0695-2