Skip to main content
SpringerLink
Log in

Fuzzy Pattern Trees for Classification Problems Using Genetic Programming

  • Conference paper
  • First Online:
Genetic Programming (EuroGP 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14631))

Included in the following conference series:

  • 147 Accesses

Abstract

Fuzzy Pattern Trees (FPTs) are tree-based structures in which the internal nodes are fuzzy operators, and the leaves are fuzzy features. This work uses Genetic Programming (GP) to evolve FPTs and assesses their performance on 20 benchmark classification problems. The results show improved accuracy for most of the problems in comparison with previous works using different approaches. Furthermore, we experiment using Lexicase Selection with FPTs and demonstrate that selection methods based on aggregate fitness, such as Tournament Selection, produce more accurate models before analysing why this is the case. We also propose new parsimony pressure methods embedded in Lexicase Selection, and analyse their ability to reduce the size of the solutions. The results show that for most problems, at least one method could reduce the size significantly while keeping a similar accuracy. We also introduce a new fuzzification scheme for categorical features with too many categories by using target encoding followed by the same scheme for numerical features, which is straightforward to implement, and avoids a much higher increase in the number of fuzzy features.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    https://github.com/bdsul/grape/tree/main/GP/ClassificationFPT.

References

  1. Propublica data store. https://www.propublica.org/datastore/dataset/compas-recidivism-risk-score-data-and-analysis

  2. Aenugu, S., Spector, L.: Lexicase selection in learning classifier systems. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 356–364 (2019). https://doi.org/10.1145/3321707.3321828

  3. Cordón, O.: A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: designing interpretable genetic fuzzy systems. Int. J. Approximate Reasoning 52(6), 894–913 (2011). https://doi.org/10.1016/j.ijar.2011.03.004

    Article  Google Scholar 

  4. de Lima, A., Carvalho, S., Dias, D.M., Naredo, E., Sullivan, J.P., Ryan, C.: Lexi2: lexicase selection with lexicographic parsimony pressure. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 929–937. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3512290.3528803

  5. de Lima, A.D., Lopes, A.J., do Amaral, J.L.M., de Melo, P.L.: Explainable machine learning methods and respiratory oscillometry for the diagnosis of respiratory abnormalities in sarcoidosis. BMC Med. Inform. Decis. Making 22(1), 274 (2022). https://doi.org/10.1186/s12911-022-02021-2

  6. Dos Santos, A.R., Amaral, J.L.M.: Synthesis of fuzzy pattern trees by cartesian genetic programming. Mathware Soft Comput.: Mag. Eur. Soc. Fuzzy Log. Technol. 22(1), 52–56 (2015)

    MathSciNet  Google Scholar 

  7. Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml

  8. Fortin, F.A., De Rainville, F.M., Gardner, M.A.G., Parizeau, M., Gagné, C.: DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13(1), 2171–2175 (2012)

    MathSciNet  Google Scholar 

  9. Gandomi, A.H., Alavi, A.H.: A new multi-gene genetic programming approach to nonlinear system modeling. Part I: materials and structural engineering problems. Neural Comput. Appl. 21(1), 171–187 (2012). https://doi.org/10.1007/s00521-011-0734-z

    Article  Google Scholar 

  10. Helmuth, T., Lengler, J., La Cava, W.: Population diversity leads to short running times of lexicase selection. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tušar, T. (eds.) PPSN 2022. LNCS, vol. 13399, pp. 485–498. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14721-0_34

    Chapter  Google Scholar 

  11. Helmuth, T., McPhee, N.F., Spector, L.: Lexicase selection for program synthesis: a diversity analysis. In: Riolo, R., Worzel, B., Kotanchek, M., Kordon, A. (eds.) Genetic Programming Theory and Practice XIII. GEC, pp. 151–167. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-34223-8_9

    Chapter  Google Scholar 

  12. Helmuth, T., Pantridge, E., Spector, L.: On the importance of specialists for lexicase selection. Genet. Program Evolvable Mach. 21(3), 349–373 (2020). https://doi.org/10.1007/s10710-020-09377-2

    Article  Google Scholar 

  13. Helmuth, T., Spector, L., Matheson, J.: Solving uncompromising problems with lexicase selection. IEEE Trans. Evol. Comput. 19(5), 630–643 (2015). https://doi.org/10.1109/TEVC.2014.2362729

    Article  Google Scholar 

  14. Hernandez, J.G., Lalejini, A., Ofria, C.: An exploration of exploration: measuring the ability of lexicase selection to find obscure pathways to optimality. In: Banzhaf, W., Trujillo, L., Winkler, S., Worzel, B. (eds.) Genetic Programming Theory and Practice XVIII. Genetic and Evolutionary Computation, pp. 83–107. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-8113-4_5

    Chapter  Google Scholar 

  15. Huang, Z., Gedeon, T.D., Nikravesh, M.: Pattern trees induction: a new machine learning method. IEEE Trans. Fuzzy Syst. 16(4), 958–970 (2008). https://doi.org/10.1109/TFUZZ.2008.924348

    Article  Google Scholar 

  16. Jackson, D.: Promoting phenotypic diversity in genetic programming. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN 2010. LNCS, vol. 6239, pp. 472–481. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15871-1_48

    Chapter  Google Scholar 

  17. Kooperberg, C.: StatLib: an archive for statistical software, datasets, and information. Am. Stat. 51(1), 98 (1997)

    Article  Google Scholar 

  18. Koshiyama, A.S., Vellasco, M.M.B.R., Tanscheit, R.: GPFIS-CLASS: a genetic fuzzy system based on genetic programming for classification problems. Appl. Soft Comput. 37, 561–571 (2015). https://doi.org/10.1016/j.asoc.2015.08.055

    Article  Google Scholar 

  19. La Cava, W., Spector, L., Danai, K.: Epsilon-lexicase selection for regression. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO 2016, pp. 741–748. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2908812.2908898

  20. Luke, S., Panait, L.: Lexicographic parsimony pressure. In: Proceedings of the 4th Annual Conference on Genetic and Evolutionary Computation, GECCO 2002, pp. 829–836. Morgan Kaufmann Publishers Inc., San Francisco (2002)

    Google Scholar 

  21. Mei, Y., Chen, Q., Lensen, A., Xue, B., Zhang, M.: Explainable artificial intelligence by genetic programming: a survey. IEEE Trans. Evol. Comput. 27(3), 621–641 (2023). https://doi.org/10.1109/TEVC.2022.3225509

    Article  Google Scholar 

  22. Miller, J.F.: Cartesian genetic programming. In: Miller, J.F. (ed.) Cartesian Genetic Programming. Natural Computing Series, pp. 17–34. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-17310-3_2

    Chapter  Google Scholar 

  23. Montana, D.J.: Strongly typed genetic programming. Evol. Comput. 3(2), 199–230 (1995). https://doi.org/10.1162/evco.1995.3.2.199

    Article  Google Scholar 

  24. Moore, J.M., Stanton, A.: Lexicase selection outperforms previous strategies for incremental evolution of virtual creature controllers. In: ECAL 2017, the Fourteenth European Conference on Artificial Life, pp. 290–297. MIT Press (2017). https://doi.org/10.1162/isal_a_050

  25. Murphy, A., Ali, M., Dias, D., Amaral, J., Naredo, E., Ryan, C.: Grammar-based fuzzy pattern trees for classification problems. In: Proceedings of the 12th International Joint Conference on Computational Intelligence, pp. 71–80. SCITEPRESS - Science and Technology Publications, Budapest (2020). https://doi.org/10.5220/0010111900710080

  26. Murphy, A., Ali, M.S., Mota Dias, D., Amaral, J., Naredo, E., Ryan, C.: Fuzzy pattern tree evolution using grammatical evolution. SN Comput. Sci. 3(6), 426 (2022). https://doi.org/10.1007/s42979-022-01258-y

    Article  Google Scholar 

  27. Murphy, A., Murphy, G., Amaral, J., MotaDias, D., Naredo, E., Ryan, C.: Towards incorporating human knowledge in fuzzy pattern tree evolution. In: Hu, T., Lourenço, N., Medvet, E. (eds.) EuroGP 2021. LNCS, vol. 12691, pp. 66–81. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72812-0_5

    Chapter  Google Scholar 

  28. Murphy, A., Murphy, G., Dias, D.M., Amaral, J., Naredo, E., Ryan, C.: Human in the loop fuzzy pattern tree evolution. SN Comput. Sci. 3(2), 163 (2022). https://doi.org/10.1007/s42979-022-01044-w

    Article  Google Scholar 

  29. Rodrigues dos Santos, A., Machado do Amaral, J.L., Ribeiro Soares, C.A., Valladão de Barros, A.: Multi-objective fuzzy pattern trees. In: 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6 (2018). https://doi.org/10.1109/FUZZ-IEEE.2018.8491689

  30. Ryan, C., Collins, J.J., Neill, M.O.: Grammatical evolution: evolving programs for an arbitrary language. In: Banzhaf, W., Poli, R., Schoenauer, M., Fogarty, T.C. (eds.) EuroGP 1998. LNCS, vol. 1391, pp. 83–96. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0055930

    Chapter  Google Scholar 

  31. Senge, R., Hüllermeier, E.: Pattern trees for regression and fuzzy systems modeling. In: International Conference on Fuzzy Systems, pp. 1–7 (2010). https://doi.org/10.1109/FUZZY.2010.5584231

  32. Senge, R., Hüllermeier, E.: Top-down induction of fuzzy pattern trees. IEEE Trans. Fuzzy Syst. 19(2), 241–252 (2011). https://doi.org/10.1109/TFUZZ.2010.2093532

    Article  Google Scholar 

  33. Shaker, A., Senge, R., Hüllermeier, E.: Evolving fuzzy pattern trees for binary classification on data streams. Inf. Sci. 220, 34–45 (2013). https://doi.org/10.1016/j.ins.2012.02.034

    Article  Google Scholar 

  34. Sobania, D., Rothlauf, F.: Program synthesis with genetic programming: the influence of batch sizes. In: Medvet, E., Pappa, G., Xue, B. (eds.) EuroGP 2022. LNCS, vol. 13223, pp. 118–129. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-02056-8_8

    Chapter  Google Scholar 

  35. Spector, L.: Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. In: Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO 2012, pp. 401–408. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2330784.2330846

  36. White, D.R., et al.: Better GP benchmarks: community survey results and proposals. Genet. Program Evolvable Mach. 14(1), 3–29 (2013). https://doi.org/10.1007/s10710-012-9177-2

    Article  Google Scholar 

  37. Wilkinson, L., Anand, A., Tuan, D.N.: CHIRP: a new classifier based on composite hypercubes on iterated random projections. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 6–14. KDD 2011, Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2020408.2020418

  38. Yi, Y., Fober, T., Hüllermeier, E.: Fuzzy operator trees for modeling rating functions. Int. J. Comput. Intell. Appl. 08(04), 413–428 (2009). https://doi.org/10.1142/S1469026809002679

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Limerick, Limerick, Ireland

    Allan de Lima, Douglas Mota Dias & Conor Ryan

  2. Technological University of the Shannon: Midlands Midwest, Limerick, Ireland

    Samuel Carvalho & Joseph P. Sullivan

  3. Rio de Janeiro State University, Rio de Janeiro, Brazil

    Jorge Amaral

Authors
  1. Allan de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samuel Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Douglas Mota Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jorge Amaral
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joseph P. Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Conor Ryan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Allan de Lima or Conor Ryan .

Editor information

Editors and Affiliations

  1. University of Torino, Grugliasco, Italy

    Mario Giacobini

  2. Victoria University of Wellington, Wellington, New Zealand

    Bing Xue

  3. University of Trieste, Trieste, Italy

    Luca Manzoni

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

de Lima, A., Carvalho, S., Dias, D.M., Amaral, J., Sullivan, J.P., Ryan, C. (2024). Fuzzy Pattern Trees for Classification Problems Using Genetic Programming. In: Giacobini, M., Xue, B., Manzoni, L. (eds) Genetic Programming. EuroGP 2024. Lecture Notes in Computer Science, vol 14631. Springer, Cham. https://doi.org/10.1007/978-3-031-56957-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56957-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56956-2

  • Online ISBN: 978-3-031-56957-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation