Benjamin P. Evans
ABSTRACT
Interpreting state-of-the-art machine learning algorithms can be difficult. For example, why does a complex ensemble predict a particular class? Existing approaches to interpretable machine learning tend to be either local in their explanations, apply only to a particular algorithm, or overly complex in their global explanations. In this work, we propose a global model extraction method which uses multi-objective genetic programming to construct accurate, simplistic and model-agnostic representations of complex black-box estimators. We found the resulting representations are far simpler than existing approaches while providing comparable reconstructive performance. This is demonstrated on a range of datasets, by approximating the knowledge of complex black-box models such as 200 layer neural networks and ensembles of 500 trees, with a single tree.
- Osbert Bastani, Carolyn Kim, and Hamsa Bastani. 2017. Interpretability via model extraction. arXiv preprint arXiv:1706.09773 (2017).Google Scholar
- Hans-Georg Beyer and Hans-Paul Schwefel. 2002. Evolution strategies-A comprehensive introduction. Natural computing 1, 1 (2002), 3--52. Google ScholarDigital Library
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems. 4349--4357. Google ScholarDigital Library
- Indranil Bose and Radha K Mahapatra. 2001. Business data mining - a machine learning perspective. Information & management 39, 3 (2001), 211--225.Google Scholar
- Cristian Bucilua, Rich Caruana, and Alexandru Niculescu-Mizil. 2006. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 535--541. Google ScholarDigital Library
- Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1721--1730. Google ScholarDigital Library
- Mark Craven and Jude W Shavlik. 1996. Extracting tree-structured representations of trained networks. In Advances in neural information processing systems. 24--30. Google ScholarDigital Library
- Hoa Khanh Dam, Truyen Tran, and Aditya Ghose. 2018. Explainable Software Analytics. CoRR abs/1802.00603 (2018). arXiv:1802.00603 http://arxiv.org/abs/1802.00603Google Scholar
- Hoa Khanh Dam, Truyen Tran, and Aditya Ghose. 2018. Explainable software analytics. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results. ACM, 53--56. Google ScholarDigital Library
- Kalyanmoy Deb. 2014. Multi-objective optimization. In Search methodologies. Springer, 403--449.Google Scholar
- Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2 (2002), 182--197. Google ScholarDigital Library
- Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research 7, Jan (2006), 1--30. Google ScholarDigital Library
- Dua Dheeru and Efi Karra Taniskidou. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/mlGoogle Scholar
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).Google Scholar
- Filip Karlo Došilović, Mario Brčić, and Nikica Hlupić. 2018. Explainable artificial intelligence: A survey. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE, 0210--0215.Google ScholarCross Ref
- Usama Fayyad and Keki Irani. 1993. Multi-interval discretization of continuous-valued attributes for classification learning. (1993).Google Scholar
- Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).Google Scholar
- Been Kim and Finale Doshi-Velez. {n. d.}. ICML 2017 tutorial on interpretable machine learning. http://people.csail.mit.edu/beenkim/icml_tutorial.htmlGoogle Scholar
- John R Koza. 1994. Genetic programming as a means for programming computers by natural selection. Statistics and computing 4, 2 (1994), 87--112.Google Scholar
- Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. 2015. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350--1371.Google ScholarCross Ref
- Thomas Loveard and Victor Ciesielski. 2001. Representing classification problems in genetic programming. In Evolutionary Computation, 2001. Proceedings of the 2001 Congress on, Vol. 2. IEEE, 1070--1077.Google ScholarCross Ref
- Tamas Madl. 2018. Sklearn interpretable tree. https://github.com/tmadl/sklearn-interpretable-treeGoogle Scholar
- Christoph Molnar. 2018. Interpretable machine learning. A Guide for Making Black Box Models Explainable (2018).Google Scholar
- David J Montana. 1995. Strongly typed genetic programming. Evolutionary computation 3, 2 (1995), 199--230. Google ScholarDigital Library
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830. Google ScholarDigital Library
- General Data Protection Regulation. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46. Official Journal of the European Union (OJ) 59, 1--88 (2016), 294.Google Scholar
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13--17, 2016. 1135--1144. Google ScholarDigital Library
- Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. 2017. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296 (2017).Google Scholar
- Juliet Popper Shaffer. 1995. Multiple hypothesis testing. Annual review of psychology 46, 1 (1995), 561--584.Google Scholar
- Latanya Sweeney. 2013. Discrimination in online ad delivery. Queue 11, 3 (2013), 10. Google ScholarDigital Library
- The H2O.ai team. 2015. h2o: Python Interface for H2O. http://www.h2o.ai Pythonpackage version 3.1.0.99999.Google Scholar
- John W Tukey. 1977. Exploratory data analysis. Vol. 2. Reading, Mass.Google Scholar
- Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, and Luis Torgo. 2013. OpenML: Networked Science in Machine Learning. SIGKDD Explorations 15, 2 (2013), 49--60. Google ScholarDigital Library
- Alfredo Vellido, José David Martín-Guerrero, and Paulo JG Lisboa. 2012. Making machine learning models interpretable.. In ESANN, Vol. 12. Citeseer, 163--172.Google Scholar
- Hongyu Yang, Cynthia Rudin, and Margo Seltzer. 2016. Scalable Bayesian rule lists. arXiv preprint arXiv:1602.08610 (2016).Google Scholar
- Matthew D Zeiler, Dilip Krishnan, Graham W Taylor, and Rob Fergus. 2010. Deconvolutional networks. (2010).Google Scholar
- Mengjie Zhang and Will Smart. 2004. Multiclass object classification using genetic programming. In Workshops on Applications of Evolutionary Computation. Springer, 369--378.Google ScholarCross Ref
Index Terms
- What's inside the black-box?: a genetic programming method for interpreting complex machine learning models
Recommendations
GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanations
AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent SystemsCounterfactual explanations are a common tool to explain artificial intelligence models. For Reinforcement Learning (RL) agents, they answer "Why not?" or "What if?" questions by illustrating what minimal change to a state is needed such that an agent ...
Counterfactual Explanations in Explainable AI: A Tutorial
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningDeep learning has shown powerful performances in many fields, however its black-box nature hinders its further applications. In response, explainable artificial intelligence emerges, aiming to explain the predictions and behaviors of deep learning ...
Benchmarking and survey of explanation methods for black box models
AbstractThe rise of sophisticated black-box machine learning models in Artificial Intelligence systems has prompted the need for explanation methods that reveal how these models work in an understandable way to users and decision makers. Unsurprisingly, ...
Comments