Rule Extraction in Trained Feedforward Deep Neural Networks - Integrating Cosine Similarity and Logic for Explainability

Abstract

Explainability is a fundamental aspect in the field of machine learning, particularly in ensuring transparency and trust in decision-making processes. As the complexity of machine learning models increases, the integration of neural and symbolic approaches has emerged as a promising solution to the explainability problem. In this context, the utilization of search methods for rule extraction in trained deep neural networks has been proven effective. This involves the examination of weight and bias values generated by the network, typically through calculating the correlation between weight vectors and outputs. The hypothesis developed in this article states that by incorporating cosine similarity in this process, the search space can be efficiently narrowed down to identify the critical path connecting inputs to results. Furthermore, to provide a more comprehensive and interpretable understanding of the decision making process, this article proposes the integration of first-order logic (FOL) in the rule extraction process. By leveraging cosine similarity and FOL, a groundbreaking algorithm that is capable of extracting and explaining the rule patterns learned by a feedforward trained neural network was designed and implemented. The algorithm was tested in three use cases showing effectiveness in providing insights into the model’s behavior.