Post-hoc Global Explanation using Hypersphere Sets

We propose a novel global explanation method for a pre-trained machine learning model. Generally, machine learning models behave as a black box. Therefore, developing a tool that reveals a model's behavior is important. Some studies have addressed this issue by approximating a black-box model with another interpretable model. Although such a model summarizes a complex model, it sometimes provides incorrect explanations because of a gap between the complex model. We define hypersphere sets of two types that respectively approximate a model based on recall and precision metrics. A high-recall set of hyperspheres provides a summary of a black-box model. A high-precision one describes the model's behavior precisely. We demonstrate from experimentation that the proposed method provides a global explanation for an arbitrary black-box model. Especially, it improves recall and precision metrics better than earlier methods.