A multi–modal unsupervised fault detection system based on power signals and thermal imaging via deep AutoEncoder neural network

In this paper a multi-modal unsupervised Deep Learning based algorithm for fault detection is proposed. Such method is applied to real data from a testing procedure implemented on an industrial production line. Both thermal images and current and power measurements coming from industrial refrigerators are collected. The considered dataset is highly unbalanced with the vast majority of samples being healthy. Thermal images are processed via a Deep Convolutional neural network. The features extracted from the thermal images are thus merged to structured data of power, current and temperature. Therefore, a Deep Auto-Encoder is trained on the dataset to signal anomalies corresponding to faults in the refrigerators. Three different methods are trained and compared: (1) an automatic method in which an expert extracts relevant features from thermal images without using the image recognition module; (2) a semi-automatic method where the convolutional neural network is applied to regions of interest within the thermal images selected by an expert operator; (3) a fully automatic method in which the Deep convolutional network processes the whole thermal image without any human intervention. The three methods show comparable results with nevertheless slight differences. © 2022 Elsevier Ltd