Shallow Neural Networks beat Deep Neural Networks trained with transfer learning: A Use Case based on training Neural Networks to identify Covid-19 in chest X-ray images

Since the start of the covid-19 health crisis, there have been many studies on the application of deep learning models in order to detect the virus on chest X-ray images. Training large neural networks on big data sets is a computationally intensive task, consuming a lot of power and needing a lot of time. Thus, usually only researchers in large institutions or companies have the necessary resources to bring the task to fruition. Other researchers employ transfer learning, a technique that is based on using pre-trained deep neural networks that have been trained on a similar dataset and retrain only their last neuron layers. However, using deep neural networks with transfer learning is not always the best option; in some cases, training a shallow neural network from scratch achieves better results. In this paper we compare training from scratch, shallow neural networks to transfer learning from deep neural models. Our experiments have been conducted on a publicly available dataset containing chest X-ray images concerning covid-19 patients, as well as non-covid-19 ones. Surprisingly enough, training from scratch shallow neural networks produced significantly better results in terms of both specificity and sensitivity. The results of the models' evaluation showed that the three shallow neural networks achieved specificity rates higher than 98%, while having a sensitivity rate of 98%, exceeding the best performing pre-trained model, the DenseNet121, which achieved a specificity rate of 91.3%, while having a sensitivity rate of 98%.