Industrial Inspection systems are an essential part of Industry 4.0. An automated inspection system can significantly improve product quality and reduce human labor while making their life easier. However, a deep learning-based camera inspection system requires a large amount of data to classify the defective products accurately. In this paper, a framework is proposed for an industrial inspection system with the help of deep learning. Additionally, A new dataset of hex-nut products is proposed containing 4000 images, i.e., 2000 defective and 2000 non-defective. Moreover, different CNN architectures, i.e., Custom CNN, Inception ResNet v2, Xception, ResNet 101 v2, ResNet 152 v2, are experimented with the concept of transfer learning on the new hex-nut dataset. Fine-tuning the CNN architectures is performed by freezing the last 14 layers, which provided the optimal architecture, i.e., Xception (last 14 layers trainable, excluding the fully connected layer). The proposed framework can efficiently separate the defective products from the non-defective products with 100% accuracy on the hex nut dataset. Furthermore, the proposed optimal Xception architecture has experimented on a publicly available casting material dataset which produced 99.72% accuracy, outperforming existing methods. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2022. Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.