The nuclear power plant (NPP) plays a crucial role in providing clean energy, significantly contributing to mitigating global warming. However, this advantage is accompanied by public concerns regarding potential accidents. The imperative for a robust safety system has led to the exploration of artificial intelligence (AI) integration in NPP to minimize human error and enhance safety efficiency during reactor accidents. This research conducts a comparative study between two deep learning (DL) models to classify five accidents based on algorithm performance and transparency. These accidents include Loss of Coolant Accident (LOCA), Loss of On-site Power (LOOP), Loss of Flow Accident (LOFA), Steam Generator Tube Rupture (SGTR), and Feed Water Line Break (FWLB). The necessary data was collected using open data from Personal Computer Transient Analyzer (PCTRAN) software, subsequently separated into two categories: digital and numerical. Image data is utilized for training Convolutional Neural Network (CNN), while quantitative data is employed for training Artificial Neural Network (ANN). Additionally, four metric parameters (Accuracy, Recall, F1-score, Precision) are used to evaluate the model's performance. Finally, eXplainable Artificial Intelligence (XAI) techniques are employed through the use of the Shapley Additive Explanation (SHAP) package to provide the model's output explainability, The results show that within the two DL algorithms, CNN presents the best performance, demonstrating an excellent accuracy reaching 99.74 and requiring less time and a smaller dataset than ANN. Furthermore, the use of XAI explains the reasons behind the models' decision-making process, leading to the establishment of trust in the algorithms' results.