Vibration signal analysis assumes a critical role in the diagnosis of faults within helical gear transmissions, facilitating early detection and mitigation of potential failures. This paper is presenting an investigation into vibration analysis and the application of the Naive Bayes (NB) machine learning approach for diagnosing tooth wear faults in helical gear transmissions. The study encompasses a thorough literature review to underscore the significance of gear fault diagnosis while identifying limitations in prior research. To address these limitations, the proposed approach integrates sophisticated vibration analysis techniques, data enhancement methods, and a machine learning algorithm. Experimental tests were conducted on a fabricated helical gear transmission system, with operational states classified using the NB-based classification model. The obtained results demonstrate the efficacy of the proposed approach, with the NB model achieving an accuracy of 93.9%. The analysis of the confusion matrix and ROC analysis provides valuable insights into the classification performance, with an impressive area under the curve of 99.1%. The findings make a notable contribution to the field of gear fault diagnosis, offering an advanced and reliable approach for real-world applications. Future research endeavors may encompass the expansion of the dataset, exploration of alternative machine learning algorithms, and the incorporation of additional diagnostic techniques to further enhance fault diagnosis capabilities..