Detection of application-layer DDoS attacks using machine learning and genetic algorithms

Application-layer Distributed Denial of Service (App-DDoS) attacks continue to be a pervasive problem in cybersecurity, despite the availability of various defensive frameworks. This research addresses the challenges associated with App-DDoS detection and presents a highly effective and adaptable solution for detecting various types of App-DDoS attacks. Motivated by the critical need for improved DDoS detection, our approach achieves dual objectives by accurately detecting both known and unknown DDoS attacks while minimizing false alarms. To achieve this, we combine Random Forest (RF), Gaussian Mixture Models (GMM) and a human with expertise in DDoS to enhance the system's resilience against evolving attack patterns. Furthermore, we prioritize high quality data curation by utilizing multiple datasets, CICIDS2017 and CICDDoS2019, and incorporating GMM to adapt effectively to varying data distributions over time. In addition, we propose a comprehensive feature selection strategy that addresses the false alarm rate and improves classifier performance by utilizing decision tree (DT) feature importance and the minimum redundancy maximum relevance (MRMR) approach. Moreover, we adopt genetic algorithms (GA) for automated hyper-parameter optimization to ensure efficient and effective DDoS detection. Quantitative analysis shows a significant reduction in false alarms to 0.12% (52 out of 45,149 samples), with the RF classifier achieving outstanding accuracy (99.9%), precision (100%), recall (99.8%), and F1 score (99.9%). Handling unknown App-DDoS attacks, our approach demonstrates remarkable performance across all datasets.