Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm

被引:7
|
作者
Alamri, Nawaf Mohammad H. [1 ]
Packianather, Michael [1 ]
Bigot, Samuel [1 ]
机构
[1] Cardiff Univ, Sch Engn, Queens Bldg,14-17 Parade, Cardiff CF24 3AA, Wales
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 04期
关键词
Deep Learning (DL); Long Short-Term Memory (LSTM); Convolutional Neural Network (CNN); Bees Algorithm (BA); Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM); NEURAL-NETWORKS;
D O I
10.3390/app13042536
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.
引用
收藏
页数:27
相关论文
共 50 条
  • [21] Long Short Term Memory Networks for Short-Term Electric Load Forecasting
    Narayan, Apurva
    Hipel, Keith W.
    2017 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2017, : 2573 - 2578
  • [22] Practical Nonlinear Model Predictive Control Algorithm for Long Short-Term Memory Networks
    Schwedersky, Bernardo B.
    Flesch, Rodolfo C. C.
    Dangui, Hiago A. S.
    IFAC PAPERSONLINE, 2019, 52 (01): : 468 - 473
  • [23] Research on Long Short-Term Memory Networks Speech Separation Algorithm Based on Beamforming
    Lan Chaofeng
    Liu Yan
    Zhao Hongyun
    Liu Chundong
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2022, 44 (07) : 2531 - 2538
  • [24] Short-Term Load Forecasting using A Long Short-Term Memory Network
    Liu, Chang
    Jin, Zhijian
    Gu, Jie
    Qiu, Caiming
    2017 IEEE PES INNOVATIVE SMART GRID TECHNOLOGIES CONFERENCE EUROPE (ISGT-EUROPE), 2017,
  • [25] Optimizing Long Short-Term Memory Network for Air Pollution Prediction Using a Novel Binary Chimp Optimization Algorithm
    Baniasadi, Sahba
    Salehi, Reza
    Soltani, Sepehr
    Martin, Diego
    Pourmand, Parmida
    Ghafourian, Ehsan
    ELECTRONICS, 2023, 12 (18)
  • [26] Enhanced Gaze Tracking Using Convolutional Long Short-Term Memory Networks
    Vo, Minh-Thanh
    Kong, Seong G.
    INTERNATIONAL JOURNAL OF FUZZY LOGIC AND INTELLIGENT SYSTEMS, 2022, 22 (02) : 117 - 127
  • [27] Predictive process model monitoring using long short-term memory networks
    De Smedt, Johannes
    De Weerdt, Jochen
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2024, 133
  • [28] Mass Spectral Substance Detections Using Long Short-Term Memory Networks
    Liu, Junxiu
    Zhang, Jinlei
    Luo, Yuling
    Yang, Su
    Wang, Jinling
    Fu, Qiang
    IEEE ACCESS, 2019, 7 : 10734 - 10744
  • [29] Deflated reputation using multiplicative long short-term memory neural networks
    Ma, Yixuan
    Zhang, Zhenji
    Li, Deming
    Tang, Mincong
    FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2021, 118 : 198 - 207
  • [30] Dialog State Tracking Using Long Short-term Memory Neural Networks
    Yang, Xiaohao
    Liu, Jia
    16TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2015), VOLS 1-5, 2015, : 1800 - 1804