An evolutionary extreme learning machine algorithm for multi-cube unit single-layer neural networks

The low-order unit type is the most commonly used neuron architecture, which multiplies each input with a corresponding weight. These types have a restriction to linear separable problems, and overcoming this issue would require more advanced units. Higher-order units like the multi-cube neuron treat the input vector as a set of multi-dimensional hyper-cubes where each cube's site corresponds to a weight. Training multi-cube unit (MCU) single-layer neural networks (SLNNs) is possible with the use of the extreme learning machine (ELM) algorithm, which has a very fast training speed because it does not use an iterative process like gradient-based methods. The training procedure begins by randomizing the hidden layer weights and thresholds, and then, with the help of the Moore-Penrose pseudo-inverse, it can analytically calculate the output layer(s) weights. Earlier works with MCU SLNNs showed significantly increased generalization performance in classification and regression problems compared to traditional low-order neuron types. The proposed evolutionary higher- order ELM (EHO-ELM) algorithm utilizes a modified self-adaptive genetic algorithm (GA) to create an SLNN containing MCUs in its hidden and output layers. EHO-ELM can automatically determine the optimal number and structure of cubic sub-units for each MCU of the neural network. Also, it can automatically tune the hidden layer weights and thresholds to increase the constructed network's generalization ability. According to our knowledge, it is the first algorithm that can optimize the hidden weights vector and the sub-cube structure of MCUs at the same time. This paper's experimental work section compares EHO-ELM in 25 datasets with 14 existing machine-learning methods. The compared approaches include ten gradient-based methods, support vector machine (SVM), and three ELM-based methods. The experimental results revealed that the proposed method had the best generalization performance. The significance of these results was verified using the Wilcoxon sign-rank test.