GEFWA: Gradient-Enhanced Fireworks Algorithm for Optimizing Convolutional Neural Networks

The efficacy of evolutionary and swarm intelligence-based black-box optimization algorithms in machine learning has increased their usage, but concerns have been raised about their low sample efficiency owing to their reliance on sampling. Consequently, improving the sample efficiency of conventional black-box optimization algorithms while retaining their strengths is crucial. To this end, we propose a new algorithm called Gradient Enhanced Fireworks Algorithm (GEFWA) that incorporates first-order gradient information into the population-based fireworks algorithm (FWA). We enhance the explosion operator with the gradient-enhanced explosion (GEE) and take advantage of attraction-based cooperation (ABC) for firework collaboration. Experimental results illustrate that GEFWA outperforms traditional first-order stochastic gradient descent-based optimization methods such as Adm and SGD when it comes to optimizing convolutional neural networks. These results demonstrate the potential of integrating gradient information into the FWA framework for addressing large-scale machine learning problems.