A deep learning model for multi-modal spatio-temporal irradiance forecast

Short-term spatio-temporal solar irradiance forecasting plays a pivotal role in scheduling and dispatching energy for distributed energy systems. Fluctuations in cloud cover can be monitored via satellite cloud imagery, which directly impacts irradiance. However, integrating and fusing multi-source heterogeneous data, such as satellite cloud images and ground monitoring data from distributed stations, remains challenging. Here, a spatio-temporal irradiance forecast model is proposed based on multi-modal deep learning model to predict global horizontal irradiance 30 min ahead. To address the feature extraction of heterogeneous data, a dual channel structure consisting of a time-series processing block and a satellite cloud image processing block is developed to enable parallel processing of multi-modal features. In order To tightly couple cloud images and historical time series at the feature level, maximum mean discrepancy of these two feature is used to help the fusion of heterogeneous data. Furthermore, a self-attention mechanism is employed to construct adaptive inter region information weights to enhance spatio-temporal representation ability. The evaluation of the method is conducted on open-access datasets from six locations in Jiangsu Province, China. Experimental results demonstrate that the proposed model efficiently utilizes heterogeneous data to improve prediction accuracy under various conditions and enhances model robustness, reducing RMSE by 2.8%-20.58%. Meanwhile, the proposed end-to-end model reduces training and deployment costs for real-world use.