Deep autoregressive models with spectral attention

被引:10
|
作者
Moreno-Pino, Fernando [1 ]
Olmos, Pablo M. [1 ]
Artes-Rodriguez, Antonio [1 ]
机构
[1] Univ Carlos III Madrid, Dept Signal Theory & Commun, Madrid, Spain
基金
欧洲研究理事会;
关键词
Attention models; Deep learning; Filtering; Global -local contexts; Signal processing; Spectral domain attention; Time series forecasting; TIME; TRANSFORMER; NETWORKS;
D O I
10.1016/j.patcog.2022.109014
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Time series forecasting is an important problem across many domains, playing a crucial role in multiple real-world applications. In this paper, we propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module, which merges global and local frequency domain information in the model's embedded space. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise. The proposed architecture has a number of useful properties: it can be effectively incorporated into well-known forecast architectures, requiring a low number of parameters and producing explainable results that improve forecasting accuracy. We test the Spectral Attention Autoregressive Model (SAAM) on several well-known forecast datasets, consistently demonstrating that our model compares favorably to state-of-the-art approaches. (c) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页数:12
相关论文
共 50 条
  • [21] Variable selection of nonparametric spatial autoregressive models via deep learning
    Zhang, Xiaodi
    Song, Yunquan
    SPATIAL STATISTICS, 2024, 64
  • [22] The Removal of Spurious Spectral Peaks From Autoregressive Models for Irregularly Sampled Data
    Broersen, Piet M. T.
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2010, 59 (01) : 205 - 214
  • [23] SELECTION OF THE ORDER OF AUTOREGRESSIVE MODELS FOR SPECTRAL-ANALYSIS OF DOPPLER ULTRASOUND SIGNALS
    SCHLINDWEIN, FS
    EVANS, DH
    ULTRASOUND IN MEDICINE AND BIOLOGY, 1990, 16 (01): : 81 - 91
  • [24] LOSS OF SPECTRAL PEAKS IN AUTOREGRESSIVE SPECTRAL ESTIMATION
    LYSNE, D
    TJOSTHEIM, D
    BIOMETRIKA, 1987, 74 (01) : 200 - 206
  • [25] Deep AutoRegressive Networks
    Gregor, Karol
    Danihelka, Ivo
    Mnih, Andriy
    Blundell, Charles
    Wierstra, Daan
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 32 (CYCLE 2), 2014, 32 : 1242 - 1250
  • [26] Bayesian autoregressive spectral estimation
    Cuevas, Alejandro
    Lopez, Sebastian
    Mandic, Danilo
    Tobar, Felipe
    2021 IEEE LATIN AMERICAN CONFERENCE ON COMPUTATIONAL INTELLIGENCE (LA-CCI), 2021,
  • [27] BIAS OF AUTOREGRESSIVE SPECTRAL ESTIMATORS
    STINE, RA
    SHAMAN, P
    JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 1990, 85 (412) : 1091 - 1098
  • [28] Autoregressive spatial spectral estimates
    Gupta, Abhimanyu
    JOURNAL OF ECONOMETRICS, 2018, 203 (01) : 80 - 95
  • [29] CONSISTENT AUTOREGRESSIVE SPECTRAL ESTIMATES
    BERK, KN
    ANNALS OF STATISTICS, 1974, 2 (03): : 489 - 502
  • [30] AN OPTIMAL AUTOREGRESSIVE SPECTRAL ESTIMATE
    SHIBATA, R
    ANNALS OF STATISTICS, 1981, 9 (02): : 300 - 306