Deep autoregressive models with spectral attention

被引:10
|
作者
Moreno-Pino, Fernando [1 ]
Olmos, Pablo M. [1 ]
Artes-Rodriguez, Antonio [1 ]
机构
[1] Univ Carlos III Madrid, Dept Signal Theory & Commun, Madrid, Spain
基金
欧洲研究理事会;
关键词
Attention models; Deep learning; Filtering; Global -local contexts; Signal processing; Spectral domain attention; Time series forecasting; TIME; TRANSFORMER; NETWORKS;
D O I
10.1016/j.patcog.2022.109014
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Time series forecasting is an important problem across many domains, playing a crucial role in multiple real-world applications. In this paper, we propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module, which merges global and local frequency domain information in the model's embedded space. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise. The proposed architecture has a number of useful properties: it can be effectively incorporated into well-known forecast architectures, requiring a low number of parameters and producing explainable results that improve forecasting accuracy. We test the Spectral Attention Autoregressive Model (SAAM) on several well-known forecast datasets, consistently demonstrating that our model compares favorably to state-of-the-art approaches. (c) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页数:12
相关论文
共 50 条
  • [11] A COMPLETE SPECTRAL CHARACTERIZATION OF QUARTER-PLANE AUTOREGRESSIVE MODELS
    LAWTON, WM
    IEEE TRANSACTIONS ON ACOUSTICS SPEECH AND SIGNAL PROCESSING, 1985, 33 (06): : 1617 - 1619
  • [12] SPECTRAL ANALYSIS OF ELECTROMYOGRAM - FITTING OF AUTOREGRESSIVE MODELS TO INTERFERENCE EMG
    BRAUER, D
    KRAMER, H
    LUN, A
    KUCHLER, G
    ACTA BIOLOGICA ET MEDICA GERMANICA, 1975, 34 (05) : 805 - 810
  • [13] Attention, please! A survey of neural attention models in deep learning
    Correia, Alana de Santana
    Colombini, Esther Luna
    ARTIFICIAL INTELLIGENCE REVIEW, 2022, 55 (08) : 6037 - 6124
  • [14] Attention, please! A survey of neural attention models in deep learning
    Alana de Santana Correia
    Esther Luna Colombini
    Artificial Intelligence Review, 2022, 55 : 6037 - 6124
  • [15] Autoregressive models for spectral analysis of short tandem repeats in DNA sequences
    Hongxia Zhou
    Hong Yan
    2006 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS, VOLS 1-6, PROCEEDINGS, 2006, : 1286 - +
  • [16] AUTOREGRESSIVE SPECTRAL MODELS OF HEART-RATE-VARIABILITY - PRACTICAL ISSUES
    BURR, RL
    COWAN, MJ
    JOURNAL OF ELECTROCARDIOLOGY, 1993, 25 : 224 - 233
  • [17] Speech spectral modeling and enhancement based on autoregressive conditional heteroscedasticity models
    Cohen, I
    SIGNAL PROCESSING, 2006, 86 (04) : 698 - 709
  • [18] Mixture autoregressive and spectral attention network for multispectral image compression based on variational autoencoder
    Kong, Fanqiang
    Ren, Guanglong
    Hu, Yunfang
    Li, Dan
    Hu, Kedi
    VISUAL COMPUTER, 2024, 40 (09): : 6295 - 6318
  • [19] Video Summarization with LSTM and Deep Attention Models
    Casas, Luis Lebron
    Koblents, Eugenia
    MULTIMEDIA MODELING, MMM 2019, PT II, 2019, 11296 : 67 - 79
  • [20] Heart Rate Modeling and Prediction Using Autoregressive Models and Deep Learning
    Staffini, Alessio
    Svensson, Thomas
    Chung, Ung-il
    Svensson, Akiko Kishi
    SENSORS, 2022, 22 (01)