Variational autoencoder-based outlier detection for high-dimensional data

被引:9
|
作者
Li, Yongmou [1 ,2 ]
Wang, Yijie [1 ,2 ]
Ma, Xingkong [2 ]
机构
[1] Natl Univ Def Technol, Natl Lab Parallel & Distributed Proc, Changsha 410073, Hunan, Peoples R China
[2] Natl Univ Def Technol, Coll Comp, Changsha 410073, Hunan, Peoples R China
基金
国家教育部科学基金资助; 中国国家自然科学基金;
关键词
Variational autoencoders; outlier detection; high-dimensional data;
D O I
10.3233/IDA-184240
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Analysis of high-dimensional data often suffers from the curse of dimensionality and the complicated correlation among dimensions. Dimension reduction methods often are used to alleviate these problems. Existing outlier detection methods based on dimension reduction usually only rely on reconstruction error to detect outlier or apply conventional outlier detection methods to the reduced data, which could deteriorate the performance of outlier detection as only considering part of the information from data. Few studies have been done to combine these two strategies to do outlier detection. In this paper, we proposed an outlier detection method based on Variational Autoencoder (VAE), which combines low-dimensional representation and reconstruction error to detect outliers. Specifically, we first model the data use VAE, then extract four outlier scores from VAE model, finally propose an ensemble method to combine the four outlier scores. The experiments conducted on six real-world datasets show that the proposed method performs better than or at least comparable to state of the art methods.
引用
收藏
页码:991 / 1002
页数:12
相关论文
共 50 条
  • [21] High-dimensional data stream outlier detection algorithm based on angle distribution
    Lu, S. (lusheng@cqupt.edu.cn), 1600, Shanghai Jiaotong University (48):
  • [22] ROBOUT: a conditional outlier detection methodology for high-dimensional data
    Farne, Matteo
    Vouldis, Angelos
    STATISTICAL PAPERS, 2024, 65 (04) : 2489 - 2525
  • [23] A Method for Measurement Data Modeling and High-Dimensional Outlier Detection Based on Large Dimensional Matrix
    Chen, Gang
    Fan, Huanhuan
    An, Baoran
    PROCEEDINGS OF THE 33RD CHINESE CONTROL AND DECISION CONFERENCE (CCDC 2021), 2021, : 2274 - 2279
  • [24] Variational autoencoder-based anomaly detection in time series data for inventory record inaccuracy
    Argun, Halil
    Alptekin, S. Emre
    TURKISH JOURNAL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCES, 2023, 31 (01) : 163 - 179
  • [25] Autoencoder-based Data Augmentation for Deepfake Detection
    Stanciu, Dan-Cristian
    Ionescu, Bogdan
    PROCEEDINGS OF THE 2ND ACM INTERNATIONAL WORKSHOP ON MULTIMEDIA AI AGAINST DISCRIMINATION, MAD 2023, 2023, : 19 - 27
  • [26] Variational AutoEncoder-Based Anomaly Detection Scheme for Load Forecasting
    Park, Sungwoo
    Jung, Seungmin
    Hwang, Eenjun
    Rho, Seungmin
    ADVANCES IN ARTIFICIAL INTELLIGENCE AND APPLIED COGNITIVE COMPUTING, 2021, : 833 - 839
  • [27] VAGA: Towards Accurate and Interpretable Outlier Detection Based on Variational Auto-Encoder and Genetic Algorithm for High-Dimensional Data
    Li, Jiamu
    Zhang, Ji
    Wang, Jian
    Zhu, Youwen
    Bah, Mohamed Jaward
    Yang, Gaoming
    Gan, Yuquan
    2021 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2021, : 5956 - 5958
  • [28] A Novel Density-Based Clustering Approach for Outlier Detection in High-Dimensional Data
    Messaoud, Thouraya Aouled
    Smiti, Abir
    Louati, Aymen
    HYBRID ARTIFICIAL INTELLIGENT SYSTEMS, HAIS 2019, 2019, 11734 : 322 - 331
  • [29] On eigenfunction approach to data mining: outlier detection in high-dimensional data sets
    Nagar, AK
    Muyeba, MK
    8TH WORLD MULTI-CONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL II, PROCEEDINGS: COMPUTING TECHNIQUES, 2004, : 251 - 256
  • [30] Sparse signal shrinkage and outlier detection in high-dimensional quantile regression with variational Bayes
    Lim, Daeyoung
    Park, Beomjo
    Nott, David
    Wang, Xueou
    Choi, Taeryon
    STATISTICS AND ITS INTERFACE, 2020, 13 (02) : 237 - 249