dSalmon: High-Speed Anomaly Detection for Evolving Multivariate Data Streams

被引:0
|
作者
Hartl, Alexander [1 ]
Iglesias, Felix [1 ]
Zseby, Tanja [1 ]
机构
[1] TU Wien Inst Telecommun, A-1040 Vienna, Austria
关键词
Outlier detection; Data streams; Unsupervised learning; !text type='Python']Python[!/text; C plus;
D O I
10.1007/978-3-031-48885-6_10
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
We introduce dSalmon, a highly efficient framework for outlier detection on streaming data. dSalmon can be used with both Python and C++, meeting the requirements of modern data science research. It provides an intuitive interface and has almost no package dependencies. dSalmon implements main stream outlier detection approaches from literature. By using pure C++ in its core and making the most of available parallelism, data is analyzed with superior processing speed. We describe design decisions and outline the software architecture of dSalmon. Additionally, we perform thorough evaluations on benchmarking datasets to measure execution time, memory requirements and energy consumption when performing outlier detection. Experiments show that dSalmon requires substantially less resources and in most cases is able to process datasets between one and three orders of magnitude faster than established Python implementations.
引用
收藏
页码:153 / 169
页数:17
相关论文
共 50 条
  • [1] Unsupervised Drift Detection on High-speed Data Streams
    Souza, Vinicius M. A.
    Chowdhury, Farhan A.
    Mueen, Abdullah
    2020 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2020, : 102 - 111
  • [2] Evolving Fuzzy Rules for Anomaly Detection in Data Streams
    Moshtaghi, Masud
    Bezdek, James C.
    Leckie, Christopher
    Karunasekera, Shanika
    Palaniswami, Marimuthu
    IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2015, 23 (03) : 688 - 700
  • [3] An adaptive algorithm for anomaly and novelty detection in evolving data streams
    Bouguelia, Mohamed-Rafik
    Nowaczyk, Slawomir
    Payberah, Amir H.
    DATA MINING AND KNOWLEDGE DISCOVERY, 2018, 32 (06) : 1597 - 1633
  • [4] Online Clustering for Evolving Data Streams with Online Anomaly Detection
    Chenaghlou, Milad
    Moshtaghi, Masud
    Leckie, Christopher
    Salehi, Mahsa
    ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PAKDD 2018, PT II, 2018, 10938 : 506 - 519
  • [5] An adaptive algorithm for anomaly and novelty detection in evolving data streams
    Mohamed-Rafik Bouguelia
    Slawomir Nowaczyk
    Amir H. Payberah
    Data Mining and Knowledge Discovery, 2018, 32 : 1597 - 1633
  • [6] Hybrid Big Data Architecture for High-Speed Log Anomaly Detection
    Nupairoj, Natawut
    Tangsatjatham, Pittayut
    JOURNAL OF INTERNET TECHNOLOGY, 2017, 18 (07): : 1681 - 1688
  • [7] Hybrid Big Data Architecture for High-Speed Log Anomaly Detection
    Tangsatjatham, Pittayut
    Nupairoj, Natawut
    2016 13TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER SCIENCE AND SOFTWARE ENGINEERING (JCSSE), 2016, : 538 - 543
  • [8] High-Speed Big Data Streams: A Literature Review
    Sneha, R. Patil
    Nagaraj, V. Dharwadkar
    SECOND INTERNATIONAL CONFERENCE ON COMPUTER NETWORKS AND COMMUNICATION TECHNOLOGIES, ICCNCT 2019, 2020, 44 : 308 - 316
  • [9] Efficient mining of high-speed uncertain data streams
    Han, Donghong
    Giraud-Carrier, Christophe
    Li, Shuoru
    APPLIED INTELLIGENCE, 2015, 43 (04) : 773 - 785
  • [10] Efficient mining of high-speed uncertain data streams
    Donghong Han
    Christophe Giraud-Carrier
    Shuoru Li
    Applied Intelligence, 2015, 43 : 773 - 785