Application Research of Energy Data Acquisition and Analysis Based on Real-time Stream Processing Platform

被引：0
|
作者：
Li, Kunming [1 ]
Ji, Cong [1 ]
Zhong, Chunlin [1 ]
Zheng, Fei [1 ]
Shao, Jun [1 ]
机构：
[1] Jiangsu Frontier Elect Technol CO LTD, Smart Grid Prod Ctr, Nanjing, Peoples R China
来源：
PROCEEDINGS OF 2017 6TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND NETWORK TECHNOLOGY (ICCSNT 2017) | 2017年
关键词：
energy saving and emission reduction; real-time stream processing platform; data acquisition; Kafka; Storm;
D O I：
暂无
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In the context of national energy-saving emission reduction strategy, the user energy efficiency has become a hot topic in academia and the business community. In order to solve the problems of large amount of data and fast changing speed in real-time transmission terminal equipment, it has strict requirement for processing timeliness. We need to introduce distributed real-time data, high-speed synchronization, acquisition and processing and analysis technology to build a real-time flow processing platform. Real-time stream processing platform uses message queue (Kafka) to receive data from different real-time sources, and the back-end uses stream processing technology (Storm) to analyze real-time data.
引用
收藏
页码：175 / 178
页数：4
相关论文
共 50 条
  • [1] Research on Real-time Processing and Stream Analysis of Unstructured Data Based on Big Data Platforms
    Liang, Huichao
    Wang, Di
    Liu, Yuan
    Mei, Lin
    Zhou, Mengxue
    Zhao, Haibin
    PROCEEDINGS OF 2024 INTERNATIONAL CONFERENCE ON MACHINE INTELLIGENCE AND DIGITAL APPLICATIONS, MIDA2024, 2024, : 96 - 101
  • [2] SpeedStream: A Real-Time Stream Data Processing Platform in The Cloud
    Li Zhao
    Zhang Chuang
    Xu Ke-fu
    2015 IEEE 34TH INTERNATIONAL PERFORMANCE COMPUTING AND COMMUNICATIONS CONFERENCE (IPCCC), 2015,
  • [3] Real-time data acquisition and processing platform for fusion experiments
    Ruiz, M
    Barrera, E
    López, S
    Machón, D
    Vega, J
    Sánchez, E
    FUSION ENGINEERING AND DESIGN, 2004, 71 (1-4) : 135 - 140
  • [4] Cloud Computing Platform Based Real-Time Processing for Stream Reasoning
    Jung, Hae Sun
    Yoon, Chul Sang
    Lee, Yong Woo
    Park, Jong Won
    Yun, Chang Ho
    2017 SIXTH INTERNATIONAL CONFERENCE ON FUTURE GENERATION COMMUNICATION TECHNOLOGIES (FGCT), 2017, : 37 - 41
  • [5] Stream Processing For Near Real-Time Scientific Data Analysis
    Choi, Jong Youl
    Kurc, Tahsin
    Logan, Jeremy
    Wolf, Matthew
    Suchyta, Eric
    Kress, James
    Pugmire, David
    Podhorszki, Norbert
    Byun, Eun-Kyu
    Ainsworth, Mark
    Pwashar, Manish
    Klasky, Scott
    2016 NEW YORK SCIENTIFIC DATA SUMMIT (NYSDS), 2016,
  • [6] Near Real-Time Big Data Stream Processing Platform Using Cassandra
    Pal, Gautam
    Li, Gangmin
    Atkinson, Katie
    2018 4TH INTERNATIONAL CONFERENCE FOR CONVERGENCE IN TECHNOLOGY (I2CT), 2018,
  • [7] Real-time stream processing for Big Data
    Wingerath, Wolfram
    Gessert, Felix
    Friedrich, Steffen
    Ritter, Norbert
    IT-INFORMATION TECHNOLOGY, 2016, 58 (04): : 186 - 194
  • [8] PARALLEL PROCESSING AND REAL-TIME DATA ACQUISITION
    TAYLOR, S
    TAYLOR, R
    IEEE TRANSACTIONS ON NUCLEAR SCIENCE, 1990, 37 (02) : 355 - 360
  • [9] Real-Time Log Analysis Using Hitachi uCosminexus Stream Data Platform
    Hayashida, Yoshiyuki
    Ioki, Nobuhiro
    Arai, Naomi
    Nishizawa, Itaru
    DATABASE SYSTEMS FOR ADVANCED APPLICATIONS, PT II, PROCEEDINGS, 2010, 5982 : 440 - 443
  • [10] A Platform for Real-time Acquisition and Analysis of Physiological Data in Hospital Emergency Departments
    Smith, Jason B.
    Reisner, Andrew T.
    Edla, Shwetha
    Liu, Jianbo
    Liddle, Stephanie
    Reifman, Jaques
    2014 36TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC), 2014, : 2678 - 2681
12345