An efficient storage and service method for multi-source merging meteorological big data in cloud environment

被引:6
|
作者
Yang, Ming [1 ,3 ]
He, Wenchun [2 ]
Zhang, Zhiqiang [2 ]
Xu, Yongjun [2 ]
Yang, Heping [2 ]
Chen, Yufeng [1 ]
Xu, Xiaolong [3 ,4 ,5 ,6 ]
机构
[1] Zhejiang Meteorol Informat Network Ctr, Hangzhou, Zhejiang, Peoples R China
[2] Natl Meteorol Informat Ctr, Beijing, Peoples R China
[3] Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing, Jiangsu, Peoples R China
[4] Nanjing Univ Informat Sci & Technol, Jiangsu Engn Ctr Network Monitoring, Nanjing, Jiangsu, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing, Jiangsu, Peoples R China
[6] Michigan State Univ, Dept Comp Sci & Engn, E Lansing, MI 48824 USA
关键词
Multi-source merging sensors data; Meteorological data storage; Meteorological data service; Distributed NoSQL; Semi/unstructured data; COMPUTATION OFFLOADING METHOD; PRIVACY PRESERVATION; NETWORKS; INTERNET;
D O I
10.1186/s13638-019-1576-0
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
With the development of the meteorological IoT (Internet of Things) and meteorological sensing network, the collected multi-source meteorological data have the characteristics of large amount of information, multidimensional and high accuracy. Cloud computing technology has been applied to the storage and service of meteorological big data. Although the constant evolution of big data storage technology is improving the storage and access of meteorological data, storage and service efficiency is still far from meeting multi-source big data requirements. Traditional methods have been used for the storage and service of meteorological data, and a number of problems still persist, such as a lack of unified storage structure, poor scalability, and poor service performance. In this study, an efficient storage and service method for multidimensional meteorological data is designed based on NoSQL big data storage technology and the multidimensional characteristics of meteorological data. In the process of data storage, multidimensional block compression technology and data structures are applied to store and transmit meteorological data. In service, heterogeneous NoSQL common components are designed to improve the heterogeneity of the NoSQL database. The results show that the proposed method has good storage transmission efficiency and versatility, and can effectively improve the efficiency of meteorological data storage and service in meteorological applications.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] An efficient storage and service method for multi-source merging meteorological big data in cloud environment
    Ming Yang
    Wenchun He
    Zhiqiang Zhang
    Yongjun Xu
    Heping Yang
    Yufeng Chen
    Xiaolong Xu
    EURASIP Journal on Wireless Communications and Networking, 2019
  • [2] An Efficient Storage Service Method for Multidimensional Meteorological Data in Cloud Environment
    Yang, Ming
    He, Wenchun
    Zhang, Zhiqiang
    Xu, Yongjun
    Chen, Yufeng
    Xu, Xiaolong
    2019 INTERNATIONAL CONFERENCE ON INTERNET OF THINGS (ITHINGS) AND IEEE GREEN COMPUTING AND COMMUNICATIONS (GREENCOM) AND IEEE CYBER, PHYSICAL AND SOCIAL COMPUTING (CPSCOM) AND IEEE SMART DATA (SMARTDATA), 2019, : 495 - 500
  • [3] Multi-source Integration and Storage Optimization Method for Big Data of Power Distribution and Utilization
    Wang L.
    Zhao T.
    Zhang Y.
    Su Y.
    Tian S.
    Gaodianya Jishu/High Voltage Engineering, 2018, 44 (04): : 1131 - 1139
  • [4] Classification and Storage Method of Marine Multi-source Transmission Data under Cloud Computing
    Xu, Hongguo
    JOURNAL OF COASTAL RESEARCH, 2020, : 84 - 86
  • [5] Multi-source remote sensing image big data classification system design in cloud computing environment
    Tong X.-Y.
    Guo C.
    Cheng H.
    International Journal of Internet Manufacturing and Services, 2020, 7 (1-2) : 130 - 145
  • [6] Research on Distributed Storage and Query Optimization of Multi-source Heterogeneous Meteorological Data
    Hu, Xiaodong
    Xu, Huanli
    Jia, Jinfang
    Wang, Xiaoying
    PROCEEDINGS OF 2018 INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTERNET OF THINGS (CCIOT 2018), 2018, : 12 - 18
  • [7] Finding Optimal Meteorological Observation Locations by Multi-Source Urban Big Data Analysis
    Liu, Tianlei
    Zhao, Guoshuai
    Wang, Huan
    Hou, Xingsong
    Qian, Xueming
    Hou, Tao
    2016 7TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA (CCBD), 2016, : 175 - 180
  • [8] Collaborative Adaptive Scheduling Scheme for Multi-Source Big Data Tasks in the Cloud
    Zheng, Lizi
    Cui, Delong
    2019 INTERNATIONAL CONFERENCE ON IMAGE AND VIDEO PROCESSING, AND ARTIFICIAL INTELLIGENCE, 2019, 11321
  • [9] A Simple Method of Coupled Merging and Downscaling for Multi-Source Daily Precipitation Data
    Zhao, Na
    Chen, Kainan
    REMOTE SENSING, 2023, 15 (18)
  • [10] Efficient Placement of Meteorological Big Data Using NSGA-III in Cloud Environment
    Huang, Tao
    Ruan, Feng
    Xue, Shengjun
    Dai, Ranran
    Yang, Qin
    2019 INTERNATIONAL CONFERENCE ON INTERNET OF THINGS (ITHINGS) AND IEEE GREEN COMPUTING AND COMMUNICATIONS (GREENCOM) AND IEEE CYBER, PHYSICAL AND SOCIAL COMPUTING (CPSCOM) AND IEEE SMART DATA (SMARTDATA), 2019, : 569 - 574