An efficient storage and service method for multi-source merging meteorological big data in cloud environment

被引:6
|
作者
Yang, Ming [1 ,3 ]
He, Wenchun [2 ]
Zhang, Zhiqiang [2 ]
Xu, Yongjun [2 ]
Yang, Heping [2 ]
Chen, Yufeng [1 ]
Xu, Xiaolong [3 ,4 ,5 ,6 ]
机构
[1] Zhejiang Meteorol Informat Network Ctr, Hangzhou, Zhejiang, Peoples R China
[2] Natl Meteorol Informat Ctr, Beijing, Peoples R China
[3] Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing, Jiangsu, Peoples R China
[4] Nanjing Univ Informat Sci & Technol, Jiangsu Engn Ctr Network Monitoring, Nanjing, Jiangsu, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing, Jiangsu, Peoples R China
[6] Michigan State Univ, Dept Comp Sci & Engn, E Lansing, MI 48824 USA
关键词
Multi-source merging sensors data; Meteorological data storage; Meteorological data service; Distributed NoSQL; Semi/unstructured data; COMPUTATION OFFLOADING METHOD; PRIVACY PRESERVATION; NETWORKS; INTERNET;
D O I
10.1186/s13638-019-1576-0
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
With the development of the meteorological IoT (Internet of Things) and meteorological sensing network, the collected multi-source meteorological data have the characteristics of large amount of information, multidimensional and high accuracy. Cloud computing technology has been applied to the storage and service of meteorological big data. Although the constant evolution of big data storage technology is improving the storage and access of meteorological data, storage and service efficiency is still far from meeting multi-source big data requirements. Traditional methods have been used for the storage and service of meteorological data, and a number of problems still persist, such as a lack of unified storage structure, poor scalability, and poor service performance. In this study, an efficient storage and service method for multidimensional meteorological data is designed based on NoSQL big data storage technology and the multidimensional characteristics of meteorological data. In the process of data storage, multidimensional block compression technology and data structures are applied to store and transmit meteorological data. In service, heterogeneous NoSQL common components are designed to improve the heterogeneity of the NoSQL database. The results show that the proposed method has good storage transmission efficiency and versatility, and can effectively improve the efficiency of meteorological data storage and service in meteorological applications.
引用
收藏
页数:12
相关论文
共 50 条
  • [31] An innovative multi-source precipitation merging method with the identification of rain and no rain
    Li L.
    Wang Y.
    Tang G.
    Gao X.
    Wang L.
    Hu Q.
    Shuikexue Jinzhan/Advances in Water Science, 2022, 33 (05): : 780 - 793
  • [32] A Method for Spatiotemporally Merging Multi-Source Precipitation Based on Deep Learning
    Fang, Wei
    Qin, Hui
    Liu, Guanjun
    Yang, Xin
    Xu, Zhanxing
    Jia, Benjun
    Zhang, Qianyi
    REMOTE SENSING, 2023, 15 (17)
  • [33] A Multi-Source Data Aggregation and Multidimensional Analysis Model for Big Data
    Liu, Pan
    Chen, Lin
    4TH ANNUAL INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY AND APPLICATIONS (ITA 2017), 2017, 12
  • [34] Research on Medical Multi-Source Data Fusion Based on Big Data
    Hu S.
    Recent Advances in Computer Science and Communications, 2022, 15 (03) : 376 - 387
  • [35] Secure Storage as a Service in Multi-Cloud Environment
    Di Pietro, Riccardo
    Scarpa, Marco
    Giacobbe, Maurizio
    Puliafito, Antonio
    AD-HOC, MOBILE, AND WIRELESS NETWORKS, ADHOC-NOW 2017, 2017, 10517 : 328 - 341
  • [36] Research on the processing method of multi-source heterogeneous data in the intelligent agriculture cloud platform
    Gao, Weimin
    Zhong, Jiaming
    Liu, Yichen
    APPLIED MATHEMATICS AND NONLINEAR SCIENCES, 2022, 8 (01) : 2367 - 2376
  • [37] Construction of a multi-source heterogeneous hybrid platform for big data
    Wang, Ying
    Liu, Yiding
    Xia, Minna
    JOURNAL OF COMPUTATIONAL METHODS IN SCIENCES AND ENGINEERING, 2021, 21 (03) : 713 - 722
  • [38] Integrating multi-source big data to infer building functions
    Niu, Ning
    Liu, Xiaoping
    Jin, He
    Ye, Xinyue
    Liu, Yu
    Li, Xia
    Chen, Yimin
    Li, Shaoying
    INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE, 2017, 31 (09) : 1871 - 1890
  • [39] Blockchain-Based Cloud Data Auditing Scheme in Multi-Cloud Storage Service Environment
    Wang, Shang
    Yu, MeiJu
    Zhao, MingZhu
    Wang, HaoTian
    PROCEEDINGS OF THE 2024 27 TH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN, CSCWD 2024, 2024, : 669 - 674
  • [40] Uncovering the spatiotemporal impacts of built environment on traffic carbon emissions using multi-source big data
    Wu, Jishi
    Jia, Peng
    Feng, Tao
    Li, Haijiang
    Kuang, Haibo
    Zhang, Junyi
    LAND USE POLICY, 2023, 129