An efficient storage and service method for multi-source merging meteorological big data in cloud environment

被引:6
|
作者
Yang, Ming [1 ,3 ]
He, Wenchun [2 ]
Zhang, Zhiqiang [2 ]
Xu, Yongjun [2 ]
Yang, Heping [2 ]
Chen, Yufeng [1 ]
Xu, Xiaolong [3 ,4 ,5 ,6 ]
机构
[1] Zhejiang Meteorol Informat Network Ctr, Hangzhou, Zhejiang, Peoples R China
[2] Natl Meteorol Informat Ctr, Beijing, Peoples R China
[3] Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing, Jiangsu, Peoples R China
[4] Nanjing Univ Informat Sci & Technol, Jiangsu Engn Ctr Network Monitoring, Nanjing, Jiangsu, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing, Jiangsu, Peoples R China
[6] Michigan State Univ, Dept Comp Sci & Engn, E Lansing, MI 48824 USA
关键词
Multi-source merging sensors data; Meteorological data storage; Meteorological data service; Distributed NoSQL; Semi/unstructured data; COMPUTATION OFFLOADING METHOD; PRIVACY PRESERVATION; NETWORKS; INTERNET;
D O I
10.1186/s13638-019-1576-0
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
With the development of the meteorological IoT (Internet of Things) and meteorological sensing network, the collected multi-source meteorological data have the characteristics of large amount of information, multidimensional and high accuracy. Cloud computing technology has been applied to the storage and service of meteorological big data. Although the constant evolution of big data storage technology is improving the storage and access of meteorological data, storage and service efficiency is still far from meeting multi-source big data requirements. Traditional methods have been used for the storage and service of meteorological data, and a number of problems still persist, such as a lack of unified storage structure, poor scalability, and poor service performance. In this study, an efficient storage and service method for multidimensional meteorological data is designed based on NoSQL big data storage technology and the multidimensional characteristics of meteorological data. In the process of data storage, multidimensional block compression technology and data structures are applied to store and transmit meteorological data. In service, heterogeneous NoSQL common components are designed to improve the heterogeneity of the NoSQL database. The results show that the proposed method has good storage transmission efficiency and versatility, and can effectively improve the efficiency of meteorological data storage and service in meteorological applications.
引用
收藏
页数:12
相关论文
共 50 条
  • [21] A Multi-Scale Method for PM2.5 Forecasting with Multi-Source Big Data
    Wenyan Yuan
    Hongchuan Du
    Jieyi Li
    Ling Li
    Journal of Systems Science and Complexity, 2023, 36 : 771 - 797
  • [22] Big Data Storage in the Cloud for Smart Environment Monitoring
    Fazio, M.
    Celesti, A.
    Puliafito, A.
    Villari, M.
    6TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT-2015), THE 5TH INTERNATIONAL CONFERENCE ON SUSTAINABLE ENERGY INFORMATION TECHNOLOGY (SEIT-2015), 2015, 52 : 500 - 506
  • [23] Hybrid encryption framework for securing big data storage in multi-cloud environment
    Viswanath, G.
    Krishna, P. Venkata
    EVOLUTIONARY INTELLIGENCE, 2021, 14 (02) : 691 - 698
  • [24] Hybrid encryption framework for securing big data storage in multi-cloud environment
    G. Viswanath
    P. Venkata Krishna
    Evolutionary Intelligence, 2021, 14 : 691 - 698
  • [25] Efficient Multimedia Data Storage in Cloud Environment
    Deshpande, Prachi
    Sharma, S. C.
    Peddoju, Sateesh K.
    Abraham, Ajith
    INFORMATICA-JOURNAL OF COMPUTING AND INFORMATICS, 2015, 39 (04): : 431 - 442
  • [26] Enhancing multi-cloud security: novel method for big data storage
    Preethi
    Bisht, Sover Singh
    Kundra, Danish
    Kandhari, Harsimrat
    Singh, Poonam
    Kaushik, Harshita
    INTERNATIONAL JOURNAL OF SYSTEM ASSURANCE ENGINEERING AND MANAGEMENT, 2025,
  • [27] Efficient multimedia data storage in cloud environment
    Deshpande, Prachi
    Sharma, S.C.
    Peddoju, Sateesh K.
    Abraham, Ajith
    Informatica (Slovenia), 2015, 39 (04): : 431 - 442
  • [28] Efficient and Secure Cloud Storage for Handling Big Data
    Kumar, Arjun
    Lee, HoonJae
    Singh, Rajeev Pratap
    2012 6TH INTERNATIONAL CONFERENCE ON NEW TRENDS IN INFORMATION SCIENCE, SERVICE SCIENCE AND DATA MINING (ISSDM2012), 2012, : 162 - 166
  • [29] Measuring Community Greening Merging Multi-Source Geo-Data
    Gu, Weiying
    Chen, Yiyong
    Dai, Muye
    SUSTAINABILITY, 2019, 11 (04)
  • [30] Research on Multi-Source Heterogeneous Big Data Fusion Method Based on Feature Level
    Chen, Yanyan
    Wang, Chenxi
    Zhou, Yuchen
    Zuo, Yuhang
    Yang, Zixuan
    Li, Hui
    Yang, Juan
    INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE, 2024, 38 (02)