Development of distributed file system for storing weather data

被引:7
|
作者
Sherstnev, V. S. [1 ]
Botygin, I. A. [1 ]
Zenzin, A. S. [1 ]
Sherstneva, A. I. [1 ]
Galanova, N. Y. [2 ]
机构
[1] Natl Res Tomsk State Univ, Lenin Ave 40, Tomsk 634050, Russia
[2] Natl Res Tomsk Polytech Univ, Lenin Ave 36, Tomsk 634050, Russia
关键词
distributed systems; distributed file system; network storage; meta-description; cross-platform; big data;
D O I
10.1117/12.2249248
中图分类号
P4 [大气科学(气象学)];
学科分类号
0706 ; 070601 ;
摘要
This article presents the results of developing a software for surface meteorological observation network storage. Multithreading was used to simulate a distributed cluster of information-computational resources during the software testing. This technology is used to create concurrently executing threads and is supported by practically every operating system. The architecture of the meteorological data network storage was designed and includes the following modules: coordinating module (monitors and displays computing nodes), database manager module (distributes meteorological data and executes user queries), client module (allows accessing the data), terminal modules (allows uploading new data), storage and processing modules (nodes for distributed data storage). The system was designed using platform-independent development tools (Java technologies), underwent simulation testing to improve the module interaction, ensure system performance, stability and reliability during processing large volumes of meteorological data.
引用
收藏
页数:6
相关论文
共 50 条
  • [31] An approach for Big Data Security based on Hadoop Distributed File system
    Mahmoud, Hadeer
    Hegazy, Abdelfatah
    Khafagy, Mohamed H.
    PROCEEDINGS OF 2018 INTERNATIONAL CONFERENCE ON INNOVATIVE TRENDS IN COMPUTER ENGINEERING (ITCE' 2018), 2018, : 109 - 114
  • [32] HDFSX: Big Data Distributed File System with Small Files Support
    EIKafrawy, Passent M.
    Sauber, Amr M.
    Hafez, Mohamed M.
    ICENCO 2016 - 2016 12TH INTERNATIONAL COMPUTER ENGINEERING CONFERENCE (ICENCO) - BOUNDLESS SMART SOCIETIES, 2016, : 131 - 135
  • [33] Sharing lattice QCD data over a widely distributed file system
    Amagasa, T.
    Aoki, S.
    Aoki, Y.
    Aoyama, T.
    Doi, T.
    Fukumura, K.
    Ishii, N.
    Ishikawa, K. -I
    Jitsumoto, H.
    Kamano, H.
    Konno, Y.
    Matsufuru, H.
    Mikami, Y.
    Miura, K.
    Sato, M.
    Takeda, S.
    Tatebe, O.
    Togawa, H.
    Ukawa, A.
    Ukita, N.
    Watanabe, Y.
    Yamazaki, T.
    Yoshie, T.
    21ST INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY AND NUCLEAR PHYSICS (CHEP2015), PARTS 1-9, 2015, 664
  • [34] Smart Medical Big Data Platform Based on Distributed File System
    Cai, Yonghua
    BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2020, 127 : 111 - 111
  • [35] Data-Intensive Workload Consolidation for the Hadoop Distributed File System
    Moraveji, Reza
    Taheri, Javid
    Reza, Mohammad
    Rizvandi, Nikzad Babaii
    Zomaya, Albert Y.
    2012 ACM/IEEE 13TH INTERNATIONAL CONFERENCE ON GRID COMPUTING (GRID), 2012, : 95 - 103
  • [36] Analysis of key technologies of distributed file system based on big data
    Junping, Zhou
    Acta Technica CSAV (Ceskoslovensk Akademie Ved), 2017, 62 (01): : 479 - 488
  • [37] Zput: a speedy data uploading approach for the Hadoop Distributed File System
    Wang, Youwei
    Wang, Weiping
    Ma, Can
    Meng, Dan
    2013 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2013,
  • [38] Erasure Code of Small File in a Distributed File System
    Chen, Xinhai
    Liu, Jie
    Xie, Peizhen
    PROCEEDINGS OF 2017 3RD IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATIONS (ICCC), 2017, : 2549 - 2554
  • [39] An Efficient Distributed Weather Data Sharing System Based on Agent
    Ma, Tinghuai
    Cao, Hao
    Guan, Donghai
    Lee, SungYoung
    INTERNATIONAL ARAB JOURNAL OF INFORMATION TECHNOLOGY, 2012, 9 (02) : 170 - 178
  • [40] Developing a File System Structure to Solve Healthy Big Data Storage and Archiving Problems Using a Distributed File System
    Erguzen, Atilla
    Unver, Mahmut
    APPLIED SCIENCES-BASEL, 2018, 8 (06):