Development of distributed file system for storing weather data

被引:7
|
作者
Sherstnev, V. S. [1 ]
Botygin, I. A. [1 ]
Zenzin, A. S. [1 ]
Sherstneva, A. I. [1 ]
Galanova, N. Y. [2 ]
机构
[1] Natl Res Tomsk State Univ, Lenin Ave 40, Tomsk 634050, Russia
[2] Natl Res Tomsk Polytech Univ, Lenin Ave 36, Tomsk 634050, Russia
关键词
distributed systems; distributed file system; network storage; meta-description; cross-platform; big data;
D O I
10.1117/12.2249248
中图分类号
P4 [大气科学(气象学)];
学科分类号
0706 ; 070601 ;
摘要
This article presents the results of developing a software for surface meteorological observation network storage. Multithreading was used to simulate a distributed cluster of information-computational resources during the software testing. This technology is used to create concurrently executing threads and is supported by practically every operating system. The architecture of the meteorological data network storage was designed and includes the following modules: coordinating module (monitors and displays computing nodes), database manager module (distributes meteorological data and executes user queries), client module (allows accessing the data), terminal modules (allows uploading new data), storage and processing modules (nodes for distributed data storage). The system was designed using platform-independent development tools (Java technologies), underwent simulation testing to improve the module interaction, ensure system performance, stability and reliability during processing large volumes of meteorological data.
引用
收藏
页数:6
相关论文
共 50 条
  • [1] Data Structures for Storing File Namespace in Distributed File System
    Long, Luu Hoang
    Choi, Eunmi
    Kim, SangBum
    Kim, Pilsung
    NCM 2008 : 4TH INTERNATIONAL CONFERENCE ON NETWORKED COMPUTING AND ADVANCED INFORMATION MANAGEMENT, VOL 1, PROCEEDINGS, 2008, : 250 - 255
  • [2] Data Adaptively Storing Approach for Hadoop Distributed File System
    Fu, Yingxun
    Wen, Shilin
    Ma, Li
    2017 2ND IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND APPLICATIONS (ICCIA), 2017, : 20 - 24
  • [3] Σ-Tree:Design of a Data Structure for Storing File Data Allocation Map in a Distributed File System
    Dewan, Hrishikesh
    Hansdah, R. C.
    Singh, Prashant
    2018 IEEE INTERNATIONAL CONFERENCE ON SMART CLOUD (SMARTCLOUD), 2018, : 90 - 98
  • [4] Architecture of scalability file system for meteorological observation data storing
    Botygin, I. A.
    Popov, V. N.
    Tartakovsky, V. A.
    Sherstnev, V. S.
    21ST INTERNATIONAL SYMPOSIUM ON ATMOSPHERIC AND OCEAN OPTICS: ATMOSPHERIC PHYSICS, 2015, 9680
  • [5] A Distributed File-Based Storage System for Improving High Availability of Space Weather Data
    Andrian, Yoga
    Kim, Hyeonwoo
    Ju, Hongtaek
    APPLIED SCIENCES-BASEL, 2019, 9 (23):
  • [6] Data Security in Hadoop Distributed File System
    Shetty, Madhvaraj M.
    Manjaiah, D. H.
    IEEE INTERNATIONAL CONFERENCE ON EMERGING TECHNOLOGICAL TRENDS IN COMPUTING, COMMUNICATIONS AND ELECTRICAL ENGINEERING (ICETT), 2016,
  • [7] A LOOSELY COUPLED DISTRIBUTED SYSTEM FOR RELIABLY STORING DATA
    BERNSTEIN, AJ
    IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1985, 11 (05) : 446 - 454
  • [8] STORING SUMMARY PATIENT DATA AS A MICROCOMPUTER FILE
    HANNAY, DR
    MITCHELL, S
    JOURNAL OF THE ROYAL COLLEGE OF GENERAL PRACTITIONERS, 1985, 35 (280): : 525 - 526
  • [9] Memory-based Data Storing Technologies on Hadoop Distribution File System
    Song, Aibo
    Zhao, Jinghua
    Tu, Jinlin
    Qian, Xuejiao
    2015 THIRD INTERNATIONAL CONFERENCE ON ADVANCED CLOUD AND BIG DATA, 2015, : 64 - 68
  • [10] An enhancement of data locality in Hadoop distributed file system
    Reddy, A. Siva Krishna
    Sujatha, Pothula
    Koti, Prasad
    Dhavachelvan, P.
    Amudhavel, J.
    BIOSCIENCE BIOTECHNOLOGY RESEARCH COMMUNICATIONS, 2018, 11 (01): : 123 - 133