A Distributed Data Management System for Data-intensive Radio Astronomy

被引：0
|
作者：
Grimstrup, Arne [1 ]
Mahadevan, Venkat [2 ]
Eymere, Olivier
Anderson, Ken [2 ]
Kiddle, Cameron [1 ]
Simmonds, Rob [1 ]
Rosolowsky, Erik [2 ]
Taylor, Andrew R. [1 ]
机构：
[1] Univ Calgary, Calgary, AB, Canada
[2] Univ British Columbia, Kelowna, BC, Canada
来源：
SOFTWARE AND CYBERINFRASTRUCTURE FOR ASTRONOMY II | 2012年 / 8451卷
关键词：
SKA; archive; distributed; architecture; VISUALIZATION;
D O I：
10.1117/12.925441
中图分类号：
P1 [天文学];
学科分类号：
0704 ;
摘要：
The next generation of telescopes, such as the Square Kilometre Array (SKA), will generate orders of magnitude more data than previous instruments, far in excess of current storage and networking system handling abilities. To address this problem, we propose an architecture where data is distributed over several archive sites, each holding only a portion of the overall data, that provides efficient and transparent access to the archive as a whole. This paper describes that architecture in detail and the design and implementation of a prototype system,based on the Integrated Rule-Oriented Data System (iRODS) software.
引用
收藏
页数：8
相关论文
共 50 条
  • [11] Protocols and services for distributed data-intensive science
    Allcock, W
    Foster, I
    Tuecke, S
    Chervenak, A
    Kesselman, C
    ADVANCED COMPUTING AND ANALYSIS TECHNIQUES IN PHYSICS RESEARCH, 2001, 583 : 161 - 163
  • [12] A Model and Survey of Distributed Data-Intensive Systems
    Margara, Alessandro
    Cugola, Gianpaolo
    Felicioni, Nicolo
    Cilloni, Stefano
    ACM COMPUTING SURVEYS, 2024, 56 (01)
  • [13] Citus: Distributed PostgreSQL for Data-Intensive Applications
    Cubukcu, Umur
    Erdogan, Ozgun
    Pathak, Sumedh
    Sannakkayala, Sudhakar
    Slot, Marco
    SIGMOD '21: PROCEEDINGS OF THE 2021 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2021, : 2490 - 2502
  • [14] Understanding performance of distributed data-intensive applications
    Miceli, Christopher
    Miceli, Michael
    Rodriguez-Milla, Bety
    Jha, Shantenu
    PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES, 2010, 368 (1926): : 4089 - 4102
  • [15] Data Management Challenges of Data-Intensive Scientific Workflows
    Deelman, Ewa
    Chervenak, Ann
    CCGRID 2008: EIGHTH IEEE INTERNATIONAL SYMPOSIUM ON CLUSTER COMPUTING AND THE GRID, VOLS 1 AND 2, PROCEEDINGS, 2008, : 687 - 692
  • [16] IBM, CERN join to create a data-intensive management system
    不详
    R&D MAGAZINE, 2003, 45 (05): : 20 - 20
  • [17] Distributed data structure templates for data-intensive remote sensing applications
    Ma, Yan
    Wang, Lizhe
    Liu, Dingsheng
    Yuan, Tao
    Liu, Peng
    Zhang, Wanfeng
    CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2013, 25 (12): : 1784 - 1797
  • [18] Pipelining/Overlapping Data Transfer for Distributed Data-Intensive Job Execution
    Jung, Eun-Sung
    Maheshwari, Ketan
    Kettimuthu, Rajkumar
    2013 42ND ANNUAL INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING (ICPP), 2013, : 791 - 797
  • [19] Distributed Data Provenance for Large-Scale Data-Intensive Computing
    Zhao, Dongfang
    Shou, Chen
    Malik, Tanu
    Raicu, Ioan
    2013 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2013,
  • [20] Improving Parallelism in Data-Intensive Workflows with Distributed Databases
    Watanabe, Elaine Naomi
    Braghetto, Kelly Rosa
    2018 IEEE INTERNATIONAL CONFERENCE ON SERVICES COMPUTING (IEEE SCC 2018), 2018, : 209 - 216
12345