A Distributed Data Management System for Data-intensive Radio Astronomy

被引:0
|
作者
Grimstrup, Arne [1 ]
Mahadevan, Venkat [2 ]
Eymere, Olivier
Anderson, Ken [2 ]
Kiddle, Cameron [1 ]
Simmonds, Rob [1 ]
Rosolowsky, Erik [2 ]
Taylor, Andrew R. [1 ]
机构
[1] Univ Calgary, Calgary, AB, Canada
[2] Univ British Columbia, Kelowna, BC, Canada
关键词
SKA; archive; distributed; architecture; VISUALIZATION;
D O I
10.1117/12.925441
中图分类号
P1 [天文学];
学科分类号
0704 ;
摘要
The next generation of telescopes, such as the Square Kilometre Array (SKA), will generate orders of magnitude more data than previous instruments, far in excess of current storage and networking system handling abilities. To address this problem, we propose an architecture where data is distributed over several archive sites, each holding only a portion of the overall data, that provides efficient and transparent access to the archive as a whole. This paper describes that architecture in detail and the design and implementation of a prototype system,based on the Integrated Rule-Oriented Data System (iRODS) software.
引用
收藏
页数:8
相关论文
共 50 条
  • [41] Unifying Data and Replica Placement for Data-intensive Services in Geographically Distributed Clouds
    Atrey, Ankita
    Van Seghbroeck, Gregory
    Mora, Higinio
    De Turck, Filip
    Volckaert, Bruno
    CLOSER: PROCEEDINGS OF THE 9TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND SERVICES SCIENCE, 2019, : 25 - 36
  • [42] A Network Performance Based Data Placement Policy in Distributed Data-Intensive Applications
    Xu, Dawei
    Miao, Xianglin
    Hu, Peng
    Luan, Zhongzhi
    2014 IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION TECHNOLOGY (CIT), 2014, : 795 - 800
  • [43] MetaFa: Metadata Management Framework for Data Sharing in Data-Intensive Applications
    Ikebe, Minoru
    Inomata, Atsuo
    Fujikawa, Kazutoshi
    Sunahara, Hideki
    DISTRIBUTED COMPUTING, ARTIFICIAL INTELLIGENCE, BIOINFORMATICS, SOFT COMPUTING, AND AMBIENT ASSISTED LIVING, PT II, PROCEEDINGS, 2009, 5518 : 655 - 658
  • [44] System dynamics simulations for data-intensive applications
    Neuwirth, Christian
    ENVIRONMENTAL MODELLING & SOFTWARE, 2017, 96 : 140 - 145
  • [45] Data throttling for data-intensive workflows
    Park, Sang-Min
    Humphrey, Marty
    2008 IEEE INTERNATIONAL SYMPOSIUM ON PARALLEL & DISTRIBUTED PROCESSING, VOLS 1-8, 2008, : 1796 - 1806
  • [46] Location, Location, Location: Data-Intensive Distributed Computing in the Cloud
    Luckeneder, Michael
    Barker, Adam
    2013 IEEE FIFTH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGY AND SCIENCE (CLOUDCOM), VOL 1, 2013, : 647 - 654
  • [47] Data-intensive Workflow Execution using Distributed Compute Resources
    Pandey, Ashish
    Wang, Songjie
    Calyam, Prasad
    2019 IEEE 27TH INTERNATIONAL CONFERENCE ON NETWORK PROTOCOLS (IEEE ICNP), 2019,
  • [48] The Quest for Scalable Support of Data-Intensive Workloads in Distributed Systems
    Raicu, Ioan
    Foster, Ian T.
    Zhao, Yong
    Little, Philip
    Moretti, Christopher M.
    Chaudhary, Amitabh
    Thain, Douglas
    HPDC'09: 18TH ACM INTERNATIONAL SYMPOSIUM ON HIGH PERFORMANCE DISTRIBUTED COMPUTING, 2009, : 207 - 216
  • [49] Modeling distributed events in data-intensive Rich Internet applications
    Carughi, Giovanni Toffetti
    Comai, Sara
    Bozzon, Alessandro
    Fraternali, Piero
    WEB INFORMATION SYSTEMS ENGINEERING - WISE 2007, PROCEEDINGS, 2007, 4831 : 593 - 602
  • [50] Multi-Layer SOA Implementation Pattern with Service and Data Proxies for Distributed Data-Intensive Application System
    Takdir
    Kistijantoro, Achmad Imam
    2014 INTERNATIONAL CONFERENCE ON ICT FOR SMART SOCIETY (ICISS), 2014, : 37 - 41