A Distributed Data Management System for Data-intensive Radio Astronomy

被引:0
|
作者
Grimstrup, Arne [1 ]
Mahadevan, Venkat [2 ]
Eymere, Olivier
Anderson, Ken [2 ]
Kiddle, Cameron [1 ]
Simmonds, Rob [1 ]
Rosolowsky, Erik [2 ]
Taylor, Andrew R. [1 ]
机构
[1] Univ Calgary, Calgary, AB, Canada
[2] Univ British Columbia, Kelowna, BC, Canada
关键词
SKA; archive; distributed; architecture; VISUALIZATION;
D O I
10.1117/12.925441
中图分类号
P1 [天文学];
学科分类号
0704 ;
摘要
The next generation of telescopes, such as the Square Kilometre Array (SKA), will generate orders of magnitude more data than previous instruments, far in excess of current storage and networking system handling abilities. To address this problem, we propose an architecture where data is distributed over several archive sites, each holding only a portion of the overall data, that provides efficient and transparent access to the archive as a whole. This paper describes that architecture in detail and the design and implementation of a prototype system,based on the Integrated Rule-Oriented Data System (iRODS) software.
引用
收藏
页数:8
相关论文
共 50 条
  • [31] A Survey of Data-Intensive Scientific Workflow Management
    Liu, Ji
    Pacitti, Esther
    Valduriez, Patrick
    Mattoso, Marta
    JOURNAL OF GRID COMPUTING, 2015, 13 (04) : 457 - 493
  • [32] Power Management of Online Data-Intensive Services
    Meisner, David
    Sadler, Christopher M.
    Barroso, Luiz Andre
    Weber, Wolf-Dietrich
    Wenisch, Thomas F.
    ISCA 2011: PROCEEDINGS OF THE 38TH ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE, 2011, : 319 - 330
  • [33] A Survey of Data-Intensive Scientific Workflow Management
    Ji Liu
    Esther Pacitti
    Patrick Valduriez
    Marta Mattoso
    Journal of Grid Computing, 2015, 13 : 457 - 493
  • [34] A high-performance distributed parallel file system for data-intensive computations
    Shen, XH
    Choudhary, A
    JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2004, 64 (10) : 1157 - 1167
  • [35] Report on Data-intensive Software Management and Mining
    Hwang, Seung-won
    SIGMOD RECORD, 2011, 40 (01) : 32 - 34
  • [36] A combination replication strategy for data-intensive services in distributed geographic information system
    Pan, Shaoming
    Xu, Zhengquan
    Meng, Qingxiang
    Chong, Yanwen
    INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS, 2017, 13 (05):
  • [37] A resource management system for data-intensive applications in desktop grid environments
    Toyama, Toshiaki
    Yamada, Yoshito
    Konishi, Katsumi
    PROCEEDINGS OF THE 18TH IASTED INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING AND SYSTEMS, 2006, : 60 - +
  • [38] The coloniality of collaboration: sources of epistemic obedience in data-intensive astronomy in Chile
    Lehuede, Sebastian
    INFORMATION COMMUNICATION & SOCIETY, 2023, 26 (02) : 425 - 440
  • [39] Scalable Data Placement of Data-intensive Services in Geo-distributed Clouds
    Atrey, Ankita
    Van Seghbroeck, Gregory
    Volckaert, Bruno
    De Turck, Filip
    CLOSER: PROCEEDINGS OF THE 8TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND SERVICES SCIENCE, 2018, : 497 - 508
  • [40] Data-Intensive Science
    Strawn, George
    IT PROFESSIONAL, 2016, 18 (05) : 66 - 68