A Distributed Data Management System for Data-intensive Radio Astronomy

被引：0
|
作者：
Grimstrup, Arne [1 ]
Mahadevan, Venkat [2 ]
Eymere, Olivier
Anderson, Ken [2 ]
Kiddle, Cameron [1 ]
Simmonds, Rob [1 ]
Rosolowsky, Erik [2 ]
Taylor, Andrew R. [1 ]
机构：
[1] Univ Calgary, Calgary, AB, Canada
[2] Univ British Columbia, Kelowna, BC, Canada
来源：
SOFTWARE AND CYBERINFRASTRUCTURE FOR ASTRONOMY II | 2012年 / 8451卷
关键词：
SKA; archive; distributed; architecture; VISUALIZATION;
D O I：
10.1117/12.925441
中图分类号：
P1 [天文学];
学科分类号：
0704 ;
摘要：
The next generation of telescopes, such as the Square Kilometre Array (SKA), will generate orders of magnitude more data than previous instruments, far in excess of current storage and networking system handling abilities. To address this problem, we propose an architecture where data is distributed over several archive sites, each holding only a portion of the overall data, that provides efficient and transparent access to the archive as a whole. This paper describes that architecture in detail and the design and implementation of a prototype system,based on the Integrated Rule-Oriented Data System (iRODS) software.
引用
收藏
页数：8
相关论文
共 50 条
  • [1] Distributed Scientific Workflow Management for Data-Intensive Applications
    Shumilov, S.
    Leng, Y.
    El-Gayyar, M.
    Cremers, A. B.
    12TH IEEE INTERNATIONAL WORKSHOP ON FUTURE TRENDS OF DISTRIBUTED COMPUTING SYSTEMS, PROCEEDINGS, 2008, : 65 - 73
  • [2] Distributed Data Access/Find System with Metadata for Data-Intensive Computing
    Ikebe, Minoru
    Inomata, Atsuo
    Fujikawa, Kazutoshi
    Sunahara, Hideki
    2008 9TH IEEE/ACM INTERNATIONAL CONFERENCE ON GRID COMPUTING, 2008, : 361 - 366
  • [3] Data-intensive workflow management: For clouds and data-intensive and scalable computing environments
    De Oliveira, Daniel C.M.
    Liu, Ji
    Pacitti, Esther
    Synthesis Lectures on Data Management, 2019, 14 (04): : 1 - 179
  • [4] Optimizing Distributed Data-Intensive Workflows
    Friese, Ryan D.
    Tallent, Nathan R.
    Schram, Malachi
    Halappanavar, Mahantesh
    Barker, Kevin J.
    2018 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2018, : 279 - 289
  • [5] Some pattern recognition challenges in data-intensive astronomy
    Djorgovski, S. G.
    Donalek, C.
    Mahabal, A.
    Williams, R.
    Drake, A. J.
    Graham, M. J.
    Glikman, E.
    18TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2006, : 856 - +
  • [6] Data-Intensive Workload Consolidation for the Hadoop Distributed File System
    Moraveji, Reza
    Taheri, Javid
    Reza, Mohammad
    Rizvandi, Nikzad Babaii
    Zomaya, Albert Y.
    2012 ACM/IEEE 13TH INTERNATIONAL CONFERENCE ON GRID COMPUTING (GRID), 2012, : 95 - 103
  • [7] Data-Intensive System Evolution
    Cleve, Anthony
    Mens, Tom
    Hainaut, Jean-Luc
    COMPUTER, 2010, 43 (08) : 110 - 112
  • [8] CoLoc: Distributed Data and Container Colocation for Data-Intensive Applications
    Renner, Thomas
    Thamsen, Lauritz
    Kao, Odej
    2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2016, : 3008 - 3015
  • [9] On the Flexibility of Data Fulfillment Locations in Data-intensive Distributed Systems
    Yu, Boyang
    Pan, Jianping
    2016 IEEE CONFERENCE ON COMPUTER COMMUNICATIONS WORKSHOPS (INFOCOM WKSHPS), 2016,
  • [10] Decoupling computation and data scheduling in distributed data-intensive applications
    Ranganathan, K
    Foster, I
    11TH IEEE INTERNATIONAL SYMPOSIUM ON HIGH PERFORMANCE DISTRIBUTED COMPUTING, PROCEEDINGS, 2002, : 352 - 358
12345