A plugin-based approach to data analysis for the AMS experiment on the ISS

被引:0
|
作者
Formato, Valerio [1 ]
机构
[1] Ist Nazl Fis Nucl, Sez Perugia, Via A Pascoli Snc, I-06123 Perugia, PG, Italy
关键词
D O I
10.1051/epjconf/201921405038
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In many HEP experiments a typical data analysis workflow requires each user to read the experiment data in order to extract meaningful information and produce relevant plots for the considered analysis. Multiple users accessing the same data result in a redundant access to the data itself, which could be factorized effectively improving the CPU efficiency of the analysis jobs and relieving stress from the storage infrastructure. To address this issue we present a modular and lightweight solution where the users code is embedded in different "analysis plugins" which are then collected and loaded at runtime for execution, where the data is read only once and shared between all the different plugins. This solution was developed for one of the data analysis groups within the AMS collaboration but is easily extendable to all kinds of analyses and workloads that need I/O access on AMS data or custom data formats and can even adapted with little effort to another HEP experiment data. This framework could then be easily embedded into a "analysis train" and we will discuss a possible implementation and different ways to optimise CPU efficiency and execution time.
引用
收藏
页数:7
相关论文
共 50 条
  • [11] Constructing lightweight and flexible pipelines using Plugin-Based Microbiome Analysis (PluMA)
    Cickovski, Trevor
    Narasimhan, Giri
    BIOINFORMATICS, 2018, 34 (17) : 2881 - 2888
  • [12] Extraction and evolution of architectural variability models in plugin-based systems
    Mathieu Acher
    Anthony Cleve
    Philippe Collet
    Philippe Merle
    Laurence Duchien
    Philippe Lahire
    Software & Systems Modeling, 2014, 13 : 1367 - 1394
  • [13] A formal contract language for plugin-based software engineering
    Dietrich, Jens
    Hosking, John
    Giles, Jonathan
    12TH IEEE INTERNATIONAL CONFERENCE ON ENGINEERING COMPLEX COMPUTER SYSTEMS, PROCEEDINGS, 2007, : 175 - +
  • [14] Plugin-based grid job management interoperation strategy
    Xu, Zhongqing
    Li, Shengli
    Wu, Song
    Shi, Xuanhua
    Huazhong Keji Daxue Xuebao (Ziran Kexue Ban)/Journal of Huazhong University of Science and Technology (Natural Science Edition), 2006, 34 (SUPPL.): : 141 - 143
  • [15] Jespipe: A Plugin-Based, Open MPI Framework for Adversarial Machine Learning Analysis
    Alemany, Sheila
    Nucciarone, Jason
    Pissinou, Niki
    2021 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2021, : 3663 - 3670
  • [16] A plugin-based architecture for simulation in the F2000 league
    Kleiner, A
    Buchheim, T
    ROBOCUP 2003: ROBOT SOCCER WORLD CUP VII, 2004, 3020 : 434 - 445
  • [17] A Parallel Plugin-Based Framework for Multi-objective Optimization
    Leon, Coromoto
    Miranda, Gara
    Segura, Carlos
    INTERNATIONAL SYMPOSIUM ON DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE 2008, 2009, 50 : 142 - 151
  • [18] Variability management of plugin-based systems using feature models
    Santos, Andre L.
    SOFTWARE-PRACTICE & EXPERIENCE, 2017, 47 (07): : 959 - 970
  • [19] PBJIM: Plugin-based job interoperation mechanism for heterogeneous grid systems
    Wu, Song
    Tao, Yongcai
    Xu, Zhongqing
    SIXTH INTERNATIONAL CONFERENCE ON GRID AND COOPERATIVE COMPUTING, PROCEEDINGS, 2007, : 830 - +
  • [20] PROSE: A Plugin-Based Framework for Paraconsistent Reasoning on Semantic Web
    Zhang, Xiaowang
    Feng, Zhiyong
    Wu, Wenrui
    Wang, Xin
    Rao, Guozheng
    INTERNATIONAL JOURNAL ON SEMANTIC WEB AND INFORMATION SYSTEMS, 2018, 14 (01) : 121 - 139