CRUSH: Data Collection and Analysis Framework for Power Capped Data Intensive Computing

被引：0
|
作者：
Gupta, Anurag [1 ]
Gupta, Sanjeev [1 ]
Ge, Rong [2 ]
Zong, Ziliang [3 ]
机构：
[1] Marquette Univ, Dept Math Stat & Comp Sci, Milwaukee, WI 53233 USA
[2] Clemson Univ, Dept Comp Sci, Clemson, SC 29631 USA
[3] Texas State Univ, Dept Comp Sci, San Marcos, TX 78666 USA
来源：
2015 SIXTH INTERNATIONAL GREEN COMPUTING CONFERENCE AND SUSTAINABLE COMPUTING CONFERENCE (IGSC) | 2015年
关键词：
CRUSH; Power Capping; Power provisioning; DVFS; RAPL; Hadoop;
D O I：
暂无
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
Hadoop system has been widely used to support large-scale data intensive computing tasks on enterprise software infrastructure. Deploying and operating Hadoop incur high costs including hardware expenditure to build computer clusters and energy consumption to run the clusters. Building sustainable and scalable systems requires optimal power management schemes. In this paper, we present a framework, namely CRUSH, which collects power and resource usage data and provides statistical analysis for power management. We also demonstrate the usage of CRUSH in studying the effects of power capping on Hadoop jobs.
引用
收藏
页数：6
相关论文
共 50 条
  • [41] Development of Data Collection and Integration Framework for Road Inventory Data
    Khan, Ghazan
    Santiago-Chaparro, Kelvin R.
    Chiturri, Madhav
    Noyce, David A.
    TRANSPORTATION RESEARCH RECORD, 2010, (2160) : 29 - 39
  • [42] Multidimensional data processing and tensor analysis for large-scale power grids in a parallel computing framework
    Liu, Changsheng
    Zhou, Mo
    Applied Mathematics and Nonlinear Sciences, 2024, 9 (01)
  • [43] A Method for Web Data Collection for Pervasive Computing
    Wang, Lihong
    Li, Qingzhong
    Deng, Li
    2008 3RD INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND APPLICATIONS, VOLS 1 AND 2, 2008, : 555 - 560
  • [44] HuREX - A framework of HRA data collection from simulators in nuclear power plants
    Jung, Wondea
    Park, Jinkyun
    Kim, Yochan
    Choi, Sun Yeong
    Kim, Seunghwan
    RELIABILITY ENGINEERING & SYSTEM SAFETY, 2020, 194
  • [45] A parallel computing framework for big data
    Chen, Guoliang
    Mao, Rui
    Lu, Kezhong
    FRONTIERS OF COMPUTER SCIENCE, 2017, 11 (04) : 608 - 621
  • [46] Software Engineering for Data Intensive Scalable Computing and Heterogeneous Computing
    Kim, Miryung
    2023 IEEE/ACM INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: FUTURE OF SOFTWARE ENGINEERING, ICSE-FOSE, 2023, : 54 - 68
  • [47] A parallel computing framework for big data
    Guoliang Chen
    Rui Mao
    Kezhong Lu
    Frontiers of Computer Science, 2017, 11 : 608 - 621
  • [48] Improvement Of Data Throughput In Data-Intensive Cloud Computing Applications
    Ibrahim, Ibrahim Adel
    Bassiouni, Mostafa
    2019 IEEE FIFTH INTERNATIONAL CONFERENCE ON BIG DATA COMPUTING SERVICE AND APPLICATIONS (IEEE BIGDATASERVICE 2019), 2019, : 49 - 54
  • [49] Data-intensive workflow management: For clouds and data-intensive and scalable computing environments
    De Oliveira, Daniel C.M.
    Liu, Ji
    Pacitti, Esther
    Synthesis Lectures on Data Management, 2019, 14 (04): : 1 - 179
  • [50] A New Data Classification Algorithm for Data-Intensive Computing Environments
    Deng, Qizhi
    Zhang, Longbo
    Qian, Xin
    Chen, Yali
    Wang, Fengying
    PROCEEDINGS OF THE 2ND INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION APPLICATIONS (ICCIA 2012), 2012, : 1351 - 1354
12345