Apache Hadoop-MapReduce on YARN framework latency

被引：0
|
作者：
El Yazidi, Abdelaziz [1 ]
Azizi, Mohamed Saad [1 ]
Benlachmi, Yassine [1 ]
Hasnaoui, Moulay Lahcen [1 ]
机构：
[1] ENSAAd Moulay Ismatl Univ, LMMI Lab, Meknes 50000, Morocco
来源：
12TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT) / THE 4TH INTERNATIONAL CONFERENCE ON EMERGING DATA AND INDUSTRY 4.0 (EDI40) / AFFILIATED WORKSHOPS | 2021年 / 184卷
关键词：
Hadoop; big data; YARN; MapReduce; Resource Manager; IoT;
D O I：
10.1016/j.procs.2021.03.100
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
Big Data is currently a fertile field for researchers and scientific companies around the world, due to the emergence of new technologies, Internet of Things (IoT) and means of communication such as social networking sites, which has led to a notable increase in the amount of data produced each day. In this paper, we discuss the challenges of Big Data and we are going to present the famous Apache Hadoop framework and we are going to test the latency at the level of execution of a MapReduce job based on YARN (Yet Another Resource Negotiator) by applying a Java wordcount application on different text files which resumes the sales of a store, which has prepared beforehand, finally going to make a synthesis of the results. (C) 2021 The Authors. Published by Elsevier B.V.
引用
收藏
页码：803 / 808
页数：6
相关论文
共 50 条
  • [31] ABS-YARN: A Formal Framework for Modeling Hadoop YARN Clusters
    Lin, Jia-Chun
    Yu, Ingrid Chieh
    Johnsen, Einar Broch
    Lee, Ming-Chang
    FUNDAMENTAL APPROACHES TO SOFTWARE ENGINEERING (FASE 2016), 2016, 9633 : 49 - 65
  • [32] A Parallel Genetic Algorithms Framework based on Hadoop MapReduce
    Ferrucci, Filomena
    Salza, Pasquale
    Kechadi, M-Tahar
    Sarro, Federica
    30TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, VOLS I AND II, 2015, : 1664 - 1667
  • [33] Implementation of Page Rank Algorithm in Hadoop MapReduce Framework
    Bhawivuga, Adhitya
    Kirana, Annisa Puspa
    2016 INTERNATIONAL SEMINAR ON INTELLIGENT TECHNOLOGY AND ITS APPLICATIONS (ISITIA): RECENT TRENDS IN INTELLIGENT COMPUTATIONAL TECHNOLOGIES FOR SUSTAINABLE ENERGY, 2016, : 231 - 235
  • [34] HybSMRP: a hybrid scheduling algorithm in Hadoop MapReduce framework
    Gandomi, Abolfazl
    Reshadi, Midia
    Movaghar, Ali
    Khademzadeh, Ahmad
    JOURNAL OF BIG DATA, 2019, 6 (01)
  • [35] Performance Enhancement of Hadoop MapReduce Framework for Analyzing BigData
    Prabhu, Swathi
    Rodrigues, Anisha P.
    Prasad, Guru M. S.
    Nagesh, H. R.
    2015 IEEE INTERNATIONAL CONFERENCE ON ELECTRICAL, COMPUTER AND COMMUNICATION TECHNOLOGIES, 2015,
  • [36] HybSMRP: a hybrid scheduling algorithm in Hadoop MapReduce framework
    Abolfazl Gandomi
    Midia Reshadi
    Ali Movaghar
    Ahmad Khademzadeh
    Journal of Big Data, 6
  • [37] Comparative Analysis of Apache Spark and Hadoop MapReduce Using Various Parameters and Execution Time
    Meena, Bhagavathula
    Sarwani, I. S. L.
    Archana, M.
    Supriya, P.
    INTELLIGENT COMPUTING AND COMMUNICATION, ICICC 2019, 2020, 1034 : 719 - 725
  • [38] Memory and Performance Aware Scheduling Design for Hadoop MapReduce Framework
    Bakka, Jagadevi
    Lingareddy, Sanjeev C.
    BIOSCIENCE BIOTECHNOLOGY RESEARCH COMMUNICATIONS, 2020, 13 (13): : 242 - 246
  • [39] An Approach to Enhance the Performance of Hadoop MapReduce Framework for Big Data
    Chandra, Subhash
    Motwani, Deepak
    2016 INTERNATIONAL CONFERENCE ON MICRO-ELECTRONICS AND TELECOMMUNICATION ENGINEERING (ICMETE), 2016, : 178 - 182
  • [40] Parallelized Genetic Operations for SBST using Hadoop MapReduce Framework
    Mayandi, Geethapriya
    Arumugam, Chamundeswari
    2014 INTERNATIONAL CONFERENCE ON ADVANCED COMMUNICATION CONTROL AND COMPUTING TECHNOLOGIES (ICACCCT), 2014, : 1686 - 1691
12345