Apache Hadoop-MapReduce on YARN framework latency

被引：0
|
作者：
El Yazidi, Abdelaziz [1 ]
Azizi, Mohamed Saad [1 ]
Benlachmi, Yassine [1 ]
Hasnaoui, Moulay Lahcen [1 ]
机构：
[1] ENSAAd Moulay Ismatl Univ, LMMI Lab, Meknes 50000, Morocco
来源：
12TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT) / THE 4TH INTERNATIONAL CONFERENCE ON EMERGING DATA AND INDUSTRY 4.0 (EDI40) / AFFILIATED WORKSHOPS | 2021年 / 184卷
关键词：
Hadoop; big data; YARN; MapReduce; Resource Manager; IoT;
D O I：
10.1016/j.procs.2021.03.100
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
Big Data is currently a fertile field for researchers and scientific companies around the world, due to the emergence of new technologies, Internet of Things (IoT) and means of communication such as social networking sites, which has led to a notable increase in the amount of data produced each day. In this paper, we discuss the challenges of Big Data and we are going to present the famous Apache Hadoop framework and we are going to test the latency at the level of execution of a MapReduce job based on YARN (Yet Another Resource Negotiator) by applying a Java wordcount application on different text files which resumes the sales of a store, which has prepared beforehand, finally going to make a synthesis of the results. (C) 2021 The Authors. Published by Elsevier B.V.
引用
收藏
页码：803 / 808
页数：6
相关论文
共 50 条
  • [41] Customized Parameter Configuration Framework for Performance Tuning in Apache Hadoop
    Mathiya, Bhavin J.
    Desai, Vinodkumar L.
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON ICT FOR SUSTAINABLE DEVELOPMENT ICT4SD 2015, VOL 2, 2016, 409 : 511 - 521
  • [42] Content Based Audiobooks Indexing using Apache Hadoop Framework
    Shetty, Sonal
    Sabarad, Akash
    Hebballi, Harish
    Husain, Moula
    Meena, S. M.
    Nagaralli, Shiddu
    PROCEEDING OF THE THIRD INTERNATIONAL SYMPOSIUM ON WOMEN IN COMPUTING AND INFORMATICS (WCI-2015), 2015, : 496 - 501
  • [43] Framework for Analyzing Web Access Logs using Hadoop and MapReduce
    Borgaonkar, Pranjali
    Kumar, Gaurav
    Yaduwanshi, Jyoti
    2018 INTERNATIONAL CONFERENCE ON RECENT INNOVATIONS IN ELECTRICAL, ELECTRONICS & COMMUNICATION ENGINEERING (ICRIEECE 2018), 2018, : 2124 - 2129
  • [44] Color and Texture Feature Extraction using Apache Hadoop Framework
    Sabarad, Akash K.
    Kankudti, Mohamed Humair
    Meena, S. M.
    Husain, Moula
    1ST INTERNATIONAL CONFERENCE ON COMPUTING COMMUNICATION CONTROL AND AUTOMATION ICCUBEA 2015, 2015, : 585 - 588
  • [45] Apache Hadoop Based Distributed Denial of Service Detection Framework
    Patil, Nilesh Vishwasrao
    Krishna, C. Rama
    Kumar, Krishan
    INFORMATION, COMMUNICATION AND COMPUTING TECHNOLOGY (ICICCT 2019), 2019, 1025 : 25 - 35
  • [46] Solving the NP-Hard Computational Problem in Bayesian Networks using Apache Hadoop MapReduce
    Jongsawat, Nipat
    Premchaiswadi, Wichian
    2013 ELEVENTH INTERNATIONAL CONFERENCE ON ICT AND KNOWLEDGE ENGINEERING (ICT&KE), 2013,
  • [47] NEWT - A Resilient BSP Framework for Iterative Algorithms on Hadoop YARN
    Kromonov, Ilja
    Jakovits, Pelle
    Srirama, Satish Narayana
    2014 INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING & SIMULATION (HPCS), 2014, : 251 - 259
  • [48] Constraint programming versus heuristic approach to MapReduce scheduling problem in Hadoop YARN for energy minimization
    Vaibhav Pandey
    Poonam Saini
    The Journal of Supercomputing, 2021, 77 : 6788 - 6816
  • [49] Constraint programming versus heuristic approach to MapReduce scheduling problem in Hadoop YARN for energy minimization
    Pandey, Vaibhav
    Saini, Poonam
    JOURNAL OF SUPERCOMPUTING, 2021, 77 (07): : 6788 - 6816
  • [50] Clustering with apache Hadoop
    Nair, S.
    Mehta, J.
    International Conference and Workshop on Emerging Trends in Technology 2011, ICWET 2011 - Conference Proceedings, 2011, : 505 - 509
12345