Apache Hadoop-MapReduce on YARN framework latency

被引：0
|
作者：
El Yazidi, Abdelaziz [1 ]
Azizi, Mohamed Saad [1 ]
Benlachmi, Yassine [1 ]
Hasnaoui, Moulay Lahcen [1 ]
机构：
[1] ENSAAd Moulay Ismatl Univ, LMMI Lab, Meknes 50000, Morocco
来源：
12TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT) / THE 4TH INTERNATIONAL CONFERENCE ON EMERGING DATA AND INDUSTRY 4.0 (EDI40) / AFFILIATED WORKSHOPS | 2021年 / 184卷
关键词：
Hadoop; big data; YARN; MapReduce; Resource Manager; IoT;
D O I：
10.1016/j.procs.2021.03.100
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
Big Data is currently a fertile field for researchers and scientific companies around the world, due to the emergence of new technologies, Internet of Things (IoT) and means of communication such as social networking sites, which has led to a notable increase in the amount of data produced each day. In this paper, we discuss the challenges of Big Data and we are going to present the famous Apache Hadoop framework and we are going to test the latency at the level of execution of a MapReduce job based on YARN (Yet Another Resource Negotiator) by applying a Java wordcount application on different text files which resumes the sales of a store, which has prepared beforehand, finally going to make a synthesis of the results. (C) 2021 The Authors. Published by Elsevier B.V.
引用
收藏
页码：803 / 808
页数：6
相关论文
共 50 条
  • [21] Apache Hadoop Yarn MapReduce Job Classification Based on CPU Utilization and Performance Evaluation on Multi-cluster Heterogeneous Environment
    Mathiya, Bhavin J.
    Desai, Vinodkumar L.
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON ICT FOR SUSTAINABLE DEVELOPMENT, ICT4SD 2015, VOL 1, 2016, 408 : 35 - 44
  • [22] A Performance Comparison of Apache Tez and MapReduce with Data Compression on Hadoop Cluster
    Rattanaopas, Kritwara
    PROCEEDINGS OF 2017 14TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER SCIENCE AND SOFTWARE ENGINEERING (JCSSE), 2017,
  • [23] Evaluation of Hadoop/Mapreduce Framework Migration Tools
    Odia, Trust
    Misra, Sanjay
    Adewumi, Adewole
    2014 ASIA-PACIFIC WORLD CONGRESS ON COMPUTER SCIENCE AND ENGINEERING (APWC ON CSE), 2014,
  • [24] Scientific data processing framework for Hadoop MapReduce
    Department of Computer and Information, Xinxiang University, Xinxiang, China
    1600, Journal of Chemical and Pharmaceutical Research, 3/668 Malviya Nagar, Jaipur, Rajasthan, India (06):
  • [25] YARN Schedulers for Hadoop MapReduce Jobs: Design Goals, Issues and Taxonomy
    Kotikam G.
    Selvaraj L.
    Recent Advances in Computer Science and Communications, 2023, 16 (06) : 44 - 55
  • [26] Straggler Mitigation in Hadoop MapReduce Framework: A Review
    Ajibade, Lukuman Saheed
    Abu Bakar, Kamalrulnizam
    Aliyu, Ahmed
    Danish, Tasneem
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (08) : 870 - 878
  • [27] High-Responsive Scheduling with MapReduce Performance Prediction on Hadoop YARN
    Liu, Yang
    Zeng, Yukun
    Piao, Xuefeng
    2016 IEEE 22ND INTERNATIONAL CONFERENCE ON EMBEDDED AND REAL-TIME COMPUTING SYSTEMS AND APPLICATIONS (RTCSA), 2016, : 238 - 247
  • [28] Extensible Video Processing Framework in Apache Hadoop
    Ryu, Chungmo
    Lee, Daecheol
    Jang, Minwook
    Kim, Cheolgi
    Seo, Euiseong
    2013 IEEE FIFTH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGY AND SCIENCE (CLOUDCOM), VOL 2, 2013, : 305 - 308
  • [29] A Neural Network based framework for Apache YARN
    Nivash, J. P.
    Raj, Ebin Deni
    Babu, L. D. Dhinesh
    Nirmala, M.
    2014 INTERNATIONAL CONFERENCE ON INFORMATION COMMUNICATION AND EMBEDDED SYSTEMS (ICICES), 2014,
  • [30] Analyzing performance of Apache Tez and MapReduce with hadoop multinode cluster on Amazon cloud
    Singh R.
    Kaur P.J.
    Journal of Big Data, 3 (1)
12345