Apache Hadoop-MapReduce on YARN framework latency

被引：0

作者：

El Yazidi, Abdelaziz ^{[1
]}

Azizi, Mohamed Saad ^{[1
]}

Benlachmi, Yassine ^{[1
]}

Hasnaoui, Moulay Lahcen ^{[1
]}

机构：

[1] ENSAAd Moulay Ismatl Univ, LMMI Lab, Meknes 50000, Morocco

来源：

12TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT) / THE 4TH INTERNATIONAL CONFERENCE ON EMERGING DATA AND INDUSTRY 4.0 (EDI40) / AFFILIATED WORKSHOPS | 2021年 / 184卷

关键词：

Hadoop; big data; YARN; MapReduce; Resource Manager; IoT;

D O I：

10.1016/j.procs.2021.03.100

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Big Data is currently a fertile field for researchers and scientific companies around the world, due to the emergence of new technologies, Internet of Things (IoT) and means of communication such as social networking sites, which has led to a notable increase in the amount of data produced each day. In this paper, we discuss the challenges of Big Data and we are going to present the famous Apache Hadoop framework and we are going to test the latency at the level of execution of a MapReduce job based on YARN (Yet Another Resource Negotiator) by applying a Java wordcount application on different text files which resumes the sales of a store, which has prepared beforehand, finally going to make a synthesis of the results. (C) 2021 The Authors. Published by Elsevier B.V.

引用

页码：803 / 808

页数：6

共 50 条

[1] Hadoop-MapReduce Job Scheduling Algorithms Survey
Mohamed, Ehab
Hong, Zheng
2016 7TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA (CCBD), 2016, : 237 - 242
[2] Hadoop-MapReduce: A Platform for Mining Large Datasets
Afzali, Maedeh
Singh, Nishant
Kumar, Suresh
PROCEEDINGS OF THE 10TH INDIACOM - 2016 3RD INTERNATIONAL CONFERENCE ON COMPUTING FOR SUSTAINABLE GLOBAL DEVELOPMENT, 2016, : 1856 - 1860
[3] Performance Control for Nonlinear Hadoop-Mapreduce Computing Systems
Lei, Jing
Song, Jia-Qing
INTEGRATED FERROELECTRICS, 2023, 233 (01) : 148 - 159
[4] Hadoop-MapReduce下的PageRank矩阵分块算法
李远方
邓世昆
闻玉彪
韩月阳
计算机技术与发展, 2011, 21 (08) : 6 - 9+13
[5] Review and comparison of Apriori algorithm implementations on Hadoop-MapReduce and Spark
Castro, Eduardo P. S.
Maia, Thiago D.
Pereira, Marluce R.
Esmin, Ahmed A. A.
Pereira, Denilson A.
KNOWLEDGE ENGINEERING REVIEW, 2018, 33 : 1 - 25
[6] Models for Internal Clustering Validation Indexes Based on Hadoop-MapReduce
Zerabi, Soumeya
Meshoul, Souham
Boucherkha, Samia
INTERNATIONAL JOURNAL OF DISTRIBUTED SYSTEMS AND TECHNOLOGIES, 2020, 11 (03) : 42 - 67
[7] Study of Hadoop-MapReduce on Google N-Gram Datasets
Bhowmick, Satyajit
Chakraborty, Suryadip
Agrawal, Dharma P.
2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), 2015, : 488 - 490
[8] Analysis, Modeling, and Simulation of Hadoop YARN MapReduce
Bressoud, Thomas C.
Tang, Qiuyi
2016 IEEE 22ND INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS (ICPADS), 2016, : 980 - 988
[9] On MapReduce Scheduling in Hadoop Yarn on Heterogeneous Clusters
Wang, Meng
Wu, Chase Q.
Cao, Huiyan
Liu, Yang
Wang, Yonggiang
Hou, Aiqin
2018 17TH IEEE INTERNATIONAL CONFERENCE ON TRUST, SECURITY AND PRIVACY IN COMPUTING AND COMMUNICATIONS (IEEE TRUSTCOM) / 12TH IEEE INTERNATIONAL CONFERENCE ON BIG DATA SCIENCE AND ENGINEERING (IEEE BIGDATASE), 2018, : 1747 - 1754
[10] Results of a Model for Hadoop YARN MapReduce Tasks
Bressoud, Thomas C.
Tang, Qiuyi
2016 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2016, : 443 - 446

← 1 2 3 4 5 →