BDPS: An Efficient Spark-Based Big Data Processing Scheme for Cloud Fog-IoT Orchestration

被引:12
|
作者
Hossen, Rakib [1 ]
Whaiduzzaman, Md [2 ,3 ]
Uddin, Mohammed Nasir [1 ]
Islam, Md. Jahidul [1 ]
Faruqui, Nuruzzaman [2 ]
Barros, Alistair [3 ]
Sookhak, Mehdi [4 ]
Mahi, Md. Julkar Nayeen [2 ]
机构
[1] Jagannath Univ, Dept Comp Sci & Engn, Dhaka 1100, Bangladesh
[2] Jahangirnagar Univ, Inst Informat Technol, Dhaka 1342, Bangladesh
[3] Queensland Univ Technol, Sch Informat Syst, Brisbane, Qld 4000, Australia
[4] Texas A&M Univ, Dept Comp Sci, Corpus Christ, TX 78412 USA
基金
澳大利亚研究理事会;
关键词
efficient data processing; depth-first search; map reduction; in-memory accelerator; spark; EDGE; ALGORITHM; INTERNET; DELAY;
D O I
10.3390/info12120517
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The Internet of Things (IoT) has seen a surge in mobile devices with the market and technical expansion. IoT networks provide end-to-end connectivity while keeping minimal latency. To reduce delays, efficient data delivery schemes are required for dispersed fog-IoT network orchestrations. We use a Spark-based big data processing scheme (BDPS) to accelerate the distributed database (RDD) delay efficient technique in the fogs for a decentralized heterogeneous network architecture to reinforce suitable data allocations via IoTs. We propose BDPS based on Spark-RDD in fog-IoT overlay architecture to address the performance issues across the network orchestration. We evaluate data processing delays from fog-IoT integrated parts using a depth-first-search-based shortest path node finding configuration, which outperforms the existing shortest path algorithms in terms of algorithmic (i.e., depth-first search) efficiency, including the Bellman-Ford (BF) algorithm, Floyd-Warshall (FW) algorithm, Dijkstra algorithm (DA), and Apache Hadoop (AH) algorithm. The BDPS exhibits low latency in packet deliveries as well as low network overhead uplink activity through a map-reduced resilient data distribution mechanism, better than in BF, DA, FW, and AH. The overall BDPS scheme supports efficient data delivery across the fog-IoT orchestration, outperforming faster node execution while proving effective results, compared to DA, BF, FW and AH, respectively.
引用
收藏
页数:22
相关论文
共 50 条
  • [21] Spark-based Cloud Data Analytics using Multi-Objective Optimization
    Song, Fei
    Zaouk, Khaled
    Lyu, Chenghao
    Sinha, Arnab
    Fan, Qi
    Diao, Yanlei
    Shenoy, Prashant
    2021 IEEE 37TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE 2021), 2021, : 396 - 407
  • [22] Spark-Based Log Data Analysis for Reconstruction of Cybercrime Events in Cloud Environment
    Hemdan, Ezz El-Din
    Manjaiah, D. H.
    PROCEEDINGS OF 2017 IEEE INTERNATIONAL CONFERENCE ON CIRCUIT ,POWER AND COMPUTING TECHNOLOGIES (ICCPCT), 2017,
  • [23] A Spark-based Analytic Pipeline for Seizure Detection in EEG Big Data Streams
    Sendi, Mohammad S. E.
    Heydarzadeh, Mehrdad
    Mahmoudi, Babak
    2018 40TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC), 2018, : 4003 - 4006
  • [24] Reducing IoT Big Data for Efficient Storage and Processing
    Katsarou, Eleftheria
    Hadjiefthymiades, Stathes
    PROCEEDINGS OF THE 8TH INTERNATIONAL CONFERENCE ON INTERNET OF THINGS, BIG DATA AND SECURITY, IOTBDS 2023, 2023, : 226 - 230
  • [25] A spatiotemporal compression based approach for efficient big data processing on Cloud
    Yang, Chi
    Zhang, Xuyun
    Zhong, Changmin
    Liu, Chang
    Pei, Jian
    Ramamohanarao, Kotagiri
    Chen, Jinjun
    JOURNAL OF COMPUTER AND SYSTEM SCIENCES, 2014, 80 (08) : 1563 - 1583
  • [26] A collaborative resource management for big IoT data processing in Cloud
    Alelaiwi, Abdulhameed
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2017, 20 (02): : 1791 - 1799
  • [27] A collaborative resource management for big IoT data processing in Cloud
    Abdulhameed Alelaiwi
    Cluster Computing, 2017, 20 : 1791 - 1799
  • [28] Fog Computing Based Efficient IoT Scheme for the Industry 4.0
    Peralta, Goiuri
    Iglesias-Urkia, Markel
    Barcelo, Marc
    Gomez, Raul
    Moran, Adrian
    Bilbao, Josu
    2017 IEEE INTERNATIONAL WORKSHOP OF ELECTRONICS, CONTROL, MEASUREMENT, SIGNALS AND THEIR APPLICATION TO MECHATRONICS (ECMSM), 2017,
  • [29] Fog-Based Smart Healthcare as a Big Data and Cloud Service for Heart Patients Using IoT
    Gill, Sukhpal Singh
    Arya, Rajesh Chand
    Wander, Gurpreet Singh
    Buyya, Rajkumar
    INTERNATIONAL CONFERENCE ON INTELLIGENT DATA COMMUNICATION TECHNOLOGIES AND INTERNET OF THINGS, ICICI 2018, 2019, 26 : 1376 - 1383
  • [30] CF-CloudOrch: container fog node-based cloud orchestration for IoT networks
    Kim, Nam Yong
    Ryu, Jung Hyun
    Kwon, Byoung Wook
    Pan, Yi
    Park, Jong Hyuk
    JOURNAL OF SUPERCOMPUTING, 2018, 74 (12): : 7024 - 7045