A Spark-based Task Allocation Solution for Machine Learning in the Edge-Cloud Continuum

被引:0
|
作者
Belcastro, Loris [1 ]
Marozzo, Fabrizio [1 ]
Presta, Aleandro [1 ]
Talia, Domenico [1 ]
机构
[1] Univ Calabria, DIMES, Arcavacata Di Rende, Italy
来源
2024 20TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING IN SMART SYSTEMS AND THE INTERNET OF THINGS, DCOSS-IOT 2024 | 2024年
关键词
Machine learning; Internet of Things; edge computing; cloud computing; edge-cloud continuum; Apache Spark;
D O I
10.1109/DCOSS-IoT61029.2024.00090
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
The widespread utilization of Internet of Things (IoT) devices has resulted in an exponential increase in data at the Internet's edges. This trend, combined with the rapid growth of machine learning (ML) applications, necessitates the execution of learning tasks across the entire spectrum of computing resources - from the device, to the edge, to the cloud. This paper investigates the execution of machine learning algorithms within the edge-cloud continuum, focusing on their implications from a distributed computing perspective. We explore the integration of traditional ML algorithms, leveraging edge computing benefits such as low-latency processing and privacy preservation, along with cloud computing capabilities offering virtually limitless computational and storage resources. Our analysis offers insights into optimizing the execution of machine learning applications by decomposing them into smaller components and distributing these across processing nodes in edge-cloud architectures. By utilizing the Apache Spark framework, we define an efficient task allocation solution for distributing ML tasks across edge and cloud layers. Experiments on a clustering application in an edgecloud setup confirm the effectiveness of our solution compared to highly centralized alternatives, in which cloud resources are extensively used for handling large volumes of data from IoT devices.
引用
收藏
页码:576 / 582
页数:7
相关论文
共 50 条
  • [31] A Smart Healthcare System for Remote Areas Based on the Edge-Cloud Continuum
    Gao, Xian
    He, Peixiong
    Zhou, Yi
    Qin, Xiao
    ELECTRONICS, 2024, 13 (21)
  • [32] Multi-Task Resource Allocation and Task Offloading via Multi-Agent Deep Reinforcement Learning in Edge-Cloud system
    Tian, Guoqing
    Wang, Xilong
    Li, Xin
    Qin, Xiaolin
    2024 5TH INTERNATIONAL CONFERENCE ON COMPUTING, NETWORKS AND INTERNET OF THINGS, CNIOT 2024, 2024, : 370 - 377
  • [33] Preemptive Scheduling for Distributed Machine Learning Jobs in Edge-Cloud Networks
    Wang, Ne
    Zhou, Ruiting
    Jiao, Lei
    Zhang, Renli
    Li, Bo
    Li, Zongpeng
    IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2022, 40 (08) : 2411 - 2425
  • [34] Architectural Vision for Quantum Computing in the Edge-Cloud Continuum
    Furutanpey, Alireza
    Barzen, Johanna
    Bechtold, Marvin
    Dustdar, Schahram
    Leymann, Frank
    Raith, Philipp
    Truger, Felix
    2023 IEEE INTERNATIONAL CONFERENCE ON QUANTUM SOFTWARE, QSW, 2023, : 88 - 103
  • [35] Hardware-Accelerated FaaS for the Edge-Cloud Continuum
    Nanos, Anastasios
    Kretsis, Aristotelis
    Mainas, Charalampos
    Ntouskos, George
    Ferikoglou, Aggelos
    Danopoulos, Dimitrios
    Kokkinis, Argyris
    Masouros, Dimosthenis
    Siozios, Kostas
    Soumplis, Polyzois
    Kokkinos, Panagiotis
    Olmos, Juan Jose Vegas
    Varvarigos, Emmanouel
    2023 IEEE 31ST INTERNATIONAL CONFERENCE ON NETWORK PROTOCOLS, ICNP, 2023,
  • [36] Demo: SmartWaste Disposal with Edge-Cloud Continuum Architecture
    Spillner, Josef
    Liu, Mengwei
    Zhan, Peng
    11TH INTERNATIONAL CONFERENCE ON THE INTERNET OF THINGS, IOT 2021, 2021, : 192 - 195
  • [37] Energy-Aware Workload Allocation for Distributed Deep Neural Networks in Edge-Cloud Continuum
    Jin, Yi
    Xu, Jiawei
    Huan, Yuxiang
    Yan, Yulong
    Zheng, Lirong
    Zou, Zhuo
    32ND IEEE INTERNATIONAL SYSTEM ON CHIP CONFERENCE (IEEE SOCC 2019), 2019, : 213 - 217
  • [38] QoS aware FaaS for Heterogeneous Edge-Cloud continuum
    Sheshadri, K. R.
    Lakshmi, J.
    2022 IEEE 15TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (IEEE CLOUD 2022), 2022, : 70 - 80
  • [39] Shepard: Dynamic Placement of Microservices in the Edge-Cloud Continuum
    Asghar, Farhan
    Fatima, Tehreem
    Siddiqui, Junaid Haroon
    Bhatti, Naveed Anwar
    Alizai, Muhammad Hamad
    MOBILE AND UBIQUITOUS SYSTEMS: COMPUTING, NETWORKING AND SERVICES, MOBIQUITOUS 2023, PT II, 2024, 594 : 43 - 62
  • [40] Towards a Reference Component Model of Edge-Cloud Continuum
    Khalyeyev, Danylo
    Bures, Tomas
    Hnetynka, Petr
    2023 IEEE 20TH INTERNATIONAL CONFERENCE ON SOFTWARE ARCHITECTURE COMPANION, ICSA-C, 2023, : 91 - 95