Service Function Chaining in LEO Satellite Networks via Multi-Agent Reinforcement Learning

被引:2
|
作者
Doan, Khai [1 ]
Avgeris, Marios [1 ]
Leivadeas, Aris [2 ,3 ]
Lambadaris, Ioannis [1 ]
Shin, Wonjae [3 ]
机构
[1] Carleton Univ, Dept Syst & Comp Engn, Ottawa, ON, Canada
[2] Ecole Technol Super, Dept Software & IT Engn, Montreal, PQ, Canada
[3] Korea Univ, Sch Elect Engn, Seoul, South Korea
关键词
Network Function Virtualization; Service Function Chaining; Satellite Networks; Multi-Agent Reinforcement Learning;
D O I
10.1109/GLOBECOM54140.2023.10437296
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Low-earth-orbit satellite networks (LSNs) offer an enhanced global connectivity and a wide range of applications such as disaster response and military operations, among others. Each specific application can be represented by a service function chain (SFC) in which each function is considered as a task in the application. Our objective is to optimize the long-term system performance by minimizing the average end-toend delay of SFC deployments in LSNs. To achieve this, we formulate a dynamic programming (DP) problem to derive an optimal placement policy. To overcome the computational intractability, the need for statistical knowledge of SFC requests, and centralized decision-making challenges, we present amulti-agent Q-learning approach where satellites act as independent agents. To facilitate performance convergence in non-stationary agents' environments, we let agents to collaborate by sharing designated learning parameters. In addition, agents update their Q-tables via two distinct rules depending on selected actions. Extensive experimentation shows that our approach achieves convergence and performance relatively close to the optimum obtained by solving the formulated DP equation.
引用
收藏
页码:7145 / 7150
页数:6
相关论文
共 50 条
  • [21] The Application of Multi-Agent Reinforcement Learning in UAV Networks
    Cui, Jingjing
    Liu, Yuanwei
    Nallanathan, Arumugam
    2019 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS WORKSHOPS (ICC WORKSHOPS), 2019,
  • [22] Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning
    Wang, Xin
    Zhao, Chen
    Huang, Tingwen
    Chakrabarti, Prasun
    Kurths, Juergen
    IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, 2023, 9 : 13 - 23
  • [23] Multi-Agent Deep Reinforcement Learning-Based Computation Offloading in LEO Satellite Edge Computing System
    Wu, Jian
    Jia, Min
    Zhang, Ningtao
    Guo, Qing
    IEEE COMMUNICATIONS LETTERS, 2024, 28 (10) : 2352 - 2356
  • [24] Emergent Social Learning via Multi-agent Reinforcement Learning
    Ndousse, Kamal
    Eck, Douglas
    Levine, Sergey
    Jaques, Natasha
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
  • [25] Intent-based multi-agent reinforcement learning for service assurance in cellular networks
    Perepu, Satheesh K.
    Martins, Jean P.
    Souza, Ricardo S.
    Dey, Kaushik
    2022 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM 2022), 2022, : 2879 - 2884
  • [26] Multi-Agent Image Classification via Reinforcement Learning
    Mousavi, Hossein K.
    Nazari, Mohammadreza
    Takac, Martin
    Motee, Nader
    2019 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2019, : 5020 - 5027
  • [27] Function approximation based multi-agent reinforcement learning
    Abul, O
    Polat, F
    Alhajj, R
    12TH IEEE INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2000, : 36 - 39
  • [28] Service Function Chain Deployment Algorithm Based on Multi-Agent Deep Reinforcement Learning
    Huang, Wanwei
    Zhang, Qiancheng
    Liu, Tao
    Xu, Yaoli
    Zhang, Dalei
    CMC-COMPUTERS MATERIALS & CONTINUA, 2024, 80 (03): : 4875 - 4893
  • [29] Batch Active Learning with Graph Neural Networks via Multi-Agent Deep Reinforcement Learning
    Zhang, Yuheng
    Tong, Hanghang
    Xia, Yinglong
    Zhu, Yan
    Chi, Yuejie
    Ying, Lei
    THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, : 9118 - 9126
  • [30] Multi-Agent Reinforcement Learning
    Stankovic, Milos
    2016 13TH SYMPOSIUM ON NEURAL NETWORKS AND APPLICATIONS (NEUREL), 2016, : 43 - 43