Deep Reinforcement Learning-Based Resource Allocation for Integrated Sensing, Communication, and Computation in Vehicular Network

In developing the sixth-generation (6G) system, integrated sensing and communication technology is becoming increasingly essential, especially for applications like autonomous driving. This paper develops an architecture for integrated sensing, communication, and computation (ISCC) in the vehicular network, where vehicles perform environment sensing, sensing data computation, and transmission. To support low-latency cooperation between vehicles and extend vehicles' sensing range, over-air-computation federated learning is employed. The optimization problem of joint beamforming design and power resource allocation in the ISCC scenario is formulated to maximize the achievable data rate while ensuring sensing and computing performance. However, solving this joint optimization problem is a great challenge due to the high coupling resource and time-varying channel environment. Therefore, a hybrid reinforcement learning scheme is proposed in this work. First, the semidefinite relaxation and Gaussian randomization techniques are leveraged to obtain the approximate solution of the aggregation beamformer. Then, the deep deterministic policy gradient algorithm is proposed to tackle the transmit beamforming design and resource allocation problem in continuous action space. Extensive simulation results validated the admirable performance of the proposed scheme in convergence and achievable sum rate compared with the benchmark schemes. In addition, the impact of variables on the optimization performance is demonstrated via numerical results.