Twin-Delayed DDPG-Based Multiuser Downlink Transmissions for RIS-Aided Wireless Communications

Reconfigurable intelligent surface (RIS) can offer a customizable wireless transmission and is regarded as an incredibly crucial enabling technology when used as reflectors for current wireless base stations (BSs) to overcome the blockage challenges of millimeter-wave (mmWave) wireless communications systems. RISs are instrumental in driving the progress of sixth-generation (6G) wireless services and beyond, with the goal of achieving gigabit-per-second (Gbps) data rates for networks operating in the mmWave frequency bands. RIS has the significant potential to diminish the effects of signal blockages and unnecessary handovers because precise phase adjustment of RIS elements enhances the scattering environments and creates multiple reflective signal paths. However, the large number of RIS elements can complicate the optimization of BS and RIS reflector configurations and result in performance degradation. This article introduces a deep reinforcement learning (DRL) strategy to dynamically configure the numerous RIS elements for multiuser downlink transmissions. The proposed method employs the twin-delayed deep deterministic policy gradient (TD3) method to solve nonconvex optimization issues where the BS receives state information from the RIS, including feedback on user channel states. For real-world systems requiring collaborative control over phase shift and beamforming matrix, the BS determines the optimal actions, which comprise the allocation of transmission power and phase shift adjustments within a Nakagami-m fading environment. The experimental results indicate that the proposed TD3-based solution outperforms other existing benchmarks.