The DOA Estimation Algorithm Based on Virtual Domain Block Matching for Nested Arrays in the Presence of Mutual Coupling

Mutual coupling severely impact Direction of Arrival (DOA) estimation, and the existence of mutual coupling can lead to significant errors in the received signal model of the array. Using algorithms to reduce it is an effective approach. Few algorithms are suitable for nested arrays, and due to incomplete elimination or the loss of some data, they performance deteriorates in the small number of snapshot and low signal-to-noise ratio environments. To address these issues, this paper proposes a Virtual Domain Block Matching DOA estimation algorithm for nested arrays in mutual coupling. The algorithm begins by partitioning the mutual coupling matrix of nested arrays, then applies a classical separation method used in Uniform Linear Array to obtain the separation parameters. Next, it reconstructs an equivalent single-snapshot received data vector in the virtual domain. A sparse reconstruction optimization model is then formulated by exploiting signal sparsity and virtual domain block matching. Solving this model provides a block representation of DOA estimation, where the $l_{2}$ norm of each block yields the DOA estimation for the actual incident signals. Extensive simulation results demonstrate that this algorithm fully utilizes the high degrees of freedom of the nested arrays to achieve underestimation under unknown mutual coupling and maintains high accuracy in small snapshot and low SNR. Additionally, the algorithm is not sensitive to the grid's size. The experimental results show that this algorithm outperforms several existing methods.