Robust DOA Estimation against Mutual Coupling with Nested Array

In this letter, we focus on the problem of direction-of-arrival (DOA) estimation with a nested array in the presence of unknown mutual coupling. Firstly, the measurement model of the nested array with mutual coupling is established, and the properties of the corresponding array covariance matrix are derived. Based on the obtained properties, the coarray output signal is then reconstructed with reduced mutual coupling. Finally, joint sparse recovery technique is employed to extract the DOAs. Our algorithm can achieve underdetermined DOA estimation with satisfactory performance under unknown mutual coupling. Numerical results demonstrate the effectiveness of the proposed algorithm.