A Residual Attention Root-MUSIC Network-Based DOA Estimator for Mixed Circular and Non-Circular Signals

A joint source enumeration and direction-of-arrival (DOA) estimation method for mixed circular and non-circular signals based on a residual attention root-MUSIC network (RARN) is proposed by leveraging both the conjugated covariance and unconjugated covariance matrices of mixed signals with the circularity difference. The RARN consists of two sub-networks: Source Enumeration Network (SEN) and Angle Estimation Network (AEN). The SEN adopts a dual-branch structure and attention mechanisms to extract distinct features from both types of signals, ensuring the accuracy of multi-task learning. Additionally, using residual connections increases the depth of the network and improves its generalization ability. Meanwhile, AEN includes the conjugated covariance and unconjugated covariance matrix reconstruction blocks with the prior source number estimates from SEN, and utilizes the root-MUSIC block for DOA estimation. Simulation results show that RARN outperforms state-of-the-art algorithms in estimation accuracy and generalization ability, especially in scenarios with low signal-to-noise ratio or small number of snapshots. © 2026 IEEE.