Enhancing Spatio-Temporal Auditory Attention Decoding with ST-AADNet

Individuals with hearing impairments often struggle to isolate and focus on a single speaker in multi-speaker environments. Neuroscience research has uncovered distinct patterns of brain activity associated with auditory attention that can be detected using electroencephalography (EEG) measurements. Existing deep learning methods developed for AAD encounter challenges in extracting effective spatial-temporal features and handling the data scarcity issues. To address these issues, this work proposes a novel neural network architecture named ST-AADNet, which integrates a convolutional neural network and a long short-term memory network to extract useful spatial-temporal features from EEG signals. Furthermore, we introduce a series of data augmentation methods tailored to enhance the model’s generalization capacity. Experimental studies on both audio-only and audio-video datasets demonstrate the superior performance of the proposed methods, securing the second position in The First Chinese Auditory Attention Decoding Challenge. © 2024 IEEE.