Frame-level Pain Intensity Assessment via Multilevel Hash-based Features and Transformer

Facial-expression-based pain assessment represents a promising computer vision application in medical diagnostics. However, improving the generalization capability of pain recognition models remains challenging due to the susceptibility of pain-related features to individual differences. The lack of large-scale pain-related datasets hinders the conventional approach relying on data diversity to enhance model generalization. Recent methods to solve this problem include feature descriptors for video classification and frame-level similarity calculation. The former captures better emotion-related features effectively, while the latter shows better generalization. We propose a two-stage framework combining the two strategies to address reference data limitations and ensure robust frame-level recognition. The initial stage introduces a simple but high-efficiency feature descriptor for emotion detail capture, which outputs a reference frame and an initial pain level score. In the second stage, we add a hash layer into the transformer and embed the transformer into the Siamese network to get the final pain scores by comparing the test frames with reference images obtained in the first stage. Experimental evaluations on UNBC McMaster and BioVid Heat Pain databases showcase our approach's state-of-the-art accuracy, robust generalization, and faster processing speed. © 2024 IEEE.