Cross-Modality Interaction Network for Equine Activity Recognition Using Imbalanced Multi-Modal Data
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Article number | 5818 |
Journal / Publication | Sensors |
Volume | 21 |
Issue number | 17 |
Online published | 29 Aug 2021 |
Publication status | Published - Sept 2021 |
Link(s)
DOI | DOI |
---|---|
Attachment(s) | Documents
Publisher's Copyright Statement
|
Link to Scopus | https://www.scopus.com/record/display.uri?eid=2-s2.0-85113827946&origin=recordpage |
Permanent Link | https://scholars.cityu.edu.hk/en/publications/publication(d2214f9d-99e4-46c4-904c-24521a5025e1).html |
Abstract
With the recent advances in deep learning, wearable sensors have increasingly been used in automated animal activity recognition. However, there are two major challenges in improving recognition performance—multi-modal feature fusion and imbalanced data modeling. In this study, to improve classification performance for equine activities while tackling these two challenges, we developed a cross-modality interaction network (CMI-Net) involving a dual convolution neural network architecture and a cross-modality interaction module (CMIM). The CMIM adaptively recalibrated the temporal- and axis-wise features in each modality by leveraging multi-modal information to achieve deep intermodality interaction. A class-balanced (CB) focal loss was adopted to supervise the training of CMI-Net to alleviate the class imbalance problem. Motion data was acquired from six neck-attached inertial measurement units from six horses. The CMI-Net was trained and verified with leave-one-out cross-validation. The results demonstrated that our CMI-Net outperformed the existing algorithms with high precision (79.74%), recall (79.57%), F1-score (79.02%), and accuracy (93.37%). The adoption of CB focal loss improved the performance of CMI-Net, with increases of 2.76%, 4.16%, and 3.92% in precision, recall, and F1-score, respectively. In conclusion, CMI-Net and CB focal loss effectively enhanced the equine activity classification performance using imbalanced multi-modal sensor data.
Research Area(s)
- equine behavior, wearable sensor, deep learning, intermodality interaction, class-balanced focal loss
Citation Format(s)
Cross-Modality Interaction Network for Equine Activity Recognition Using Imbalanced Multi-Modal Data. / Mao, Axiu; Huang, Endai; Gan, Haiming et al.
In: Sensors, Vol. 21, No. 17, 5818, 09.2021.
In: Sensors, Vol. 21, No. 17, 5818, 09.2021.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Download Statistics
No data available