Spatiotemporal graph convolutional network for automated detection and analysis of social behaviours among pre-weaning piglets

In the pig industry, social behaviors of preweaning piglets are critical indicators of their livability, growth, health, and welfare status, for which there is an urgent need for using precision livestock farming tools. In this study, a novel method based on graph convolutional networks (GCNs) was developed to characterize preweaning piglet social behaviors such as snout–snout as well as snout-body social nosing and snout–snout as well as snout-body aggressive/playing behavior. Using an integrated CNN-based network, the proposed method first detected and tracked individual piglets. After that, a self-adaptive spatial affinity kernel function was used to detect suspected social behaviors and spatiotemporal graphs with high-quality node features (coordinates, node-estimation confidence, distance from each node to the centroid, and node motion) were built for pairwise piglets for further analysis. The spatiotemporal graph sequences were fed into a self-adaptive GCN combined with an attention mechanism to classify the suspected social behaviours. Our method performed well in detecting piglet social behaviours with a recall of 0.9405, a precision of 0.9669, and an F1 score of 0.9535. In an 8-h video episode, the time budges of snout–snout as well as snout-body social nosing and snout–snout as well as snout-body aggressive/playing behaviour were 33.82%, 38.34%, 13.74%, and 14.10%, respectively. The findings show that detecting body part-associated social behaviours in piglets using a GCN is feasible, yielding practical computer vision technologies for improved piglet behaviour monitoring and management.