Improved Mask R-CNN with Attention U-Net Feature Extractor for Pronucleus Instance Segmentation in Fertilized Egg Embryo

The pronucleus is the nucleus that formed in the fertilized egg embryo during the early stage following the fusion of the oocyte and sperm nuclei. The presence of pronucleus is a crucial indicator of successful fertilization. Among assisted reproductive technology, the intelligent detection of the pronucleus is essential for assessing embryo quality and for subsequent clinical analysis. The minimal contrast between pronucleus and the cytoplasmic background, along with numerous extraneous artifacts, poses a challenge for pronucleus instance segmentation. This paper proposes an improved mask R-CNN with attention U-Net feature extractor (AUM-R-CNN) for pronucleus instance segmentation, featuring a cell-guided region attention U-Net and a context-driven object relation detection head. As a feature extractor, the improved U-Net architecture is enhanced by the cell-guided region attention branch. This feature extractor focuses on the cytoplasmic region to promote feature extraction of pronucleus. The object relation detection head performs joint inference on proposals, introducing contextual relational information. This enhances the semantic consistency pronuclear proposals features, reducing the interference of other target objects in the cytoplasm. Furthermore, data augmentation techniques for pronuclear stage embryo images address limited training data. Experimental results show that the proposed AUM-R-CNN can achieve better performance on pronucleus instance segmentation tasks in the embryo images than the existing state-of-the-art methods. Particularly, the AUM-R-CNN demonstrates superior performance in complex scenarios with overlapping pronucleus and dynamic cytoplasmic environments. © 2025 IEEE.