Multistream Fusion Segmentation and Classification of Prostate Lesions from Magnetic Resonance Images

Prostate cancer is a significant contributor to cancer-related deaths in men. Detecting prostate cancer early can greatly increase the likelihood of successful treatment. However, detecting and assessing prostate lesions from multiparametric magnetic resonance images (MRI) is time-consuming and variable across radiologists with different levels of experience. We present an integrated framework for segmenting and classifying prostate lesions from MRI. The proposed approach is in contrast with most existing automated prostate analysis approaches, which treat segmentation and classification of prostate lesions as two separate tasks with no interactions between them. In the proposed framework, preliminary lesion boundaries were first segmented from T2-weighted (T2W) and diffusion-weighted images (DWI) by a three-stream network. The region of interest (ROI) enclosing the segmented lesion was fed to a weakly supervised classification network, which predicted the Gleason grade of the lesion and provided the class activation maps (CAMs) corresponding to multiple MRI modalities. Finally, MR images of different modalities with the corresponding CAMs were fed to a six-stream network to generate an enhanced lesion mask. Our experiments showed that CAMs generated by the proposed weakly supervised classifier improved segmentation performance. Our proposed method has a great potential to improve the accuracy and efficiency of prostate MRI interpretation workflow. © 2024 SPIE.