Prostate cancer segmentation from MRI by a multistream fusion encoder

Mingjie Jiang, Baohua Yuan, Weixuan Kou, Wen Yan, Harry Marshall, Qianye Yang, Tom Syer, Shonit Punwani, Mark Emberton, Dean C. Barratt, Carmen C. M. Cho, Yipeng Hu, Bernard Chiu*

*Corresponding author for this work

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

5 Citations (Scopus)

Abstract

Background: Targeted prostate biopsy guided by multiparametric magnetic resonance imaging (mpMRI) detects more clinically significant lesions than conventional systemic biopsy. Lesion segmentation is required for planning MRI-targeted biopsies. The requirement for integrating image features available in T2-weighted and diffusion-weighted images poses a challenge in prostate lesion segmentation from mpMRI. 

Purpose: A flexible and efficient multistream fusion encoder is proposed in this work to facilitate the multiscale fusion of features from multiple imaging streams. A patch-based loss function is introduced to improve the accuracy in segmenting small lesions. 

Methods: The proposed multistream encoder fuses features extracted in the three imaging streams at each layer of the network, thereby allowing improved feature maps to propagate downstream and benefit segmentation performance. The fusion is achieved through a spatial attention map generated by optimally weighting the contribution of the convolution outputs from each stream. This design provides flexibility for the network to highlight image modalities according to their relative influence on the segmentation performance. The encoder also performs multiscale integration by highlighting the input feature maps (low-level features) with the spatial attention maps generated from convolution outputs (high-level features). The Dice similarity coefficient (DSC), serving as a cost function, is less sensitive to incorrect segmentation for small lesions. We address this issue by introducing a patch-based loss function that provides an average of the DSCs obtained from local image patches. This local average DSC is equally sensitive to large and small lesions, as the patch-based DSCs associated with small and large lesions have equal weights in this average DSC. 

Results: The framework was evaluated in 931 sets of images acquired in several clinical studies at two centers in Hong Kong and the United Kingdom. In particular, the training, validation, and test sets contain 615, 144, and 172 sets of images, respectively. The proposed framework outperformed single-stream networks and three recently proposed multistream networks, attaining F1 scores of 82.2 and 87.6% in the lesion and patient levels, respectively. The average inference time for an axial image was 11.8 ms. 

Conclusion: The accuracy and efficiency afforded by the proposed framework would accelerate the MRI interpretation workflow of MRI-targeted biopsy and focal therapies. 

© 2023 American Association of Physicists in Medicine.

Original languageEnglish
Pages (from-to)5489-5504
JournalMedical Physics
Volume50
Issue number9
Online published20 Mar 2023
DOIs
Publication statusPublished - Sept 2023

Funding

This work is supported by the Innovation and Technology Commission of Hong Kong (Project No. ITS/387/18) and Research Grant Council of HKSAR, China (Project No. CityU 11205421). The article is related to a patent filed as a US nonprovisional patent titled “Multistream Fusion Encoder for Prostate Cancer Segmentation And Classification” (Inventors: B. Chiu, C.C.M. Cho, M. Jiang, and B. Yuan). Mark Emberton receives research support from the United Kingdom's National Institute of Health Research (NIHR) UCLH/UCL Biomedical Research Centre. He became an NIHR Senior Investigator in 2015. He acts as a Consultant/Trainer/Lecturer to the following companies: Sonacare Inc USA, Profound Medical Inc. Canada, Angiodynamics Inc. USA, NINA Medical Inc. Israel.

Research Keywords

  • multistream fusion encoder
  • patch-based loss function
  • prostate lesion segmentation

Fingerprint

Dive into the research topics of 'Prostate cancer segmentation from MRI by a multistream fusion encoder'. Together they form a unique fingerprint.

Cite this