Combiner and HyperCombiner networks : Rules to combine multimodality MR images for prostate cancer localisation

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

1 Scopus Citations
View graph of relations

Author(s)

  • Ziyi Shen
  • Qianye Yang
  • Tom Syer
  • Zhe Min
  • Shonit Punwani
  • Mark Emberton
  • David Atkinson
  • Dean C. Barratt
  • Yipeng Hu

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number103030
Journal / PublicationMedical Image Analysis
Volume91
Online published21 Nov 2023
Publication statusPublished - Jan 2024

Abstract

One of the distinct characteristics of radiologists reading multiparametric prostate MR scans, using reporting systems like PI-RADS v2.1, is to score individual types of MR modalities, including T2-weighted, diffusion-weighted, and dynamic contrast-enhanced, and then combine these image-modality-specific scores using standardised decision rules to predict the likelihood of clinically significant cancer. This work aims to demonstrate that it is feasible for low-dimensional parametric models to model such decision rules in the proposed Combiner networks, without compromising the accuracy of predicting radiologic labels. First, we demonstrate that either a linear mixture model or a nonlinear stacking model is sufficient to model PI-RADS decision rules for localising prostate cancer. Second, parameters of these combining models are proposed as hyperparameters, weighing independent representations of individual image modalities in the Combiner network training, as opposed to end-to-end modality ensemble. A HyperCombiner network is developed to train a single image segmentation network that can be conditioned on these hyperparameters during inference for much-improved efficiency. Experimental results based on 751 cases from 651 patients compare the proposed rule-modelling approaches with other commonly-adopted end-to-end networks, in this downstream application of automating radiologist labelling on multiparametric MR. By acquiring and interpreting the modality combining rules, specifically the linear-weights or odds ratios associated with individual image modalities, three clinical applications are quantitatively presented and contextualised in the prostate cancer segmentation application, including modality availability assessment, importance quantification and rule discovery. © 2023 Elsevier B.V.

Research Area(s)

  • Combiner networks, Decision rule modelling, Multiparametric MRI, Prostate cancer localisation

Citation Format(s)

Combiner and HyperCombiner networks: Rules to combine multimodality MR images for prostate cancer localisation. / Yan, Wen; Chiu, Bernard; Shen, Ziyi et al.
In: Medical Image Analysis, Vol. 91, 103030, 01.2024.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review