HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data

Xingjian Chen; Jiecong Lin; Yuchen Wang; Weitong Zhang; Weidun Xie; Zetian Zheng; Ka-Chun Wong

doi:10.1093/bioinformatics/btae343

HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data

Xingjian Chen, Jiecong Lin, Yuchen Wang, Weitong Zhang, Weidun Xie, Zetian Zheng, Ka-Chun Wong^*

^*Corresponding author for this work

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

2 Citations (Scopus)

Abstract

Motivation: Tissue context and molecular profiling are commonly used measures in understanding normal development and disease pathology. In recent years, the development of spatial molecular profiling technologies (e.g. spatial resolved transcriptomics) has enabled the exploration of quantitative links between tissue morphology and gene expression. However, these technologies remain expensive and time-consuming, with subsequent analyses necessitating high-throughput pathological annotations. On the other hand, existing computational tools are limited to predicting only a few dozen to several hundred genes, and the majority of the methods are designed for bulk RNA-seq.

Results: In this context, we propose HE2Gene, the first multi-task learning-based method capable of predicting tens of thousands of spot-level gene expressions along with pathological annotations from H&E-stained images. Experimental results demonstrate that HE2Gene is comparable to state-of-the-art methods and generalizes well on an external dataset without the need for re-training. Moreover, HE2Gene preserves the annotated spatial domains and has the potential to identify biomarkers. This capability facilitates cancer diagnosis and broadens its applicability to investigate gene-disease associations.

Availability and implementation: The source code and data information has been deposited at https://github.com/Microbiods/HE2Gene.

Original language	English
Article number	btae343
Journal	Bioinformatics
Volume	40
Issue number	6
Online published	5 Jun 2024
DOIs	https://doi.org/10.1093/bioinformatics/btae343
Publication status	Published - Jun 2024

Funding

This research was substantially sponsored by the research project (Grant No. 32170654) supported by the National Natural Science Foundation of China and was substantially supported by the Shenzhen Research Institute, City University of Hong Kong. The work described in this paper was substantially supported by the grant from the Research Grants Council of the Hong Kong Special Administrative Region [CityU 11203723]. The work described in this paper was partially supported by the grants from City University of Hong Kong (2021SIRG036, CityU 9667265, CityU 11203221) and Innovation and Technology Commission (ITB/FBL/9037/22/S).

Research Keywords

Humans
Transcriptome
Gene Expression Profiling/methods
Computational Biology/methods
Machine Learning
RNA/metabolism

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

RGC Funding Information

RGC-funded

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access Output

10.1093/bioinformatics/btae343

223773109.pdfFinal Published version, 3.1 MBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@article{1e88733ebc5b42a1a29d786b9617da66,

title = "HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data",

abstract = "Motivation: Tissue context and molecular profiling are commonly used measures in understanding normal development and disease pathology. In recent years, the development of spatial molecular profiling technologies (e.g. spatial resolved transcriptomics) has enabled the exploration of quantitative links between tissue morphology and gene expression. However, these technologies remain expensive and time-consuming, with subsequent analyses necessitating high-throughput pathological annotations. On the other hand, existing computational tools are limited to predicting only a few dozen to several hundred genes, and the majority of the methods are designed for bulk RNA-seq.Results: In this context, we propose HE2Gene, the first multi-task learning-based method capable of predicting tens of thousands of spot-level gene expressions along with pathological annotations from H&E-stained images. Experimental results demonstrate that HE2Gene is comparable to state-of-the-art methods and generalizes well on an external dataset without the need for re-training. Moreover, HE2Gene preserves the annotated spatial domains and has the potential to identify biomarkers. This capability facilitates cancer diagnosis and broadens its applicability to investigate gene-disease associations.Availability and implementation: The source code and data information has been deposited at https://github.com/Microbiods/HE2Gene.{\textcopyright} The Author(s) 2024. Published by Oxford University Press. ",

keywords = "Humans, Transcriptome, Gene Expression Profiling/methods, Computational Biology/methods, Machine Learning, RNA/metabolism",

author = "Xingjian Chen and Jiecong Lin and Yuchen Wang and Weitong Zhang and Weidun Xie and Zetian Zheng and Ka-Chun Wong",

year = "2024",

month = jun,

doi = "10.1093/bioinformatics/btae343",

language = "English",

volume = "40",

journal = "Bioinformatics",

issn = "1367-4803",

publisher = "Oxford University Press",

number = "6",

}

TY - JOUR

T1 - HE2Gene

T2 - image-to-RNA translation via multi-task learning for spatial transcriptomics data

AU - Chen, Xingjian

AU - Lin, Jiecong

AU - Wang, Yuchen

AU - Zhang, Weitong

AU - Xie, Weidun

AU - Zheng, Zetian

AU - Wong, Ka-Chun

PY - 2024/6

Y1 - 2024/6

N2 - Motivation: Tissue context and molecular profiling are commonly used measures in understanding normal development and disease pathology. In recent years, the development of spatial molecular profiling technologies (e.g. spatial resolved transcriptomics) has enabled the exploration of quantitative links between tissue morphology and gene expression. However, these technologies remain expensive and time-consuming, with subsequent analyses necessitating high-throughput pathological annotations. On the other hand, existing computational tools are limited to predicting only a few dozen to several hundred genes, and the majority of the methods are designed for bulk RNA-seq.Results: In this context, we propose HE2Gene, the first multi-task learning-based method capable of predicting tens of thousands of spot-level gene expressions along with pathological annotations from H&E-stained images. Experimental results demonstrate that HE2Gene is comparable to state-of-the-art methods and generalizes well on an external dataset without the need for re-training. Moreover, HE2Gene preserves the annotated spatial domains and has the potential to identify biomarkers. This capability facilitates cancer diagnosis and broadens its applicability to investigate gene-disease associations.Availability and implementation: The source code and data information has been deposited at https://github.com/Microbiods/HE2Gene.© The Author(s) 2024. Published by Oxford University Press.

AB - Motivation: Tissue context and molecular profiling are commonly used measures in understanding normal development and disease pathology. In recent years, the development of spatial molecular profiling technologies (e.g. spatial resolved transcriptomics) has enabled the exploration of quantitative links between tissue morphology and gene expression. However, these technologies remain expensive and time-consuming, with subsequent analyses necessitating high-throughput pathological annotations. On the other hand, existing computational tools are limited to predicting only a few dozen to several hundred genes, and the majority of the methods are designed for bulk RNA-seq.Results: In this context, we propose HE2Gene, the first multi-task learning-based method capable of predicting tens of thousands of spot-level gene expressions along with pathological annotations from H&E-stained images. Experimental results demonstrate that HE2Gene is comparable to state-of-the-art methods and generalizes well on an external dataset without the need for re-training. Moreover, HE2Gene preserves the annotated spatial domains and has the potential to identify biomarkers. This capability facilitates cancer diagnosis and broadens its applicability to investigate gene-disease associations.Availability and implementation: The source code and data information has been deposited at https://github.com/Microbiods/HE2Gene.© The Author(s) 2024. Published by Oxford University Press.

KW - Humans

KW - Transcriptome

KW - Gene Expression Profiling/methods

KW - Computational Biology/methods

KW - Machine Learning

KW - RNA/metabolism

UR - http://www.scopus.com/inward/record.url?scp=85195628822&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85195628822&origin=recordpage

U2 - 10.1093/bioinformatics/btae343

DO - 10.1093/bioinformatics/btae343

M3 - RGC 21 - Publication in refereed journal

C2 - 38837395

SN - 1367-4803

VL - 40

JO - Bioinformatics

JF - Bioinformatics

IS - 6

M1 - btae343

ER -

HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data

Abstract

Funding

Research Keywords

Publisher's Copyright Statement

RGC Funding Information

UN SDGs

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: DNA Motif Knowledge Extraction and Distillation from Big Deep Learning Models in Regulatory Genomics

Cite this

HE2Gene: image-to-RNA translation via multi-task learning for spatial transcriptomics data

Abstract

Funding

Research Keywords

Publisher's Copyright Statement

RGC Funding Information

UN SDGs

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: DNA Motif Knowledge Extraction and Distillation from Big Deep Learning Models in Regulatory Genomics

Cite this