scGREAT: Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics

Yuchen Wang; Xingjian Chen; Zetian Zheng; Lei Huang; Weidun Xie; Fuzhou Wang; Zhaolei Zhang; Ka-Chun Wong

doi:10.1016/j.isci.2024.109352

scGREAT: Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics

Yuchen Wang
, Xingjian Chen
, Zetian Zheng
, Lei Huang
, Weidun Xie
, Fuzhou Wang
, Zhaolei Zhang
, Ka-Chun Wong^*

^*Corresponding author for this work

Department of Computer Science

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

22 Citations (Scopus)

27 Downloads (CityUHK Scholars)

Abstract

Gene regulatory networks (GRNs) involve complex and multi-layer regulatory interactions between regulators and their target genes. Precise knowledge of GRNs is important in understanding cellular processes and molecular functions.Recent breakthroughs in single-cell sequencing technology made it possible to infer GRNs at single-cell level. Existing methods, however, are limited by expensive computations, and sometimes simplistic assumptions. To overcome these obstacles, we propose scGREAT, an framework to infer GRN using gene Embeddings And Transformer from single cell transcriptomics. scGREAT starts by constructing gene expression and gene biotext dictionaries from scRNA-seq data and gene text information. The representation of TF gene pairs is learned through optimizing embedding spaceby transformer-based engine. Results illustrated scGREAT outperformed other contemporary methods on benchmarks.Besides, gene representations from scGREAT provide valuable gene regulation insights, and external validation on spatial transcriptomics illuminated the mechanism behind scGREAT annotation. Moreover, scGREAT identified several TFtarget regulations corroborated in studies.

© 2024 The Authors.

Original language	English
Article number	109352
Journal	iScience
Volume	27
Issue number	4
Online published	28 Feb 2024
DOIs	https://doi.org/10.1016/j.isci.2024.109352
Publication status	Published - 19 Apr 2024

Funding

This research was substantially sponsored by the research projects (Grant No. 32170654 and Grant No. 32000464) supported by the National Natural Science Foundation of China and was substantially supported by the Shenzhen Research Institute, City University of Hong Kong. The work described in this paper was substantially supported by the grant from the Research Grants Council of the Hong Kong Special Administrative Region [CityU 11203723]. This project was substantially funded by the Strategic Interdisciplinary Research Grant of City University of Hong Kong (Project No. 2021SIRG036). The work described in this paper was partially supported by the grant from City University of Hong Kong (CityU 9667265). The three anonymous reviewers are thanked for their time and efforts, improving numerous aspects of the current study.

Publisher's Copyright Statement

This full text is made available under CC-BY-NC-ND 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/

RGC Funding Information

RGC-funded

Access Output

10.1016/j.isci.2024.109352

186577789.pdfFinal Published version, 5.46 MBLicence: CC BY-NC-ND

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

GRF: DNA Motif Knowledge Extraction and Distillation from Big Deep Learning Models in Regulatory Genomics
WONG, K. C. (Principal Investigator / Project Coordinator)
1/01/24 → …
Project: Research

Cite this

@article{485211155bf84f009f020d28d33a8bd5,

title = "scGREAT: Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics",

abstract = "Gene regulatory networks (GRNs) involve complex and multi-layer regulatory interactions between regulators and their target genes. Precise knowledge of GRNs is important in understanding cellular processes and molecular functions.Recent breakthroughs in single-cell sequencing technology made it possible to infer GRNs at single-cell level. Existing methods, however, are limited by expensive computations, and sometimes simplistic assumptions. To overcome these obstacles, we propose scGREAT, an framework to infer GRN using gene Embeddings And Transformer from single cell transcriptomics. scGREAT starts by constructing gene expression and gene biotext dictionaries from scRNA-seq data and gene text information. The representation of TF gene pairs is learned through optimizing embedding spaceby transformer-based engine. Results illustrated scGREAT outperformed other contemporary methods on benchmarks.Besides, gene representations from scGREAT provide valuable gene regulation insights, and external validation on spatial transcriptomics illuminated the mechanism behind scGREAT annotation. Moreover, scGREAT identified several TFtarget regulations corroborated in studies. {\textcopyright} 2024 The Authors.",

author = "Yuchen Wang and Xingjian Chen and Zetian Zheng and Lei Huang and Weidun Xie and Fuzhou Wang and Zhaolei Zhang and Ka-Chun Wong",

year = "2024",

month = apr,

day = "19",

doi = "10.1016/j.isci.2024.109352",

language = "English",

volume = "27",

number = "4",

}

TY - JOUR

T1 - scGREAT

T2 - Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics

AU - Wang, Yuchen

AU - Chen, Xingjian

AU - Zheng, Zetian

AU - Huang, Lei

AU - Xie, Weidun

AU - Wang, Fuzhou

AU - Zhang, Zhaolei

AU - Wong, Ka-Chun

PY - 2024/4/19

Y1 - 2024/4/19

N2 - Gene regulatory networks (GRNs) involve complex and multi-layer regulatory interactions between regulators and their target genes. Precise knowledge of GRNs is important in understanding cellular processes and molecular functions.Recent breakthroughs in single-cell sequencing technology made it possible to infer GRNs at single-cell level. Existing methods, however, are limited by expensive computations, and sometimes simplistic assumptions. To overcome these obstacles, we propose scGREAT, an framework to infer GRN using gene Embeddings And Transformer from single cell transcriptomics. scGREAT starts by constructing gene expression and gene biotext dictionaries from scRNA-seq data and gene text information. The representation of TF gene pairs is learned through optimizing embedding spaceby transformer-based engine. Results illustrated scGREAT outperformed other contemporary methods on benchmarks.Besides, gene representations from scGREAT provide valuable gene regulation insights, and external validation on spatial transcriptomics illuminated the mechanism behind scGREAT annotation. Moreover, scGREAT identified several TFtarget regulations corroborated in studies. © 2024 The Authors.

AB - Gene regulatory networks (GRNs) involve complex and multi-layer regulatory interactions between regulators and their target genes. Precise knowledge of GRNs is important in understanding cellular processes and molecular functions.Recent breakthroughs in single-cell sequencing technology made it possible to infer GRNs at single-cell level. Existing methods, however, are limited by expensive computations, and sometimes simplistic assumptions. To overcome these obstacles, we propose scGREAT, an framework to infer GRN using gene Embeddings And Transformer from single cell transcriptomics. scGREAT starts by constructing gene expression and gene biotext dictionaries from scRNA-seq data and gene text information. The representation of TF gene pairs is learned through optimizing embedding spaceby transformer-based engine. Results illustrated scGREAT outperformed other contemporary methods on benchmarks.Besides, gene representations from scGREAT provide valuable gene regulation insights, and external validation on spatial transcriptomics illuminated the mechanism behind scGREAT annotation. Moreover, scGREAT identified several TFtarget regulations corroborated in studies. © 2024 The Authors.

U2 - 10.1016/j.isci.2024.109352

DO - 10.1016/j.isci.2024.109352

M3 - RGC 21 - Publication in refereed journal

C2 - 38510148

SN - 2589-0042

VL - 27

JO - iScience

JF - iScience

IS - 4

M1 - 109352

ER -

scGREAT: Transformer-Based Deep-Language Model for Gene Regulatory Network Inference from Single-Cell Transcriptomics

Abstract

Funding

Publisher's Copyright Statement

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Fingerprint

Projects

GRF: DNA Motif Knowledge Extraction and Distillation from Big Deep Learning Models in Regulatory Genomics

Cite this