FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems

Hu Wan; Xuan Sun; Yufei Cui; Chia-Lin Yang; Tei-Wei Kuo; Chun Jason Xue

doi:10.1145/3476886.3477511

FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems

Hu Wan
, Xuan Sun
, Yufei Cui
, Chia-Lin Yang
, Tei-Wei Kuo
, Chun Jason Xue

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

19 Citations (Scopus)

Abstract

We present FlashEmbedding, a hardware/software co-design solution for storing embedding tables on SSDs for large-scale recommendation inference under memory capacity-limited systems. FlashEmbedding leverages an embedding semantic-aware SSD, an embedding-oriented software cache, and pipeline techniques to improve the overall performance. We evaluate the performance of FlashEmbedding with our FPGA-based prototype SSD on a real-world public dataset. FlashEmbedding achieves up to 17.44× lower latency in embedding lookups and 2.89× lower end-to-end latency than baseline solution in a memory capacity-limted system.

Original language	English
Title of host publication	APSys '21
Subtitle of host publication	Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems
Publisher	Association for Computing Machinery
Pages	9-16
ISBN (Print)	9781450386982
DOIs	https://doi.org/10.1145/3476886.3477511
Publication status	Published - 2021
Event	12th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys 2021 - Virtual, Hong Kong, China Duration: 24 Aug 2021 → 25 Aug 2021

Publication series

Name	APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems

Conference

Conference	12th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys 2021
Place	Hong Kong, China
Period	24/08/21 → 25/08/21

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

Embedding
Recommender systems
Solid-state drive (SSD)

Access Output

10.1145/3476886.3477511

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Wan, H., Sun, X., Cui, Y., Yang, C.-L., Kuo, T.-W., & Xue, C. J. (2021). FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems. In APSys '21: Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems (pp. 9-16). (APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems). Association for Computing Machinery. https://doi.org/10.1145/3476886.3477511

@inproceedings{63aedf77d5df4fab874d099ce7deee7e,

title = "FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems",

abstract = "We present FlashEmbedding, a hardware/software co-design solution for storing embedding tables on SSDs for large-scale recommendation inference under memory capacity-limited systems. FlashEmbedding leverages an embedding semantic-aware SSD, an embedding-oriented software cache, and pipeline techniques to improve the overall performance. We evaluate the performance of FlashEmbedding with our FPGA-based prototype SSD on a real-world public dataset. FlashEmbedding achieves up to 17.44× lower latency in embedding lookups and 2.89× lower end-to-end latency than baseline solution in a memory capacity-limted system.",

keywords = "Embedding, Recommender systems, Solid-state drive (SSD)",

author = "Hu Wan and Xuan Sun and Yufei Cui and Chia-Lin Yang and Tei-Wei Kuo and Xue, \{Chun Jason\}",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; 12th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys 2021 ; Conference date: 24-08-2021 Through 25-08-2021",

year = "2021",

doi = "10.1145/3476886.3477511",

language = "English",

isbn = "9781450386982",

series = "APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems",

publisher = "Association for Computing Machinery",

pages = "9--16",

booktitle = "APSys '21",

address = "United States",

}

Wan, H , Sun, X , Cui, Y, Yang, C-L, Kuo, T-W & Xue, CJ 2021, FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems. in APSys '21: Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems. APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems, Association for Computing Machinery, pp. 9-16, 12th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys 2021, Hong Kong, China, 24/08/21. https://doi.org/10.1145/3476886.3477511

FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems. / Wan, Hu ; Sun, Xuan ; Cui, Yufei et al.
APSys '21: Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems. Association for Computing Machinery, 2021. p. 9-16 (APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - FlashEmbedding

T2 - 12th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys 2021

AU - Wan, Hu

AU - Sun, Xuan

AU - Cui, Yufei

AU - Yang, Chia-Lin

AU - Kuo, Tei-Wei

AU - Xue, Chun Jason

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2021

Y1 - 2021

N2 - We present FlashEmbedding, a hardware/software co-design solution for storing embedding tables on SSDs for large-scale recommendation inference under memory capacity-limited systems. FlashEmbedding leverages an embedding semantic-aware SSD, an embedding-oriented software cache, and pipeline techniques to improve the overall performance. We evaluate the performance of FlashEmbedding with our FPGA-based prototype SSD on a real-world public dataset. FlashEmbedding achieves up to 17.44× lower latency in embedding lookups and 2.89× lower end-to-end latency than baseline solution in a memory capacity-limted system.

AB - We present FlashEmbedding, a hardware/software co-design solution for storing embedding tables on SSDs for large-scale recommendation inference under memory capacity-limited systems. FlashEmbedding leverages an embedding semantic-aware SSD, an embedding-oriented software cache, and pipeline techniques to improve the overall performance. We evaluate the performance of FlashEmbedding with our FPGA-based prototype SSD on a real-world public dataset. FlashEmbedding achieves up to 17.44× lower latency in embedding lookups and 2.89× lower end-to-end latency than baseline solution in a memory capacity-limted system.

KW - Embedding

KW - Recommender systems

KW - Solid-state drive (SSD)

UR - https://www.scopus.com/pages/publications/85118174515

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85118174515&origin=recordpage

U2 - 10.1145/3476886.3477511

DO - 10.1145/3476886.3477511

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781450386982

T3 - APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems

SP - 9

EP - 16

BT - APSys '21

PB - Association for Computing Machinery

Y2 - 24 August 2021 through 25 August 2021

ER -

Wan H , Sun X , Cui Y, Yang CL, Kuo TW, Xue CJ. FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems. In APSys '21: Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems. Association for Computing Machinery. 2021. p. 9-16. (APSys - Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems). Epub 2021 Aug 24. doi: 10.1145/3476886.3477511

FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Accelerating Large-Scale Recommender Systems

Cite this

FlashEmbedding: Storing embedding tables in SSD for large-scale recommender systems

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Student theses

Accelerating Large-Scale Recommender Systems

Cite this