A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments

Jichao Wang; Ran Zhang; Guanqiang Qi; Lanqing Hong

doi:10.1109/IEEM50564.2021.9672905

A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments

Jichao Wang
, Ran Zhang
, Guanqiang Qi
, Lanqing Hong

Department of Data Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

5 Citations (Scopus)

Abstract

The hard disk drives (HDD) are essential devices lying in primary layers of diverse information infrastructure. Long-term disk failure predictions are crucial to the stability and robustness of storage systems for data centers. In this paper, a domain adaption method is developed to improve prediction performance in out-of-distribution disk datasets. We propose heuristic invariant risk minimization (HIRM) with a new loss function to deal with imbalanced data. The HIRM combined with machine learning models are verified to promote the accuracy and stability in out-of-distribution (OoD) data. When hard disks with new SMART feature distribution are introduced into the data center, the proposed HIRM algorithm achieves better results than vanilla neural networks. A numerical example using the data from the BackBlaze data center is shown to illustrate the application of our HIRM model. The aims of each person are different.

Original language	English
Title of host publication	2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)
Publisher	IEEE
Pages	1661-1664
ISBN (Electronic)	9781665437714
ISBN (Print)	9781665437721
DOIs	https://doi.org/10.1109/IEEM50564.2021.9672905
Publication status	Published - Dec 2021
Event	2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM2021) - Virtual, Singapore Duration: 13 Dec 2021 → 16 Dec 2021 https://www.ieem.org/public.asp?page=index.asp

Publication series

Name	IEEE International Conference on Industrial Engineering and Engineering Management, IEEM

Conference

Conference	2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM2021)
Abbreviated title	IEEE IEEM21
Place	Singapore
Period	13/12/21 → 16/12/21
Internet address	https://www.ieem.org/public.asp?page=index.asp

Research Keywords

Disk failure prediction
Domain generalization
IoT
Predictive maintenance

Access Output

10.1109/IEEM50564.2021.9672905

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Wang, J., Zhang, R., Qi, G., & Hong, L. (2021). A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments. In 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) (pp. 1661-1664). (IEEE International Conference on Industrial Engineering and Engineering Management, IEEM). IEEE. https://doi.org/10.1109/IEEM50564.2021.9672905

@inproceedings{5f58fc9e1fbe4083bbc5cbbbb9dc2b75,

title = "A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments",

abstract = "The hard disk drives (HDD) are essential devices lying in primary layers of diverse information infrastructure. Long-term disk failure predictions are crucial to the stability and robustness of storage systems for data centers. In this paper, a domain adaption method is developed to improve prediction performance in out-of-distribution disk datasets. We propose heuristic invariant risk minimization (HIRM) with a new loss function to deal with imbalanced data. The HIRM combined with machine learning models are verified to promote the accuracy and stability in out-of-distribution (OoD) data. When hard disks with new SMART feature distribution are introduced into the data center, the proposed HIRM algorithm achieves better results than vanilla neural networks. A numerical example using the data from the BackBlaze data center is shown to illustrate the application of our HIRM model. The aims of each person are different.",

keywords = "Disk failure prediction, Domain generalization, IoT, Predictive maintenance",

author = "Jichao Wang and Ran Zhang and Guanqiang Qi and Lanqing Hong",

year = "2021",

month = dec,

doi = "10.1109/IEEM50564.2021.9672905",

language = "English",

isbn = "9781665437721",

series = "IEEE International Conference on Industrial Engineering and Engineering Management, IEEM",

publisher = "IEEE",

pages = "1661--1664",

booktitle = "2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)",

address = "United States",

note = "2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM2021), IEEE IEEM21 ; Conference date: 13-12-2021 Through 16-12-2021",

url = "https://www.ieem.org/public.asp?page=index.asp",

}

Wang, J, Zhang, R, Qi, G & Hong, L 2021, A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments. in 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE International Conference on Industrial Engineering and Engineering Management, IEEM, IEEE, pp. 1661-1664, 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM2021), Singapore, 13/12/21. https://doi.org/10.1109/IEEM50564.2021.9672905

A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments. / Wang, Jichao; Zhang, Ran; Qi, Guanqiang et al.
2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE, 2021. p. 1661-1664 (IEEE International Conference on Industrial Engineering and Engineering Management, IEEM).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments

AU - Wang, Jichao

AU - Zhang, Ran

AU - Qi, Guanqiang

AU - Hong, Lanqing

PY - 2021/12

Y1 - 2021/12

N2 - The hard disk drives (HDD) are essential devices lying in primary layers of diverse information infrastructure. Long-term disk failure predictions are crucial to the stability and robustness of storage systems for data centers. In this paper, a domain adaption method is developed to improve prediction performance in out-of-distribution disk datasets. We propose heuristic invariant risk minimization (HIRM) with a new loss function to deal with imbalanced data. The HIRM combined with machine learning models are verified to promote the accuracy and stability in out-of-distribution (OoD) data. When hard disks with new SMART feature distribution are introduced into the data center, the proposed HIRM algorithm achieves better results than vanilla neural networks. A numerical example using the data from the BackBlaze data center is shown to illustrate the application of our HIRM model. The aims of each person are different.

AB - The hard disk drives (HDD) are essential devices lying in primary layers of diverse information infrastructure. Long-term disk failure predictions are crucial to the stability and robustness of storage systems for data centers. In this paper, a domain adaption method is developed to improve prediction performance in out-of-distribution disk datasets. We propose heuristic invariant risk minimization (HIRM) with a new loss function to deal with imbalanced data. The HIRM combined with machine learning models are verified to promote the accuracy and stability in out-of-distribution (OoD) data. When hard disks with new SMART feature distribution are introduced into the data center, the proposed HIRM algorithm achieves better results than vanilla neural networks. A numerical example using the data from the BackBlaze data center is shown to illustrate the application of our HIRM model. The aims of each person are different.

KW - Disk failure prediction

KW - Domain generalization

KW - IoT

KW - Predictive maintenance

UR - https://www.scopus.com/pages/publications/85125419625

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85125419625&origin=recordpage

U2 - 10.1109/IEEM50564.2021.9672905

DO - 10.1109/IEEM50564.2021.9672905

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781665437721

T3 - IEEE International Conference on Industrial Engineering and Engineering Management, IEEM

SP - 1661

EP - 1664

BT - 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

PB - IEEE

T2 - 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM2021)

Y2 - 13 December 2021 through 16 December 2021

ER -

Wang J, Zhang R, Qi G, Hong L. A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments. In 2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE. 2021. p. 1661-1664. (IEEE International Conference on Industrial Engineering and Engineering Management, IEEM). doi: 10.1109/IEEM50564.2021.9672905

A Heuristic-IRM Method on Hard Disk Failure Prediction in Out-of-distribution Environments

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this