A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Hiu Tung Wong; Chi-Sing Leung; Sam Kwong

doi:10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Hiu Tung Wong, Chi-Sing Leung^*, Sam Kwong

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

1 Citation (Scopus)

Abstract

The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

Original language	English
Title of host publication	Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings
Editors	Derong Liu, Shengli Xie, Yuanqing Li, Dongbin Zhao, El-Sayed M. El-Alfy
Publisher	Springer, Cham
Pages	424-433
Volume	10634 LNCS
ISBN (Electronic)	978-3-319-70087-8
ISBN (Print)	978-3-319-70086-1
DOIs	https://doi.org/10.1007/978-3-319-70087-8_45
Publication status	Published - Nov 2017
Event	24th International Conference on Neural Information Processing (ICONIP 2017) - Guangzhou, China Duration: 14 Nov 2017 → 18 Nov 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10634 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Neural Information Processing (ICONIP 2017)
Country/Territory	China
City	Guangzhou
Period	14/11/17 → 18/11/17

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

Additive noise
Feedforward networks
Multiplicative noise

Access Output

10.1007/978-3-319-70087-8_45

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Wong, H. T., Leung, C.-S., & Kwong, S. (2017). A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. In D. Liu, S. Xie, Y. Li, D. Zhao, & E.-S. M. El-Alfy (Eds.), Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings (Vol. 10634 LNCS, pp. 424-433). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS). Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_45

Wong, Hiu Tung ; Leung, Chi-Sing ; Kwong, Sam. / A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. editor / Derong Liu ; Shengli Xie ; Yuanqing Li ; Dongbin Zhao ; El-Sayed M. El-Alfy. Vol. 10634 LNCS Springer, Cham, 2017. pp. 424-433 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c1d22816c5b340f7811267825d14f7e1,

title = "A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks",

abstract = "The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.",

keywords = "Additive noise, Feedforward networks, Multiplicative noise",

author = "Wong, {Hiu Tung} and Chi-Sing Leung and Sam Kwong",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; 24th International Conference on Neural Information Processing (ICONIP 2017) ; Conference date: 14-11-2017 Through 18-11-2017",

year = "2017",

month = nov,

doi = "10.1007/978-3-319-70087-8_45",

language = "English",

isbn = "978-3-319-70086-1",

volume = "10634 LNCS",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Cham",

pages = "424--433",

editor = "Derong Liu and Xie, {Shengli } and Yuanqing Li and Dongbin Zhao and El-Alfy, {El-Sayed M. }",

booktitle = "Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings",

}

Wong, HT , Leung, C-S & Kwong, S 2017, A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. in D Liu, S Xie, Y Li, D Zhao & E-SM El-Alfy (eds), Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. vol. 10634 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10634 LNCS, Springer, Cham, pp. 424-433, 24th International Conference on Neural Information Processing (ICONIP 2017), Guangzhou, China, 14/11/17. https://doi.org/10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. / Wong, Hiu Tung ; Leung, Chi-Sing ; Kwong, Sam.
Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. ed. / Derong Liu; Shengli Xie; Yuanqing Li; Dongbin Zhao; El-Sayed M. El-Alfy. Vol. 10634 LNCS Springer, Cham, 2017. p. 424-433 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

AU - Wong, Hiu Tung

AU - Leung, Chi-Sing

AU - Kwong, Sam

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2017/11

Y1 - 2017/11

N2 - The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

AB - The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

KW - Additive noise

KW - Feedforward networks

KW - Multiplicative noise

UR - http://www.scopus.com/inward/record.url?scp=85035088364&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85035088364&origin=recordpage

U2 - 10.1007/978-3-319-70087-8_45

DO - 10.1007/978-3-319-70087-8_45

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-3-319-70086-1

VL - 10634 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 424

EP - 433

BT - Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings

A2 - Liu, Derong

A2 - Xie, Shengli

A2 - Li, Yuanqing

A2 - Zhao, Dongbin

A2 - El-Alfy, El-Sayed M.

PB - Springer, Cham

T2 - 24th International Conference on Neural Information Processing (ICONIP 2017)

Y2 - 14 November 2017 through 18 November 2017

ER -

Wong HT , Leung CS , Kwong S. A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. In Liu D, Xie S, Li Y, Zhao D, El-Alfy ESM, editors, Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS. Springer, Cham. 2017. p. 424-433. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this