Effect of Logistic Activation Function and Multiplicative Input Noise on DNN-kWTA Model

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 12 - Chapter in an edited book (Author)peer-review

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationNeural Information Processing
Subtitle of host publication29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part IV
EditorsMohammad Tanveer, Sonali Agarwal, Seiichi Ozawa, Asif Ekbal, Adam Jatowt
Place of PublicationSingapore
PublisherSpringer Nature Singapore Pte Ltd.
Pages202-214
ISBN (electronic)978-981-99-1639-9
ISBN (print)978-981-99-1638-2
Publication statusPublished - 2023

Publication series

NameCommunications in Computer and Information Science
Volume1791
ISSN (Print)1865-0929
ISSN (electronic)1865-0937

Conference

Title29th International Conference on Neural Information Processing (ICONIP 2022)
LocationVirtual
PlaceIndia
CityIndore
Period22 - 26 November 2022

Abstract

The dual neural network-based (DNN) k-winner-take-all (kWTA) model is one of the simplest analog neural network models for the kWTA process. This paper analyzes the behaviors of the DNN-kWTA model under these two imperfections. The two imperfections are, (1) the activation function of IO neurons is a logistic function rather than an ideal step function, and (2) there are multiplicative Gaussian noise in the inputs. With the two imperfections, the model may not be able to perform correctly. Hence it is important to estimate the probability of the imperfection model performing correctly. We first derive the equivalent activation function of IO neurons under the two imperfections. Next, we derive the sufficient conditions for the imperfect model to operate correctly. These results enable us to efficiently estimate the probability of the imperfect model generating correct outputs. Additionally, we derive a bound on the probability that the imperfect model generates the desired outcomes for inputs with a uniform distribution. Finally, we discuss how to generalize our findings to handle non-Gaussian multiplicative input noise. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023.

Research Area(s)

  • DNN-kWTA, Logistic activation function, Multiplicative Input Noise, Threshold logic units (tlus)

Bibliographic Note

Information for this record is supplemented by the author(s) concerned.

Citation Format(s)

Effect of Logistic Activation Function and Multiplicative Input Noise on DNN-kWTA Model. / Lu, Wenhao; Leung, Chi-Sing; Sum, John.
Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part IV. ed. / Mohammad Tanveer; Sonali Agarwal; Seiichi Ozawa; Asif Ekbal; Adam Jatowt. Singapore: Springer Nature Singapore Pte Ltd., 2023. p. 202-214 (Communications in Computer and Information Science; Vol. 1791).

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 12 - Chapter in an edited book (Author)peer-review