Properties of Standard and Sketched Kernel Fisher Discriminant

Jiamin Liu; Wangli Xu; Fode Zhang; Heng Lian

doi:10.1109/TPAMI.2023.3242681

Properties of Standard and Sketched Kernel Fisher Discriminant

Jiamin Liu
, Wangli Xu
, Fode Zhang
, Heng Lian

Department of Mathematics

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

3 Citations (Scopus)

Abstract

Kernel Fisher discriminant (KFD) is a popular tool as a nonlinear extension of Fisher's linear discriminant, based on the use of the kernel trick. However, its asymptotic properties are still rarely studied. We first present an operator-theoretical formulation of KFD which elucidates the population target of the estimation problem. Convergence of the KFD solution to its population target is then established. However, the complexity of finding the solution poses significant challenges when n is large and we further propose a sketched estimation approach based on a m x n sketching matrix which possesses the same asymptotic properties (in terms of convergence rate) even when m
is much smaller than n. Some numerical results are presented to illustrate the performances of the sketched estimator. © 2023 IEEE.

Original language	English
Pages (from-to)	10596-10602
Number of pages	8
Journal	IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume	45
Issue number	8
Online published	6 Feb 2023
DOIs	https://doi.org/10.1109/TPAMI.2023.3242681
Publication status	Published - Aug 2023

Funding

The work of Wangli Xu was supported in part by Beijing Natural Science Foundation under Grant Z200001 and in part by the National Natural Science Foundation of Chine under Grant 11971478. The work of Fode Zhang was supported in part by the National Natural Science Foundationo of China under Grant 12071372. The work of Heng Lian was supported in part by NSF of Jiangxi Province under Grant 20223BCJ25017, and in part by Hong Kong RGC general research fund under Grants 11300519, 11300721, and 11311822.

Research Keywords

Convergence
Eigenvalues and eigenfunctions
Estimation
Kernel
Kernel method
random projection
reproducing kernel Hilbert space
Sociology
Standards
Urban areas
variance operator

RGC Funding Information

RGC-funded

Access Output

10.1109/TPAMI.2023.3242681

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@article{1ec758dba670457dab89d64f670d201f,

title = "Properties of Standard and Sketched Kernel Fisher Discriminant",

abstract = "Kernel Fisher discriminant (KFD) is a popular tool as a nonlinear extension of Fisher's linear discriminant, based on the use of the kernel trick. However, its asymptotic properties are still rarely studied. We first present an operator-theoretical formulation of KFD which elucidates the population target of the estimation problem. Convergence of the KFD solution to its population target is then established. However, the complexity of finding the solution poses significant challenges when n is large and we further propose a sketched estimation approach based on a m x n sketching matrix which possesses the same asymptotic properties (in terms of convergence rate) even when m is much smaller than n. Some numerical results are presented to illustrate the performances of the sketched estimator. {\textcopyright} 2023 IEEE.",

keywords = "Convergence, Eigenvalues and eigenfunctions, Estimation, Kernel, Kernel method, random projection, reproducing kernel Hilbert space, Sociology, Standards, Urban areas, variance operator",

author = "Jiamin Liu and Wangli Xu and Fode Zhang and Heng Lian",

year = "2023",

month = aug,

doi = "10.1109/TPAMI.2023.3242681",

language = "English",

volume = "45",

pages = "10596--10602",

journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",

issn = "0162-8828",

publisher = "IEEE Computer Society",

number = "8",

}

TY - JOUR

T1 - Properties of Standard and Sketched Kernel Fisher Discriminant

AU - Liu, Jiamin

AU - Xu, Wangli

AU - Zhang, Fode

AU - Lian, Heng

PY - 2023/8

Y1 - 2023/8

N2 - Kernel Fisher discriminant (KFD) is a popular tool as a nonlinear extension of Fisher's linear discriminant, based on the use of the kernel trick. However, its asymptotic properties are still rarely studied. We first present an operator-theoretical formulation of KFD which elucidates the population target of the estimation problem. Convergence of the KFD solution to its population target is then established. However, the complexity of finding the solution poses significant challenges when n is large and we further propose a sketched estimation approach based on a m x n sketching matrix which possesses the same asymptotic properties (in terms of convergence rate) even when m is much smaller than n. Some numerical results are presented to illustrate the performances of the sketched estimator. © 2023 IEEE.

AB - Kernel Fisher discriminant (KFD) is a popular tool as a nonlinear extension of Fisher's linear discriminant, based on the use of the kernel trick. However, its asymptotic properties are still rarely studied. We first present an operator-theoretical formulation of KFD which elucidates the population target of the estimation problem. Convergence of the KFD solution to its population target is then established. However, the complexity of finding the solution poses significant challenges when n is large and we further propose a sketched estimation approach based on a m x n sketching matrix which possesses the same asymptotic properties (in terms of convergence rate) even when m is much smaller than n. Some numerical results are presented to illustrate the performances of the sketched estimator. © 2023 IEEE.

KW - Convergence

KW - Eigenvalues and eigenfunctions

KW - Estimation

KW - Kernel

KW - Kernel method

KW - random projection

KW - reproducing kernel Hilbert space

KW - Sociology

KW - Standards

KW - Urban areas

KW - variance operator

UR - https://www.scopus.com/pages/publications/85148476705

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85148476705&origin=recordpage

U2 - 10.1109/TPAMI.2023.3242681

DO - 10.1109/TPAMI.2023.3242681

M3 - RGC 21 - Publication in refereed journal

SN - 0162-8828

VL - 45

SP - 10596

EP - 10602

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

IS - 8

ER -

Properties of Standard and Sketched Kernel Fisher Discriminant

Abstract

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Low-rank Nonparametric Regression and Application to Reinforcement Learning

GRF: Distributed Estimation with Random Projection in Reproducing Kernel Hilbert Spaces

GRF: Low-rank tensor as a Dimension Reduction Tool in Complex Data Analysis

Cite this

Properties of Standard and Sketched Kernel Fisher Discriminant

Abstract

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Low-rank Nonparametric Regression and Application to Reinforcement Learning

GRF: Distributed Estimation with Random Projection in Reproducing Kernel Hilbert Spaces

GRF: Low-rank tensor as a Dimension Reduction Tool in Complex Data Analysis

Cite this