Improving text classification using local Latent Semantic Indexing

Tao Liu; Zheng Chen; Benyu Zhang; Wei-Ying Ma; Gongyi Wu

doi:10.1109/icdm.2004.10096

Improving text classification using local Latent Semantic Indexing

Tao Liu
, Zheng Chen
, Benyu Zhang
, Wei-Ying Ma
, Gongyi Wu

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

77 Citations (Scopus)

Abstract

Latent Semantic Indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called "Local Relevancy Weighted LSI" to improve text classification by performing a separate Single Value Decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension. © 2004 IEEE.

Original language	English
Title of host publication	Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004
Pages	162-169
DOIs	https://doi.org/10.1109/icdm.2004.10096
Publication status	Published - 2004
Externally published	Yes
Event	Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004 - Brighton, United Kingdom Duration: 1 Nov 2004 → 4 Nov 2004 https://ieeexplore.ieee.org/xpl/conhome/9681/proceeding

Publication series

Name	Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004

Conference

Conference	Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004
Place	United Kingdom
City	Brighton
Period	1/11/04 → 4/11/04
Internet address	https://ieeexplore.ieee.org/xpl/conhome/9681/proceeding

Bibliographical note

Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

Access Output

10.1109/icdm.2004.10096

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{1903d89af72c4996a584780df3179e3d,

title = "Improving text classification using local Latent Semantic Indexing",

abstract = "Latent Semantic Indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called {"}Local Relevancy Weighted LSI{"} to improve text classification by performing a separate Single Value Decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension. {\textcopyright} 2004 IEEE.",

author = "Tao Liu and Zheng Chen and Benyu Zhang and Wei-Ying Ma and Gongyi Wu",

note = "Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].; Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004 ; Conference date: 01-11-2004 Through 04-11-2004",

year = "2004",

doi = "10.1109/icdm.2004.10096",

language = "English",

isbn = "0769521428",

series = "Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004",

pages = "162--169",

booktitle = "Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004",

url = "https://ieeexplore.ieee.org/xpl/conhome/9681/proceeding",

}

Liu, T, Chen, Z, Zhang, B, Ma, W-Y & Wu, G 2004, Improving text classification using local Latent Semantic Indexing. in Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004. Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004, pp. 162-169, Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004, Brighton, United Kingdom, 1/11/04. https://doi.org/10.1109/icdm.2004.10096

Improving text classification using local Latent Semantic Indexing. / Liu, Tao; Chen, Zheng; Zhang, Benyu et al.
Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004. 2004. p. 162-169 (Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Improving text classification using local Latent Semantic Indexing

AU - Liu, Tao

AU - Chen, Zheng

AU - Zhang, Benyu

AU - Ma, Wei-Ying

AU - Wu, Gongyi

N1 - Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

PY - 2004

Y1 - 2004

N2 - Latent Semantic Indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called "Local Relevancy Weighted LSI" to improve text classification by performing a separate Single Value Decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension. © 2004 IEEE.

AB - Latent Semantic Indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called "Local Relevancy Weighted LSI" to improve text classification by performing a separate Single Value Decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension. © 2004 IEEE.

UR - https://www.scopus.com/pages/publications/19544372770

UR - https://www.scopus.com/pages/publications/19544372770#tab=impact

U2 - 10.1109/icdm.2004.10096

DO - 10.1109/icdm.2004.10096

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 0769521428

SN - 9780769521428

T3 - Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004

SP - 162

EP - 169

BT - Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004

T2 - Proceedings - Fourth IEEE International Conference on Data Mining, ICDM 2004

Y2 - 1 November 2004 through 4 November 2004

ER -

Improving text classification using local Latent Semantic Indexing

Abstract

Publication series

Conference

Bibliographical note

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this