Classification Performance Using Principal Component Analysis and Different Value of the Ratio R

Jasmina Novakovic, Sinisa Rankov

Abstract


A comparison between several classification algorithms with feature extraction on real dataset is presented. Principal Component Analysis (PCA) has been used for feature extraction with different values of the ratio R, evaluated and compared using four different types of classifiers on two real benchmark data sets. Accuracy of the classifiers is influenced by the choice of different values of the ratio R. There is no best value of the ratio R, for different datasets and different classifiers accuracy curves as a function of the number of features used may significantly differ. In our cases feature extraction is especially effective for classification algorithms that do not have any inherent feature selections or feature extraction build in, such as the nearest neighbour methods or some types of neural networks.

Keywords


feature extraction, linear feature extraction methods, principal component analysis, classification algorithms, classification accuracy

Full Text:

PDF

References


A.L. Blum, R.L. Rivest, Training a 3-node neural networks is NP-complete, Neural Networks, 5:117 - 127, 1992.
http://dx.doi.org/10.1016/S0893-6080(05)80010-3

N. Wyse, R. Dubes, A.K. Jain, A critical evaluation of intrinsic dimensionality algorithms. In E.S. Gelsema and L.N. Kanal, editors, Pattern Recognition in Practice, pp 415–425. Morgan Kaufmann Publishers, Inc., 1980.

M. Ben-Bassat, Pattern recognition and reduction of dimensionality, In P. R. Krishnaiah and L. N. Kanal, editors, Handbook of statistics-II, pp 773-791. North Holland, 1982.

L.Breiman, J.H. Friedman, R.H. Olshen, Stone C.J., Classification and Regression Trees, Wadsworth and Brooks, Monterey, CA, 1984.

J.R. Quinlan, C4.5: Programs for machine learning, San Mateo, Morgan Kaufman, 1993.

W. Duch, R. Adamczak, K. Grabczewski, A new methodology of extraction, optimization and application of crisp and fuzzy logical rules, IEEE Transactions on Neural Networks, vol. 12, pp. 277-306, 2001.
http://dx.doi.org/10.1109/72.914524

I.T. Jolliffe, Principal Component Analysis, Springer-Verlag, New York, 1986.
http://dx.doi.org/10.1007/978-1-4757-1904-8

L. Sirovich, M. Kirby, Low dimensional procedure for the characterization of human faces, Journal of the Optical Society of America, 4(3) 519-524, 1987.
http://dx.doi.org/10.1364/JOSAA.4.000519

M. Turk, A. Pentland, Eigen faces for recognition, J. of Cognitive Neuroscience 3(1), 1991.
http://dx.doi.org/10.1162/jocn.1991.3.1.71

B. Moghaddam, A. Pentland, B. Starner, View-based and modular eigenspaoes for face recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 84-91, 1994.

P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection, Proceedings of the European Conference on Computer Vision, 1996.

H. Murase, S.K. Nayar, Learning and recognition of 3D objects from appearance, IEEE 2nd Qualitative Vision Workshop, pp 39-50, New York, NY, June 1993.

M. J. Black, D. Jepson, Eigen-tracking: Robust matching and tracking of articulated objects using a view-based representation, Proceedings of the European Conference on Computer Vision (ECCV), pp 329-342, Cambridge, England, 1996.

J.J. Atick, P.A. Griffin, N.A. Redlich, Statistical approach to shape-from-shading: deriving 3d face surfaces from single 2d images, Neural Computation, 1997.

M. Kantardzic,Data Mining: Concepts, Models, Methods, and Algorithms, John Wiley & Sons, 2003.

P. Comon, Independent component analysis, a new concept? Signal processing pages 36(3), pp 11-20, 1994.

A.J. Bell, T.J. Sejnowski, An information maximization approach to blind separation and blind deconvolution, Neural Computation, pp 1129-1159, 1995.
http://dx.doi.org/10.1162/neco.1995.7.6.1129

C. Bregler, S.M. Omohundro, Nonlinear manifold learning for visual speech recognition, iccv, Boston, Jun 1995.

T. Heap, D. Hogg, Wormholes in shape space: Tracking through discontinuous changes in shape, iccv, 1998.

T. Hastie, W. Stuetzle, Principal curves, Journal of Americam Statistical Association 84, pp 502-516, 1989.
http://dx.doi.org/10.1080/01621459.1989.10478797

M.A. Kramer, Non linear principal component analysis using autoassociative neural networks, AI Journal 37(2), pp 233-243, 1991.

A.R. Webb, An approach to nonlinear principal components-analysis using radially symmetrical kernel functions, Statistics and computing 6(2), pp 159-168, 1996.
http://dx.doi.org/10.1007/BF00162527

V. Silva, J.B. Tenenbaum, J.C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, 290, December 2000.

J.M. Winn, C.M. Bishop, Non-linear bayesian image modelling, Proceedings of the European Conference on Computer Vision, Dublin, Ireland, June 2000.

M. Kuramochi, G. Karypis. Gene classification using expression profiles: a feasibility study, International Journal on Artificial Intelligence Tools, 14(4):641-660, 2005.
http://dx.doi.org/10.1142/S0218213005002302

P. Domingos, M. Pazzani, Feature selection and transduction for prediction of molecular bioactivity for drug design, Machine Learning, 29:103-130, 1997.
http://dx.doi.org/10.1023/A:1007413511361

E. P. Xing, M. L. Jordan, R. M. Karp Feature selection for high-dimensional genomic microarray data, Proceedings of the 18th International Conference on Machine Learning, 601-608, 2001.

C.M. Bishop, Neural Network for Pattern Recognition, Oxford University Press Inc., New York, 1995.

L. Wang, X. Fu, Data Mining with Computational Intelligence, Springer-Verlag Berlin Heidelberg, Germany, pages 9-14, 2005.




DOI: https://doi.org/10.15837/ijccc.2011.2.2180



Copyright (c) 2017 Jasmina Novakovic, Sinisa Rankov

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CC-BY-NC  License for Website User

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]


INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2018: IF=1.585..

IJCCC is indexed in Scopus from 2008 (CiteScore2018 = 1.56):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.

 

 Impact Factor in JCR2018 (Clarivate Analytics/SCI Expanded/ISI Web of Science): IF=1.585 (Q3). Scopus: CiteScore2018=1.56 (Q2); Editors-in-Chief: Ioan DZITAC & Florin Gheorghe FILIP.