Selective Feature Generation Method for Classification of Low-Dimensional Data

Sang-Il Choi, Sang Tae Choi, Haanju Yoo


We propose a method that generates input features to effectively classify low-dimensional data. To do this, we first generate high-order terms for the input features of the original low-dimensional data to form a candidate set of new input features. Then, the discrimination power of the candidate input features is quantitatively evaluated by calculating the ‘discrimination distance’ for each candidate feature. As a result, only candidates with a large amount of discriminative information are selected to create a new input feature vector, and the discriminant features that are to be used as input to the classifier are extracted from the new input feature vectors by using a subspace discriminant analysis. Experiments on low-dimensional data sets in the UCI machine learning repository and several kinds of low-resolution facial image data show that the proposed method improves the classification performance of low-dimensional data by generating features.


patter classification, feature selection, discriminant analysis

Full Text:



Baker, S.; Sim, T.; Bsat, M. (2003); The CMU pose, illumination, and expression database, IEEE Transaction on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI. 2003.1251154, 25(12), 1615-1618, 2003.

Belhumeur, P. N.; Hespanha, J. P.; Kriegman, D. J. (1997); Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/34.598228, 19(7), 711-720, 1997.

Blake, C.; Merz, C. J. (1998); UCI Repository of machine learning databases,, 1998.

Cevikalp, H.; Neamtu, M.; Wilkes, M.; Barkana, A. (2005); Discriminative common vectors for face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI.2005.9, 27(1), 4-13, 2005.

Chen, S.; Zhu, Y.; Zhang, D.; Yang, J.-Y. (2005); Feature extraction approaches based on matrix pattern: MatPCA and MatFLDA, Pattern Recognition Letters, DOI: 10.1016/j.patrec.2004.10.009, 26(8), 1157-1167, 2005.

Choi, S.-I. (2015); Feature generation method for low-resolution face recognition, Journal of Korea Multimedia Society, 18(9):1039-1046, 2015.

Choi, S.-I.; Choi, C.-H.; Jeong, G.-M.; Kwak, N. (2012); Pixel selection based on discriminant features with application to face recognition, Pattern Recognition Letters, DOI: 10.1016/j.patrec.2012.01.005, 33(9), 1083-1092, 2012.

Choi, S.-I.; Oh, J.; Choi, C.-H.; Kim, C. (2012); Input variable selection for feature extraction in classification problems, Signal Processing, ISSN: 01651684, DOI: 10.1016/j.sigpro.2011.08.023, 92(3), 636-648, 2012.

Cortes, C.; Vapnik, V. (1995); Support-vector networks, Machine Learning, DOI: 10.1023/A:1022627411411, 20(3), 273-297, 1995.

Cover, T. M. (1965); Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE Transactions on Electronic Computers, ISSN: 03677508, DOI: 10.1109/PGEC.1965.264137, (3):326-334, 1965.

Duda, R. O.; Hart, P. E.; Stork, D. G. (2001); Pattern classification. 2nd, New York, 55, 2001.

Georghiades, A. S.; Belhumeur, P. N.; Kriegman, D. J. (2001); From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/34.927464, 23(6), 643-660, 2001.

Gonzalez, R.; Woods, R. (2002); Digital image processing, A. Dowrkin, Ed. Upper Saddle River, New Jersey 07458, Prentice Hall, 2002.

Jain, A. K.; Duin, R. P. W.; Mao, J. (2000); Statistical pattern recognition: A review, IEEE Transactions on Pattern Analysis and Machine Intelligence, I, DOI: 10.1109/34.824819, 22(1), 4-37, 2000.

Keinosuke, F. (1990); Introduction to statistical pattern recognition, Academic Press Inc., 1990.

Keys, R. (1981); Cubic convolution interpolation for digital image processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, I, DOI: 10.1109/TASSP.1981.1163711, 29(6), 1153-1160, 1981.

Kim, C. (2007); Pattern recognition using composite features, Ph. D. Thesis, Seoul National University, 2007.

Kim, C.; Choi, C.-H. (2007); A discriminant analysis using composite features for classification problems, Pattern Recognition, DOI: 10.1016/j.patcog.2007.02.008, 40(11), 2958-2966, 2007.

Kim, C.; Oh, J. Y.; Choi, C.-H. (2005); Combined subspace method using global and local features for face recognition, Neural Networks, 2005. IJCNN'05. Proceedings. 2005 IEEE International Joint Conference on, DOI: 10.1109/IJCNN.2005.1556212, 4, 2030-2035, 2005.

Kwak, N.; Oh, J. (2009); Feature extraction for one-class classification problems: Enhancements to biased discriminant analysis, Pattern Recognition, I DOI: 10.1016/j.patcog.2008.07.002, 42(1), 17-26, 2009.

Liang, J.; Yang, S.; Winstanley, A. (2008); Invariant optimal feature selection: A distance discriminant and feature ranking based solution, Pattern Recognition, DOI: 10.1016/j.patcog.2007.10.018, 41(5), 1429-1439, 2008.

Lin, F.; Zhou, X.; Zeng, W. (2016); Sparse online learning for collaborative filtering, International Journal of Computers Communications & Control, 11(2), 248-258, 2016.

Marimont, R.; Shapiro, M. (1979); Nearest neighbour searches and the curse of dimensionality, IMA Journal of Applied Mathematics, DOI: 10.1093/imamat/24.1.59, 24(1), 59-70, 1979.

Mika, S.; Ratsch, G.; Weston, J.; Scholkopf, B.; Mullers, K.-R. (1999); Fisher discriminant analysis with kernels, Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop, 41-48, 1999.

Phillips, P. J.; Wechsler, H.; Huang, J.; Rauss, P. J. (1998); The FERET database and evaluation procedure for face-recognition algorithms, Image and vision computing, 16(5), 295-306, 1998.

Suto, J.; Oniga, S.; Pop Sitar, P. (2016); Feature analysis to human activity recognition, International Journal of Computers Communications & Control, ISSN: 18419836, 12(1), 116-130, 20106.

Turk, M.; Pentland, A. (1991); Eigenfaces for recognition, Journal of Cognitive Neuroscience, DOI: 10.1162/jocn.1991.3.1.71, 3(1), 71-86, 1991.

Viriri, S.; Lagerwall, B. (2016); Increasing face recognition rates using novel classification algorithms, International Journal of Computers Communications & Control, 11(3), 381-393, 2016.

Xiong, H.; Swamy, M.; Ahmad, M.O. (2005); Two-dimensional FLD for face recognition, Pattern Recognition, ISSN: 00313203, DOI: 10.1016/j.patcog.2004.12.003, 38(7), 1121-1124, 2005.

Yang, J.; Frangi, A. F.; Yang, J.-y.; Zhang, D.; Jin, Z. (2005); KPCA plus LDA: a complete kernel fisher discriminant framework for feature extraction and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI.2005.33, 27(2), 230-244, 2005.

Zhou, X. S.; Huang, T. S. (2001); Small sample learning during multimedia retrieval using biasmap, Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, DOI: 10.1109/CVPR.2001.990450, 1, 111-117, 2001.

Center for Computational Vision and Control, Yale University, The Yale FaceDatabase,


Copyright (c) 2018 Sang-Il Choi, Sang Tae Choi, Haanju Yoo

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CC-BY-NC  License for Website User

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]

INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2016: IF=1.374. .

IJCCC is indexed in Scopus from 2008 (SNIP2016 = 0.701, SJR2016 =0.319):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.