A New Linear Classifier Based on Combining Supervised and Unsupervised Techniques


  • Luminiţa State University of PiteÅŸti Faculty of Mathematics and Computer Science Romania, 110040 PiteÅŸti, 1 Tí¢rgu din Vale St.
  • Iuliana Paraschiv-Munteanu University of Bucharest Faculty of Mathematics and Computer Science Romania, 010014 Bucharest, 14 Academiei St.


support vector machine, classification, unsupervised learning, supervised learning, k-means algorithm


The aim of the research reported in the paper is to obtain an alternative approach in using Support Vector Machine (SVM) in case of nonlinearly separable data based on using the k-means algorithm instead of the standard kernel based approach.
The SVM is a relatively new concept in machine learning and it was introduced by Vapnik in 1995. In designing a classifier, two main problems have to be solved, on one hand the option concerning a suitable structure and on the other hand the selection of an algorithm for parameter estimation.
The algorithm for parameter estimation performs the optimization of a convenable selected cost function with respect to the empirical risk which is directly related to the representativeness of the available learning sequence. The choice of the structure is made such that to maximize the generalization capacity, that is to assure good performance in classifying new data coming from the same classes. In solving these problems one has to establish a balance between the accuracy in encoding the learning sequence and the generalization capacities because usually the over-fitting prevents the minimization of the empirical risk.


S. Abe, Support Vector Machines for Pattern Classification, Springer-Verlag, 2005.

C.J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, 2, pp. 121-167, 1998. http://dx.doi.org/10.1023/A:1009715923555

C. Cortes, V. Vapnik, Support-vector networks, Machine Learning, 20(3):273-297, 1995. http://dx.doi.org/10.1007/BF00994018

N. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge University Press, 2000. http://dx.doi.org/10.1017/CBO9780511801389

G. Gan, C. Ma, J. Wu, Data Clustering: Theory, Algorithms and Applications, SIAM, 2007. http://dx.doi.org/10.1137/1.9780898718348

S.R. Gunn, Support Vector Machines for Classification and Regression, University of Southampton, Technical Report, 1998.

L. State, I. Paraschiv-Munteanu, I., Introducere in teoria statistică a recunoaÅŸterii formelor, Editura Universitaţii din PiteÅŸti, 2009.

J.B. MacQueen, Some Methods for classification and Analysis of Multivariate Observations, Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, University of California Press, pp. 281-297, 1967.

I. Paraschiv-Munteanu, Support Vector Machine in solving pattern recognition tasks, Proceedings of First Doctoral Student Workshop in Computer Science, University of PiteÅŸti, May 2009.

I. Paraschiv-Munteanu, Theoretical approach in performance evaluation of a classsification system, Scientific Bulletin of the University of PiteÅŸti, Series Mathematics and Informatics, No. 14, pp. 203-218, 2008.

N. Popescu-Bodorin, Fast K-Means Image Quantization Algorithm and Its Application to Iris Segmentation", Scientific Bulletin of the University of PiteÅŸti, Series Mathematics and Informatics, No. 14, 2008.

R. Stoean, C. Stoean, M. Preuss, D. Dumitrescu, Evolutionary multi-class support vector machines for classification, International Journal of Computers Communications & Control, Vol.1 Supplement: Suppl. S, pp. 423-428, 2006.

V.N. Vapnik, The Nature of Statistical Learning Theory, New York, Springer Verlag, 1995. http://dx.doi.org/10.1007/978-1-4757-2440-0

V.N. Vapnik, Statistical Learning Theory, New York, Wiley-Interscience, 1998.

R. Xu, D.C.II Wunsch, Clustering, Wiley&Sons, 2009.



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.