Structural Regular Multiple Criteria Linear Programming for Classification Problem

Authors

  • Zhiquan Qi Research Center on Fictitious Economy & Data Science, Chinese Academy of Sciences, Beijing 100190, China
  • Yong Shi 1. Research Center on Fictitious Economy & Data Science, Chinese Academy of Sciences, Beijing 100190, China and 2.College of Information Science & Technology, University of Nebraska at Omaha Omaha, NE 68182, USA

Keywords:

classification, RMCLP, structural information of data, SVM

Abstract

Classification problem has attracted an increasing amount of interest. Various classifiers have been proposed in the last decade, such as ANNs, LDA, and SVM. Regular Multiple Criteria Linear Programming (RMCLP) is an effective classification method, which was proposed by Shi and his colleagues and have been applied to handle different real-life data mining problems. In this paper, inspired by the application potential of RMCLP, we propose a novel Structural RMCLP (called SRMCLP) method for classification problem. Unlike RMCLP, SRMCLP is sensitive to the structure of the data distribution and can construct more reasonable classifiers by exploiting these prior data distribution information within classes. The corresponding optimization problem of SRMCLP can be solved by a standard quadratic programming. The effectiveness of the proposed method is demonstrated via experiments on synthetic and available benchmark datasets.

Author Biography

Zhiquan Qi, Research Center on Fictitious Economy & Data Science, Chinese Academy of Sciences, Beijing 100190, China

Department of Mathematics and Computer Science

References

Vapnik V.N. The Nature of Statistical Learning Theory. 2nd ed. New York: Springer, 2000. http://dx.doi.org/10.1007/978-1-4757-3264-1

Deng N.Y., Tian Y.J. Support vector machines: Theory, Algorithms and Extensions. Science Press, Beijing, 2009.

Fisher R.A. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7(2): 179-188, 1936. http://dx.doi.org/10.1111/j.1469-1809.1936.tb02137.x

Mangasarian O.L. Generalized support vector machines. Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 2000.

Freed N., Glover F. Simple but powerful goal programming models for discriminant problems. European Journal of Operational Research 7: 44-60, 1981. http://dx.doi.org/10.1016/0377-2217(81)90048-5

Freed N., Glover F. Evaluating alternative linear programming models to solve the two-group discriminant problem. Decision Science 17: 151-162, 1986. http://dx.doi.org/10.1111/j.1540-5915.1986.tb00218.x

Olson D., Shi Y. Introduction to Business Data Mining. McGraw-Hill/Irwin, 2007.

Shi Y., Tian Y.J., Chen X.J., Zhang P. A Regularized Multiple Criteria Linear Program for Classification. In: ICDM Workshops, pp 253-258, 2007.

Kou G., Shi Y., Wang S.Y. Multiple criteria decision making and decision support systems - Guest editor's introduction. Decision Support Systems 51(2): 247-249, 2011. http://dx.doi.org/10.1016/j.dss.2010.11.027

Zhang D., Tian Y.J., Shi Y. A regression method by multiple criteria linear programming. In: 19th International Conference on Multiple Criteria Decision Making (MCDM), pp 7-12, 2008.

Shi Y., Wise W., Lou M. Multiple Criteria Decision Making in Credit Card Portfolio Management. Multiple Criteria Decision Making in New Millennium, pp 427-436, 2001.

Shi Y., Peng Y., Xu W. Data mining via multiple criteria linear programming: applications in credit card portfolio management. International Journal of Information Technology and Decision Making 1: 131-151, 2002. http://dx.doi.org/10.1142/S0219622002000038

Zhang J., Zhuang W., Yan N. Classification of HIV-1 Mediated Neuronal Dendritic and Synaptic Damage Using Multiple Criteria Linear Programming. Neuroinformatics 2: 303- 326, 2004 http://dx.doi.org/10.1385/NI:2:3:303

Kwak W., Shi Y., Eldridge S. Bankruptcy prediction for Japanese firms: using multiple criteria linear programming data mining approach. International Journal of Data Mining and Business Intelligence, 2006.

Yeung D., Wang D., Ng W., Tsang E., Wang X., Structured large margin machines: sensitive to data distributions, Machine Learning 68 (2): 171-200, 2007. http://dx.doi.org/10.1007/s10994-007-5015-9

Xue H., Chen S., Yang Q., Structural support vector machine, in: The 15th International Symposium on Neural Networks, pp. 501-511, 2008.

Xue H., Chen S., Yang Q., Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier, Neural Networks, IEEE Transactions on 22 (4): 573- 587, 2011.

Ward, J.R. Hierarchical Grouping to Optimize an Objective Function, Journal of the American Statistical Association, 58 (301): 236-244, 1963. http://dx.doi.org/10.1080/01621459.1963.10500845

G. R. G. Lanckriet, L. E. Ghaoui, C. Bhattacharyya, M. I. Jordan, A robust minimax approach to classification, Journal of Machine Learning Research, 3: 555-582, 2002.

Kzhuang K. H., Yang H., King I., Learning large margin classifiers locally and globally, in: In The Twenty-First International Conference on Machine Learning, pp. 401-408, 2004.

Murphy P.M. and Aha D.W. UCI machine learning repository, 1992.

Published

2014-09-16

Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.