A Novel Fuzzy ARTMAP Architecture with Adaptive FeatureWeights based on Onicescu’s Informational Energy

  • Răzvan Andonie Computer Science Department Central Washington University, Ellensburg, USA and Department of Electronics and Computers Transylvania University of Bra¸sov, Romania
  • Lucian Mircea Sasu Applied Informatics Department Transylvania University of Bra¸sov, Romania
  • Angel Cațaron Department of Electronics and Computers Transylvania University of Bra¸sov, Romania

Abstract

Fuzzy ARTMAP with Relevance factor (FAMR) is a Fuzzy ARTMAP (FAM) neural architecture with the following property: Each training pair has a relevance factor assigned to it, proportional to the importance of that pair during the learning phase. Using a relevance factor adds more flexibility to the training phase, allowing ranking of sample pairs according to the confidence we have in the information source or in the pattern itself. We introduce a novel FAMR architecture: FAMR with Feature Weighting (FAMRFW). In the first stage, the training data features are weighted. In our experiments, we use a feature weighting method based on Onicescu’s informational energy (IE). In the second stage, the obtained weights are used to improve FAMRFW training. The effect of this approach is that category dimensions in the direction of relevant features are decreased, whereas category dimensions in the direction of non-relevant feature are increased. Experimental results, performed on several benchmarks, show that feature weighting can improve the classification performance of the general FAMR algorithm.

References

[1] R. Andonie and A. Ca¸taron. Feature ranking using supervised neural gas and informational energy. In Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN2005), Canada, Montreal, July 31 - August 4, 2005.
http://dx.doi.org/10.1109/ijcnn.2005.1556036

[2] R. Andonie, A. Ca¸taron, and L. Sasu. Fuzzy ARTMAP with feature weighting. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), Innsbruck, Austria, Febr. 11-13, 2008, 91–96.

[3] R. Andonie and F. Petrescu. Interacting systems and informational energy. Foundation of Control Engineering, 11, 1986, 53–59.

[4] R. Andonie and L. Sasu. Fuzzy ARTMAP with input relevances. IEEE Transactions on Neural Networks, 17, 2006, 929–941.
http://dx.doi.org/10.1109/TNN.2006.875988

[5] A. Asuncion and D. J. Newman. UCI machine learning repository, 2007. University of California, Irvine, School of Information and Computer Sciences http://www.ics.uci.edu/»mlearn/MLRepository.html

[6] I. Dzi¸tac and B. E. B˘arbat. Artificial intelligence + distributed systems = agents. International Journal Computers, Communications, and Control, 4, 2009, 17–26.
http://dx.doi.org/10.15837/ijccc.2009.1.2410

[7] G. A. Carpenter, S. Grossberg, N. Markuzon, J. H. Reynolds, and D. B. Rosen. Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps. IEEE Transactions on Neural Networks, 3, 1992, 698–713.
http://dx.doi.org/10.1109/72.159059

[8] G. A. Carpenter, B. L. Milenova, and B. W. Noeske. Distributed ARTMAP: A neural network for fast distributed supervised learning. Neural Networks, 11, 1998, 793–813.
http://dx.doi.org/10.1016/S0893-6080(98)00019-7

[9] G. A. Carpenter and W. Ross. ART-EMAP: A neural network architecture for learning and prediction by evidence accumulation. IEEE Transactions on Neural Networks, 6, 1995, 805–818.
http://dx.doi.org/10.1109/72.392245

[10] D. Charalampidis, G. Anagnostopoulos, M. Georgiopoulos, and T. Kasparis. Fuzzy ART and Fuzzy ARTMAP with adaptively weighted distances. In Proceedings of the SPIE, Applications and Science of Computational Intelligence, Aerosense, 2002.
http://dx.doi.org/10.1117/12.458723

[11] I. Dagher, M. Georgiopoulos, G. L. Heileman, and G. Bebis. An ordering algorithm for pattern presentation in Fuzzy ARTMAP that tends to improve generalization performance. IEEE Transactions on Neural Networks, 10, 1999, 768–778.
http://dx.doi.org/10.1109/72.774217

[12] I. Dagher, M. Georgiopoulos, G. L. Heileman, and G. Bebis. Fuzzy ARTVar: An improved fuzzy ARTMAP algorithm. In Proceedings IEEE World Congress Computational Intelligence WCCI'98, Anchorage, 1998, 1688–1693.
http://dx.doi.org/10.1109/ijcnn.1998.687110

[13] J. C. Principe et al. Information-theoretic learning. In S. Haykin, editor, In Unsupervised Adaptive Filtering. Wiley, New York, 2000.

[14] E. Gomez-Sanchez, Y. A. Dimitriadis, J. M. Cano-Izquierdo, and J. Lopez-Coronado. ¹ARTMAP: Use of mutual information for category reduction in fuzzy ARTMAP. IEEE Transactions on Neural Networks, 13, 2002, 58–69.
http://dx.doi.org/10.1109/72.977271

[15] S. Guia¸su. Information theory with applications. McGraw Hill, New York, 1977.

[16] B. Hammer, D. Schunk, T. Bojer, and T. K. von Toschanowitz. Relevance determination in learning vector quantization. In Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2001), Bruges, Belgium, 2001, 271–276.

[17] B. Hammer, M. Strickert, and T. Villmann. Supervised neural gas with general similarity measure. Neural Processing Letters, 21, 2005, 21–44.
http://dx.doi.org/10.1007/s11063-004-3255-2

[18] B. Hammer and T. Villmann. Generalized relevance learning vector quantization. Neural Networks, 15, 2002, 1059–1068.
http://dx.doi.org/10.1016/S0893-6080(02)00079-5

[19] C. P. Lim and R. Harrison. ART-Based Autonomous Learning Systems: Part I - Architectures and Algorithms. In L. C. Jain, B. Lazzerini, and U. Halici, editors, Innovations in ART Neural Networks. Springer, 2000.
http://dx.doi.org/10.1007/978-3-7908-1857-4_6

[20] C. P. Lim and R. F. Harrison. An incremental adaptive network for on-line supervised learning and probability estimation. Neural Networks, 10, 1997, 925–939.
http://dx.doi.org/10.1016/S0893-6080(96)00123-2

[21] S. Marriott and R. F. Harrison. A modified fuzzy ARTMAP architecture for the approximation of noisy mappings. Neural Networks, 8, 1995, 619–641.
http://dx.doi.org/10.1016/0893-6080(94)00110-8

[22] T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. Neural-gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4, 1993, 558–569.
http://dx.doi.org/10.1109/72.238311

[23] S. Min-Kyu, J. Murata, and K. Hirasawa. Function approximation using LVQ and fuzzy sets. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Tucson, AZ, 2001, 1442–1447.

[24] O. Onicescu. Theorie de l'information. Energie informationnelle. C. R. Acad. Sci. Paris, Ser. A–B, 263, 1966, 841—842.

[25] O. Parsons and G. A. Carpenter. ARTMAP neural networks for information fusion and data mining: map production and target recognition methodologies. Neural Networks, 16, 2003, 1075–1089.
http://dx.doi.org/10.1016/S0893-6080(03)00007-8

[26] M. Taghi, V. Baghmisheh, and P. Nikola. A Fast Simplified Fuzzy ARTMAP Network. Neural Processing Letters, 17, 2003, 273–316.
http://dx.doi.org/10.1023/A:1026004816362

[27] S. J. Verzi, G. L. Heileman, M. Georgiopoulos, and M. J. Healy. Boosted ARTMAP. In Proceedings IEEE World Congress Computational Intelligence WCCI'98, 1998, 396–400.
http://dx.doi.org/10.1109/ijcnn.1998.682299

[28] J. Williamson. Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks, 9, 1996, 881–897.
http://dx.doi.org/10.1016/0893-6080(95)00115-8
Published
2009-06-01
How to Cite
ANDONIE, Răzvan; SASU, Lucian Mircea; CAȚARON, Angel. A Novel Fuzzy ARTMAP Architecture with Adaptive FeatureWeights based on Onicescu’s Informational Energy. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, [S.l.], v. 4, n. 2, p. 104-117, june 2009. ISSN 1841-9844. Available at: <http://univagora.ro/jour/index.php/ijccc/article/view/2418>. Date accessed: 16 july 2020. doi: https://doi.org/10.15837/ijccc.2009.2.2418.

Keywords

Fuzzy ARTMAP, feature weighting, LVQ, Onicescu’s informational energy