A Knowledge Base Completion Model Based on Path Feature Learning

  • Xixun Lin Jilin University
  • Yanchun Liang Jilin University
  • Limin Wang Jilin University of Finance and Economics
  • Xu Wang Jilin University
  • Mary Qu Yang university of arkansas at little rock
  • Renchu Guan Jilin University

Abstract

Large-scale knowledge bases, as the foundations for promoting the development of artificial intelligence, have attracted increasing attention in recent years. These knowledge bases contain billions of facts in triple format; yet, they suffer from sparse relations between entities. Researchers proposed the path ranking algorithm (PRA) to solve this fatal problem. To improve the scalability of knowledge inference, PRA exploits random walks to find Horn clauses with chain structures to predict new relations given existing facts. This method can be regarded as a statistical classification issue for statistical relational learning (SRL). However, large-scale knowledge base completion demands superior accuracy and scalability. In this paper, we propose the path feature learning model (PFLM) to achieve this urgent task. More precisely, we define a two-stage model: the first stage aims to learn path features from the existing knowledge base and extra parsed corpus; the second stage uses these path features to predict new relations. The experimental results demonstrate that the PFLM can learn meaningful features and can achieve significant and consistent improvements compared with previous work.

References

[1] Agirre, E.; Lacalle, O.; Soroa, A. (2014); Random walks for knowledge-based word sense disambiguation, Computational Linguistics, 40, 57–84, 2014.
https://doi.org/10.1162/COLI_a_00164

[2] Berant, J.; Chou, A.; Frostig, R.; Liang, P. (2013); Semantic parsing on Freebase from question-answer pairs, Proceedings of EMNLP, 1533–1544, 2013.

[3] Bollacker, K.; Evans C.; Paritosh, P.; Sturge, T.; Taylor, J. (2008); Freebase: a collaboratively created graph database for structuring human knowledge, Proceedings of KDD, 1247–1250, 2008.

[4] Bordes, A.; Usunier, N.; García-Durán, A.; Weston, J.; Yakhnenko O. (2013); Translating embeddings for modeling multi-relational data, Proceedings of NIPS, 2787–2795, 2013.

[5] Cao, F.; Liu, B.; Park, D. (2013); Image classification based on effective extreme learning machine, Neurocomputing, 102, 90–97, 2013.
https://doi.org/10.1016/j.neucom.2012.02.042

[6] Carlson, A.; Betteridge, J.; Kisiel, B.; Settles, B.; Hruschka, E.; Mitchell T. (2010); Toward an architecture for never-ending language learning, Proceedings of AAAI, 1306–1313, 2010.

[7] Gardner, M.; Talukdar, P.; Kisiel, B.; Mitchell, T. (2013); Improving learning and inference in a large knowledge-base using latent syntactic cues, Proceedings of EMNLP, 833–838, 2013.

[8] Gardner, M.; Talukdar, P.; Krishnamurthy, J.; Mitchell, T. (2014); Incorporating vector space similarity in random walk inference over knowledge bases, Proceedings of EMNLP, 833–838, 2014.

[9] Gardner, M.; Mitchell, T. (2015); Efficient and expressive knowledge base completion using subgraph feature extraction, Proceedings of EMNLP, 1488–1498, 2015.

[10] Getoor, L.; Taskar, B. (2007); Introduction to statistical relational learning, MIT press, 2007.

[11] Glassick, C.E.; Huber, M.T.; Maeroff, G.I. (2015); DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia, Semantic Web, 6, 167–195, 2015.

[12] Guo, S.; Wang, Q.; Wang, B.; Wang, L.; Guo, L. (2015); Semantically smooth knowledge graph embedding, Proceedings of ACL, 84–94, 2015.

[13] Hoffmann, R.; Zhang, C.; Ling, X.; Zettlemoyer, L.; Weld, D. (2011); Knowledge-based weak supervision for information extraction of overlapping relations, Proceedings of ACL, 541–550, 2011.

[14] Huang, G.; Wang, D.; Lan, Y. (2011); Extreme learning machines: a survey, International Journal of Machine Learning and Cybernetics, 2, 107–122, 2011.
https://doi.org/10.1007/s13042-011-0019-y

[15] Huang, G.; Zhou, H.; Ding, X.; Zhang, R. (2012); Extreme learning machine for regression and multiclass classification, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42, 513–529, 2012.
https://doi.org/10.1109/TSMCB.2011.2168604

[16] Huang, G.; Zhu, Q.; Siew, C. (2006); Extreme learning machine: theory and applications, Neurocomputing, 70, 489–501, 2006.
https://doi.org/10.1016/j.neucom.2005.12.126

[17] Lanckriet, G.; Cristianini, N.; Bartlett, P.; Ghaoui, L.; Jordan, M. (2004); Learning the kernel matrix with semidefinite programming, Journal of Machine Learning Research, 5, 27–72, 2004.

[18] Landwehr, N.; Kersting, K.; Raedt, L. (2005); nFOIL: Integrating naıve bayes and FOIL, Proceedings of AAAI, 795–800, 2005.

[19] Lao, N.; Mitchell, T.; Cohen, W. (2011); Random walk inference and learning in a large scale knowledge base, Proceedings of EMNLP, 529–539, 2011.

[20] Lao, N.; Minkov, E.; Cohen, W. (2015); Learning relational features with backward random walks, Proceedings of ACL, 666–675, 2015.

[21] Lao, N.; Subramanya, A.; Pereira, F.; Cohen, W. (2012); Reading the web with learned syntactic-semantic inference rules, Proceedings of EMNLP, 1017–1026, 2012.

[22] Lao, N.; Mitamura, T.; Mitchell, T.; Zuo, W. (2012); Efficient random walk inference with knowledge bases, PhD Thesis, 2012.

[23] Lavrac, N.; Dzeroski, S. (1994), Inductive logic programming, Proceedings of Workshop on Logic Programming, 146–160, 1994.

[24] Lee, K.; Man, Z.; Wang, D.; Cao, Z. (2013); Classification of bioinformatics dataset using finite impulse response extreme learning machine for cancer diagnosis, Neural Computing and Applications, 22, 457–468, 2013.
https://doi.org/10.1007/s00521-012-0847-z

[25] Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. (2015); Learning entity and relation embeddings for knowledge graph completion, Proceedings of AAAI, 2181–2187, 2015.

[26] Ma, C.; OuYang J.; Chen, H.; Ji, J. (2016); A novel kernel extreme learning machine algorithm based on self-adaptive artificial bee colony optimisation strategy, International Journal of Systems Science, 47, 1342–1357, 2016.
https://doi.org/10.1080/00207721.2014.924602

[27] Nickel, M.; Murphy, K.; Tresp, V.; Gabrilovich, E. (2015); A review of relational machine learning for knowledge graphs, Proceedings of IEEE, 104, 11-33, 2015.

[28] Nickel, M.; Tresp, V.; Kriegel, H. (2011); A three-way model for collective learning on multi-relational data, Proceedings of ICML, 809–816, 2011.

[29] Nickel, M.; Rosasco, L.; Poggio, T. (2016); Holographic embeddings of knowledge graphs, Proceedings of AAAI, 1955–1961, 2016.

[30] Niu, F.; Ré C.; Doan, A.; Shavlik, J. (2011); Tuffy: Scaling up statistical inference in markov logic networks using an rdbms, Proceedings of the VLDB Endowment, 4, 373–384, 2011.
https://doi.org/10.14778/1978665.1978669

[31] Quinlan, J. (1990); Learning logical definitions from relations, Machine Learning, 5, 239– 266, 1990.
https://doi.org/10.1007/BF00117105

[32] Richardson, M.; Domingos, P. (2006); Markov logic networks, Machine Learning, 62, 107– 136, 2006.
https://doi.org/10.1007/s10994-006-5833-1

[33] Socher, R.; Chen, D.; Manning, C.; Ng, A. (2013); Reasoning with neural tensor networks for knowledge base completion, Proceedings of NIPS, 926–934, 2013.

[34] Su, L.; Yao, M. (2013); Extreme learning machine with multiple kernels, Proceedings of ICCA, 424–429, 2013.
https://doi.org/10.1109/ICCA.2013.6565148

[35] Suchanek, F.; Kasneci, G.; Weikum, G. (2007); Yago: a core of semantic knowledge, Proceedings of WWW, 697–706, 2007.

[36] Wang, Q.; Mao, Z. Wang, B.; Guo, L. (2017); Knowledge graph embedding: a Survey of approaches and applications, IEEE Transactions on Knowledge and Data Engineering, 2724–2743, 2017.

[37] Wang, W.; Mazaitis, K.; Cohen, W. (2013); Programming with personalized pagerank: a locally groundable first-order probabilistic logic, Proceedings of CIKM, 2129–2138, 2013.

[38] West, R.; Gabrilovich, E.; Murphy, K.; Sun, S.; Gupta, R.; Lin, D. (2014); Knowledge base completion via search-based question answering, Proceedings of WWW, 515–526, 2014.

[39] Zheng, W.; Qian, Y.; Lu, H. (2013); Text categorization based on regularization extreme learning machine, Neural Computing and Applications, 22, 447–456, 2013.
https://doi.org/10.1007/s00521-011-0808-y

[40] Zong, W.; Huang, G. (2011); Face recognition based on extreme learning machine, Neurocomputing, 74, 2541–2551, 2011.
https://doi.org/10.1016/j.neucom.2010.12.041
Published
2018-02-12
How to Cite
LIN, Xixun et al. A Knowledge Base Completion Model Based on Path Feature Learning. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, [S.l.], v. 13, n. 1, p. 71-82, feb. 2018. ISSN 1841-9844. Available at: <http://univagora.ro/jour/index.php/ijccc/article/view/3104>. Date accessed: 05 july 2020. doi: https://doi.org/10.15837/ijccc.2018.1.3104.

Keywords

knowledge base completion, random walks, path features, extreme learning machine