A Knowledge Base Completion Model Based on Path Feature Learning

Xixun Lin, Yanchun Liang, Limin Wang, Xu Wang, Mary Qu Yang, Renchu Guan

Abstract


Large-scale knowledge bases, as the foundations for promoting the development of artificial intelligence, have attracted increasing attention in recent years. These knowledge bases contain billions of facts in triple format; yet, they suffer from sparse relations between entities. Researchers proposed the path ranking algorithm (PRA) to solve this fatal problem. To improve the scalability of knowledge inference, PRA exploits random walks to find Horn clauses with chain structures to predict new relations given existing facts. This method can be regarded as a statistical classification issue for statistical relational learning (SRL). However, large-scale knowledge base completion demands superior accuracy and scalability. In this paper, we propose the path feature learning model (PFLM) to achieve this urgent task. More precisely, we define a two-stage model: the first stage aims to learn path features from the existing knowledge base and extra parsed corpus; the second stage uses these path features to predict new relations. The experimental results demonstrate that the PFLM can learn meaningful features and can achieve significant and consistent improvements compared with previous work.

Keywords


knowledge base completion, random walks, path features, extreme learning machine

Full Text:

PDF

References


Agirre, E.; Lacalle, O.; Soroa, A. (2014); Random walks for knowledge-based word sense disambiguation, Computational Linguistics, 40, 57–84, 2014.
https://doi.org/10.1162/COLI_a_00164

Berant, J.; Chou, A.; Frostig, R.; Liang, P. (2013); Semantic parsing on Freebase from question-answer pairs, Proceedings of EMNLP, 1533–1544, 2013.

Bollacker, K.; Evans C.; Paritosh, P.; Sturge, T.; Taylor, J. (2008); Freebase: a collaboratively created graph database for structuring human knowledge, Proceedings of KDD, 1247–1250, 2008.

Bordes, A.; Usunier, N.; García-Durán, A.; Weston, J.; Yakhnenko O. (2013); Translating embeddings for modeling multi-relational data, Proceedings of NIPS, 2787–2795, 2013.

Cao, F.; Liu, B.; Park, D. (2013); Image classification based on effective extreme learning machine, Neurocomputing, 102, 90–97, 2013.
https://doi.org/10.1016/j.neucom.2012.02.042

Carlson, A.; Betteridge, J.; Kisiel, B.; Settles, B.; Hruschka, E.; Mitchell T. (2010); Toward an architecture for never-ending language learning, Proceedings of AAAI, 1306–1313, 2010.

Gardner, M.; Talukdar, P.; Kisiel, B.; Mitchell, T. (2013); Improving learning and inference in a large knowledge-base using latent syntactic cues, Proceedings of EMNLP, 833–838, 2013.

Gardner, M.; Talukdar, P.; Krishnamurthy, J.; Mitchell, T. (2014); Incorporating vector space similarity in random walk inference over knowledge bases, Proceedings of EMNLP, 833–838, 2014.

Gardner, M.; Mitchell, T. (2015); Efficient and expressive knowledge base completion using subgraph feature extraction, Proceedings of EMNLP, 1488–1498, 2015.

Getoor, L.; Taskar, B. (2007); Introduction to statistical relational learning, MIT press, 2007.

Glassick, C.E.; Huber, M.T.; Maeroff, G.I. (2015); DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia, Semantic Web, 6, 167–195, 2015.

Guo, S.; Wang, Q.; Wang, B.; Wang, L.; Guo, L. (2015); Semantically smooth knowledge graph embedding, Proceedings of ACL, 84–94, 2015.

Hoffmann, R.; Zhang, C.; Ling, X.; Zettlemoyer, L.; Weld, D. (2011); Knowledge-based weak supervision for information extraction of overlapping relations, Proceedings of ACL, 541–550, 2011.

Huang, G.; Wang, D.; Lan, Y. (2011); Extreme learning machines: a survey, International Journal of Machine Learning and Cybernetics, 2, 107–122, 2011.
https://doi.org/10.1007/s13042-011-0019-y

Huang, G.; Zhou, H.; Ding, X.; Zhang, R. (2012); Extreme learning machine for regression and multiclass classification, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42, 513–529, 2012.
https://doi.org/10.1109/TSMCB.2011.2168604

Huang, G.; Zhu, Q.; Siew, C. (2006); Extreme learning machine: theory and applications, Neurocomputing, 70, 489–501, 2006.
https://doi.org/10.1016/j.neucom.2005.12.126

Lanckriet, G.; Cristianini, N.; Bartlett, P.; Ghaoui, L.; Jordan, M. (2004); Learning the kernel matrix with semidefinite programming, Journal of Machine Learning Research, 5, 27–72, 2004.

Landwehr, N.; Kersting, K.; Raedt, L. (2005); nFOIL: Integrating naıve bayes and FOIL, Proceedings of AAAI, 795–800, 2005.

Lao, N.; Mitchell, T.; Cohen, W. (2011); Random walk inference and learning in a large scale knowledge base, Proceedings of EMNLP, 529–539, 2011.

Lao, N.; Minkov, E.; Cohen, W. (2015); Learning relational features with backward random walks, Proceedings of ACL, 666–675, 2015.

Lao, N.; Subramanya, A.; Pereira, F.; Cohen, W. (2012); Reading the web with learned syntactic-semantic inference rules, Proceedings of EMNLP, 1017–1026, 2012.

Lao, N.; Mitamura, T.; Mitchell, T.; Zuo, W. (2012); Efficient random walk inference with knowledge bases, PhD Thesis, 2012.

Lavrac, N.; Dzeroski, S. (1994), Inductive logic programming, Proceedings of Workshop on Logic Programming, 146–160, 1994.

Lee, K.; Man, Z.; Wang, D.; Cao, Z. (2013); Classification of bioinformatics dataset using finite impulse response extreme learning machine for cancer diagnosis, Neural Computing and Applications, 22, 457–468, 2013.
https://doi.org/10.1007/s00521-012-0847-z

Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. (2015); Learning entity and relation embeddings for knowledge graph completion, Proceedings of AAAI, 2181–2187, 2015.

Ma, C.; OuYang J.; Chen, H.; Ji, J. (2016); A novel kernel extreme learning machine algorithm based on self-adaptive artificial bee colony optimisation strategy, International Journal of Systems Science, 47, 1342–1357, 2016.
https://doi.org/10.1080/00207721.2014.924602

Nickel, M.; Murphy, K.; Tresp, V.; Gabrilovich, E. (2015); A review of relational machine learning for knowledge graphs, Proceedings of IEEE, 104, 11-33, 2015.

Nickel, M.; Tresp, V.; Kriegel, H. (2011); A three-way model for collective learning on multi-relational data, Proceedings of ICML, 809–816, 2011.

Nickel, M.; Rosasco, L.; Poggio, T. (2016); Holographic embeddings of knowledge graphs, Proceedings of AAAI, 1955–1961, 2016.

Niu, F.; Ré C.; Doan, A.; Shavlik, J. (2011); Tuffy: Scaling up statistical inference in markov logic networks using an rdbms, Proceedings of the VLDB Endowment, 4, 373–384, 2011.
https://doi.org/10.14778/1978665.1978669

Quinlan, J. (1990); Learning logical definitions from relations, Machine Learning, 5, 239– 266, 1990.
https://doi.org/10.1007/BF00117105

Richardson, M.; Domingos, P. (2006); Markov logic networks, Machine Learning, 62, 107– 136, 2006.
https://doi.org/10.1007/s10994-006-5833-1

Socher, R.; Chen, D.; Manning, C.; Ng, A. (2013); Reasoning with neural tensor networks for knowledge base completion, Proceedings of NIPS, 926–934, 2013.

Su, L.; Yao, M. (2013); Extreme learning machine with multiple kernels, Proceedings of ICCA, 424–429, 2013.
https://doi.org/10.1109/ICCA.2013.6565148

Suchanek, F.; Kasneci, G.; Weikum, G. (2007); Yago: a core of semantic knowledge, Proceedings of WWW, 697–706, 2007.

Wang, Q.; Mao, Z. Wang, B.; Guo, L. (2017); Knowledge graph embedding: a Survey of approaches and applications, IEEE Transactions on Knowledge and Data Engineering, 2724–2743, 2017.

Wang, W.; Mazaitis, K.; Cohen, W. (2013); Programming with personalized pagerank: a locally groundable first-order probabilistic logic, Proceedings of CIKM, 2129–2138, 2013.

West, R.; Gabrilovich, E.; Murphy, K.; Sun, S.; Gupta, R.; Lin, D. (2014); Knowledge base completion via search-based question answering, Proceedings of WWW, 515–526, 2014.

Zheng, W.; Qian, Y.; Lu, H. (2013); Text categorization based on regularization extreme learning machine, Neural Computing and Applications, 22, 447–456, 2013.
https://doi.org/10.1007/s00521-011-0808-y

Zong, W.; Huang, G. (2011); Face recognition based on extreme learning machine, Neurocomputing, 74, 2541–2551, 2011.
https://doi.org/10.1016/j.neucom.2010.12.041




DOI: http://dx.doi.org/10.15837/ijccc.2018.1.3104



Copyright (c) 2018 Xixun Lin, Yanchun Liang, Limin Wang, Xu Wang, Mary Qu Yang, Renchu Guan

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CC-BY-NC  License for Website User

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]


INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2016: IF=1.374. .

IJCCC is indexed in Scopus from 2008 (SNIP2016 = 0.701, SJR2016 =0.319):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.