Abstract
Background: Knowledge representation learning aims at mapping entity and relational data in knowledge graphs to a low-dimensional space in the form of vectors. The existing work has mainly focused on structured information representation of triples or introducing only one additional kind of information, which has large limitations and reduces the representation efficiency.
Objective: This study aims to combine entity description information and textual relationship description information with triadic structure information, and then use the linear mapping method to linearly transform the structure vector and text vector to obtain the joint representation vector.
Methods: A knowledge representation learning (DRKRL) model that fuses external information for semantic enhancement is proposed, which combines entity descriptions and textual relations with a triadic structure. For entity descriptions, a vector representation is performed using a Bi-directional Long- and Short-term Memory Network (Bi-LSTM) model and an attention mechanism. For the textual relations, a convolutional neural network is used to vectorially encode the relations between entities, and then an attention mechanism is used to obtain valuable information as complementary information to the triad.
Results: Link prediction and triadic group classification experiments were conducted on the FB15K, FB15K-237, WN18, WN18RR, and NELL-995 datasets. Theoretical analysis and experimental results show that the DRKRL model proposed in this paper has higher accuracy and efficiency compared with existing models.
Conclusion: Combining entity description information and textual relationship description information with triadic structure information can make the model have better performance and effectively improve the knowledge representation learning ability.
Graphical Abstract
[1]
X. Chen, S. Jia, and Y. Xiang, "A review: Knowledge reasoning over knowledge graph", Expert Syst. Appl., vol. 141, p. 112948, .
2020 [http://dx.doi.org/10.1016/j.eswa.2019.112948]
2020 [http://dx.doi.org/10.1016/j.eswa.2019.112948]
[2]
C. Peng, F. Xia, M. Naseriparsa, and F. Osborne, "Knowledge graphs: Opportunities and challenges", Artif. Intell. Rev., pp. 1-32, 2023.
[PMID: 37362886]
[PMID: 37362886]
[3]
J. Yan, C. Wang, W. Cheng, M. Gao, and A. Zhou, "A retrospective of knowledge graphs", Front. Comput. Sci., vol. 12, no. 1, pp. 55-74, 2018.
[http://dx.doi.org/10.1007/s11704-016-5228-9]
[http://dx.doi.org/10.1007/s11704-016-5228-9]
[4]
S. Ji, S. Pan, E. Cambria, P. Marttinen, and P.S. Yu, "A survey on knowledge graphs: Representation, acquisition, and applications", IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 2, pp. 494-514, 2022.
[http://dx.doi.org/10.1109/TNNLS.2021.3070843] [PMID: 33900922]
[http://dx.doi.org/10.1109/TNNLS.2021.3070843] [PMID: 33900922]
[5]
K. Bollacker, C. Evans, and P. Paritosh, "Freebase: A collaboratively created graph database for structuring human knowledge", In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data . . Vancouver Canada: ACM, 2008, pp. 1247-1250
[6]
J. Lehmann, R. Isele, M. Jakob, A. Jentzsch, D. Kontokostas, P.N. Mendes, S. Hellmann, M. Morsey, P. van Kleef, S. Auer, and C. Bizer, "DBpedia – A large-scale, multilingual knowledge base extracted from Wikipedia", Semant. Web, vol. 6, no. 2, pp. 167-195, 2015.
[http://dx.doi.org/10.3233/SW-140134]
[http://dx.doi.org/10.3233/SW-140134]
[7]
F.M. Suchanek, G. Kasneci, and G. Weikum, "Yago: A core of semantic knowledge", In Proceedings of the 16th International Conference on World Wide Web 2007: 697-706Banff Alberta Canada: ACM,, , 2007, pp. 697-706
[8]
T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, B. Yang, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling, "Never-ending learning", Commun. ACM, vol. 61, no. 5, pp. 103-115, 2018.
[http://dx.doi.org/10.1145/3191513]
[http://dx.doi.org/10.1145/3191513]
[9]
R. Wang, J. Deng, X. Guan, and Y. He, "A framework of genealogy knowledge reasoning and visualization based on a knowledge graph", Libr. Hi Tech, 2023.
[http://dx.doi.org/10.1108/LHT-05-2022-0265]
[http://dx.doi.org/10.1108/LHT-05-2022-0265]
[10]
Z. Chen, Y. Wang, B. Zhao, J. Cheng, X. Zhao, and Z. Duan, "Knowledge graph completion: A review", IEEE Access, vol. 8, pp. 192435-192456, 2020.
[http://dx.doi.org/10.1109/ACCESS.2020.3030076]
[http://dx.doi.org/10.1109/ACCESS.2020.3030076]
[11]
A. Jaya Kumar, C. Schmidt, and J. Köhler, "A knowledge graph based speech interface for question answering systems", Speech Commun., vol. 92, pp. 1-12, 2017.
[http://dx.doi.org/10.1016/j.specom.2017.05.001]
[http://dx.doi.org/10.1016/j.specom.2017.05.001]
[12]
L. Jiang, and Z. Meng, "Knowledge-based visual question answering using multi-modal semantic graph", Electronics, vol. 12, no. 6, p. 1390, 2023.
[http://dx.doi.org/10.3390/electronics12061390]
[http://dx.doi.org/10.3390/electronics12061390]
[13]
X. Chang, P. Ren, P. Xu, Z. Li, X. Chen, and A. Hauptmann, "A comprehensive survey of scene graphs: Generation and application", IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 1-26, 2023.
[http://dx.doi.org/10.1109/TPAMI.2021.3137605] [PMID: 34941499]
[http://dx.doi.org/10.1109/TPAMI.2021.3137605] [PMID: 34941499]
[14]
Y. Xiao, L. Zhang, J. Huang, L. Zhang, and J. Wan, "An information retrieval-based joint system for complex chinese knowledge graph question answering", Electronics, vol. 11, no. 19, p. 3214, 2022.
[http://dx.doi.org/10.3390/electronics11193214]
[http://dx.doi.org/10.3390/electronics11193214]
[15]
D. Genest, and M. Chein, "A content-search information retrieval process based on conceptual graphs", Knowl. Inf. Syst., vol. 8, no. 3, pp. 292-309, 2005.
[http://dx.doi.org/10.1007/s10115-004-0179-0]
[http://dx.doi.org/10.1007/s10115-004-0179-0]
[16]
X. Zhao, A. Li, R. Jiang, K. Chen, and Z. Peng, "Householder transformation-based temporal knowledge graph reasoning", Electronics, vol. 12, no. 9, p. 2001, 2023.
[http://dx.doi.org/10.3390/electronics12092001]
[http://dx.doi.org/10.3390/electronics12092001]
[17]
Z. Liu, M. Sun, and Y. Lin, "Knowledge representation learning: A review", J. Comput. Res. Develop., vol. 53, no. 2, pp. 247-261, 2016.
[18]
Q. He, S. Liu, and Y. Liu, "Optimal recommendation models based on knowledge representation learning and graph attention networks", IEEE Access, vol. 11, pp. 19809-19818, 2023.
[http://dx.doi.org/10.1109/ACCESS.2023.3248618]
[http://dx.doi.org/10.1109/ACCESS.2023.3248618]
[19]
F. Chen, G. Yin, Y. Dong, G. Li, and W. Zhang, "KHGCN: Knowledge-enhanced recommendation with hierarchical graph capsule network", Entropy, vol. 25, no. 4, p. 697, 2023.
[http://dx.doi.org/10.3390/e25040697] [PMID: 37190485]
[http://dx.doi.org/10.3390/e25040697] [PMID: 37190485]
[20]
B. Zhu, T. Bao, R. Han, H. Cui, J. Han, L. Liu, and T. Peng, "An effective knowledge graph entity alignment model based on multiple information", Neural Netw., vol. 162, pp. 83-98, 2023.
[http://dx.doi.org/10.1016/j.neunet.2023.02.029] [PMID: 36893693]
[http://dx.doi.org/10.1016/j.neunet.2023.02.029] [PMID: 36893693]
[21]
X. Yu, Q. Peng, F. Jiang, J. Du, H. Liang, and J. Liu, "Multi-head attention and knowledge graph based dual target graph collaborative filtering network", Neural Process. Lett., vol. 55, pp. 9155-9177, 2023.
[http://dx.doi.org/10.1007/s11063-023-11197-1]
[http://dx.doi.org/10.1007/s11063-023-11197-1]
[22]
G. Xu, Q. Zhang, D. Yu, S. Lu, and Y. Lu, "JKRL: Joint knowledge representation learning of text description and knowledge graph", Symmetry, vol. 15, no. 5, p. 1056, 2023.
[http://dx.doi.org/10.3390/sym15051056]
[http://dx.doi.org/10.3390/sym15051056]
[23]
Z. Han, F. Chen, H. Zhang, Z. Yang, W. Liu, Z. Shen, and H. Xiong, "An attention-based representation learning model for multiple relational knowledge graph", Expert Syst., vol. 40, no. 6, p. e13234, .
2023 [http://dx.doi.org/10.1111/exsy.13234]
2023 [http://dx.doi.org/10.1111/exsy.13234]
[24]
Y. Sun, A. Chen, and C. Chen, "“A joint model for representation learning of tibetan knowledge graph based on encyclopedia”, ACM Trans. Asian Low-Resour", Lang. Inf. Process., vol. 20, no. 2, pp. 1-17, 2021.
[25]
H.J. Song, and S.B. Park, "Enriching translation-based knowledge graph embeddings through continual learning", IEEE Access, vol. 6, pp. 60489-60497, 2018.
[http://dx.doi.org/10.1109/ACCESS.2018.2874656]
[http://dx.doi.org/10.1109/ACCESS.2018.2874656]
[26]
H. Cai, V.W. Zheng, and K.C.C. Chang, "A comprehensive survey of graph embedding: Problems, techniques, and applications", IEEE Trans. Knowl. Data Eng., vol. 30, no. 9, pp. 1616-1637, 2018.
[http://dx.doi.org/10.1109/TKDE.2018.2807452]
[http://dx.doi.org/10.1109/TKDE.2018.2807452]
[27]
T. Zhang, T. Xue, and X. Sun, "Overview on knowledge graph embedding technology research", J. Softw., vol. 34, no. 1, pp. 277-311, 2021.
[28]
J. Li, J. Xiang, and J. Cheng, "EARR: Using rules to enhance the embedding of knowledge graph", Expert Syst. Appl., vol. 232, p. 120831, .
2023 [http://dx.doi.org/10.1016/j.eswa.2023.120831]
2023 [http://dx.doi.org/10.1016/j.eswa.2023.120831]
[29]
Q. Wang, Z. Mao, B. Wang, and L. Guo, "Knowledge graph embedding: A survey of approaches and applications", IEEE Trans. Knowl. Data Eng., vol. 29, no. 12, pp. 2724-2743, 2017.
[http://dx.doi.org/10.1109/TKDE.2017.2754499]
[http://dx.doi.org/10.1109/TKDE.2017.2754499]
[30]
X. Ge, Y.C. Wang, B. Wang, and C.C.J. Kuo, "CORE: A knowledge graph entity type prediction method via complex space regression and embedding", Pattern Recognit. Lett., vol. 157, pp. 97-103, 2022.
[http://dx.doi.org/10.1016/j.patrec.2022.03.024]
[http://dx.doi.org/10.1016/j.patrec.2022.03.024]
[31]
M. Li, Z. Sun, and W. Zhang, "SLAN: Similarity-aware aggregation network for embedding out-of-knowledge-graph entities", Neurocomputing, vol. 491, pp. 186-196, 2022.
[http://dx.doi.org/10.1016/j.neucom.2022.03.063]
[http://dx.doi.org/10.1016/j.neucom.2022.03.063]
[32]
R. Ma, F. Guo, Z. Li, and L. Zhao, "Knowledge graph random neural networks for recommender systems", Expert Syst. Appl., vol. 201, p. 117120, .
2022 [http://dx.doi.org/10.1016/j.eswa.2022.117120]
2022 [http://dx.doi.org/10.1016/j.eswa.2022.117120]
[33]
A. Bordes, N. Usunier, and A. Garcia-Duran, Translating embeddings for modeling multi-relational data.", In: Proc of Advances in Neural Information Processing Systems (NIPS)., MIT Press: Cambridge, MA, 2013, pp. 2787-2795.
[34]
W. Zhen, J. Zhang, and J. Feng, "Knowledge graph embedding by translating on hyperplanes", In Proc of the 28th AAAI Confon Artificial Intelligence, 2014, pp. 1112-1119
[35]
Y. Lin, Z. Liu, and M. Sun, "Learning entity and relation embeddings for knowledge graph completion", In Proc of the 29th AAAI Conf on Artificial Intelligence, 2015, pp. 2181-2187
[36]
G. Ji, S. He, and L. Xu, "Knowledge graph embedding via dynamic mapping matrix", In Proc of the 53rd Annual Meeting of the ACL and the 7th Int Joint Conf on Natural Language Processing., vol. 1, Stroudsburg, PA ACL, 2015, pp. 687-696
[37]
X. Han, M. Huang, and H. Yu, "A generative mixture model for knowledge graph embedding", In Proc of the 54th Annual Meeting of the ACL, vol. 1, Stroudsburg, PA ACL, 2016, pp. 2316-2325
[38]
W. Zhen, J. Zhang, and J. Feng, "Knowledge graph and text jointly embedding", In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1591-1601
[39]
R. Xie, Z. Liu, and J. Jia, "Representation learning of knowledge graphs with entity descriptions", In Proc of the 30th AAAI Conf on Artificial Intelligence . Palo Alto, CA: AAAI Press, 2016, pp. 2659-2665
[40]
R. Xie, Z. Liu, and M. Sun, "Representation learning of knowledge graphs with hierarchical types", In Proc of the 25th Int Joint Conf on Artificial Intelligence (IJCAI) . Palo Alto, CA: AAAI, 2016, pp. 2965-2971
[41]
F. Li, X. He, and R. Dong, "Multi-hop inference model for knowledge graphs incorporating semantic information", Patt. Recogn. Artif. Intellig., vol. 35, no. 11, pp. 1025-1032, 2022.
[42]
Qiaosong. Chen, and Aodong. Guo, "Recommendation model by integrating knowledge graph and image features", Dianzi Yu Xinxi Xuebao, vol. 44, pp. 1-11, 2022.
[43]
X. Yang, and N. Wang, "A confidence-aware and path-enhanced convolutional neural network embedding framework on noisy knowledge graph", Neurocomputing, vol. 545, p. 126261, .
2023 [http://dx.doi.org/10.1016/j.neucom.2023.126261]
2023 [http://dx.doi.org/10.1016/j.neucom.2023.126261]
[44]
M. Sabet, M. Pajoohan, and M.R. Moosavi, "Representation learning of knowledge graphs with correlation-based methods", Inf. Sci., vol. 641, p. 119043, .
2023 [http://dx.doi.org/10.1016/j.ins.2023.119043]
2023 [http://dx.doi.org/10.1016/j.ins.2023.119043]
[45]
Q. Fang, X. Zhang, J. Hu, X. Wu, and C. Xu, "Contrastive multi-modal knowledge graph representation learning", IEEE Trans. Knowl. Data Eng., vol. 35, no. 9, pp. 8983-8996, 2023.
[http://dx.doi.org/10.1109/TKDE.2022.3220625]
[http://dx.doi.org/10.1109/TKDE.2022.3220625]
[46]
Z. Lyu, Y. Wu, and J. Lai, "Knowledge enhanced graph neural networks for explainable recommendation", IEEE Trans. Knowl. Data Eng., vol. 35, pp. 4954-4968, 2023.
[47]
D Q. Nguyen, TD. Nguyen, and DQ. Nguyen, “A novel embedding model for knowledge base completion based on convolutional neural network”, Proc of the North American Chapter of the Association for Computational Linguistics., Stroudsburg, PA : ACL, 2018, pp. 327-333.
[48]
T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel, "Convolutional 2d knowledge graph embeddings", Proc. Conf. AAAI Artif. Intell., vol. 32, no. 1, 2018.
[http://dx.doi.org/10.1609/aaai.v32i1.11573]
[http://dx.doi.org/10.1609/aaai.v32i1.11573]
[50]
S. Vashishth, S. Sanyal, V. Nitin, N. Agrawal, and P. Talukdar, "InteractE: Improving convolution-based knowledge graph embeddings by increasing feature interactions", Proc. Conf. AAAI Artif. Intell., vol. 34, no. 3, pp. 3009-3016, 2020.
[http://dx.doi.org/10.1609/aaai.v34i03.5694]
[http://dx.doi.org/10.1609/aaai.v34i03.5694]
[51]
N. Thomas, Kipf,Max Welling. Semi-Supervised Classification with Graph Convolutional Networks ", CoRR, 2016. abs/1609.02907., .
[52]
M. Schlichtkrull, T.N. Kipf, and P. Bloem, "Modeling relational data with graph convolutional networks European semantic web conference. Springer, Cham, 2018: 593-607",
[53]
F. Che, D. Zhang, J. Tao, M. Niu, and B. Zhao, "ParamE: Regarding neural network parameters as relation embeddings for knowledge graph completion", Proc. Conf. AAAI Artif. Intell., vol. 34, no. 3, pp. 2774-2781, 2020.
[http://dx.doi.org/10.1609/aaai.v34i03.5665]
[http://dx.doi.org/10.1609/aaai.v34i03.5665]
[54]
K. Toutanova, D. Chen, and P. Pantel, "Representing text for joint embedding of text and knowledge bases", In Proceedings of the 2015 conference on empirical methods in natural language processing 2015 2015:, pp. 1499-1509
[55]
G. Ji, and L. Kang, "Knowledge graph completion with adaptive sparse transfer matrix", In Proceedings of the 30th AAAI, 2016, pp. 985-991
[56]
F. Miao, and Z. Qiang, "and Emily Chang, "Transition-based knowledge
graph embedding with relational mapping properties", Proc of the
28th Pacific Asia Conf on Language, Information and Computing.
Stroudsburg, PA: ACL 2014: 328-337",
[57]
X. Han, M. Huang, and H. Yu, TransA: An adaptive approach for knowledge graph embedding ", arXiv preprint, arXiv, pp. 1509- 05490,, 2015.
[58]
X. Lv, L. Hou, and J. Li, "Differentiating concepts and instances for knowledge graph embedding , Proceedings of the 2018 conference
on empirical methods in natural language processing. brussels,
Belgium: Association for computational linguistics 2018: 1971-
1979",
[59]
Y. Lin, Z. Liu, and H. Luan, "Modeling relation paths for representation learning of knowledge bases", In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics 2015:, pp. 705-714
[60]
M. Nickel, V. Tresp, and H.P. Kriegel, "A three-way model for collective learning on multi-relational data", In Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 809-816
[61]
B. Yang, W. Yih, and X. He, Embedding entities and relations for learning and inference in knowledge bases ", arXiv preprint, arXiv, pp. 1412-6575, 2014.
[62]
T. Trouillon, J. Welbl, and S. Riedel, "Complex embeddings for simple link prediction", In Proc of the 33rd Int Conf on Machine Learning (ICML) New York: ACM, 2016, pp. 2071-2080
[63]
Z. Sun, Z. Deng, and J. Nie, Rotate: Knowledge graph embedding by relational rotation in complex space ", arXiv preprint, arXiv, pp. 1902-10197,, 2019.
[64]
F. Jun, M. Huang, and Y. Yang, "GAKE: Graph aware knowledge embedding Proceedings of Coling 2016, the 26th International
Conference on Computational Linguistics: Technical Papers. 2016:
641-651",
[65]
R. Xie, Z. Liu, and H. Luan, "Image-embodied knowledge representation learning", In Proc of the 26th Int Joint Conf on Artificial Intelligence San Francisco: Morgan Kaufmann, 2017, pp. 3140-3146
[66]
X. Hu, Z. Wang, and Y. Sun, "Knowledge graph representation method combined with semantic parsing", J. Comput. Res. Develop., vol. 59, no. 12, pp. 2878-2888, 2022.
[67]
Y. Ning, Z. Gang, and J. Lu, "A representation learning method of knowledge graph integrating relation path and entity description information", Journal of Computer Research and Development, vol. 59, no. 09, pp. 1966-1979, 2022.
[68]
L. Song, S. Shu, and X. Hao, "Knowledge representation learning method integrating textual description and hierarchical type", J. Zhejiang Univ. Eng. Sci., vol. 57, no. 05, pp. 911-920, 2023.
[69]
Y. Mao, and H. Chen, "Rule-guided compositional representation learning on knowledge graphs with hierarchical types", Mathematics, vol. 9, no. 16, p. 1978, 2021.
[http://dx.doi.org/10.3390/math9161978]
[http://dx.doi.org/10.3390/math9161978]
[70]
S. Ying, D. Ning, and T. Zheng Haitao, "Modeling relation paths for knowledge graph completion", IEEE Trans. Knowl. Data Eng., vol. 33, no. 11, pp. 3607-3617, 2020.
[71]
K. Toutanova, and D. Chen, "Observed versus latent features for knowledge base and text inference", In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, 2015, pp. 57-66
Related Books
Practical Digital Forensics: A Guide for Windows and Linux Users
The Future of Computing: Ubiquitous Applications and Technologies
Metal Complexes: Environmental and Biomedical Applications
Artificial Intelligence: A Multidisciplinary Approach towards Teaching and Learning
Smart Home and Industrial IoT Devices: Critical Perspectives on Cyberthreats, Frameworks and Protocols
A Practitioner's Approach to Problem-Solving using AI
A Handbook of Computational Linguistics: Artificial Intelligence in Natural Language Processing
Green Industrial Applications of Artificial Intelligence and Internet of Things
Cyber Forensics and Investigation on Smart Devices
Digital Innovation Adoption: Architectural Recommendations and Security Solutions
