Learning Chain Of Thoughts Prompts for Predicting Entities, Relations, and even Literals on Knowledge Graphs
Summary: arXiv:2604.12651v1 Announce Type: cross
Abstract
Knowledge graph embedding (KGE) models have demonstrated considerable effectiveness in link prediction tasks. However, they often encounter significant challenges when it comes to handling unseen entities, relations, and particularly literals. This limitation restricts their applicability in dynamic and heterogeneous graph environments. In contrast, pretrained large language models (LLMs) exhibit a remarkable ability to generalize through the use of prompting techniques.
Introduction to RALP
In response to these challenges, we present RALP (Relation-Aware Language Prompting), which reformulates the link prediction task as a prompt learning problem. RALP leverages string-based chain-of-thought (CoT) prompts to function as scoring mechanisms for triples. This innovative approach not only enhances the capability to predict missing entities and relations but also facilitates the prediction of entire triples.
Methodology
RALP employs Bayesian Optimization through the MIPRO algorithm to efficiently identify effective prompts. Remarkably, it achieves this with fewer than 30 training examples and operates without requiring gradient access. During the inference phase, RALP utilizes the learned prompts to predict missing entities, relations, or complete triples while assigning confidence scores based on the quality of these prompts.
Evaluation and Results
Our evaluation of RALP encompasses a variety of benchmarks, including transductive tasks, numerical data, and OWL instance retrieval challenges. The results reveal that RALP significantly enhances the performance of state-of-the-art KGE models, achieving over a 5% improvement in Mean Reciprocal Rank (MRR) across multiple datasets. Furthermore, it demonstrates superior generalization capabilities through the generation of high-quality inferred triples.
OWL Reasoning Tasks
In specific OWL reasoning tasks that involve complex class expressions such as ∃ hasChild.Female and ≥ 5 hasChild.Female, RALP achieves an impressive Jaccard similarity score exceeding 88%. These findings underline the efficacy of prompt-based reasoning using LLMs as a versatile alternative to traditional embedding-based methods.
Open Source Implementation
In our commitment to advancing the field, we have released the complete implementation, training, and evaluation pipeline as open source. Researchers and practitioners can access and utilize this resource at the following link: https://github.com/dice-group/RALP.
Conclusion
RALP represents a significant advancement in the realm of knowledge graph embeddings by integrating the strengths of prompt learning and large language models. This innovative approach not only enhances the predictive capabilities for unseen entities and relations but also provides a robust framework for dealing with dynamic graph structures. The open-source release of RALP sets the stage for further research and applications in this evolving field.