KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning
Summary: arXiv:2604.12487v1 Announce Type: cross
Abstract
Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging.
Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps.
Introduction
In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified “thinking” phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths and perform backtracking when necessary.
Key Features of KG-Reasoner
- End-to-End Integration: Unlike traditional methods, KG-Reasoner combines the reasoning process into one coherent phase, eliminating the limitations of fixed pipelines.
- Dynamic Path Exploration: The model is equipped to explore multiple reasoning paths dynamically, adapting to the complexity of the queries posed.
- Reinforcement Learning Training: The use of RL allows the LLM to learn from its reasoning experiences, improving its performance over time.
- Backtracking Capability: The model can backtrack during the reasoning process, ensuring that previous decisions can be revisited and revised if necessary.
Performance Evaluation
Experiments conducted on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to state-of-the-art methods. The results indicate a significant improvement in handling complex queries that require nuanced reasoning across multiple hops in a knowledge graph.
Conclusion
KG-Reasoner represents a significant advancement in the domain of knowledge graph reasoning and natural language understanding. By integrating reinforcement learning with a unified reasoning framework, KG-Reasoner not only enhances the flexibility and coherence of decision-making but also sets a new standard for future research in this area.
Access the Code
For those interested in exploring the implementation of KG-Reasoner, the codes are available at the following repository: KG-Reasoner GitHub Repository.