When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression
Summary: arXiv:2604.03557v1 Announce Type: new
Abstract
Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. Although such failures are widely observed, the mechanisms by which decoder-only Transformers produce them remain poorly understood. We model next-token prediction as a graph search process over an underlying graph, where entities correspond to nodes and learned transitions form edges. From this perspective, contextual reasoning is a constrained search over a sampled subgraph (intrinsic reasoning), while context-free queries rely on memorized structures in the underlying graph (extrinsic reasoning). We show that reasoning hallucinations arise from two fundamental mechanisms: Path Reuse, where memorized knowledge overrides contextual constraints during early training, and Path Compression, where frequently traversed multi-step paths collapse into shortcut edges in later training. Together, these mechanisms provide a unified explanation for reasoning hallucinations in LLMs and connect to well-known behaviors observed in downstream applications.
Introduction
The emergence of reasoning hallucinations in large language models has become a key concern in the field of artificial intelligence. These hallucinations manifest as coherent yet factually incorrect outputs, prompting researchers to delve deeper into the underlying mechanisms that lead to such phenomena. Understanding these mechanisms is essential for improving the reliability and accuracy of LLMs.
Mechanisms Behind Hallucinations
Our research identifies two primary mechanisms that contribute to the occurrence of reasoning hallucinations:
- Path Reuse: This mechanism highlights how memorized knowledge can sometimes override contextual constraints during early stages of training. As LLMs are exposed to vast amounts of data, they learn to associate certain responses with specific prompts. However, this memorization can lead to outputs that do not align with the current context.
- Path Compression: In later training phases, frequently traversed multi-step paths in the graph can collapse into shortcut edges. This compression can simplify the reasoning process but may also contribute to inaccuracies if the shortcuts bypass critical contextual information.
Graph Search Perspective
We propose modeling next-token prediction as a graph search process. In this model:
- Entities are represented as nodes.
- Learned transitions between these entities form the edges of the graph.
This graph-based approach allows for a clearer understanding of how contextual reasoning and extrinsic reasoning operate. Contextual reasoning is viewed as a constrained search over a sampled subgraph, while context-free queries depend on memorized structures within the underlying graph.
Implications for Future Research
The findings from this research have significant implications for the development of future LLMs. By understanding the mechanics of reasoning hallucinations, researchers can work towards creating models that better adhere to contextual constraints and reduce the likelihood of generating unsupported conclusions. This could lead to advancements in various applications, such as conversational agents, educational tools, and more.
Conclusion
In conclusion, the exploration of reasoning hallucinations in large language models through the lens of graph theory provides valuable insights into the mechanisms at play. By addressing path reuse and path compression, we can pave the way for more reliable and accurate AI systems.