Introduction
Recent advances in autoregressive language models have brought to light the phenomenon of hallucination, where models generate outputs that diverge from factual information. A new paper, titled “Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation,” explores this phenomenon through a novel experimental approach.
Key Findings
The research presents compelling evidence that hallucination is an early trajectory commitment influenced by asymmetric attractor dynamics. The authors introduce a method called same-prompt bifurcation, which involves repeatedly sampling identical inputs to track spontaneous divergence in generated outputs. This approach allows the researchers to isolate trajectory dynamics from prompt-level confounds.
Methodology
The study was conducted using the Qwen2.5-1.5B model across 61 prompts distributed among six distinct categories. The findings reveal that:
- 27 prompts, representing 44.3%, exhibited bifurcation, where factual and hallucinated trajectories began to diverge at the first generated token.
- The divergence was quantitatively measured with Kullback-Leibler divergence (KL), showing KL = 0 at step 0 and KL > 1.0 at step 1.
Causal Asymmetry
One of the most significant revelations of the study is the pronounced causal asymmetry observed through activation patching across 28 layers of the model. Key findings include:
- Injecting a hallucinated activation into a correct trajectory resulted in output corruption in 87.5% of trials, particularly at layer 20.
- Conversely, recovering a correct trajectory from a hallucinated activation only succeeded in 33.3% of trials at layer 24.
- Both results significantly exceed the baseline corruption rate of 10.4% (p = 0.025) and random-patch control outcomes of 12.5%.
Intervention Dynamics
Further investigations using window patching techniques indicated that correcting a hallucinated output requires sustained multi-step interventions, while corrupting a correct trajectory necessitates only a single perturbation. This highlights the complexity of interventions needed to navigate these dynamics.
Prompt Encoding Insights
The researchers also probed the prompt encoding, revealing that step-0 residual states could predict the per-prompt hallucination rate with a Pearson correlation of r = 0.776 at layer 15 (p < 0.001 compared to a 1000-permutation null). Unsupervised clustering identified five distinct regime-like groups, with a specific focus on a saddle-adjacent cluster containing 12 out of 13 bifurcating false-premise prompts. This suggests that the basin structure is organized around regime commitments that are already discernible at the initial step of encoding.
Conclusion
The findings from this study characterize hallucination as a locally stable attractor basin. The entry into this basin appears to be probabilistic and rapid, while exiting requires coordinated interventions across multiple layers and steps. The research underscores the significance of prompt encoding in influencing the selection of these attractor basins, providing new insights into the dynamics of transformer-based language generation.