Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives
Summary: arXiv:2603.29997v1 | Announce Type: cross
Abstract: Analogical reasoning is a key driver of human generalization in problem-solving and argumentation. Yet, analogies between narrative structures remain challenging for machines. Cognitive engines for structural mapping are not directly applicable, as they assume pre-extracted entities, whereas LLMs’ performance is sensitive to prompt format and the degree of surface similarity between narratives. This gap motivates a key question: What is the impact of enhancing structural mapping with LLM-derived abstractions on their analogical reasoning ability in narratives?
Introduction
Analogical reasoning is a fundamental cognitive process that allows humans to draw connections between disparate ideas, facilitating problem-solving and argumentation. However, teaching machines to understand and utilize analogies within narrative structures presents significant challenges. Traditional cognitive engines often fail in this regard due to their reliance on pre-extracted entities, which do not align well with the dynamic nature of language models (LLMs).
Proposed Framework: YARN
To address these challenges, we propose a modular framework named YARN (Yielding Abstractions for Reasoning in Narratives). YARN leverages the capabilities of LLMs to decompose narratives into manageable units, abstract these units, and subsequently align them across different stories to facilitate analogical reasoning.
Methodology
The YARN framework operates on four defined levels of abstraction, which capture both the general meaning of narrative units and their functional roles within the story. This approach is grounded in established research on framing and aims to enhance the analogical reasoning capabilities of machine learning models.
Experimental Results
Our experiments demonstrate that the use of abstractions consistently enhances model performance. Specifically, the results showed:
- YARN achieved competitive or superior performance when compared to end-to-end LLM baselines.
- Closer error analysis revealed critical areas where abstraction poses challenges, such as:
- Determining the correct level of abstraction.
- Incorporating implicit causality within narratives.
- Identifying and categorizing analogical patterns effectively.
Conclusion
The YARN framework serves as a significant step forward in enhancing the capacity of LLMs for analogical reasoning in narratives. By systematically varying experimental settings, YARN allows for a deeper analysis of component contributions and their impact on overall performance. As part of our commitment to advancing research in this field, we are making the code for YARN openly available for further exploration and development.
Future Work
Future research will focus on refining the abstraction levels and addressing the challenges identified during error analysis. We aim to develop methods that can better incorporate implicit causality and improve the categorization of analogical patterns, ultimately enhancing the effectiveness of machine understanding of narratives.