One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models
Summary: arXiv:2604.18839v1 Announce Type: cross
Abstract: Looped transformers scale computational depth without increasing parameter count by repeatedly applying a shared transformer block and can be used for iterative refinement, where each loop rewrites a full fixed-size prediction in parallel. On difficult problems, such as those that require search-like computation, reaching a highly structured solution starting from noise can require long refinement trajectories. Learning such trajectories is challenging when training specifies only the target solution and provides no supervision over the intermediate refinement path.
Diffusion models tackle this issue by corrupting data with varying magnitudes of noise and training the model to reverse it in a single step. However, this process misaligns training and testing behaviour. We introduce Denoising Recursion Models, a method that similarly corrupts data with noise but trains the model to reverse the corruption over multiple recursive steps. This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation.
Introduction to Denoising Recursion Models
The research surrounding Denoising Recursion Models (DRMs) represents a significant advancement in the realm of artificial intelligence and machine learning. By leveraging the strengths of looped transformers, DRMs enable more profound reasoning capabilities. While traditional approaches may limit themselves to single-step transformations, the recursive framework of DRMs allows for multiple iterations that enhance the model’s ability to generate structured outputs.
Challenges in Traditional Approaches
Traditional diffusion models have shown promise in various applications but often face obstacles related to training and testing alignment. Key challenges include:
- Lack of Intermediate Supervision: Training typically focuses solely on the final output, neglecting the crucial intermediate steps that lead to that output.
- Single-Step Misalignment: The training process does not reflect the iterative nature of many real-world problems, leading to performance inconsistencies.
- Limited Exploration: The models may adopt a greedy approach, optimizing for immediate rewards rather than considering longer-term outcomes.
The Advantages of Denoising Recursion Models
DRMs address these challenges through several innovative mechanisms:
- Multiple Recursive Steps: By allowing the model to refine its predictions over multiple iterations, DRMs create a more effective learning environment.
- Alignment of Training and Testing: This approach ensures that the training phase mirrors the conditions of the testing phase, improving overall performance.
- Encouragement of Strategic Thinking: The non-greedy nature of the recursion encourages the model to consider broader strategies, leading to more sophisticated outputs.
Experimental Results and Conclusion
Extensive experiments conducted with the Denoising Recursion Model indicate that it outperforms the previously established Tiny Recursion Model (TRM) on the ARC-AGI benchmark, demonstrating breakthrough performance in tasks requiring complex reasoning.
In conclusion, Denoising Recursion Models represent a significant step forward in the field of AI, providing enhanced reasoning capabilities through innovative recursive strategies. As research progresses, these models may pave the way for even more advanced applications in artificial intelligence.