Riemann-Bench: A Benchmark for Moonshot Mathematics
Summary: arXiv:2604.06802v1 Announce Type: new
Abstract: Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competition-style problem solving. However, competition mathematics represents only a narrow slice of mathematical reasoning: problems are drawn from limited domains, require minimal advanced machinery, and can often reward insightful tricks over deep theoretical knowledge.
We introduce Riemann-Bench, a private benchmark of 25 expert-curated problems designed to evaluate AI systems on research-level mathematics that goes far beyond the olympiad frontier. Problems are authored by Ivy League mathematics professors, graduate students, and PhD-holding IMO medalists, and routinely took their authors weeks to solve independently. Each problem undergoes double-blind verification by two independent domain experts who must solve the problem from scratch and yields a unique, closed-form solution assessed by programmatic verifiers.
Key Features of Riemann-Bench
- Expert-Curated Problems: The benchmark consists of 25 carefully selected problems, ensuring a high level of difficulty and relevance to advanced mathematical research.
- Double-Blind Verification: Each problem is verified by two independent experts to maintain integrity and ensure a rigorous evaluation process.
- Unique Solutions: Problems are designed to have unique, closed-form solutions, providing a clear metric for success.
- Unconstrained Evaluation: Frontier models are evaluated as unrestricted research agents, with full access to coding tools and open-ended reasoning capabilities.
- Statistical Rigor: Performance is assessed using an unbiased statistical estimator computed over 100 independent runs per problem.
Results and Implications
Our evaluations reveal that all frontier models currently score below 10%, exposing a substantial gap between olympiad-level problem solving and genuine research-level mathematical reasoning. This performance gap highlights the limitations of current AI systems in addressing complex mathematical challenges that go beyond competition-style problems.
By keeping the benchmark fully private, we ensure that measured performance reflects authentic mathematical capability rather than memorization of training data. This approach aims to foster a deeper understanding of mathematical reasoning in AI systems and to identify the areas where further advancements are necessary.
Conclusion
As AI continues to evolve, the need for robust benchmarks like Riemann-Bench becomes increasingly critical. By challenging AI systems with genuine research-level mathematics, we can better assess their capabilities and guide future developments in the field. The introduction of Riemann-Bench serves as a significant step towards bridging the gap between competition mathematics and the rich, complex landscape of advanced mathematical reasoning.