ALL-FEM: Agentic Large Language models Fine-tuned for Finite Element Methods
In the rapidly evolving field of computational engineering, finite element (FE) analysis plays a pivotal role in the design and verification of manufactured objects. As outlined in the recent research paper on arXiv (arXiv:2603.21011v2), the integration of artificial intelligence into FE workflows presents a significant advancement, leading to the development of ALL-FEM—an autonomous simulation system that utilizes agentic AI combined with domain-specific large language models (LLMs).
Understanding the Challenge
FE analysis is crucial for simulating complex physical systems, encompassing a range of applications from solid mechanics to fluid dynamics and multiphysics interactions. However, the conventional approach to implementing FE codes and analyzing simulation results requires a high level of expertise across various disciplines, including:
- Numerical analysis
- Continuum mechanics
- Programming and software development
Traditional LLMs can produce FE code; however, they often encounter challenges such as hallucination of results, a lack of understanding of variational structures, and an inability to connect problem statements with verified solutions.
The ALL-FEM Solution
ALL-FEM addresses these limitations by introducing a fine-tuned LLM framework specifically designed for generating FEniCS code across various applications, including solid mechanics, fluid dynamics, and multiphysics scenarios. The research team constructed a comprehensive corpus consisting of over 1000 verified FEniCS scripts. This corpus was developed by:
- Combining more than 500 curated expert codes
- Employing a retrieval-augmented, multi-LLM pipeline to generate and filter codes
- Covering diverse partial differential equations (PDEs), geometries, and boundary conditions
Fine-Tuning and Evaluation
The corpus was utilized to fine-tune LLMs with varying parameter sizes, ranging from 3 billion to 120 billion parameters. The ALL-FEM framework orchestrates specialized agents powered by these fine-tuned LLMs to:
- Formulate problems as PDEs
- Generate and debug corresponding code
- Visualize simulation results effectively
The system was rigorously evaluated against 39 benchmarks, which included complex problems in:
- Linear and nonlinear elasticity
- Plasticity
- Newtonian and non-Newtonian flow
- Thermofluids and fluid-structure interaction
- Phase separation and transport on moving domains
Results and Implications
Embedded within a multi-agent workflow that incorporates runtime feedback, the best-performing fine-tuned model (GPT OSS 120B) achieved a remarkable code-level success rate of 71.79%. This performance notably surpassed a non-agentic deployment of GPT 5 Thinking, demonstrating the effectiveness of the agentic approach.
By showcasing that relatively small, fine-tuned LLMs can facilitate the automation of FE workflows through agentic frameworks, ALL-FEM presents a promising blueprint for the future of autonomous simulation systems in computational science and engineering. This innovative approach not only streamlines the FE analysis process but also opens new avenues for research and development in the field.