EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs
Summary: arXiv:2604.19761v1 Announce Type: new
Abstract
Modern machine learning is still largely organized around a single recipe: choose a parameterized model family and optimize its weights. Although highly successful, this paradigm is too narrow for many structured prediction problems, where the main bottleneck is not parameter fitting but discovering what should be computed from the data. Success often depends on identifying the right transformations, statistics, invariances, interaction structures, temporal summaries, gates, or nonlinear compositions, especially when objectives are non-differentiable, evaluation is cross-validation-based, interpretability matters, or continual adaptation is required.
The EvoForest Approach
We present EvoForest, a hybrid neuro-symbolic system for end-to-end open-ended evolution of computation. Rather than merely generating features, EvoForest jointly evolves reusable computational structures, callable function families, and trainable low-dimensional continuous components inside a shared directed acyclic graph. This innovative approach offers several distinct benefits:
- Intermediate Nodes: Store alternative implementations that can be dynamically adjusted based on performance feedback.
- Callable Nodes: Encode reusable transformation families such as projections, gates, and activations, enhancing flexibility in computation.
- Output Nodes: Define candidate predictive computations, allowing for a diverse range of outputs based on input data.
- Global Parameters: Persistent global parameters can be refined through gradient descent, ensuring continuous learning and adaptation.
Evaluation Process
For each graph configuration, EvoForest evaluates the discovered computation and employs a lightweight Ridge-based readout to score the resulting representation against a non-differentiable cross-validation target. This process is crucial for ensuring the robustness and reliability of the model’s predictions.
Feedback Mechanism
The evaluator also produces structured feedback that guides future LLM-driven mutations. This feedback loop is essential for the iterative improvement of the computational structure, allowing EvoForest to adapt and refine its approach over time.
Results and Achievements
In the 2025 ADIA Lab Structural Break Challenge, EvoForest achieved an impressive score of 94.13% ROC-AUC after just 600 evolution steps. This performance significantly exceeded the publicly reported winning score of 90.14% under the same evaluation protocol, showcasing the effectiveness of the EvoForest paradigm in tackling complex machine learning tasks.
Conclusion
The introduction of EvoForest represents a significant advancement in machine learning methodologies. By focusing on the evolution of computational graphs rather than merely optimizing existing models, EvoForest opens new avenues for addressing structured prediction problems. As the field continues to evolve, the insights gained from this innovative approach could lead to more robust, interpretable, and adaptable machine learning systems.