Neural Global Optimization via Iterative Refinement from Noisy Samples
Summary: arXiv:2604.03614v1 Announce Type: cross
Abstract: Global optimization of black-box functions from noisy samples is a fundamental challenge in machine learning and scientific computing. Traditional methods such as Bayesian Optimization often converge to local minima on multi-modal functions, while gradient-free methods require many function evaluations. We present a novel neural approach that learns to find global minima through iterative refinement.
Our model takes noisy function samples and their fitted spline representation as input, then iteratively refines an initial guess toward the true global minimum. Trained on randomly generated functions with ground truth global minima obtained via exhaustive search, our method achieves a mean error of 8.05 percent on challenging multi-modal test functions, compared to 36.24 percent for the spline initialization, a 28.18 percent improvement. The model successfully finds global minima in 72 percent of test cases with error below 10 percent, demonstrating learned optimization principles rather than mere curve fitting.
Our architecture combines encoding of multiple modalities including function values, derivatives, and spline coefficients with iterative position updates, enabling robust global optimization without requiring derivative information or multiple restarts.
Introduction
Global optimization is a critical area of research with applications spanning various fields, including engineering, finance, and artificial intelligence. The ability to find global minima efficiently in the presence of noise is particularly important, as many real-world problems involve uncertain data. Traditional optimization techniques often struggle in these scenarios, leading to suboptimal solutions.
Challenges in Traditional Methods
- Bayesian Optimization: While effective in many cases, it often converges to local minima, particularly in complex multi-modal functions.
- Gradient-free Methods: These approaches can be computationally expensive, requiring numerous function evaluations to identify optimal solutions.
Proposed Neural Approach
Our proposed method leverages neural networks to enhance the global optimization process. The architecture is designed to iteratively refine an initial guess based on noisy samples, utilizing a combination of the following:
- Noisy Function Samples: The model learns from the inherent noise in the data to improve its predictions.
- Spline Representation: The fitted spline serves as a smooth approximation of the function, guiding the optimization process.
- Iterative Refinement: The model adjusts its guess incrementally, honing in on the global minimum.
Results and Performance
In extensive testing, our model demonstrated a mean error of 8.05 percent on challenging test functions, significantly outperforming traditional spline initialization methods which achieved a mean error of 36.24 percent. Furthermore, the proposed architecture successfully identified global minima in 72 percent of test cases with an error margin of less than 10 percent.
Conclusion
The findings from our research indicate that the proposed neural optimization approach not only enhances the accuracy of finding global minima but also reduces the dependency on derivative information and multiple restarts. This positions our model as a promising solution for tackling the complexities involved in global optimization of noisy functions.
Future work will focus on expanding the applicability of this method to broader classes of functions and exploring further enhancements in model architecture to improve performance.