StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
In the world of machine learning, optimization algorithms play a crucial role in training models effectively. Recently, sign-based optimization algorithms, particularly SignSGD, have gained traction due to their impressive results in distributed learning and training large foundation models. However, SignSGD presents challenges, particularly when dealing with non-smooth objectives, which are common in modern machine learning applications. These objectives arise from various sources, including Rectified Linear Units (ReLUs), max-pools, and mixture-of-experts.
To address the inherent limitations of SignSGD, researchers have introduced a novel algorithm known as StoSignSGD. This innovative approach integrates structural stochasticity into the sign operator while ensuring that the update steps remain unbiased. The implications of this development are significant, particularly in the context of online convex optimization.
Theoretical Advancements
The theoretical framework surrounding StoSignSGD demonstrates its ability to resolve the non-convergence issues that plague SignSGD. Through rigorous analysis, it has been shown that StoSignSGD achieves a sharp convergence rate that aligns with established lower bounds. This advancement is particularly noteworthy as it assures practitioners of the algorithm’s reliability in achieving convergence.
When delving into the more complex realm of non-convex non-smooth optimization, StoSignSGD introduces generalized stationary measures. These measures encompass previous definitions and provide a pathway to prove that StoSignSGD surpasses existing complexity bounds, offering improvements by dimensional factors.
Empirical Performance
Beyond theoretical promises, StoSignSGD has demonstrated robust empirical performance across a variety of large language model (LLM) training scenarios. One standout feature of StoSignSGD is its stability in low-precision FP8 pretraining, a challenging setting where traditional optimizers like AdamW often falter. In this context, StoSignSGD has achieved impressive speedups ranging from 1.44x to 2.14x compared to established baseline methods.
Furthermore, when applied to fine-tuning 7 billion parameter LLMs on mathematical reasoning tasks, StoSignSGD has shown significant performance enhancements over both AdamW and SignSGD. These results not only validate the effectiveness of StoSignSGD but also highlight its potential as a preferred optimization method in challenging scenarios.
Innovative Framework and Ablation Study
To further dissect the mechanisms propelling StoSignSGD’s success, the researchers have developed a sign conversion framework. This framework allows for the transformation of any general optimizer into its unbiased, sign-based counterpart. By utilizing this framework, the researchers have deconstructed the fundamental components of StoSignSGD and conducted a comprehensive ablation study. This study empirically validates the design choices made in the algorithm’s development, providing insights into the factors contributing to its superior performance.
Conclusion
In conclusion, StoSignSGD represents a significant advancement in the field of optimization algorithms for machine learning. By addressing the limitations of SignSGD and demonstrating both theoretical and empirical superiority, StoSignSGD is poised to become a vital tool in the training of large language models. As research continues to evolve, the implications of this work will undoubtedly influence future developments in the optimization landscape.