KRONE: Hierarchical and Modular Log Anomaly Detection
Summary: arXiv:2602.07303v2 Announce Type: replace-cross
Abstract: Log anomaly detection is crucial for uncovering system failures and security risks. Although logs originate from nested component executions with clear boundaries, this structure is lost when stored as flat sequences. As a result, state-of-the-art methods often miss true dependencies within executions while learning spurious correlations across unrelated events. We propose KRONE, the first hierarchical anomaly detection framework that automatically derives execution hierarchies from flat logs to enable modular, multi-level anomaly detection.
Introduction
In today’s complex software ecosystems, the ability to accurately detect anomalies in log data is essential for maintaining system integrity and security. Traditional methods often fail to recognize the inherent structure of logs, leading to potential oversights in identifying critical dependencies and correlations. KRONE addresses these challenges by introducing a sophisticated framework that not only recognizes but also utilizes the hierarchical nature of log data.
KRONE Log Abstraction Model
At the heart of the KRONE framework lies the KRONE Log Abstraction Model. This innovative model extracts application-specific semantic hierarchies from flat log sequences. By doing so, it recursively decomposes these sequences into coherent execution units called KRONE Seqs. This transformation facilitates a shift from sequence-level anomaly detection to a modular approach that focuses on KRONE Seq-level tasks.
Hybrid Modular Detection Strategy
For each test KRONE Seq, the framework employs a hybrid modular detection strategy that includes:
- Local-Context Detector: An efficient, level-independent detector designed for rapid filtering of anomalies.
- Nested-Aware Detector: This component captures cross-level semantic dependencies, enhancing the accuracy of anomaly detection.
- LLM-based Anomaly Detection: Leveraging large language models, KRONE provides advanced anomaly detection coupled with explanatory insights.
Optimization Techniques
KRONE enhances its detection capabilities through several optimization techniques, including:
- Cached Result Reuse: This allows the framework to avoid redundant computations, improving efficiency.
- Early-Exit Strategies: By implementing these strategies along the hierarchy, KRONE can quickly terminate searches when anomalies are detected, further enhancing performance.
Experimental Results
Comprehensive experiments conducted on three public benchmarks and an industrial dataset from ByteDance Cloud reveal that KRONE significantly outperforms existing methods. Key findings include:
- Accuracy improvements ranging from 42.49% to 87.98%.
- F1 score enhancement by 10.07%, increasing from 82.76% to 92.83%.
- Data efficiency, achieving a 117.3x reduction in resource usage.
- Resource efficiency improvements with a 43.7x reduction in required computational resources.
- Minimal LLM usage, at only 1.1% to 3.3% of the test data.
Conclusion
KRONE represents a significant advancement in the field of log anomaly detection, combining hierarchical modeling with modular detection strategies to improve accuracy, efficiency, and interpretability. As systems grow increasingly complex, the need for such innovative solutions becomes paramount. For more information, visit the following resources: