AEG: A Baremetal Framework for AI Acceleration via Direct Hardware Access in Heterogeneous Accelerators
Summary: arXiv:2604.09565v1 Announce Type: cross
Abstract
This paper introduces a unified, hardware-independent baremetal runtime architecture designed to enable high-performance machine learning (ML) inference on heterogeneous accelerators, such as AI Engine (AIE) arrays, without the overhead of an underlying real-time or general-purpose operating system. Existing edge-deployment frameworks, such as TinyML, often rely on real-time operating systems (RTOS), which introduce unnecessary complexity and performance bottlenecks.
Introduction
In the rapidly evolving field of artificial intelligence, the need for efficient and high-performance machine learning frameworks is paramount. Traditional frameworks often impose limitations due to their reliance on complex operating systems. The AEG framework aims to eliminate these constraints by providing a baremetal runtime that optimally utilizes the capabilities of heterogeneous accelerators.
Key Features of AEG
- Decoupled Runtime Architecture: AEG fundamentally decouples the runtime from hardware specifics by flattening complex control logic into linear, executable Runtime Control Blocks (RCBs).
- Control as Data Paradigm: This paradigm allows high-level models, including Adaptive Data Flow (ADF) graphs, to be executed by a generic engine through a minimal Runtime Hardware Abstraction Layer (RHAL).
- Runtime Platform Management (RTPM): AEG integrates RTPM to manage system-level orchestration, ensuring smooth operation and coordination across components.
- Lightweight Network Stack: The framework includes a lightweight network stack to aid in communication without the baggage of traditional protocols.
- Runtime In-Memory File System (RIMFS): RIMFS effectively manages data in OS-free environments, ensuring efficient data handling.
Performance Evaluation
To demonstrate the efficacy of the AEG framework, a ResNet-18 image classification implementation was conducted. The experimental results highlighted significant advantages over traditional Linux-based Vitis AI deployment:
- Compute Efficiency: AEG achieved 9.2× higher compute efficiency (throughput per AIE tile) compared to Vitis AI.
- Data Movement Overhead: The framework realized a 3–7× reduction in data movement overhead, optimizing resource utilization.
- Latency Variance: AEG showcased near-zero latency variance, with a coefficient of variation (CV) of 0.03%.
- Accuracy: The system achieved 68.78% Top-1 accuracy on the ImageNet dataset using only 28 AIE tiles, a stark contrast to Vitis AI’s requirement of 304 tiles.
Conclusion
The AEG framework represents a significant advancement in the field of AI acceleration, providing an efficient and effective solution for machine learning inference on heterogeneous accelerators. By eliminating the reliance on complex operating systems and optimizing resource management, AEG sets a new standard for performance in edge-deployment frameworks.