Introduction
Generative Artificial Intelligence (GenAI) has made significant strides in recent years, particularly in applications like image editing, object removal, and prompt-guided image transformation. These features are increasingly being integrated into mobile applications, providing users with powerful tools for creative expression. However, the deployment of Large Vision Models (LVMs) on resource-constrained devices presents significant challenges, primarily due to their high memory and computational requirements.
The Challenge of Deploying LVMs
While Low-Rank Adapters (LoRAs) have emerged as a solution for parameter-efficient task adaptation, current mobile deployment pipelines generally compile separate model binaries for each LoRA along with a copy of the foundation model. This approach leads to:
- Redundant storage requirements
- Increased runtime overhead
- Complexity in managing multiple models
As a result, there is a pressing need for a more efficient approach that can reduce the memory footprint and enhance the performance of generative vision tasks on edge devices.
Proposed Solution: Unified Framework
In this work, we introduce a unified framework designed to enable multi-task GenAI inference on edge devices through a single shared model. The cornerstone of our approach is the innovative treatment of LoRA weights as runtime inputs rather than embedding them within the compiled model graph. This allows for:
- Dynamic task switching at runtime
- Elimination of recompilation needs
- Reduction in storage and overhead costs
Introducing QUAD: Quantization with Unified Adaptive Distillation
To facilitate efficient on-device execution, we propose QUAD (Quantization with Unified Adaptive Distillation), a quantization-aware training strategy that aligns multiple LoRA adapters under a shared quantization profile. This innovative method not only streamlines the deployment process but also enhances the overall performance of the models.
Implementation and Evaluation
Our system has been implemented with a lightweight runtime stack that is fully compatible with mobile Neural Processing Units (NPUs). We conducted extensive evaluations across multiple chipsets to assess the effectiveness of our approach. The experimental results yielded remarkable findings:
- Up to a 6x reduction in memory footprint
- Latency improvements of up to 4x
- High visual quality maintained across various GenAI tasks
Conclusion
In summary, our unified framework, combined with the QUAD strategy, represents a significant advancement in the deployment of Generative Vision Models on edge devices. By allowing for dynamic task switching and efficient on-device execution, we are paving the way for more streamlined, powerful, and accessible GenAI applications on mobile platforms. As the demand for innovative AI-driven features continues to grow, solutions like this will be crucial in making advanced technology available to a broader audience.