SecureRouter: Encrypted Routing for Efficient Secure Inference
Summary: arXiv:2604.15499v1 | Announce Type: cross
Abstract: Cryptographically secure neural network inference typically relies on secure computing techniques such as Secure Multi-Party Computation (MPC), enabling cloud servers to process client inputs without decrypting them. Although prior privacy-preserving inference systems co-design network optimizations with MPC, they remain slow and costly, limiting real-world deployment.
A major bottleneck is their use of a single, fixed transformer model for all encrypted inputs, ignoring that different inputs require different model sizes to balance efficiency and accuracy. We present SecureRouter, an end-to-end encrypted routing and inference framework that accelerates secure transformer inference through input-adaptive model selection under encryption.
SecureRouter establishes a unified encrypted pipeline that integrates a secure router with an MPC-optimized model pool, enabling coordinated routing, inference, and protocol execution while preserving full data and model confidentiality. The framework includes:
- Training-Phase Components:
- MPC-cost-aware secure router that predicts per-model utility and cost from encrypted features.
- MPC-optimized model pool whose architectures and quantization schemes are co-trained to minimize MPC communication and computation overhead.
- Inference-Phase Components:
- Coordinated routing and inference execution to maximize efficiency during secure computations.
- Preservation of data confidentiality throughout the process.
Compared to prior work, SecureRouter achieves a latency reduction by 1.95x with negligible accuracy loss, offering a practical path toward scalable and efficient secure AI inference. The advancements made through SecureRouter not only address the challenges faced by existing systems but also pave the way for broader applications of secure AI technologies in various sectors.
In addition to its impressive performance metrics, SecureRouter’s open-source implementation is available for public use, fostering collaboration and further development within the research community. Interested developers and researchers can access the code at: https://github.com/UCF-ML-Research/SecureRouter.
The development of SecureRouter represents a significant milestone in the field of secure AI inference, emphasizing the importance of efficiency and adaptability in cryptographic systems. As the demand for privacy-preserving technologies continues to rise, innovations like SecureRouter are crucial for ensuring that secure AI solutions can be effectively implemented in real-world applications.