Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation
Abstract: Pretrained models have become standard in both vision and language, yet they typically do not provide reliable measures of confidence. Existing uncertainty estimation methods, such as deep ensembles and MC dropout, are often too computationally expensive to deploy in practice. Evidential Deep Learning (EDL) offers a more efficient alternative, but it requires models to be trained to output evidential quantities from the start, which is rarely true for pretrained networks. To enable EDL-style uncertainty estimation in pretrained models, we propose the Evidential Transformation Network (ETN), a lightweight post-hoc module that converts a pretrained predictor into an evidential model. ETN operates in logit space: it learns a sample-dependent affine transformation of the logits and interprets the transformed outputs as parameters of a Dirichlet distribution for uncertainty estimation. We evaluate ETN on image classification and large language model question-answering benchmarks under both in-distribution and out-of-distribution settings. ETN consistently improves uncertainty estimation over post-hoc baselines while preserving accuracy and adding only minimal computational overhead.
Introduction
The integration of pretrained models into various applications has revolutionized fields such as computer vision and natural language processing. However, one significant challenge persists: the inability of these models to provide reliable confidence measures. This gap in performance has prompted researchers to explore alternative methods for uncertainty estimation.
Current Challenges in Uncertainty Estimation
Traditional methods for uncertainty estimation, including:
- Deep Ensembles
- Monte Carlo (MC) Dropout
These techniques, while effective, are often computationally prohibitive for real-world applications. As a result, there is a pressing need for more efficient methods that do not compromise on performance.
Introducing the Evidential Transformation Network (ETN)
The Evidential Transformation Network (ETN) emerges as a promising solution to this problem. Unlike conventional methods, ETN is designed as a lightweight post-hoc module that can seamlessly convert a pretrained model into an evidential model. This transformation is crucial for enabling reliable uncertainty estimation without the need for extensive retraining.
How ETN Works
ETN operates within logit space, where it learns a sample-dependent affine transformation of the logits generated by the pretrained model. The transformed outputs are then interpreted as parameters of a Dirichlet distribution. This approach allows ETN to provide a robust framework for uncertainty estimation, significantly enhancing the model’s ability to quantify confidence.
Evaluation and Results
To validate the efficacy of ETN, extensive evaluations were conducted across various benchmarks, including:
- Image Classification Tasks
- Large Language Model Question-Answering Tasks
These evaluations were conducted under both in-distribution and out-of-distribution settings. The results demonstrated that ETN consistently outperformed existing post-hoc baselines in terms of uncertainty estimation while maintaining comparable accuracy levels. Notably, the computational overhead introduced by ETN was minimal, making it a practical choice for deployment.
Conclusion
The Evidential Transformation Network represents a significant advancement in the field of uncertainty estimation for pretrained models. By providing a lightweight and effective solution, ETN not only enhances the reliability of predictions but also opens up new avenues for the application of pretrained models in critical areas where understanding uncertainty is paramount.
For further details, please refer to the original paper available on arXiv: arXiv:2604.08627v1.