Towards Rigorous Explainability by Feature Attribution
Summary: arXiv:2604.15898v1 Announce Type: new
Abstract: For around a decade, non-symbolic methods have been the option of choice when explaining complex machine learning (ML) models. Unfortunately, such methods lack rigor and can mislead human decision-makers. In high-stakes uses of ML, the lack of rigor is especially problematic. One prime example of provable lack of rigor is the adoption of Shapley values in explainable artificial intelligence (XAI), with the tool SHAP being a ubiquitous example. This paper overviews the ongoing efforts towards using rigorous symbolic methods of XAI as an alternative to non-rigorous non-symbolic approaches, concretely for assigning relative feature importance.
Introduction
Explainable artificial intelligence (XAI) has become a pivotal area of research as machine learning models gain prominence in decision-making processes across various domains, including healthcare, finance, and criminal justice. The need for transparency in these models has propelled scholars and practitioners to seek methods that elucidate the decision-making process of complex algorithms.
The Limitations of Non-Symbolic Approaches
Despite their widespread adoption, non-symbolic methods, such as SHAP and LIME, suffer from a fundamental lack of rigor. These methods often provide explanations that can be misleading or overly simplistic, which poses significant risks in high-stakes environments. The reliance on Shapley values, while popular, exemplifies this issue. It operates under assumptions that do not always hold true, leading to potentially incorrect attributions of feature importance.
Symbolic Methods: A Promising Alternative
In response to the shortcomings of non-symbolic methods, researchers are increasingly looking towards rigorous symbolic methods for feature attribution. These methods offer a framework that can guarantee reliability and validity in the explanations provided. The focus on symbolic approaches emphasizes the need for a theoretical foundation that can be rigorously analyzed and validated.
Key Advantages of Rigorous Symbolic Methods
- Provable Guarantees: Symbolic methods can provide proofs regarding the correctness of the feature attribution, ensuring that the explanations are trustworthy.
- Robustness: These methods are generally less susceptible to the biases and inconsistencies that can arise from non-symbolic techniques.
- Consistency: Symbolic approaches can maintain consistency in feature importance across different models, making them more reliable for decision-making.
- Comprehensibility: By utilizing a structured and rigorous framework, symbolic methods can often yield explanations that are easier for humans to understand.
Conclusion
The ongoing shift towards rigorous explainability in AI highlights the growing recognition of the importance of reliable and valid explanations in machine learning. As the implications of AI technologies expand, ensuring that decision-makers can trust the outputs of these models becomes paramount. The exploration of symbolic methods for feature attribution represents a significant step forward in achieving this goal, paving the way for more transparent and accountable AI systems.
For further reading, the detailed insights and findings can be accessed in the full paper available on arXiv.