Structured Exploration and Exploitation of Label Functions for Automated Data Annotation
Summary: arXiv:2604.08578v1 Announce Type: cross
High-quality labeled data is critical for training reliable machine learning and deep learning models, yet manual annotation remains costly and error-prone. Programmatic labeling addresses this challenge by utilizing label functions (LFs), which are heuristic rules that automatically generate weak labels for training datasets. However, existing automated LF generation methods primarily depend on large language models (LLMs) to synthesize surface-level heuristics or utilize model-based synthesis over hand-crafted primitives. These methodologies often lead to limited coverage and unreliable label quality, creating a need for more robust solutions.
Introducing EXPONA
In response to the aforementioned challenges, we present EXPONA, an innovative automated framework for programmatic labeling. EXPONA redefines the LF generation process by balancing diversity and reliability through a systematic approach. This framework explores multi-level LFs, encompassing surface, structural, and semantic perspectives, thereby enhancing the labeling process significantly.
Key Features of EXPONA
- Multi-level LF Exploration: EXPONA delves into various levels of LFs, which allows for a broader understanding of the data and improves the overall labeling accuracy.
- Reliability-aware Mechanisms: The framework implements sophisticated mechanisms that suppress noisy or redundant heuristics while preserving diverse and complementary signals.
- Automated and Efficient: By automating the label function generation, EXPONA reduces the time and effort typically associated with manual data annotation.
Experimental Validation
To validate the effectiveness of EXPONA, extensive experiments were conducted on eleven classification datasets spanning various domains. The results were impressive:
- EXPONA achieved nearly complete label coverage of up to 98.9%.
- There was an improvement in weak label quality by up to 87%.
- Downstream performance gains reached up to 46% in the weighted F1 score.
Conclusion
The experimental results demonstrate that EXPONA’s combination of multi-level LF exploration and reliability-aware filtering contributes to a more consistent label quality and improved downstream performance across diverse tasks. By effectively balancing coverage and precision in the generated LF set, EXPONA stands out as a promising solution for automated data annotation, paving the way for more reliable machine learning applications.