Unifying Speech Editing Detection and Content Localization via Prior-Enhanced Audio LLMs
Summary: arXiv:2601.21463v2 Announce Type: replace-cross
Abstract: Existing speech editing detection (SED) datasets are predominantly constructed using manual splicing or limited editing operations, resulting in restricted diversity and poor coverage of realistic editing scenarios. Meanwhile, current SED methods rely heavily on frame-level supervision to detect observable acoustic anomalies, which fundamentally limits their ability to handle deletion-type edits, where the manipulated content is entirely absent from the signal.
To address these challenges, we present a unified framework that bridges speech editing detection and content localization through a generative formulation based on Audio Large Language Models (Audio LLMs). We first introduce AiEdit, a large-scale bilingual dataset (approximately 140 hours) that covers addition, deletion, and modification operations using state-of-the-art end-to-end speech editing systems, providing a more realistic benchmark for modern threats.
Innovative Approach
Building upon this, we reformulate SED as a structured text generation task, enabling joint reasoning over edit type identification and content localization. This novel approach allows for a more comprehensive understanding of the underlying acoustic evidence.
Methodology
- Prior-Enhanced Prompting Strategy: To enhance the grounding of generative models in acoustic evidence, we propose a prior-enhanced prompting strategy that injects word-level probabilistic cues derived from a frame-level detector. This strategy aims to leverage existing acoustic data to improve the performance of the generative models.
- Acoustic Consistency-Aware Loss: Furthermore, we introduce an acoustic consistency-aware loss that explicitly enforces the separation between normal and anomalous acoustic representations in the latent space. This loss function is designed to improve the model’s robustness in distinguishing between edited and unedited audio segments.
Experimental Results
Experimental results demonstrate that the proposed approach consistently outperforms existing methods across both detection and localization tasks. The integration of AiEdit and the novel methodologies has shown significant improvements in handling various types of speech editing, including challenging scenarios that were previously inadequately addressed.
Conclusion
This research marks a significant advancement in the fields of speech editing detection and content localization. By unifying these two domains through the use of Audio LLMs, we not only enhance the capabilities of existing tools but also lay the groundwork for future developments in audio processing and manipulation detection. The implications of this work extend to various applications, including media forensics, security, and content authenticity verification.
Overall, our findings highlight the potential of leveraging large language models in the audio domain, paving the way for more sophisticated and reliable approaches to speech editing detection.