Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming
In recent years, live video streaming has gained widespread popularity across various social media platforms. Quality of Experience (QoE), which reflects end-users’ satisfaction and overall experience, plays a critical role for media service providers to optimize large-scale live compression and transmission strategies to achieve a perceptually optimal rate-distortion trade-off.
Although many QoE metrics for video-on-demand (VoD) have been proposed, there remain significant challenges in developing QoE metrics specifically for live video streaming. To bridge this gap, a comprehensive study of subjective and objective QoE evaluations for live video streaming has been conducted.
Introduction to the Study
This study introduces the first live video streaming QoE dataset, known as the TaoLive QoE. The dataset consists of 42 source videos collected from real live broadcasts and 1,155 corresponding distorted versions degraded due to a variety of streaming distortions. These include conventional streaming distortions such as compression and stalling, as well as live streaming-specific distortions like frame skipping and variable frame rates.
Methodology
Two primary methodologies were employed in this study: subjective QoE evaluation and objective QoE assessment.
- Subjective QoE Study: A human study was conducted to derive subjective QoE scores of videos in the TaoLive QoE dataset. Participants were asked to evaluate the viewing experience, allowing researchers to gather valuable data on user satisfaction.
- Objective QoE Study: Existing QoE models were benchmarked on the TaoLive QoE dataset as well as publicly available QoE datasets for VoD scenarios. This analysis highlighted that current models struggle to accurately assess video QoE, particularly for live content.
Challenges Identified
The study revealed several challenges in accurately measuring QoE for live video streaming. Notably, conventional models often fail to account for the unique characteristics associated with live content, which can lead to misleading assessments. Issues such as variable network conditions and real-time user interactions further complicate the evaluation process.
Proposed Solution: Tao-QoE
In response to the challenges identified, the study proposes an end-to-end QoE evaluation model known as Tao-QoE. This innovative model integrates multi-scale semantic features and optical flow-based motion features to predict retrospective QoE scores. Importantly, it eliminates reliance on traditional statistical quality of service (QoS) features, offering a more accurate representation of user experience.
Conclusion
The findings from this comprehensive study are expected to significantly advance the field of live video streaming QoE evaluation. By developing a robust dataset and proposing a novel evaluation model, this research paves the way for media service providers to optimize their streaming strategies, ultimately enhancing viewer satisfaction.
As live video streaming continues to evolve, ongoing research and improvements in QoE assessment will be crucial in meeting the demands of users and ensuring high-quality viewing experiences.