A Screening Methodology for Crowdsourcing Video QoE Evaluation

被引:0
|
作者
Anegekuh, Louis [1 ]
Sun, Lingfen [1 ]
Ifeachor, Emmanuel [1 ]
机构
[1] Univ Plymouth, Ctr Signal Proc & Multimedia Commun, Plymouth PL4 8AA, Devon, England
来源
2014 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM 2014) | 2014年
关键词
QoE; QoS; Crowdsourcing; HEVC; Subjective test; Video streaming; Algorithm;
D O I
暂无
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Recently, crowdsourcing has emerged as a cheaper and quicker alternative to traditional laboratory based Quality of Experience (QoE) evaluation for video streaming services. Crowdsourcing is a process of recruiting anonymous members of the public to solve/perform different tasks without supervision. This approach of seeking solutions from the public has been enhanced by ubiquitous internet connectivity, low cost and the ease to recruit workers without geographical restrictions. Although crowdsourcing makes it possible to save cost and reach a large group of people to perform subjective video quality testing, challenges such as validity of results and the trustworthiness of workers still remain unresolved. In this paper, we attempt to address some of these challenges by developing a screening algorithm that is able to determine the validity and trustworthiness of crowdsourcing video QoE evaluation results. This algorithm is based on evaluator's extracted data (e.g. Network IP addresses, network devices and browser information, time spent on the crowdsourcing platform, date and grading patterns). To determine the performance of this algorithm, we carried out a separate controlled laboratory based subjective video quality testing. This test enables us to determine how screened crowdsourcing results correlate with lab based results. Preliminary results show that crowdsourcing results can be improved by up to 59% when the proposed screening algorithm is applied. Moreover, the results support our assertion that crowdsourcing can provide reliable video QoE measurements if proper screening is performed on the test results.
引用
收藏
页码:1152 / 1157
页数:6
相关论文
共 50 条
  • [21] QoE and Video Quality Evaluation for HTTP Based Adaptive Streaming
    Arsenovic, Milica
    Rimac-Drlje, Snjezana
    2019 27TH TELECOMMUNICATIONS FORUM (TELFOR 2019), 2019, : 503 - 506
  • [22] A Continuous QoE Evaluation Framework for Video Streaming Over HTTP
    Eswara, Nagabhushan
    Manasa, K.
    Kommineni, Avinash
    Chakraborty, Soumen
    Sethuram, Hemanth P.
    Kuchi, Kiran
    Kumar, Abhinav
    Channappayya, Sumohana S.
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2018, 28 (11) : 3236 - 3250
  • [23] QoECenter: A Visual Platform for QoE Evaluation of Streaming Video Services
    Zhang, Lingyan
    Wang, Shangguang
    Yang, Fangchun
    Chang, Rong N.
    2017 IEEE 24TH INTERNATIONAL CONFERENCE ON WEB SERVICES (ICWS 2017), 2017, : 212 - 219
  • [24] Video Artifact Evaluation Based on QoS and Objective QoE Parameters
    Garcia, Laura
    Jimenez, Jose M.
    Taha, Miran
    Lloret, Jaime
    2017 INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTING, COMMUNICATIONS AND INFORMATICS (ICACCI), 2017, : 1048 - 1054
  • [25] QoE performance evaluation of YouTube video streaming in mobile broadband networks
    Argyropoulos, Savvas
    Fotiou, Nikos
    Polyzos, George C.
    2018 IEEE 19TH INTERNATIONAL SYMPOSIUM ON A WORLD OF WIRELESS, MOBILE AND MULTIMEDIA NETWORKS (WOWMOM), 2018,
  • [26] An Objective Multi-layer QoE Evaluation For TCP Video Streaming
    Yu, Peng
    Liu, Fan
    Geng, Yang
    Li, Wenjing
    Qiu, Xuesong
    PROCEEDINGS OF THE 2015 IFIP/IEEE INTERNATIONAL SYMPOSIUM ON INTEGRATED NETWORK MANAGEMENT (IM), 2015, : 1255 - 1260
  • [27] Temporal Reasoning Guided QoE Evaluation for Mobile Live Video Broadcasting
    Chen, Pengfei
    Li, Leida
    Wu, Jinjian
    Zhang, Yabin
    Lin, Weisi
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 3279 - 3292
  • [28] QoE and Energy Consumption Evaluation of Adaptive Video Streaming on Mobile Device
    Bezerra, Charles
    de Carvalho, Artur
    Borges, Demetrio
    Barbosa, Newton
    Pontes, Jonas
    Tavares, Eduardo
    2017 14TH IEEE ANNUAL CONSUMER COMMUNICATIONS & NETWORKING CONFERENCE (CCNC), 2017,
  • [29] QoE-Oriented Performance Evaluation of Video Streaming over WiMAX
    Migliorini, Daniele
    Mingozzi, Enzo
    Vallati, Carlo
    WIRED-WIRELESS INTERNET COMMUNICATIONS, PROCEEDINGS, 2010, 6074 : 240 - 251
  • [30] QoE-based Evaluation Model on Video Streaming Service Quality
    Shen, Yun
    Liu, Yitong
    Qiao, Nan
    Sang, Lin
    Yang, Dacheng
    2012 IEEE GLOBECOM WORKSHOPS (GC WKSHPS), 2012, : 1314 - 1318