360° video quality assessment based on saliency-guided viewport extraction

被引:0
|
作者
Fanxi Yang
Chao Yang
Ping An
Xinpeng Huang
机构
[1] Shanghai University,School of Communication and Information Engineering
来源
Multimedia Systems | 2024年 / 30卷
关键词
Video quality assessment; video; Viewport selection; Saliency prediction;
D O I
暂无
中图分类号
学科分类号
摘要
Due to the distortion of projection generated during the production of 360∘\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$360^{\circ }$$\end{document} video, most quality assessment algorithms used for 2D video have the problem of performance degradation. In this paper, we propose a full-reference 360∘\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$360^{\circ }$$\end{document} video quality assessment method, utilizing saliency to guide viewport extraction to eliminate the projection distortion. To be more specific, we first predict the visual saliency of each frame with a 360∘\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$360^{\circ }$$\end{document} saliency prediction network and then select the viewport that optimally represents the video frame through the optimal viewport positioning module (OVPM). Furthermore, we propose the attention-based three-dimensional convolutional neural network (3D CNN) quality assessment network to evaluate the video quality, in which 3D CNN convolution and attention modules can better capture the quality degradation of distorted viewports. Experimental results show that our method achieves superior performance in 360∘\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$360^{\circ }$$\end{document} video quality assessment tasks.
引用
收藏
相关论文
共 50 条
  • [1] 360° video quality assessment based on saliency-guided viewport extraction
    Yang, Fanxi
    Yang, Chao
    An, Ping
    Huang, Xinpeng
    MULTIMEDIA SYSTEMS, 2024, 30 (02)
  • [2] Saliency-Guided Video Deinterlacing
    Trocan, Maria
    Coudoux, Francois-Xavier
    COMPUTATIONAL COLLECTIVE INTELLIGENCE (ICCCI 2015), PT II, 2015, 9330 : 24 - 33
  • [3] Viewport Proposal CNN for 360° Video Quality Assessment
    Li, Chen
    Xu, Mai
    Jiang, Lai
    Zhang, Shanyi
    Tao, Xiaoming
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 10169 - 10178
  • [4] Saliency-Guided Deep Framework for Image Quality Assessment
    Hou, Weilong
    Gao, Xinbo
    IEEE MULTIMEDIA, 2015, 22 (02) : 46 - 55
  • [5] Viewport-Dependent Saliency Prediction in 360° Video
    Qiao, Minglang
    Xu, Mai
    Wang, Zulin
    Borji, Ali
    IEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23 : 748 - 760
  • [6] Saliency-Guided Quality Assessment of Screen Content Images
    Gu, Ke
    Wang, Shiqi
    Yang, Huan
    Lin, Weisi
    Zhai, Guangtao
    Yang, Xiaokang
    Zhang, Wenjun
    IEEE TRANSACTIONS ON MULTIMEDIA, 2016, 18 (06) : 1098 - 1110
  • [7] Blind 360-degree image quality assessment via saliency-guided convolution neural network
    Qiu, Miaomiao
    Shao, Feng
    OPTIK, 2021, 240
  • [8] Saliency-Guided Local Full-Reference Image Quality Assessment
    Varga, Domonkos
    SIGNALS, 2022, 3 (03): : 483 - 496
  • [9] No-reference stereoscopic image quality assessment based on saliency-guided binocular feature consolidation
    Xu, Xiaogang
    Zhao, Yang
    Ding, Yong
    ELECTRONICS LETTERS, 2017, 53 (22) : 1468 - 1469
  • [10] BE NATURAL: A SALIENCY-GUIDED DEEP FRAMEWORK FOR IMAGE QUALITY
    Hou, Weilong
    Gao, Xinbo
    2014 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME), 2014,