Benchmarking Multimodal Sentiment Analysis

被引：31

作者：

Cambria, Erik ^{[1
]}

Hazarika, Devamanyu ^{[2
]}

Poria, Soujanya ^{[3
]}

Hussain, Amir ^{[4
]}

Subramanyam, R. B. V. ^{[2
]}

机构：

[1] Nanyang Technol Univ, Sch Comp Sci & Engn, Singapore, Singapore

[2] Natl Inst Technol, Warangal, Andhra Pradesh, India

[3] Nanyang Technol Univ, Temasek Labs, Singapore, Singapore

[4] Univ Stirling, Sch Nat Sci, Stirling, Scotland

来源：

COMPUTATIONAL LINGUISTICS AND INTELLIGENT TEXT PROCESSING, CICLING 2017, PT II | 2018年 / 10762卷

关键词：

Multimodal sentiment analysis; Emotion detection; Deep learning; Convolutional neural networks; EMOTION RECOGNITION;

D O I：

10.1007/978-3-319-77116-8_13

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We propose a deep-learning-based framework for multimodal sentiment analysis and emotion recognition. In particular, we leverage on the power of convolutional neural networks to obtain a performance improvement of 10% over the state of the art by combining visual, text and audio features. We also discuss some major issues frequently ignored in multimodal sentiment analysis research, e.g., role of speaker-independent models, importance of different modalities, and generalizability. The framework illustrates the different facets of analysis to be considered while performing multimodal sentiment analysis and, hence, serves as a new benchmark for future research in this emerging field.

引用

页码：166 / 179

页数：14

共 50 条

[41] Multimodal Sentiment Analysis Using Deep Learning
Sharma, Rakhee
Le Ngoc Tan
Sadat, Fatiha
2018 17TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA), 2018, : 1475 - 1478
[42] Multimodal PEAR Chain-of-Thought Reasoning for Multimodal Sentiment Analysis
Li, Yan
Lan, Xiangyuan
Chen, Haifeng
Lu, Ke
Jiang, Dongmei
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (09)
[43] MAG plus : AN EXTENDED MULTIMODAL ADAPTATION GATE FOR MULTIMODAL SENTIMENT ANALYSIS
Zhao, Xianbing
Chen, Yixin
Li, Wanting
Gao, Lei
Tang, Buzhou
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 4753 - 4757
[44] Toward Robust Multimodal Sentiment Analysis using multimodal foundational models
Zhao, Xianbing
Poria, Soujanya
Li, Xuejiao
Chen, Yixin
Tang, Buzhou
EXPERT SYSTEMS WITH APPLICATIONS, 2025, 276
[45] SKEAFN: Sentiment Knowledge Enhanced Attention Fusion Network for multimodal sentiment analysis
Zhu, Chuanbo
Chen, Min
Zhang, Sheng
Sun, Chao
Liang, Han
Liu, Yifan
Chen, Jincai
INFORMATION FUSION, 2023, 100
[46] Sentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis
Qian, Fan
Han, Jiqing
He, Yongjun
Zheng, Tieran
Zheng, Guibin
FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2023), 2023, : 12966 - 12978
[47] UniMF: A Unified Multimodal Framework for Multimodal Sentiment Analysis in Missing Modalities and Unaligned Multimodal Sequences
Huan, Ruohong
Zhong, Guowei
Chen, Peng
Liang, Ronghua
IEEE TRANSACTIONS ON MULTIMEDIA, 2024, 26 : 5753 - 5768
[48] Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis
Han, Wei
Chen, Hui
Poria, Soujanya
2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 9180 - 9192
[49] Hierarchical Interactive Multimodal Transformer for Aspect-Based Multimodal Sentiment Analysis
Yu, Jianfei
Chen, Kai
Xia, Rui
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2023, 14 (03) : 1966 - 1978
[50] Disentangled variational auto-encoder for multimodal fusion performance analysis in multimodal sentiment analysis
Chen, Rongfei
Zhou, Wenju
Hu, Huosheng
Fei, Zixiang
Fei, Minrui
Zhou, Hao
KNOWLEDGE-BASED SYSTEMS, 2024, 301

← 1 2 3 4 5 →