Out of Time: Automated Lip Sync in the Wild

被引：217

作者：

Chung, Joon Son ^{[1
]}

Zisserman, Andrew ^{[1
]}

机构：

[1] Univ Oxford, Visual Geometry Grp, Dept Engn Sci, Oxford, England

来源：

COMPUTER VISION - ACCV 2016 WORKSHOPS, PT II | 2017年 / 10117卷

基金：

英国工程与自然科学研究理事会;

关键词：

SPEECH; SYNCHRONIZATION; TRANSLATION;

D O I：

10.1007/978-3-319-54427-4_19

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

The goal of this work is to determine the audio-video synchronisation between mouth motion and speech in a video. We propose a two-stream ConvNet architecture that enables the mapping between the sound and the mouth images to be trained end-to-end from unlabelled data. The trained network is used to determine the lip-sync error in a video. We apply the network to two further tasks: active speaker detection and lip reading. On both tasks we set a new state-of-the-art on standard benchmark datasets.

引用

页码：251 / 263

页数：13

共 50 条

[41] Out of sync, out of society: Political beliefs and social networks
Joo, Won-tak
Fletcher, Jason
NETWORK SCIENCE, 2020, 8 (03) : 445 - 468
[42] Out of sight, out of sync: Understanding conflict in distributed teams
Hinds, PJ
Bailey, DE
ORGANIZATION SCIENCE, 2003, 14 (06) : 615 - 632
[43] Interpolation of packet loss and lip sync error on IP media
Mued, L
Lines, B
Furnell, S
CCCT 2003, VOL 5, PROCEEDINGS: COMPUTER, COMMUNICATION AND CONTROL TECHNOLOGIES: II, 2003, : 249 - 253
[44] WILD MAMMALS BILL RUNS OUT OF TIME
不详
VETERINARY RECORD, 1995, 137 (21) : 527 - 528
[45] Evaluation of a Korean Lip-Sync System for an Android Robot
Hyung, Hyun-Jun
Ahn, Byeong-Kyu
Choi, Dongwoon
Lee, Dukyeon
Lee, Dong-Wook
2016 13TH INTERNATIONAL CONFERENCE ON UBIQUITOUS ROBOTS AND AMBIENT INTELLIGENCE (URAI), 2016, : 78 - 82
[46] Animating Lip-Sync Characters With Dominated Animeme Models
Chen, Yu-Mei
Huang, Fu-Chun
Guan, Shuen-Huei
Chen, Bing-Yu
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2012, 22 (09) : 1344 - 1353
[47] Lip Reading in the Wild
Chung, Joon Son
Zisserman, Andrew
COMPUTER VISION - ACCV 2016, PT II, 2017, 10112 : 87 - 103
[48] DTV lip-sync test using time indexed audio and video signals without effect on program
Han, CH
Kim, ES
Jang, SW
Sohng, KI
ICCE: 2003 INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS, DIGEST OF TECHNICAL PAPERS, 2003, : 194 - 195
[49] Heterogeneous Sensor Fusion With Out Of Sync Data
Chen, Biao
Varshney, Pramod K.
Zulch, Peter
Distasio, Marcello
Niu, Ruixin
Shen, Dan
Lu, Jingyang
Chen, Genshe
2020 IEEE AEROSPACE CONFERENCE (AEROCONF 2020), 2020,
[50] Out of sync: antimicrobial drug development for children
LANCET CHILD & ADOLESCENT HEALTH, 2024, 8 (08): : 629 - 629

← 1 2 3 4 5 →