Spatio-temporal alignment of sequences

被引:103
作者
Caspi, Y [1 ]
Irani, M [1 ]
机构
[1] Weizmann Inst Sci, Dept Appl Math & Comp Sci, IL-76100 Rehovot, Israel
关键词
sequence-to-sequence alignment; space-time analysis; direct methods; feature-based methods;
D O I
10.1109/TPAMI.2002.1046148
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper studies the problem of sequence-to-sequence alignment, namely, establishing correspondences in time and in space between two different video sequences of the same dynamic scene. The sequences are recorded by uncalibrated video cameras which are either stationary or jointly moving, with fixed (but unknown) internal parameters and relative intercamera external parameters. Temporal variations between image frames (such as moving objects or changes in scene illumination) are powerful cues for alignment, which cannot be exploited by standard image-to-image alignment techniques. We show that, by folding spatial and temporal cues into a single alignment framework, situations which are inherently ambiguous for traditional image-to-image alignment methods, are often uniquely resolved by sequence-to-sequence alignment. Furthermore, the ability to align and integrate information across multiple video sequences both in time and in space gives rise to new video applications that are not possible when only image-to-image alignment is used.
引用
收藏
页码:1409 / 1424
页数:16
相关论文
共 34 条
[1]  
[Anonymous], CMURITR0103
[2]  
[Anonymous], P EUR C COMP VIS ECC
[3]  
BAKER S, 2001, P IEEE C COMP VIS PA
[4]   A 3-FRAME ALGORITHM FOR ESTIMATING 2-COMPONENT IMAGE MOTION [J].
BERGEN, JR ;
BURT, PJ ;
HINGORANI, R ;
PELEG, S .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1992, 14 (09) :886-896
[5]  
Birchfield S, 1996, KLT IMPLEMENTATION K
[6]   THE LAPLACIAN PYRAMID AS A COMPACT IMAGE CODE [J].
BURT, PJ ;
ADELSON, EH .
IEEE TRANSACTIONS ON COMMUNICATIONS, 1983, 31 (04) :532-540
[7]  
BURT PJ, 1993, P 4 INT C COMP VIS B, P173, DOI DOI 10.1109/ICCV.1993.378222
[8]  
Caspi Y, 2000, PROC CVPR IEEE, P682, DOI 10.1109/CVPR.2000.854940
[9]  
CASPI Y, 2001, P 8 INT C COMP VIS V, V2, P76
[10]   RANDOM SAMPLE CONSENSUS - A PARADIGM FOR MODEL-FITTING WITH APPLICATIONS TO IMAGE-ANALYSIS AND AUTOMATED CARTOGRAPHY [J].
FISCHLER, MA ;
BOLLES, RC .
COMMUNICATIONS OF THE ACM, 1981, 24 (06) :381-395