Deep Dual Relation Modeling for Egocentric Interaction Recognition

Egocentricinteractionrecognitionaims to recognize the camera wearer's interactionswith the interactorwho faces the camera wearer in egocentric videos. In such a humanhuman interactionanalysisproblem, it is crucial to explore the relationsbetween the camera wearerand the interactor. However most existing works directly model the interactions as a whole and lack modeling the relations between the two interactingpersons. To exploit the strong relations for egocentric interactionrecognition,we introducea dual relation modelingframework which learns to model the relations between the camera wearerand the interactorbased on the individual action representationsof the two persons. Specifically, we develop a novel interactive LSTM module, the key component of our framework, to explicitly model the relations between the two interactingpersons based on their individual action representations,which are collaboratively learned with an interactorattention module and a global-localmotion module. Experimental results on three egocentric interactiondatasetsshow the effectiveness ofour method and advantage over state-of-the-arts.