HTTNet: hybrid transformer-based approaches for trajectory prediction

Forecasting future trajectories of intelligent agents presents a formidable challenge, necessitating the analysis of intricate scenarios and uncertainties arising from agent interactions. Consequently, it is judicious to contemplate the establishment of inter-agent relationships and the assimilation of contextual semantic information. In this manuscript, we introduce HTTNet, a comprehensive framework that spans three dimensions of information modeling: (1) the temporal dimension, where HTTNet employs a time encoder to articulate time sequences, comprehending the influences of past and future trajectories; (2) the social dimension, where the trajectory encoder facilitates the input of trajectories from multiple agents, thereby streamlining the modeling of interaction information among intelligent agents; (3) the contextual dimension, where the TF-map encoder integrates semantic scene input, amplifying HTTNet cognitive grasp of scene information. Furthermore, HTTNet integrates a hybrid modeling paradigm featuring CNN and transformer, transmuting map scenes into feature information for the transformer. Qualitative and quantitative analyses on the nuScenes and interaction datasets highlight the exceptional performance of HTTNet, achieving 1.03 minADE10 and a 0.31 miss rate on nuScenes, underscoring its effectiveness in multi-agent trajectory prediction in complex scenarios.