Evaluation of Crowd Models in Low Density Scenarios Using Real-world Crowd Data

In this paper, we evaluate the simulation accuracy of five crowd models: (a) RVO2; (b) social force; (c) approximate nearest neighbor search (ANN); (d) perception-action graph (PAG); and (e) clustering-based model (CLUST) by comparing the simulation results against the real world motion data quantitatively on six metrics: (a) travel time; (b) travel distance; (c) deviation; (d) speed change; (e) angle change; and (f) energy. We use real pedestrians' motion data in two scenarios with different crowd densities and main walking directions as the ground truth. The results demonstrate that the CLUST model outperforms other models in terms of most metrics, while the PAG model has the worst accuracy in all metrics. The performance of the social force model depends largely on the scenario. We also conduct a qualitative comparison of five models on a simple scenario with only two agents, in order to give an indication of the differences and similarities between models. We find that the simulated trajectories of the RVO2 and social force models are more symmetric and regular than that generated by the ANN, PAG and CLUST models. And the ANN, PAG and CLUST models' trajectories reflect the motion behaviors of the input data used to train the models. Finally, we compare the simulation frame rates of five models on two real-world scenarios and show that by applying certain data pre-processing techniques, the PAG and CLUST models can achieve better run-time performances than the ANN model, but still run slower than the RVO2 and social force models.