A Hierarchical Temporal Memory Based End-to-End Autonomous Driving System

Achieving human-level driving performance in complex environments remains a major challenge in the field of deep learning (DL)-based end-to-end autonomous driving systems (ADS). In ADS, generalization to rare edge cases poses a serious safety concern with DL-based models. The leading solution to this problem is the construction of larger models and datasets, an approach known as scaling. However, limitations in the computational power available to autonomous vehicles, coupled with the under-representation of safety-critical edge cases in large autonomous driving datasets, raise questions over the suitability of scaling for ADS. In this work, we investigate the performance of an alternate, computationally less-demanding, machine learning (ML) algorithm, hierarchical temporal memory (HTM). Existing HTM models use rudimentary encoding schemes that have thus far limited their application to simple inputs. Motivated by this shortcoming, we first propose a bespoke convolutional neural network (CNN)-based encoding scheme suited to the input data used in ADS. We then integrate this encoding scheme into a novel DL-HTM end-to-end ADS. The proposed DL-HTM-based end-to-end ADS is trained and evaluated against a conventional DL end-to-end ADS based on the widely used AlexNet model from the literature. Our evaluation results show that the proposed DL-HTM model achieves comparable performance with far fewer trainable parameters than the conventional DL-based end-to-end ADS. Results also indicate that the proposed model demonstrates a superior capacity for learning underrepresented classes, i.e., edge cases, in the dataset. Copyright © 2024 by ASME.