Prospective Role of Foundation Models in Advancing Autonomous Vehicles.

With the development of artificial intelligence and breakthroughs in deep learning, large-scale foundation models (FMs), such as generative pre-trained transformer (GPT), Sora, etc., have achieved remarkable results in many fields including natural language processing and computer vision. The application of FMs in autonomous driving holds considerable promise. For example, they can contribute to enhancing scene understanding and reasoning. By pre-training on rich linguistic and visual data, FMs can understand and interpret various elements in a driving scene, and provide cognitive reasoning to give linguistic and action instructions for driving decisions and planning. Furthermore, FMs can augment data based on the understanding of driving scenarios to provide feasible scenes of those rare occurrences in the long tail distribution that are unlikely to be encountered during routine driving and data collection. The enhancement can subsequently lead to improvement in the accuracy and reliability of autonomous driving systems. Another testament to the potential of FMs' applications lies in world models, exemplified by the DREAMER series, which showcases the ability to comprehend physical laws and dynamics. Learning from massive data under the paradigm of self-supervised learning, world models can generate unseen yet plausible driving environments, facilitating the enhancement in the prediction of road users' behaviors and the off-line training of driving strategies. In this paper, we synthesize the applications and future trends of FMs in autonomous driving. By utilizing the powerful capabilities of FMs, we strive to tackle the potential issues stemming from the long-tail distribution in autonomous driving, consequently advancing overall safety in this domain.

A modeling method for two-dimensional two-wheeler driving behavior during severe conflict interaction at intersections.

The safety of two-wheelers is a serious public safety issue nowadays. Two-wheelers usually have severe conflict interaction with vehicles at intersections, such as running red lights, which is very likely to cause traffic accidents. Therefore, a model of two-wheeler driving behavior in conflicting interactions can provide guidance for traffic safety management on one hand, and can be used for the development and testing of autonomous vehicles on the other. However, the existing models perform poorly when interacting with vehicles. To address the problems, this paper proposes a modeling method (an improved social force model, ISFM) for two-dimensional two-wheeler driving simulation for conflict interaction at intersections. Based on analysis of naturalistic driving study data, when two-wheelers encounter with vehicles, their driving intentions and trajectories can be categorized into two groups, which are yielding and overtaking. Therefore, the vehicle-related social forces are designed to be a set of two forces rather than a repulsion force in original SFM, which is a yielding force based on the relative distance between the two-wheeler and the vehicle, and an overtaking force based on the velocity of the two-wheeler itself. This opens up the possibilities for modeling the multi-modal driving intention of two-wheelers encountering with cross traffic. Based on ISFM, a bicycle model, a powered two-wheeler (PTW) model and a model of a group of PTWs, are then constructed. Compared to the original SFM, ISFM increases the precision of driving intention prediction by 19.7 % (yielding situation) and 25.0 % (overtaking situation), and reduces the root mean square error between simulated and actual trajectories by 7.8 % and 14.8 % on the bicycle model and the PTW model, respectively. Meanwhile, the model of a group of PTWs also performs well. Finally, the results of ablation experiments also validate the effectiveness of the social force designed based on velocity.

A synthetic digital city dataset for robustness and generalisation of depth estimation models.

Existing monocular depth estimation driving datasets are limited in the number of images and the diversity of driving conditions. The images of datasets are commonly in a low resolution and the depth maps are sparse. To overcome these limitations, we produce a Synthetic Digital City Dataset (SDCD) which was collected under 6 different weather driving conditions, and 6 common adverse perturbations caused by the data transmission. SDCD provides a total of 930 K high-resolution RGB images and corresponding perfect observed depth maps. The evaluation shows that depth estimation models which are trained on SDCD provide a clearer, smoother, and more precise long-range depth estimation compared to those trained on one of the best-known driving datasets KITTI. Moreover, we provide a benchmark to investigate the performance of depth estimation models in different adverse driving conditions. Instead of collecting data from the real world, we generate the SDCD under severe driving conditions with perfect observed data in the digital world, enhancing depth estimation for autonomous driving.