Intelligent Robotics

Supervisor: Prof. Ville Kyrki (ville.kyrki@aalto.fi)
Advisor: Dr. Shoaib Azam (shoaib.azam@aalto.fi), Daulet Baimukashev  (daulet.baimukashev@aalto.fi)

Keywords: World Model, Diffusion Models, Model-based Imitation Learning, Safe planning

Project Description

Building efficient representation models is crucial for autonomous driving. These representation models serve as a building block for understanding the environment and are also used to predict the trajectories of the autonomous vehicle. In the literature, different approaches have been developed to represent the environment, but they have limitations in imbuing common sense reasoning into autonomous vehicles. One possible way to address this challenge is to build world models representing the environment.

This research aims to explore whether learning and leveraging world models can also be beneficial in visual representation learning for autonomous driving in a closed-loop settings. For this purpose, in this research CARLA simulator will be used to evaluate the world model representation in predicting the trajectories by developing diffusion-based imitation learning learning models.

Deliverables

• Review of relevant literature
• Developing of world model representation using diffusion models
• Comparison of the implementation with the state-of-the-art methods in closed-loop settings

Practical Information

Pre-requisites: Python(high), Deep Learning (high)

Tools: PyTorch

Simulators: Carla

Datasets: Carla generated datasets, nuScenes

Start: Available immediately

References

• Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (https://arxiv.org/abs/2301.08243)
• Learning and Leveraging World Models in Visual Representation Learning (https://arxiv.org/abs/2403.00504)
• World Models for Autonomous Driving: An Initial Survey (https://arxiv.org/abs/2403.02622)
• Model-Based Imitation Learning for Urban Driving(https://arxiv.org/abs/2210.07729)