Master Thesis on “Explaining Driving Situations with Natural Language”

29/06/2023

Supervisor: Prof. Ville Kyrki (ville.kyrki@aalto.fi)

Advisor: Dr. Tsvetomila Mihaylova (tsvetomila.mihaylova@aalto.fi)

Keywords: autonomous driving, natural language processing

Enabling autonomous vehicles to perform in the real world is an active area of research.
One of the open challenges in the field is providing human-friendly explanations of the decisions of the autonomous system. This can be necessary in cases, where, for example, a human operator needs to take control of the vehicle in a difficult situation. A natural way of providing explanations to a human is by using natural language.

Project Description

The goal of this master thesis is to explore existing approaches, datasets and models that provide textual explanations of driving situations, to implement a state-of-the-art model and to validate it on predefined driving conflict situations. The successful realization of the thesis would contribute to integrating textual explanations in a system that aims to redirect the vehicle control to a human operator in uncertain situations and explain those situations in an accessible way. The thesis allows for integrating student ideas, and, potentially, for contributing to expanding the research in the field.

Deliverables

  • Literature review of existing datasets and models for explaining driving situations in natural language; pointing directions for future work
  • Implementation of an existing state-of-the-art model
  • Definitions of several conflict scenarios and implementation in CARLA
  • Validations of the scenarios with the implemented model

Practical Information

Pre-requisites: Python, PyTorch, experience with machine learning

Simulators to be used: CARLA

Start: Available immediately

References

Tags: