Supervisor: Prof. Ville Kyrki (firstname.lastname@example.org)
Advisor: Dr. Tsvetomila Mihaylova (email@example.com)
Keywords: autonomous driving, natural language processing
Enabling autonomous vehicles to perform in the real world is an active area of research.
One of the open challenges in the field is providing human-friendly explanations of the decisions of the autonomous system. This can be necessary in cases, where, for example, a human operator needs to take control of the vehicle in a difficult situation. A natural way of providing explanations to a human is by using natural language.
The goal of this master thesis is to explore existing approaches, datasets and models that provide textual explanations of driving situations, to implement a state-of-the-art model and to validate it on predefined driving conflict situations. The successful realization of the thesis would contribute to integrating textual explanations in a system that aims to redirect the vehicle control to a human operator in uncertain situations and explain those situations in an accessible way. The thesis allows for integrating student ideas, and, potentially, for contributing to expanding the research in the field.
- Literature review of existing datasets and models for explaining driving situations in natural language; pointing directions for future work
- Implementation of an existing state-of-the-art model
- Definitions of several conflict scenarios and implementation in CARLA
- Validations of the scenarios with the implemented model
Pre-requisites: Python, PyTorch, experience with machine learning
Simulators to be used: CARLA
Start: Available immediately
- ADAPT: Action-aware Driving Caption Transformer, https://arxiv.org/abs/2302.00673v1
- Textual Explanations for Automated Commentary Driving, https://arxiv.org/abs/2304.08178v1
- Sense–Assess–eXplain (SAX): Building Trust in Autonomous Vehicles in Challenging Real-World Driving Scenarios, https://www.robots.ox.ac.uk/~mobile/Papers/EVSAV_SAX.pdf
- Textual Explanations for Self-Driving Vehicles, https://arxiv.org/abs/1807.11546v1