Master Thesis on “Highway autopilot using Deep Reinforcement Learning and Graph Neural Networks”

Volvo Group Trucks Technology

Location: Göteborg, Sweden (remote work occasionally allowed)
Time Schedule: Jan 2023- Jun 2023

Supervisors: Leo Laine (leo.laine@volvo.com), Ville Kyrki (ville.kyrki@aalto.fi)
Advisors: Gökhan Alcan (gokhan.alcan@aalto.fi), Deepthi Pathare (deepthi.pathare@volvo.com), Erik Börve (erik.borve@volvo.com)

More than one million people die in traffic accidents every year. One solution to improve traffic safety could be to develop safe autonomous vehicles that avoid dangerous human driving behaviors [1]. The main challenge of autonomous vehicles is decision making in complex environments such as lane changing in highways or crossing an intersection. Learning from own experiences and considering the states of surrounding vehicles is important in the decision-making process. Promising results in this area have been achieved using Deep Reinforcement Learning (DRL) [2]. An example project is available on Github: Link.

Fig 1. Highway driving scenario (Source: [2]).

In recent years Graph Neural Networks (GNNs) have become increasingly popular for modeling non-euclidean systems. Some popular applications combine DRL with GNNs to find control policies for multiple interacting agents [3,4]. GNNs have also achieved promising results in terms of describing the interactions between different vehicles in typical driving scenarios [5]. This thesis aims to use DRL with GNNs to create a highway autopilot that sets the vehicle speed, decides when to do lane changes and estimates the uncertainty of the decisions.

Fig 2. Representation of non-euclidean interactions in a driving scenario (Source: [5]).

The purpose of this thesis can be summarized as:

  1. Accomplish a safe autopilot that sets the speed and performs lane changes based on state information of the own vehicle and the surrounding vehicles.
  2. Investigate the feasibility of utilizing DRL together with a GNN to accomplish a safe driving scenario.
  3. Extend the existing DRL research project (Link) to include the use of GNNs and add an exit ramp scenario.
  4. Additionally, estimate the uncertainty of the decisions that are generated by the DRL controller.

The aim of this thesis is to use advanced machine learning methods to solve problems within autonomous driving. The work will include programming, machine learning, control theory, numerical optimization and vehicle simulation. The work will be carried out at Volvo Group Trucks Technology, Sweden. The thesis is recommended for one or two students with a strong background in machine learning and Python/C++ with a good mathematical background. Prior experience with control theory and modeling/simulation is meritorious.

If you find this proposal interesting:

References

[1] Heather E Rosen et al. “Global road safety 2010–18: An analysis of Global Status Reports”.In:Injury (2022). URL:https://apps.who.int/iris/bitstream/handle/10665/276462/9789241565684-eng.pdf

[2] Carl-Johan E Hoel. “Decision-Making in Autonomous Driving using Reinforcement Learning”. PhD thesis. Chalmers Tekniska Högskola (Sweden),2021. URL:https://research.chalmers.se/publication/526543/file/526543_Fulltext.pdf

[3] Almasan, Paul et al. “Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case.” arXiv: Networking and Internet Architecture (2020): URL:https://arxiv.org/pdf/1910.07421.pdf

[4]Li, Qingbiao et al. “Graph Neural Networks for Decentralized Multi-Robot Path Planning.” 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020): 11785-11792. URL: https://arxiv.org/pdf/1912.06095.pdf

[5] Eunsan Jo, Myoungho Sunwoo, and Minchul Lee. “Vehicle Trajectory Prediction Using Hierarchical Graph Neural Network for Considering Interaction among Multimodal Maneuvers”. In:Sensors 21.16 (2021).  URL: https://www.mdpi.com/1424-8220/21/16/5354