Intelligent Robotics

Supervisor: Prof. Ville Kyrki (ville.kyrki@aalto.fi)

Advisor: Shivam Chaubey (shivam.chaubey@aalto.fi), Dr. Francesco Verdoja (francesco.verdoja@aalto.fi)

Keywords: Learning from Demonstration, Inverse Optimal Control, Control System

Project Description

We are currently seeking a motivated and talented master’s student to join our Intelligent Robotics group at Aalto University. The selected student will investigate and improve upon a developed method that learns cost and constraints explicitly as part of their master’s thesis. Learning generalizable policies with LfDs methods can be challenging as these methods often do not explicitly guarantee constraint
satisfaction. This is a significant issue, since constraints are used to represent essential safety requirements, performance specifications, or other critical operational factors, and without guarantees of constraint satisfaction, the safety and efficacy of robotic systems cannot be ensured. Additionally, learning constraints can help generalize different skills. For example, in a ball-dropping from a cup task, traditional imitation learning may fail if the environment changes, such as the inclusion of obstacles. This would require multiple demonstrations to collect data and train the policy. Conversely, if an explicit constraint from a previous task is known, new constraints introduced in the environment can be accommodated within an optimal control framework, enhancing adaptability and generalization.

This project offers an excellent opportunity to gain hands-on experience with robotic manipulators, real-time control, and inverse optimal control, and to contribute to cutting-edge research.

Deliverables

• Identifying suitable tasks and collecting data for different types of tasks.
• Finding bottlenecks in the existing method for practical real-world scenarios.
• Developing and proposing ideas to improve the current method.

Practical Information

Pre-requisites: Optimal Control, Reinforcement Learning, Control Systems

Programming Skills: Python, MATLAB, ROS

Theoretical Background: Optimal Control, Reinforcement Learning, Control Systems

Start: Available immediately

References

• Jointly Learning Cost and Constraints from Demonstrations for Safe Trajectory Generation. (https://arxiv.org/abs/2405.03491)
• Constrained Inverse Optimal Control With Application to a Human Manipulation Task. (https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8938707)
• C-learn: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy. (https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7989466)