Recent advancements in differential simulators offer a promising approach to enhancing the sim2real transfer of reinforcement learning (RL) agents by enabling the computation of gradients of the simulator’s dynamics with respect to its parameters. However, the application of these gradients is often limited to specific scenarios. In this thesis, we address these limitations by proposing methods to obtain accurate gradients through the use of a privileged value function. This approach provides valuable insights into the effectiveness of differential gradients and demonstrates that, in certain cases, it can significantly improve sim2real performance. To illustrate this, we develop an adversary that identifies the worst-case domain parameters for a given policy using local gradients. Our experiments are conducted on the Pendulum swing-up environment. This thesis forms the basis for the exploration of further possibilities of leveraging differential simulator gradients.

Our challenge was to define a format for a mission plan, and develop an application which allows the users of the robot to create a mission plan effectively. This application should assist the user in the process of creating a mission plan as much as possible. Our client was Allseas who was already the sponsor of the LOBSTER project. During the research phase we learned that our application would be best suited for an offline environment, as the user would not necessarily have an internet connection due to being at sea. Furthermore, we learned that the application had to be easy to use and not too complex. Throughout the course of this project we worked with Scrum, our repository was hosted on GitHub, with Travis CI integrated to test our code. Using this we could see our coverage and errors could not enter our final product. In our team we cooperated with an open culture to prevent frustrations. We made agreements on work hours and software methodologies. Still, we faced challenges which ranged from underestimating time needed for certain features to an increasing complexity of code and data gathering. The final product is a web application which can be accessed offline. This application can be successfully used to create missions for Autonomous Underwater Vehicles, which we showed in our final usability evaluation. The LOBSTER team will be using our product to plan their missions in the future. However, before it can be fully used, the low-level control software of the LOBSTER robots needs to be finished by the LOBSTER team.