Intelligent agents must pursue their goals in complex environments with partial information and often limited computational capacity. Reinforcement learning methods have achieved great success by creating agents that optimize engineered reward functions, but which often struggle to learn in sparse-reward environments, generally require many environmental interactions to perform well, and are typically computationally very expensive. Active inference is a model-based approach that directs agents to explore uncertain states while adhering to a prior model of their goal behaviour. This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency due to its objective function, which encourages directed exploration of uncertain states. Moreover, our model is computationally very light and can operate in a fully online manner while achieving comparable performance to offline RL methods. We showcase the capabilities of our model by solving the mountain car problem, where we demonstrate its superior exploration properties and its robustness to observation noise, which in fact improves performance. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives and improves performance over previous active inference approaches.

This research proposes a new differentiator for estimating higher order derivatives of an input signal. The main reason why higher order derivatives are necessary is that Active Inference makes use of generalized coordinates. This means that it keeps internally track of higher order temporal derivatives of states, inputs and measurements. The difficult part of generalized coordinates are the generalized measurements, because these should be measured from a physical system. However, it is not always possible to measure all states of a system and it is definitely not possible to measure all higher order derivatives. A solution to this is to estimate the derivatives of states by using real time differentiators.

A short literature study about differentiators is conducted and found that the state of the art differentiator is the algebraic estimation approach differentiator (AEAD). However, this and other differentiators have the problem that when they are converted to discrete time, they cannot track polynomial signals anymore. For that reason, this thesis proposes a new differentiator: the recurrent
low pass algebraic differentiator (RLPAD).

The proposed differentiator is compared with the (AEAD) in two experiments. The first experiment evaluates the performance based on three analytical inputs with known higher order derivatives. The three inputs are: a polynomial, a sine and a combination of the two. Additionally these three inputs are corrupted by Gaussian white noise. This experiment concludes that the proposed method outperforms the (AEAD) significantly when a polynomial or polynomial combined with sine input is used. They perform similar for sine inputs, although (RLPAD) is considered slightly better.

The second experiment evaluates how the two differentiators perform when real sensor data is used. In order to conduct the experiment, the raw data is smoothed by a Savitzky-Golay filter to create a ground truth about the derivatives. This experiment concludes that the differentiators have an optimal set of parameters, where either the one or the other performs better. The proposed method performs a lot better when few derivatives are estimated. On the other hand (AEAD) performs better when there are more derivatives estimated. However, this is not true when the noise gets stronger. (AEAD) is more sensitive to noise than the proposed method.

Based on the experiments, the proposed method (RLPAD) is the better differentiator for creating generalized measurements in Active Inference for three reasons. The first is that it can track polynomial signals. The second is that it has stronger noise attenuation. And the third reason is that it is computationally faster.

The inertial parameters of a vehicle, which include the mass, centre of gravity position and the moments of inertia, influences the dynamics of the vehicle. Currently, the modelling of the vehicle is done by assuming fixed, conservative, values for the inertial parameters. Knowing the exact values may increase the performance, safety and comfort of the vehicle. A literature review has been conducted, where different methods for online inertial parameter estimation have been graded based on the amount of parameters it is able to estimate, the sensors used and the accuracy of the methods. Rozyn's method seems best for online inertial parameter estimation. Rozyn proposes a method which can estimate the inertial parameters from vertical acceleration data using a state variable method, modal analysis and a simple vehicle model. Rozyn's method can be summarised in four steps: •Extract the free decay response from acceleration data. •Construct the state transition matrix. •Construct the system characteristic matrix. •Estimate the inertial parameters using the constructed characteristic matrix and simplified vehicle model. The main shortcoming of Rozyn's method is the road profile which is used for the simulation, which is described in the ISO 8608 norm. The ISO 8608 description is a stationary Gaussian process. This means that the road profile random variables are normally distributed. Furthermore, the properties of the road profile (mean and variance) does not change over time. In practice however, road profiles never follow a stationary Gaussian process, but are much more random, with more variance between different sections. Another, more realistic, road profile description is proposed by Bogsjö where the road profile follows a non-stationary Laplace distribution. Another shortcoming of Rozyn's paper is that it only shows results for only one condition. For the simulation, the vehicle is driving 100 km/h and a measurement period of 1,000 seconds is used. Unknown is the influence of the velocity of the vehicle on the results. It is to be expected that the accuracy decreases for shorter measurement periods, but by how much is also unknown. In this thesis, Rozyn's algorithm is explained and implemented using a half car vehicle model. Rozyn's algorithm is validated using the ISO 8608 road profile description on similar conditions. The algorithm is then tested using the ISO 8608 road profile description where the velocity of the vehicle is varied between 30 and 100 km/h and the measurement periods between 30 and 120 seconds. This is done 100 times for each condition. This results in 100 estimates of the inertial parameters of each condition. From these results, the average and standard deviation between the estimates can be calculated. This is also done for the alternative Laplace road description. The resulting standard deviations are plotted in surface plots, as function of the varying velocity and measurement period. The results show that the standard deviation between the different estimated parameters when using the Laplace description for the road profile are up to 5 times higher compared to the ISO 8608 road profile description. The results also show that the performance of the algorithm is heavily dependent on the measurement time. A measurement time of at less than 60 seconds is not recommended, due to the large deviation in the estimated parameters. For the mass and centre of gravity position, the performance is independent of the velocity of the vehicle. However, the pitch moment of inertia shows a slight dependency on the velocity, with lower deviations between the different estimates for higher velocities. The algorithm can still be used on non-stationary road profiles. However, more and longer measurements are needed for the algorithm to return with an accurate estimation of the inertial parameters. Even then, some errors in the estimated parameters in the order of 10% are present.