This thesis studies the Canonical Polyadic Decomposition (CPD) constrained kernel machine for large scale learning, i.e. learning with a large number of samples. The kernel machine optimization problem is solved in the primal space, such that the complexity of the problem scales linearly in the number of samples as opposed to scaling cubically in the dual space. Product feature maps are applied to transform the input data. The weights are constrained to be a CPD, so the number of weights scales linearly in the number of features. The CPD introduces a nonlinearity, so nonlinear optimization must be applied.
It is studied in which situation it is more advantageous to apply iterative all-at-once opti- mization compared to Alternating Least Squares (ALS) to solve the CPD constrained kernel machine problem. Specifically, all-at-once gradient descent methods are studied. An efficient analytical algorithm for the all-at-once gradient is derived. Furthermore, it is shown that automatic differentiation (AD) can also be applied, but it is found to be slower than the analytical method.
The selection of a step size is found to be challenging. It is shown that the magnitude of the gradient of the mean squared error (MSE) term decreases for an increasing number of features. As a result, selecting the step size becomes more difficult for more features. To overcome this, the Line search and the Adam method are studied. A general expression for the exact line search solution is derived. It can be applied to compute the optimal step size for any step direction and any number of features. However, the Adam method performs better in terms of loss after training, convergence and the training run time. The mini-batch Adam method is used to evaluate the performance of all-at-once optimization for large scale learning.
It is found that the Adam method no longer performs well for data sets with around 16 features or more, likely due to the decrease in the magnitude of the gradient of the MSE term. On large scale data sets with fewer features, the Adam method outperforms ALS in terms of run time until convergence while achieving similar training and validation losses. The Adam method reached convergence on a data set with 11 million samples within ten minutes. Furthermore, it is shown that the scaling of the run time of the Adam method in terms of the feature map order and the CP-rank is more than an order lower than the scaling of ALS when the methods are run on a GPU. This makes to Adam method more suitable for more complex models.

Optical microscopes are fundamentally limited to a resolution of several hundreds of nanometers by the diffraction of light. Single-molecule localization microscopy (SMLM) circumvents this limit by sparsely exciting fluorescent molecules at different time instances. Single molecules can subsequently be localized with improved precision over the diffraction limit. Due to their high frame rate, single-photon avalanche diode (SPAD) arrays are imagers that can be used for SMLM. Most SPADs have to recharge after each photon arrival. During this recharging period, the SPAD is insensitive to more photon arrivals. As a result, SPAD arrays will measure zero or one photons for each pixel in each frame, whereas scientific complementary metal-oxide semiconductor (sCMOS) imagers and electron multiplying charge-coupled devices (EMCCD) have a discrete frame output. Here, we describe the photon arrivals in the image formation model of the SPAD array as a binomial process rather than as a Poissonian process. In addition, we quantify the minimum theoretical uncertainty of single-molecule localizations using a binomial Cramér-Rao lower bound and benchmark it with simulated and experimental data. We show that if the expected photon count is larger than one for all pixels within one standard deviation of a Gaussian point spread function, the binomial CRLB gives a 46% higher theoretical uncertainty than the Poissonian CRLB. Without saturation, which is the case for most SMLM applications, the binomial CRLB model gives the same uncertainty as the Poissonian CRLB. Therefore, the binomial CRLB can be used to predict and benchmark localization uncertainty for SMLM with SPAD arrays for all practical emitter intensities.

Least Squares Support Vector Machines (LS-SVMs) are state-of-the-art learning algorithms that have been widely used for pattern recognition. The solution for an LS-SVM is found by solving a system of linear equations, which involves the computational complexity of O(N^3). When datasets get larger, solving LS-SVM problems with standard methods becomes burdensome or even unfeasible. The Tensor Train (TT) decomposition provides an approach to representing data in highly compressed formats without loss of accuracy. By converting vectors and matrices in the TT format, the storage and computational requirements can be greatly reduced. In this thesis, we develop a Bayesian learning method in the TT format to solve large-scale LS-SVM problems, which involves the computation of a matrix inverse. This method allows us to include the information we know about the model parameters in the prior distribution. As a result, we are able to obtain a probability distribution of the parameters, which enables us to construct confidence levels of the predictions. In the numerical experiment, we show that the developed method performs competitively with the current methods.

Cyber-attacks long have been a topic reserved for sci-fi movies and books. With the advance of internet and the globalisation of technology supply chains, as well as the growing political and economic pressure around the world, cyber warfare has become the new weapon of choice for covert state operations, but also rogue organizations. In the last 2 decades multiple major industries have suffered some kind of outage – power generation, manufacturing, oil & gas, transport, and others. A typical industrial system is designed for a life span of more than 20 years, making future issues hard to protect against at the planning phase. No widespread efforts exist to identify such threats, to detect attacks and counteract them. This thesis proposes a practical approach for lower level protection of control systems based on linear watermarking as a transparent process to provide detection for any malicious activity that might significantly impact the operations of the plant.

The world population is growing rapidly and the demand for healthy food grows with it. Greenhouse cultivation provides an efficient way to grow crops in a protected and controlled environment. In the past, many greenhouse control algorithms have been developed. How- ever, the majority of these algorithms rely on an explicit parametric model description of the greenhouse. These models are often based on physical laws such as conservation of mass and energy and contain many parameters which should be identified. Due to the complex and highly non-linear dynamics of greenhouses, these models might not be applicable to control greenhouses other than the one for which the model has been designed and identified. Hence, in current horticultural practice these control algorithms are scarcely used. Therefore, the need rises for a control algorithm which does not rely on a parametric system representa- tion but rather on input/output data of the greenhouse system, hereby establishing a way to control the system with unknown or unmodeled dynamics. A recently proposed algo- rithm, Data-Enabled Predictive Control (DeePC), is able to replace system identification, state estimation and future trajectory prediction by one single optimization framework. The algorithm exploits a non-parametric model constructed solely from input/output data of the system. This algorithm is employed in order to control the greenhouse climate. It is shown that in numerical simulation the DeePC algorithm is able to control the greenhouse climate. A comparison is made with the conventional Nonlinear Model Predictive Control (NMPC) algorithm in order to show the differences between a predictive control algorithm that has direct access to the non-linear greenhouse simulation model and a purely data-driven predic- tive control algorithm. Both algorithms are compared based on reference tracking accuracy and computational time. Furthermore, it is shown in numerical simulation that the DeePC algorithm is able to cope with changing dynamics within the greenhouse system throughout the crop cycle.

Sensors are used all around us in various industries, for instance: agricultural, medical,aerospace and automotive. It is important for these industries to have reliable sensor data because the functionality of technologies depends on it. In this work, the industry of inter-est is automotive, specifically in the field of Cooperative Adaptive Cruise Control (CACC). The control of vehicles depends on measurement data from radars, which are carried by each vehicle in a platoon. If these measurements are faulty, it could affect CACC and cause a crash. This work models the Radio Control (RC) vehicle, implements CACC and aims to identify such faults in the radar measurement data before they could impact the behavior of the platoon. The results are obtained in simulation, comprising the mathematical model of the vehicles, the implemented CACC controller and a virtual radar exposed to 4 different faults, with which the chosen method for fault detection is evaluated. The results of the CACC operating in ideal conditions and with faulty measurement data are depicted. The work ends with analysing the results and concluding, whether the chosen method is capable of identifying the faulty measurement data.

Deep Reinforcement Learning enables us to control increasingly complex and high-dimensional problems. Modelling and control design is longer required, which paves the way to numerous in- novations, such as optimal control of evermore sophisticated robotic systems, fast and efficient scheduling and logistics, effective personal drug dosing schemes that minimise complications, as well as applications not yet conceived. Yet, this potential is obstructed by the need for vast amounts of data. Without it, deep Reinforcement Learning (RL) cannot work. If we want to advance RL re- search and its applications, a primary concern is to improve this sample efficiency. Otherwise, all potential is restricted to settings where interaction is abundant, whilst this is seldom the case in real-world scenarios. In this thesis we will study binary corrective feedback as a general and intuitive manner to in- corporate human intuition and domain knowledge in model-free machine learning. In accordance with our conclusions drawn from literature, we will present two algorithms, namely Probabilistic Merging of Policies (PMP) and its extension Predictive PMP (PPMP). Both methods estimate the abili- ties of their inbuilt Reinforcement Learning (RL) entity by computing the covariance over multiple output heads of the actor network. Subsequently, the corrections are quantified by comparing the uncertainty in what is learned with the inaccuracy of the given feedback. The resulting new action estimates will immediately be applied as probabilistic conditional exploration. The first algorithm is a surprisingly clean and straightforward way to accelerate an off-policy RL baseline and as well improves on existing work that learns from corrections only. Its extension Predictive Probabilistic Merging of Policies (PPMP) predicts the corrected samples. This gives the most substantial improve- ments, whilst the required feedback is further reduced. We demonstrate our algorithms in combination with Deep Deterministic Policy Gradient (DDPG) on continuous control problems of the OpenAI Gym. We show that the greatest part of the otherwise ignorant learning process is indeed evaded. Moreover, we achieve drastic improvements in final performance, robustness to erroneous feedback and feedback efficiency both for simulated and real human feedback, and show that our method is able to outperform the demonstrator.

This research focuses on proving the presence of coloured noise on the longitudinal, lateral and rotational velocity in steady-state cornering of a skid steer mobile robot. Furthermore, it also focuses on the creation of a Gaussian filter which is able to recreate the characteristics of the measured coloured noise. This is done to determine the characteristics of the Gaussian filter based on real-life experimental values, which defines the Gaussian filter that creates coloured process noise in the Active Inference algorithm. To accomplish the main research goal, the following is done: - Create a linear dynamical model of the jackal robot and optimize the model in such a way that its velocity states resemble the experimental values of the velocities for the given experimental inputs. - Show the presence of coloured noise in the dynamical behaviour of the jackal robot. - Create a Gaussian filter and determine its characteristics, which can recreate the coloured noise found in the velocity states of the jackal robot in steady state turning. To accomplish this, experiments are done capturing the linear acceleration, rotational ve- locities, position and heading of the skid steer mobile robot using both the internal sensors of the robot as well as an external motion capturing system. A linear model of the jackal robot is constructed and discretised as the noise is a function of the difference between the experimental value for the next step state [k + 1]exp and the prediction of the next step state [k + 1]est. The prediction is made by inputting the experimental values for step [k] into the discretised linear model together with the model input. The difference between [k + 1]exp and [k+1]est is the noise on the velocities during the steady-state cornering experiments. Optimal values for Gaussian filter are found by fitting the autocorrelations of coloured noise which is made by filtering white noise with a range of Gaussian filters on the autocorrelation of the measured coloured noise. The result of this research is proof of the presence of coloured noise in skid steer mobile robots which in turn indicates that using Active Inference on these types of robots is a worthwhile approach to control and state estimation.

Formation flying is a phenomenon observed very often in the natural world, e.g. birds flying in a flock. The past decade has given a lot of emphasis on the research for the control of autonomous Unmanned Aerial Vehicles (UAVs) of the fixed-wing kind, in an effort to emulate the behavior of natural flocks. Emulating this behavior requires the construction of path following and formation control laws with the capability of adapting to changing situations, in a similar way as natural flocks can do. This thesis is devoted to studying Adaptive Vector Field Guidance laws and Adaptive Formation laws for fixed-wing UAVs. Formation control relies on an adaptive hierarchical formation control method for uncertain heterogeneous nonlinear agents with Euler-Lagrange (EL) dynamics. It is shown that various formations (T-V-Y formations) can be established using this method, tested using a Matlab/Simulink environment. Additionally, a distinguishing feature of this thesis is the development of a 3D-Simulation platform to perform a hardware in the loop (HITL) simulations (i.e. using the control hardware on board of an actual UAV): a Raspberry Pi is used to run the formation control algorithm and to communicate with a Pixhawk Cube autopilot board which contains the low-level control algorithm. The autopilot board is then connected to a 3D Simulator (Gazebo) and Ground Control System (QGroundControl). The proposed HITL platform promises to facilitate the testing and validation of guidance and formation laws in a much more realistic way than a Matlab/Simulink environment can do.

Fleet Cleaner B.V. is a company that specializes in the cleaning of ship hulls of oceanic trade vessels using a mobile robot. The mobile robot mainly operates out of sight of the operator, making accurate localization of the robot crucial to its operation. However, the absence of absolute localization and sources for error build-up like wheel slip and sensor noise, increase the error between the estimated location and true location over time. Therefore, the goal of this thesis is to minimize the amount of error build-up between the estimated position and the true position of the robot, to allow more efficient operation of the robot and to ultimately allow the robot to operate autonomously.

The main sources for error build-up are determined by simulating the position estimator with modeled sensor- and perturbation models and evaluating their individual effect on the position estimator. Algorithms that combat the error build-up are established by synthesizing working principles from literature and extensively testing these principles using simulated data and finally verifying them using real but limited data.

The analysis of the sensor noise and perturbations revealed that the EKF is best suited to estimate the position of the robot and that the main sources for error build-up are wheel slip and IMU heading drift. The addition of a slip detection and velocity correction algorithm reduced the average error build-up per meter traveled by a factor of four. The addition of the heading correction algorithm reduced the error build-up by a factor of three and mitigates the need for intermittent resetting of the robot heading by the operator.

The velocity and heading correction algorithms improve the position estimator allowing the operator to more efficiently control the robot. In order to further reduce the error build-up, it is recommended to add two more wheel encoders to the robot. However, the error will still be unbounded, making the position estimator unsuitable for autonomous operation.