The thesis presents a novel approach to optimizing input computation for minimizing classification error in image classification tasks. It leverages the capabilities of the Soft Actor-Critic (SAC) algorithm, a reinforcement learning method tailored for continuous action spaces. The focus is on developing a real-time adaptable feedback loop that continuously learns and adjusts inputs based on classifier output probabilities. Key to this approach is the incorporation of a Gated Recurrent Unit (GRU) architecture within the SAC framework to capture temporal dependencies, addressing the challenge of ever-increasing state dimensions.

The Shack-Hartmann wavefront sensor is a widely-used device to measure the light wavefront. Cur- rently, the sensor is used as a gradient sensor, which is achieved by placing the microlens array in the plane conjugate to the deformable mirror and the aberration. The resulting spot pattern is then transformed into a gradient through the use of centroiding: the measurement of the relative movement of all spots compared to the reference image. As an alternative to the traditional alignment, alternative alignment moves the Shack-Hartmann wave- front sensor: the CCD is now optically conjugated to the deformable mirror and the aberrations, and not the gradients but the curvature of the wavefront is measured. This changes the behaviour of the Shack-Hartmann sensor: instead of the amount of spots remaining the same and the CCD region oc- cupied by them changing, the amount of spots starts to change but the CCD region occupied by them remains the same in the face of aberrations. Curvature measurements can be used directly to actuate bimorph and membrane deformable mirrors, bypassing the need for complex control schemes. As an alternative to centroiding Fourier demodulation can also be used to convert the information measured by the wavefront sensor into gradients. For large aberrations where the spots stray out of their subapertures and the centroiding algorithm breaks down this method is especially suited. In this thesis it is shown that a synergy exists between alternative alignment and Fourier demodulation. Large aberrations no longer occupy a huge region on the CCD, instead changing the amount of spots, and the movement of the spots outside of their subapertures is no longer a problem as there is no need for rigidly defined subapertures. The combined use of alternative alignment and Fourier demodulation was tested on simulations, both for square and circular apertures, and for cases with extra added Poisson noise. Finally real-life experi- ments were also conducted using circular apertures.

Deformable Mirror (DM) is the key part in the AO system. Characterization of DM is important as it maps between input voltage signal and output mirror deformation. The precision of characterization largely affects the control performance. While characterizing the DM with wavefront sensors is easy and straightforward, there is very few research on the characterizing the DM in a Wavefront sensorless (WFSless) setup. Existing research focuses on using Phase Diversity Phase Retrieval (PDPR) methods to retrieve phase, while introducing diversity phase by using various additional hardware.

This thesis tries to use PDPR methods to characterize the DM in a WFSless setup without using any additional hardware. Diversity phase are introduced by the DM itself and calibrated afterwards. Simulation results show that the proposed method is capable of characterizing the DM. Experiments on the setup are also conducted and further validates the proposed method, while unstable setup is accountable for the undesired results of the characterization.

Counter-acting image quality degradation caused by phase aberrations through physical correction requires the phase field to be known. As imaging hardware captures real-valued intensity, defined as the wave amplitude squared, obtaining this lost phase information is known as the phase retrieval problem and is a non-linear and non-convex optimisation problem.

Literature treats this problem from two points of view: reconstruction using indirect phase sampling, and reconstruction using Fourier amplitude sampling. The former employs wavefront sensors such as the Shack-Hartmann sensor, which encodes phase gradient information in the form of the displacement of imaged spots. These methods, while fast, discard information such as interference and are limited to low-order reconstruction. The latter, also called wavefront sensorless phase retrieval, uses the full point-spread function (PSF) to obtain high-accuracy reconstruction at the cost of computational speed, number of intensity images required, or limited aberration magnitude. These two points of view have remained largely separated, but can be made compatible through a modelling technique called Shack-Hartmann diversity. This thesis explores the merging of wavefront sensorless methods with Shack-Hartmann intensity patterns to leverage more information from a single captured image frame.

Firstly, a low-order modal reconstruction technique is presented applied to phase gradient fields obtained from Fourier demodulation of a Shack-Hartmann intensity pattern. A method of minimising the amount of redundant data-points used for reconstruction is illustrated through the removal of Fourier-interpolated data to speed up performance.

Secondly, a novel extension of the Fourier demodulation technique to hexagonal Shack-Hartmann arrays is presented, allowing phase gradient extraction and modal reconstruction of hexagonal array intensity images using Fourier demodulation.

Thirdly, Shack-Hartmann diversity is used to extend intensity-based modal phase retrieval using Taylor approximation of the intensity function to Shack-Hartmann intensity patterns. This bridges the gap between wavefront sensorless methods and Shack-Hartmann intensity patterns.

Lastly, a novel hybrid method is presented for high-accuracy phase reconstruction based on applying the above intensity-based method to a single Shack-Hartmann intensity pattern with low-order pre-conditioning obtained from Fourier demodulation. The method is demonstrated to obtain highly accurate reconstruction on low-order aberrations, and better reconstruction accuracy on small-magnitude high-order aberrations with dominating large-magnitude low-order terms than traditional methods. Potential use cases are discussed, such as open-loop turbulence reconstruction for use in turbulence modelling.

With the introduction of machine learning algorithms in industry came a new improvement to excising production line. Image recognisance can now be used to detect defects of the product made by these production lines. However these kinds of flaw detection can be fooled by mirror-like specular light effect on these products, which can reduce the reliability of these detection methods. In this thesis two approaches are taken to
minimize these spurious specular light features. Here I look at images of welding spots taken from a car-door production line of Fiat Chrysler Automotive Italy. First a median filter is used on multiple images of welding spots, which each have a different illumination. Secondly the same images were fed into a modified algorithm developed by Antonello et al,2018 , originally used to split fore-and background of videos, to remove these spurious features. Then I tried to improve the output of this algorithm so that oversaturated pixels in the output were replaced by a combination of pixel values of the input images that were not oversaturated.
Here is found that the median filter does not satisfy the desired goal of this thesis. The algorithm however gives a good filtered output of the input images and therefore are suitable to possibly be used for computer vision applications. The improvements made to the output of the algorithm do not correct the oversaturated pixels the right way, however the method I propose here seems to have some promising results if some improvement to this method are made, mainly concerning the normalisation of these pixels.

Phase Retrieval Problem is very common and has a lot of applications in fields such as microscope, astronomy, crystallography, optical imaging etc.. In this problem, PSF and the pupil function are given, and the goal is to reconstruct the phase of the object. For camera which has 8 bits color depth, the maximum number of colors that can be displayed at any one time is 2^8 , i.e. 256. When the minimal exposure time of the camera is longer than "correct" exposure time or the intensity is changing, e.g. dynamic focusing, too much intensity is measured by the camera and causes saturation, i.e. over-exposure. Over-exposure causes distortion of the image, thereby losing the information. However, on the other hand, the overall brightness of the PSF image will be enhanced due to over-exposure. It highlight structure and increase the number of informative pixels of the image. We can use this to enhance the information in the PSF image to offset the negative effects of noise or background in the image.
Gerchberg-Saxton algorithm is a solution to phase retrieval problem and is a nice example of projection based algorithm. The algorithm has a perfect performance in the case that the PSF is the modules square of the object, i.e. without noise or other preconditioners effect. It is demonstrated in [Nishizaki, 2019] that an over-exposure preconditioner can improve the performance for phase restoration of a machine-learning model based algorithm. In this master thesis, we propose modification of the model of PSF amplitude in Gerchbrg-Saxton algorithm to make the algorithm be able to cope with over-exposed PSF image. This approach is tested both by numerical simulation and with experimental data.

Specular light reflections are the mirror-like reflections from a material interface. They appear in the observation of any illuminated surface. Specular reflections can be set apart from the diffuse reflection type, which has a random distribution of reflection directions. The radiance of the specular reflected light is governed by the Fresnel ratio and depends on both geometrical and spectral properties. A clear distinction can be made between the reflection properties of dielectric and the much more reflective non-dielectric material types. In certain cases the presence of specular reflections is useful, for example in object identification or computer graphics applications. However, often the presence of specular reflection causes bright spots on the image of an object, which results in loss of detail in these areas. Particularly in computer vision applications it is important to only observe the intrinsic diffuse reflection. Hence, a variety of specular reflection removal methods have been developed. Color-space analysis specular reflection removal methods are able to accurately recover the intrinsic diffuse color of the image of an object. Optimization based methods based on a non-negative matrix factorization of the data are considered. The reduced characteristic representation of the data allows the specular and diffuse reflection component to be separated more easily. A multi-image approach is considered because it provides more available information on the intrinsic diffuse reflection component, which is typically a relatively weak signal for non-dielectric surface reflections. A specular reflection removal method is developed, that combines a new reflection model for colored illumination with a sparse Non-negative Matrix Factorization (NMF) optimization in a multi-image framework. The proposed method is evaluated on synthetic data and real data images of non-dielectric materials acquired conform the proposed reflection model.

When objects are imaged, often aberrations slip into the captured images. Many techniques exist to remove part of those aberrations using hardware, software or a combination of those. This research will focus on Phase Diversity for Blind Multi-Frame Deconvolution. In Blind Multi-Frame Deconvolution both the object and the aberrations are estimated from multiple images of the same object. Previous research has shown, Phase Diversity can be used to improve the quality of the reconstructed object in combination with Blind Multi-Frame Deconvolution. Using Defocus Phase Diversity is currently mainly used in literature.

In this research new Phase Diversity methods are proposed. When there is no knowledge on the object or aberrations, Phase Diversities are proposed to improve the object reconstruction compared to Defocus Phase Diversity.

We also propose a new method to optimize the Phase Diversity for the N + 1th image, when already N images have been taken. This method first determines the low values in the frequency spectra of the images. Next it estimates a Phase Diversity that results in a Modulation Transfer Function to enhances those frequencies. This method leads to an improved restoration quality in terms of Structural Similarity and Peak Signal to Noise Ratio when there is Gaussian noise or Poisson noise with a higher Signal to Noise Ratio. Recommendations are
done to improve the method for lower Signal to Noise Ratios.

In the field of Adaptive Optics (AO) a new development has been introduced recently: the Wavefront Sensorless Adaptive Optics (WFSless AO) approach, which only uses camera measurements to optimize image quality and does not use a Shack-Hartman (SH) Wavefront Sensor (WFS) [Booth, 2006, Débarre et al., 2009, Hinnen et al., 2008]. In order to optimize the image quality of a point source or an extended object, the residual wavefront should be minimised as much as possible. The Model Based Approach [Booth, 2007, Linhai and Rao, 2011] is a very promising approach which can be used for this optimisation, resulting in a significant reduction in the amount of measurements. This approach uses an initial measurement and then excites all the modes that need to be optimised separately and takes a measurement of the intensity distribution after each excitation. Then, the Second-Moment (SM)’s of the measured intensity distributions are computed and used for the computation of the optimal control input, which is the coefficient vector in a Zonal representation of the wavefront. The control vector can be used to minimise the residual wavefront and thus optimise the image quality. However, in this approach it is assumed that the incoming wavefront does not change between the first and last measurement. This assumption is called the frozen window time and it is not valid for real-time problems. In [Lianghua et al., 2017] a solution was proposed for this problem, by using a precomputed independent set of modes, which allows a decoupling of the modes and computations and reduces the frozen window time. Unfortunately, this research introduces more measurements which leads to more delay. In this thesis a new approach will be presented, the Continuous Model Based Approach. This approach tries to solve both problems simultaneously: a reduction of the number of measurements and a reduction of the frozen window time. The number of measurements will be reduced by combining the excitation and optimisation step and the frozen window time will be reduced by using a two step optimisation, based on [Lianghua et al., 2017]. First, the validity of this approach will be proven mathematically and with simulations, then it will be tested in experiments. In order to make the step to real-time experiments, also an online calibration method will be introduced, proven and tested. This is also a new technique, based on the Model Based Approach [Linhai and Rao, 2011] and can be used to obtain the correlation matrix for any set of modes, focussed on actuator responses.

Turbulent layers high in the atmosphere cause anisoplanatic phase aberrations, that are responsible for image degradation [6, 33, 46]. Adaptive Optics (AO) aims to correct for these aberrations by sensing them with a wavefront sensor and performing the correction with phase conjugate devices [18]. Various wavefront sensors have been designed and successfully implemented over the years, including the curvature sensor, interferometers, the Shack-Hartmann (SH) sensor and the pyramid sensor [7, 32, 40, 51]. In recent work the plenoptic sensor, an intermediate between the SH and pyramid sensor designs [10, 32, 37], has been proposed to improve on modern wavefront sensing. It has been reported to be beneficial in presence of strong or complex
wavefronts, such as deep turbulence conditions [18, 47]. The goal of this thesis is to investigate the SH and plenoptic wavefront sensors, with the
specific application on the correction of anisoplanatic aberrations that vary throughout the field of view. The contribution of this thesis to the field of AO is the development of a simulation toolbox in MATLAB. This toolbox is designed to simulate the SH and plenoptic sensors, providing a comparative study between the two. The comparative simulations reinforced results obtained from a corresponding Literature Survey. The SH sensor can outperform the plenoptic sensor on many occasions. If both sensors share the same microlens array (MLA) the plenoptic sensor scores worse in terms of performance metrics. By adjusting the MLA of the plenoptic sensor its dynamic range and sensitivity can be improved, such that it outperforms the SH sensor. Additionally, it was shown that the plenoptic sensor performs best for strong aberrations, simulated using
randomly-generated Kolmogorov screens. On the other hand, it fails in the presence of weak aberrations where the SH performs best. The developed toolbox allows for iso- and anisoplanatic aberrations to be reconstructed in a single frame, by differentiating between aperture plane reconstruction and phase screen retrieval. Multiple approaches are implemented in the toolbox, such that different reconstruction methods can be selected depending on experimental conditions.

The Delft Center for Systems and Control (DCSC) 'Smart Optics' aim to achieve higher resolution imaging through Adaptive Optics (AO). Adaptive optics is a modern technique for detecting and correcting real-time wavefront aberrations and is widely used in biomedical imaging and astronomical imaging. Wavefront sensing lies at the core of Adaptive Optics and is known to pose some challenges. Measurement of the wavefront cannot be done directly and has to be estimated through an intensity distribution on a detector. One approach to wavefront sensing is by using a Shack-Hartmann (SH) sensor. A Shack-Hartmann sensor (a pupil-plane sensor) subdivides the wavefront into N spatial areas using sub-apertures. The individual slopes across all sub-apertures are integrated to reconstruct the wavefront. The major advantage of using a Shack-Hartmann sensor is its fast operation speed, caused by the linear relationship between local slopes and original wavefront. This enables real-time wavefront reconstruction. The Shack-Hartmann sensor however, has some limitations. Its ability to reconstruct higher-order aberrations is restricted by the amount of lenses within the micro-lens array. Furthermore, a centroiding algorithm is used to compute the local slopes. Going from spots to centroids decreases the amount of informative pixels and greatly limits its wavefront reconstruction potential. Moreover, these centroiding algorithms often add a measure of uncertainty since spots can have irregular shapes or cross-over/overlap. In this Master Thesis a novel approach to phase reconstruction from the raw SH measurement is proposed. Here, we show that Deep Learning techniques in combination with a micro-lens array can surpass traditional SH phase reconstruction methods and alleviate their current limitations. The proposed method uses the entire Shack-Hartmann Pattern (HP) as input to a neural network, supplying the network with more information than existing Deep Learning SHWR methods, which still rely on centroids. Using this approach, we can combine the accuracy of sensor-less techniques with the speed of a Shack-Hartmann sensor. Three different neural network architectures are considered in this thesis. Two of these neural networks (Alex-Net and Xception) are adapted to output a series of Zernike coefficients. Using these estimated Zernike coefficients, a wavefront can be reconstructed. The remaining neural network, U-Net, performs a direct pixel-wise estimation of the phase-map. The input Shack-Hartmann patterns are created using different micro-lens array (MLA) geometries, consisting of 25-, 256- or 900 lenses. The networks are evaluated on their ability to reconstruct a combination of 32- or 100- Zernike coefficients.

Multivariate images are built up by measuring multiple features or variables simultaneously while recording a measurement’s location. An example of such images is Imaging Mass Spectrometry (IMS) data. IMS is a technique for recording the mass-over-charge ratio of molecules in (biological) samples while also recording the molecules’ spatial location. High dimensionality in multivariate images (e.g. many recorded features per pixel) often makes direct human interpretation infeasible and computational analysis impractical. For this reason, unsupervised and data-driven factorization is often applied prior to any exploration of the data, with the goal of reducing its dimensionality. However, one of the more promising factorization methods for multivariate images, Maximum Autocorrelation Factorization (MAF), still depends on some input from the user. Unlike most factorization methods, that focus solely on the spectral content of the observations, MAF also utilizes the spatial structure of the input data to generate matrix factors that try to capture both spatial and spectral patterns in the data. The factors of MAF represent the content of a matrix, ranked according to spatial autocorrelation, i.e. how rapidly they vary spatially. The idea for application of MAF in a bioimaging context is that naturally occurring, signals tend to form larger, more uniform areas and therefore change more slowly spatially, compared to noisy, non-biological measurement patterns. Since MAF factors are ordered according to autocorrelation, noisy measurement (with low autocorrelation) tend to get demoted in the order of factors, effectively separating them from biological data components (with positive autocorrelation). The goal of this thesis is to build upon the MAF algorithm and remove the current need for user input, making an extension of MAF. This novel factorization method is named Extended Maximum Autocorrelation Factorization (EMAF). Similarly to MAF, EMAF is invariant to linear transformations, utilizes spatial and spectral information of the dataset to determine its factors, and produces uncorrelated factors at all distances under certain conditions. The EMAF algorithm is fully unsupervised and produces factors ranked according to spatial autocorrelation. Unlike MAF, EMAF does not unnecessarily promote spatial artifacts oriented in one particular direction over other directions. The exact formulation of EMAF, derivations of the mentioned traits, and practical examples of EMAF are found in the following thesis. Preliminary experiments show that EMAF returns factors of improved quality compared to MAF.