The goal of this project was to simulate the colour change over time of Girl with a Pearl Earring from Johannes Vermeer. Using art history knowledge and computational methods this was proven to be possible. The combination of these methods is unique and promising.

As a first for the Girl, the paints that Vermeer used have been reconstructed and artificially aged by UV light at AkzoNobel. Their colour change was then measured and turned out to be as expected. The mechanisms of indigo, yellow and red lake fading seems plausible. A start in reconstructing the painting has been made by digitally applying these colour changes on the original painting.

The digital reconstruction showed that the contrast increased and the Girl is more lively and looks ready to step out of the painting, there is a clear three dimensional effect. The colours are more defined instead of faded and patchy. More contrast can be seen, which is for example shown in the white collar. It also makes the edges between the face and background sharper. Overall, the painting shows more Houding and seems to be more like Vermeer probably intended it to be. A reconstruction made with BRDF should be more scientifically accurate than the reconstruction presented here.

Important points for future research are more opaque samples, which are created in more batches. Another point is that the measured light intensities should not be in lux and to be quick in the BRDF measurements after UV aging. Finally, it would be interesting to reconstruct other paintings, from Vermeer or from another painter.

Hyperspectral techniques have found application for the investigation of objects and samples in many scientific fields. Data evaluation approaches commonly either process them as stacks of randomly ordered spectra or as flat images. This thesis aims at combining both, lateral and spectral, aspects of hyperspectral data during data evaluation. Different approaches, such as lateral distance and neighbor pixel augmentation, will be explored with modern factorization techniques, such as t-stochastic neighbor embedding and self-organizing maps, and supported by artificial neural networks. The routines developed are applied on selected X-ray fluorescence data (XRF) sets from the field of materials science and cultural heritage. The altered augmented data set concept consists in augmenting the spectrum of a central pixel with the mean spectrum of its eight neighboring pixels. This new method is applied to the data set of a bi-modal Ti-6Al-6V-2Sn alloy. Clustering the optimized augmented data set with t-SNE reveals the existence of four components: phase with strong titanium signal, phase with strong vanadium signal, the first phase on top of the second phase and the second phase on top of the first phase. The later was not identified by clustering with t-SNE the data set augmented with all the spectra of its eight neighboring pixels. Clustering the XRF data set of a 13th century B.C. Egyptian mural painting with Self Organising Map (SOM) indicates areas with different thicknesses of copper and iron containing pigments. Clustering with Fast interpolation-based t-SNE (FIt-SNE) also reveals information about the painting sequence and the composition of a mixture of pigments. The pigments are not separable with a simple observation of the XRF elemental distribution images. Moreover, FIt-SNE dramatically accelerates t-SNE and can provide well-separated clusters. These properties are especially useful for large data sets, containing mixture of pigments. Finally, I introduce an optimised Artificial Neural Network (ANN) for training the photograph of a painting with the elemental distribution images, obtained by its XRF data set. The elemental distribution images are upscaled with Laplacian Pyramid Super-Resolution Network. In this way, we can identify with more accuracy the exact location of the damages and retouches on a 17th century A.C. easel painting (portrait of Hortense Manchini), since the result image is of high resolution.

The scientific analysis of historical paintings has been traditionally restricted to the analysis of paint cross-section samples. This invasive method provides extensive information but is inherently limited in scope due to the extreme heterogeneity of paintings. In the last decade, non-invasive spot analyses and spectral imaging methods have become increasingly widespread in cultural heritage science. Two of these methods are macroscopic X-ray fluorescence imaging spectroscopy (MA-XRF) and reflectance imaging spectroscopy (RIS). These methods allow for 2D-scanning the entire surface of a painting and provide complementary information on elemental and molecular composition and distribution of the paint. However, these methods are often used only for qualitative analysis of the paint based on relative distribution maps, revealing only limited information about the paint layer stratigraphy. This thesis is an exploration of a combined approach for quantitative analysis of paint composition and layer stratigraphy using MA-XRF and RIS. The research used a set of specially prepared paint samples of mixtures and multiple layer applications based on historically relevant pigments which were scanned using MA-XRF and RIS in the visible and Near IR range (400-2500 nm). The spectral data acquired were processed and analyzed in a variety of ways, including Non-negative Matrix Factorization, Non-Linear Least Square Fitting, among other methods; in an attempt to gather quantitative compositional and stratigraphic data. In these trials, using characteristic reflectance features in the visible range together with a comparison of highly and lightly absorbed X-ray fluorescence lines allowed for the identification and quantification of surface specific compounds related to the top paint layer. Further comparison of fluorescence lines and absorption features in the Near IR range provided a potential avenue for quantification of subsurface paint layers. The results confirm that the combination of these methods allows to reveal the paint stratigraphy. The project provides samples and data sets which may serve as the basis for the development of a robust algorithm to address this issue in the future.

A novel combination of analytical techniques for measuring optical properties of glaze and paint as input for computer modelling is used to investigate the influence of the green glaze layer present on top of the black underpainting in the background of Girl with a Pearl Earring. The absorption spectra of paint layers from paint reconstructions and samples of the original painting were determined using micro spectrophotometry. Additionally, Bidirectional Reflectance Distribution Function (BRDF) measurements were done on paint reconstructions to determine the reflection distribution from multiple illumination and observation angles. Already existing height and layer thickness information was compiled in a digital 3D model of the painting which, in combination with the optical properties, was used as a base for rendering using the Mitsuba render engine. The experimental results clearly show that the green glaze layer under specific angles darkens the appearance of the black paint onto which it is applied. The computer simulation shows promising results for digitally recreating the painting.

It is assumed that 17th century paintings were painted by a very systematic approach which involved the use of very specific palettes. For every pictorial unit, such a skin, a very specific palette was created. In this case the palette would then only be used to paint the skin and nothing else. The result is that for each pictorial unit a separate palette is used. This assumption can be verified by characterizing the all the paint mixtures present in a painting. Paint derives it colour from the pigments present within them. Many pigments used in the 17^th century contain characteristic heavy elements. The usage macro X-ray fluorescence (XRF) scanning is gaining popularity as a non-destructive method for determining elements present in a paintings. By the creation of elemental maps the pigments can be identified and localized in a painting. These maps already allow for visualizing many interesting features such as retouches, inpaintings and even hidden layers behind a painting. However the usage of the XRF data have never been used for identifying and visualizing the palettes used while painting.
In this work it is attempted to identify and visualize the palettes used in the Man with the red baret by Rembrandt van Rijn, by an in-depth analysis of raw XRF data. The raw data was model fitted using a simulated annealing algorithm to retrieve spectral line intensities in each point of the painting. The fitted spectral intensities was then visualized using t-distributed stochastic neighbour embedding (t-SNE). With help of the t-SNE visualisation the painting is divided into 24 sections. The suggestion is that these 24 sections each contain a different and unique paint mixture. This suggestion is further investigated by the use of the two-sample χ²-test between the sections’ spectral composition. The combination of model fitting alongside with t-SNE on a XRF spectra can indeed visualize different paint mixtures and give strong suggestion of the different palettes used in Man with the red baret.

Scientific investigation of paintings has been facilitated by the development of advanced non-de\-struc\-tive imaging methods. Characterization of painting stratigraphy traditionally requires extraction of small paint samples, thereby limiting its use to a few locations on a painting due to its destructive nature. Alternatively, non-destructive analysis of paint layer stratigraphy and structure across an entire painting often requires highly specialized and costly equipment, and/or the transport of priceless artworks. In addition, most methods are also typically limited to a localized point analysis.

This document proposes an alternative method for the substructure examination of paintings using a mobile macro-XRF spectrometer and a stereographic approach with reduced step sizes. This is coupled with a novel data analysis method which will enable a global study of the topographical features of hidden paint layers. As a prototype to test the feasibility of the method, we utilized a two-layer test sample consisting of pastose bone black pigment on pastose lead white, with an aluminum substrate. High resolution 3D optical microscopy was utilized to establish a ground truth for the thicknesses of both paint layers.

Through successful registration of MA-XRF scans obtained with varying detector geometries into a single hyrbid image, we were able to find a strong correlation between the quanitative height data obtained with optical microscopy and our hybrid XRF image. Our findings indicate that utilizing this approach for visualizing hidden paint layer topographies proves to be very promising. Coupled with novel data fusion algorithms and visualization techniques, additional insight about painterly technique can thus be gained by utilizing existing MA-XRF scanners to scan a painting in multiple orientations.

Rembrandt was a unique painter of his era, and one of his most outstanding pieces is The Jewish Bride. He mastered the art of manipulating pigments to create unique paint mixtures and textures. This can be seen in his painting, The Jewish Bride.
In the background of The Jewish Bride, Rembrandt mixed five pigments together: bone black, red earth, smalt (a blue pigment), yellow earth and yellow lake. Smalt and yellow lake are unstable pigments in oil and their colour fades over time. Due to the degradation process of smalt, the oil matrix and the top layer have both changed its appearance into a transparent monochrome brown. The colour loss and change of colour affect the pictorial impression created by the painting.
The goal of this thesis is to digital colour reconstruct the background of The Jewish Bride to show how it might have looked when non-degraded smalt and yellow lake were present.
In this work, the painting is analysed with different micro and macro techniques: microscopy, SEM-EDX, Macro-XRF, and hyperspectral imaging. The outcomes of microscopy, SEM-EDX and Macro-XRF are combined to show the distribution and to quantify the amount of smalt and yellow lake in the background.
Knowing the distribution of smalt and yellow lake in the painting made it possible to digital mix the colours back to their primary appearance. To achieve this, the hyperspectral data were used, together with the Kubelka-Munk theory. This was later transferred to RGB values, which results in a digital colour reconstruction.
The digital colour reconstructions show a trend of what the painting might have looked like, at the time Rembrandt painted it.