This research is an early exploration into the potential of deep neural networks predicting physically-based building performance metrics on 3D urban geometry. Specifically, this work aims to predict annual solar irradiation using networks trained on point clouds. It is expected that the proposed method will allow designers to optimize their designs based on solar performance, due to the significantly lower inference time, in comparison to traditional simulation models. Furthermore, this research paves the way for generative models, which include the evaluation of performance such as solar irradiation, wind and acoustics.

Prior research has suggested several methods to predict solar irradiation on 2D building data and low resolution 3D buildings. These researches have in common that there is a lack of real observed irradiation data on buildings. Therefore, this research proposes several methods to efficiently synthesize an irradiation dataset based on 3D building geometry, which samples are larger in scale and resolution in comparison to earlier work.

Based on the synthesized datasets, several models have been trained to predict the irradiation values. Experiments have shown that a finetuned version of the model is able to predict irradiation with an average RMSE of 21 kWh/m2 with an average inference time of 0.7 seconds, on a patch of 100x100 meters. Furthermore, this thesis provides an analysis on how patch size and point sampling technique affect the prediction error of the model.

Finally, this research provides an implementation of the network in a user-friendly client-server ecosystem, that can assist architects and engineers to fine-tune their building designs in urban environments.

In the summer of 2023, heatwaves became quite prominent in the south of Europe. Due to the extreme heat, the health of those citizens was affected. The Netherlands Meteorological Institute predicts an increase in heatwaves in the future for the Netherlands as well. The main research question is how to propose a strategy for a liveable environment by designing public spaces while mitigating heat stress for vulnerable target groups in the context of Bospolder Tussendijken in Rotterdam. This research questions also how a reproducible tool could help identify heat stress and test design interventions in Dutch cities. The research included a literature review, expert consultations, scenario planning, modelling of the urban environment and mapping techniques.
Comparing the heat stress software reproducibility, computation time, possibility to test design interventions and the scale of modelling were important. Improvements in the reproducibility of the PET map of Koopmans et al. (2020) are made by creating an open-accessible QGIS plugin applicable to Dutch cities. This helps urban designers to indicate and test their design interventions. Refinement of the wind calculation contributed to speeding up calculation times of the wind for neighbourhood and city scale areas. Future research should focus on some refinement in PET calibration to work properly, and advanced wind modelling is required for urban design practices.
The application in the Rotterdam test case study emphasizes the importance of maintaining liveability now and in the future. By enhancing social liveability and physical liveability within a network of heat-mitigating interventions liveability is guaranteed. By revealing the vulnerable groups and their social interactions on a summer day, the most frequently used routes are qualified for refurbishment. Based on the current quality of social space and walkable environment, ownership and degree of open space on the street level, the interventions are chosen for the situation.
The research emphasized the importance of identifying heat stress in public spaces and the need for urgent action to maintain the quality of life in the future. By integrating informed strategies from multiple fields like Geomatics and Urbanism a climate-adaptive and healthy environment can take shape.

Existing content-based image retrieval models work well for natural photos, but not for images of architectural floor plans.
Previous work on floor plan retrieval has focused on graph-based methods, rather than image-based floor plans.
Training a CNN-based representation learning framework on segmented floor plan images with standard image augmentations does not result in semantically meaningful retrievals.
This work shows that a CNN-based representation learning model can learn features for retrieving floor plans that have similar graphs given the right training signal. Two methods were investigated here: GeomPerturb, a data augmentation that perturbs the underlying geometry of a floor plan, and a weakly supervised method with labels based on the graph edit distance between a pair of floor plans. The results show that while GeomPerturb learns representations that are correlated with the floor plan graph, training with GED labels leads to better retrievals both in terms of the floor plan graph and with respect to room shapes.

This thesis investigates the transition of traditional houses into Smart Homes, emphasizing the improvement of residents’ well-being, particularly for those aged 50 to 60. It is set against the rapid growth of Smart Home Devices enabled by the Internet of Things, facilitating easy control and monitoring of home features. There has been a significant increase in the market, with global shipments of such devices rising by 12% in 2021, reflecting a trend towards more connected and automated homes.
Smart Homes are becoming increasingly popular with people spending more time at home and the rise in digitalization. These homes represent technology and address the needs of an aging population, energy efficiency, and personal well-being enhancement. Yet, this shift to digital comes with challenges, including reliance on the internet and potential cybersecurity risks. The dissertation critically reviews these issues and looks at possibilities applied in the Built Environment.
Approaching the integration of Artificial Intelligence in the Built Environment, the study delves into the evolution from Smart Homes to Intelligent Homes. In this transition, AI plays a crucial role in creating environments that are adaptive and capable of making predictions. This shift towards more advanced technology raises important issues regarding privacy, ethical implications, and finding the right balance between convenience and security.
The research is driven by a detailed problem statement emphasizing the urgent need for innovative housing solutions in light of an aging society and healthcare system pressures. The case study developed in this thesis is in the Netherlands, where projections suggest that by 2035, over 24% of the population will be 65 and above. The study aims to transform an existing house into a Smart Home, focusing on improving indoor climate and resident well-being. Recognizing the subjective nature of health and well-being, it notes that the advantages of a Smart Home can differ among individuals. The study is significant for its potential contribution to the development of assistive technologies and services, ultimately seeking to improve life quality and support aging in place for future generations. With a combination of interdisciplinary fields such as Architecture and the Built Environment, design, and Artificial Intelligence, this thesis explored the possibilities for the future by taking a computational approach, including focusing on the indoor environment, comfort, and other things related to the technical aspect of the building.

In contrast to the prevalent focus on real photos in computer vision research, we present a contribution by making the Ot & Sien dataset machine learning-ready for object detection tasks in illustrations. We refer to the new dataset as Ot & Sien++ that is composed of scanned images of children’s book illustrations, thereby venturing into an unexplored domain. The primary objective of this research is to investigate the generalization capabilities of existing object detection models to this unique dataset and establish benchmarks for this dataset.
To evaluate the performance of existing object detection models on our proposed dataset, we employed the widely used YOLOv5 as a benchmark. To mitigate the inherent imbalance of the dataset, various data augmentation techniques were applied. The results demonstrated the effectiveness of the object detection model and data augmentation in the context of children’s book illustrations. In addition, this research also explored applying few-shot learning models to the dataset. Baseline models were investigated to examine the potential of few-shot learning in the context of object detection in illustrations.
The proposed dataset elicits new challenges in object detection and will serve as a valuable resource for researchers in this domain. Our dataset can be found at https://data.4tu.nl/datasets/d1f3ca5c-f1e4-48f5-9a04-0564572d2b9c/1.

The use of artificial intelligence (AI), digital tools and fabrication technologies in architecture has followed a paradigm shift, enhancing its design and construction processes and performances. While it has enhanced the performability and efficiency aspects of architecture, it has removed the aesthetics and symbolism of ornamentation and individuality of buildings. This research questions what the new relationships and arrangements for architecture and ornament are, specifically focusing on the use of AI and digital technology, to understand how these instruments can be used to bring craft, structure, functionality, and symbolism back to contemporary ornamentation in architecture.

VPR describes a task where an agent (e.g., a robot) attempts to recognize its current location by comparing the incoming visual data from its sensor(s) (query images), usually a camera, to geotagged reference images. Both query and reference images are described using a feature extractor, and the query descriptor is matched to its closest reference descriptor in the feature space. Within VPR there are many different VPR techniques that have been proposed throughout the years with many different types of architecture and trained on different datasets. With the many test datasets available, there exists no VPR technique that is able to reach state-of-the-art performance on all these datasets. For this reason, existing work has argued it can be beneficial to utilize an ensemble-based method to combine multiple VPR techniques and achieve better VPR performance. Some of these Ensemble-based methods have already been proposed. These ensemble-methods combine individual VPR techniques and weigh their predictions using these same predictions to give an indication of their confidence. This calculation, however, is strictly based on predictions obtained from applying the VPR techniques on test data at inference time. Generally within VPR research, the dataset that was used to train the VPR technique is often different from the dataset it is tested on. This means there is a domain shift between the training and test data. This domain shift is not taken into account when weighting the predictions of VPR techniques in an ensemble using these existing methods. In this work, we analyze how this degree of domain shift between train and test data, which can be observed by looking at the relative location of descriptors in the feature space, impacts downstream VPR performance. Intuitively, one would expect better VPR performance in a situation where the degree of train-test domain shift is minimal. Our analysis shows that this is indeed the case. We propose two different methods that utilize this degree of domain shift to calculate the weights given to the VPR techniques in an ensemble. First, we propose a generative method. Here weights are given to the VPR techniques based on the likelihood that the query sample originated from the same distribution as the training dataset of the technique and is in distribution. This way each individual technique is given a weight. Secondly, we propose a discriminative method. Here weights are given to the training datasets used to train the techniques in the ensemble. These training datasets are given weights based on relative proximity to a query sample in the feature space, an indicator for the degree of domain-shift between the training dataset and the query sample. all VPR techniques are given the weight corresponding to their training dataset. We compare these proposed approaches to other ensemble-based baselines and individual VPR techniques. The quantitative results show that our proposed methods generally outperform the ensemblebased baselines and the individual VPR techniques. We also propose further future work. One of the generative methods still delivers lower performance than could be possible, caused by applying this method to high-dimensionality descriptors. A solution for this issue should lead to higher VPR performance using this method. Additionally, we suggest future work to expand on the datasets used in this research, to strengthen the claims made, and verify that results and trends found to hold up when testing and training using other datasets.

This work addresses visual localization for intelligent vehicles. The task of cross-view matching-based localization is to estimate the geo-location of a vehicle-mounted camera by matching the captured street view image with an overhead-view satellite map containing the vehicle's local surroundings. This local satellite view image can be obtained using any rough localization prior, e.g., from a global navigation satellite system or temporal filtering. Existing cross-view matching methods are global image descriptor-based and achieve considerably lower localization performance than structure-based methods with 3D maps. Whereas structure-based methods utilized global image descriptors in the past, recent structure-based work has shown that significantly better localization performance can be achieved using local image descriptors to find pixel-level correspondences between the query street view image and the 3D map. Hence, using local image descriptors may be the key to improving the localization performance of cross-view matching methods. However, the street and the satellite view do exhibit not only very different visual appearances but also have distinctive geometric configurations. As a result, finding correspondences between the two views is not a trivial task. We observe that the geometric relationship between the street and satellite view implies that every vertical line in the street view image has a corresponding azimuth direction in the satellite view image. Based on this prior, we devise a novel neural network architecture called SliceNet that extracts local image descriptors from both images and matches these to compute a dense spatial distribution for the camera's location. Specifically, the geometric prior is used as a weakly supervised signal to enable SliceNet to learn the correspondences between the two views. As an additional task, we also show that the extracted local image descriptors can be used to determine the heading of the camera. SliceNet outperforms global image descriptor-based cross-view matching methods and achieves state-of-the-art localization results on the VIGOR dataset. Notably, the proposed method reduces the median metric localization error by 21% and 4% compared to the state-of-the-art methods when generalizing, respectively, in the same area and across areas.

Three-dimensional building models play a pivotal role in shaping the digital twin of our world. With the advance of sensing technologies, unprecedented data acquisition capabilities on capturing the built environment have surfaced, with photogrammetry and light detection and ranging being the two important sources, both of which can acquire point clouds of buildings. A point cloud is anisotropically distributed in space, which---though conveying spatial information itself---has to be converted into a surface model for a wider spectrum of usage. This conversion is often referred to as reconstruction. Despite the enhanced availability of point cloud data in the built environment, how to reconstruct high-quality building surface models remains non-trivial in remote sensing, computer vision, and computer graphics. Most reconstruction methods are dedicated to smooth surfaces represented by dense triangles, irrespective of the piecewise planarity that dominates the geometry of real-world buildings. Although some works claim the possibility of reconstructing piecewise-planar shapes from point clouds, they either struggle to comply with specific geometric constraints, or suffer from serious scalability issues. There is no versatile solution yet for building reconstruction. In this thesis, we propose a novel framework for reconstructing compact, watertight, polygonal building models from point clouds. Our approach comprises three functional blocks: (a) a cell complex is generated via adaptive space partitioning that provides a polyhedral embedding as the candidate set; (b) an implicit field is learnt by a deep neural network that facilitates building occupancy estimation; (c) a Markov random field is formulated for surface extraction via combinatorial optimisation, where an efficient graph-cut solver is applied. We extensively evaluate the proposed method in comparison with state-of-the-art methods in shape reconstruction, surface approximation and geometry simplification. Experimental results reveal that, with our neural-guided strategy, high-quality building models can be obtained with significant advantages over fidelity, compactness and computational efficiency. The method shows robustness to noise and insufficient measurements due to occlusions, and generalise reasonably well from synthetic scans to real-world measurements. Moreover, our method remains generic to not only buildings, but any piecewise-planar objects.

3D indoor reconstruction has been an important research area in the field of computer vision and photogrammetry. While the initial techniques developed for this purpose use sensor devices and multiple images for data acquisition and extracting 3D information and representation of the scene, with the advent of deep learning techniques, there has been good progress in extracting 3D information of an indoor scene reconstruction using a single image. This has potential in minimizing user efforts and cost for data acquisition. The current state-of-the-art method involves two main components, the global depth map and plane instances. After investigating the current state-of-the-art methods, it is observed that there is inconsistency in reconstructed surface boundaries and depth estimation over the curvature and edges of the objects present in the scene, despite having good 3D representation in the surrounding regions. We devise a loss function for optimizing depth estimation during supervision of the neural network by providing geometric awareness to the pixels at local level based on its neighborhood properties defined by spatial compatibility and color similarity. A similar function is used during 3D reconstruction for orientation consistency in the point cloud. Based on the quantitative and qualitative analysis, it is observed that the proposed approach helps in improving the 3D reconstruction from a single image in an indoor environment.

A cross-domain visual place recognition (VPR) task is proposed in this work, i.e., matching images of the same architectures depicted in different domains. VPR is commonly treated as an image retrieval task, where a query image from an unknown location is matched with relevant instances from geo-tagged gallery database. Different from conventional VPR settings where the query images and gallery images come from the same domain, we propose a more common but challenging setup where the query images are collected under a new unseen condition. The two domains involved in this work are contemporary street view images of Amsterdam from the Mapillary dataset (source domain) and historical images of the same city from Beeldbank dataset (target domain). We tailored an age-invariant feature learning CNN that can focus on domain invariant objects and learn to match images based on a weakly supervised ranking loss. We propose an attention aggregation module that is robust to domain discrepancy between the train and the test data. Further, a multi-kernel maximum mean discrepancy (MK-MMD) domain adaptation loss is adopted to improve the cross-domain ranking performance. Both attention and adaptation modules are unsupervised while the ranking loss uses weak supervision. Visual inspection shows that the attention module focuses on built forms while the dramatically changing environment are less weighed. Our proposed CNN achieves state of the art results (99% accuracy) on the single-domain VPR task and 20\% accuracy at its best on the cross-domain VPR task, revealing the difficulty of age-invariant VPR.

We propose a framework to interpret deep convolutional models for visual place classification. Given a deep place classification model, our proposed method produces visual explanations and saliency maps that reveal the understanding of images by the model. To evaluate the interpretability, t-SNE algorithm is used for mapping and visualization of these latent visual explanations. Moreover, we use pre-trained semantic segmentation networks to label all objects appearing in the visual explanations for our discriminative models. This work has two main goals. The first one is to investigate the consistency of visual explanations by different models. The second goal is to investigate whether visual explanations are meaningful and interpretable or not in an unsupervised manner. We find that varying the CNN architecture leads to variations in the discriminative visual explanations, but these visual explanations are interpretable.

This work proposes a method for matching images from different domains in an unsupervised manner, and detecting outlier samples in the target domain at the same time. This matching problem is made difficult by i) the different domain images that are related but under different conditions (e.g. photos of the same location captured in different illuminations), ii) unsupervised settings with paired-image information available only for one of the domains, iii) the existing
of outliers that makes the two domains not fully overlap. To this end, we propose an end-to-end architecture that can match cross domain images in an unsupervised manner and handle not fully overlapping domains by outlier
detection. Our architecture is composed of three subnetworks, two of which are fed with pairs of source images to learn the ”match” information. The other subnetwork is fed with target images, and works together with the other two subnetworks to learn domain invariant representations of the source samples and the target inlier samples by applying a weighted multi-kernel Maximum Mean Discrepancy (weighted MK-MMD). We propose the weighted MK-MMD, together with an entropy loss, for outlier detection. The entropy loss iteratively outputs the probability of a target sample to be an inlier during training. And the probabilities are used as weights in our weighted MK-MMD for aligning only the target inlier samples with the source samples. Extensive experimental evidence on Office [26] dataset and our proposed datasets Shape, Pitts-CycleGAN shows that the proposed approach yields state-of-the-art cross domain image matching and outlier detection performance on different benchmarks.

Vegetation phenology is the interaction between vegetation activities and ecosystem. Accurate monitoring of vegetation phenology is required to build models and enhance the understanding of the relationship between creatures and climate-environment. PhenoCam is a ground-level, webcam based images database recording the growing of various vegetations, PhenoCam and multiple modeling methods have been utilized to study vegetation phenology since 2000s. In this paper, it first time the deep learning models are applied to detect the phenological transition dates of vegetation. Four different deep learning models: Convolution Neural Network (CNN), Siamese Network, 3-D Fully Convolution Neural Network (FCN) and Regression Network are used to study the vegetation phenology, based on these approaches, the transition dates of vegetation activities within annual time can be determined from webcam-based images, some of these deep learning methods are more accurate than traditional modeling method in detecting the transition dates.

This report describes the process of the Bachelorproject(TI3806) done for ‘De Energiebespaarders’, a startup in Amsterdam striving to make homes more energy efficient through accessible advice and installation of insulation or solar panels. The goal of the project was to apply machine learning to improve their system for identifying house features; windows, doors, and walls, and calculate their surface areas to improve the advice they can give to customers. The old system could produce good results, but was time-consuming to use and sensitive in regard to user input. We chose to implement a new system that makes the process automatic. In the report, our design process and our chosen implementation is described. Our new system makes use of a convolutional neural network to give pixels a label of wall(blue), window(red), door(purple), or nothing(black), without the need for the user to click the individual features beforehand. The results are promising and can save a lot of time, but the results are still inconsistent at times. Therefore, the report also contains recommendations for improvement of this new system, as De Energiebespaarders have shown interest in further developing our system.