Ray tracing has experienced increasing adoption in various spaces of computer graphics. The ReSTIR (Reservoir-based Spatiotemporal Importance Resampling) family of techniques has enabled several orders of magnitude speedups in light transport simulation algorithms which rely on ray tracing.

We introduce an extension to WRS (Weighted Reservoir Sampling), a key component of ReSTIR, to reduce the occurrence of duplicate samples in multi-sample reservoirs. Further, we show how samples from multiple reservoirs can be combined in an MIS-style estimator as opposed to resampling from them. Lastly, we combine these two components to compute optimal weights for this estimator in a similar vein to OMIS (Optimal Multiple Importance Sampling).

Our direct lighting proof of concept implementation demonstrates the efficacy of our WRS extension, lowering the variance of ReSTIR, particularly with difficult lighting arrangements. Further, our MIS-style estimator shows a significant improvement compared to ReSTIR. In problem domains where resampling produces a function value for integration, such as path tracing, this comes at no additional cost. Lastly however, optimal weights do not appear to be beneficial for our approach.

The prevalence of partial differential equations (PDEs) in modeling physics and the low speeds of numerical solvers demands more efficient solving methods. For this purpose, machine learning based methods have been proposed, but these are typically discrete, difficult to interpret and require large amounts of ground truth data to train. To address these issues, we propose a novel machine learning based solver that builds upon advancements in Gaussian based reconstruction. Our method represents the solution to a time-dependent target PDE purely in terms of Gaussians, making it completely continuous and meshless. These Gaussians are evolved by means of an autoregressive neural network that is applied to each Gaussian, integrating information from a local neighborhood of Gaussians. This enables the change of position, scale, rotation and value of the Gaussians to model the solution in a particle-like manner. Extensive experiments show the potential for Gaussians to model arbitrary physical phenomena. We also compare our approach to ground truth data and various state of the art methods, which demonstrates that the method performs well on short-term prediction, but does not match the state of the art for long-term prediction.

The bilateral filter is an edge-aware image filter. While it has a variety of applications, its naive implementation is quadratic in nature, hindering the ability to efficiently process multi-megapixel images. If performance is needed, like in a real-time setting, an approximation is necessary. Current literature on Fourier series-based approximations does not explore the capabilities of graphics processing units (GPUs) as viable platforms for this computational problem. This paper proposes an approach for implementing such filtering on a GPU by conducting a series of separable convolutions, and also investigates the use of different range kernels. Our adaption of the bilateral filter is found to be more two times faster than readily available solutions, with frame times showing that real-time performance is possible for large spatial kernel sizes and image resolutions.

Unlike traditional blur filters, the bilateral filter exhibits non-linear blur behaviour as its kernel size increases. This atypical blur behaviour makes it challenging to find a good σr . This paper investigates the underlying reasons for this behaviour and proposes methods to align the bilateral filter’s blur scaling linearly with its spatial filter size. Using local frequency analyses to quantify blur levels, we introduce an approach that finds the best σr through iterative search. Results demonstrate that the pro- posed method effectively counters the atypical blur behaviour. However, the proposed method does not perform sufficiently when handling very large kernel sizes. The proposed method can be used to abstract away the σr parameter when seeking linear blur behaviour in the bilateral filter. Further re- search is needed to make it functional for very large kernel sizes.

This paper introduces the Quadrilateral filter, an advanced extension of the Bilateral and Trilateral filters aimed at addressing limitations in high-gradient regions of images. While the Bilateral filter effectively preserves edges during smoothing, it struggles with intensity variations, leading to blunted image details. The trilateral filter improves upon this by incorporating local plane geometry approximations but assumes linear pixel intensity distributions, limiting its effectiveness. The proposed Quadrilateral filter utilizes curvature-based geometry approximations to enhance noise reduction, contrast preservation, artifact reduction, and image reconstruction by accounting for nonlinear pixel value distributions. The development of this filter represents the main contribution of the paper while exploring whether the established Bilateral and Trilateral filters’ performance can be further improved through curvature-based local geometry approximations. The findings demonstrate improvements in image quality and detail preservation, with broad implications for applications in image de-noising, tone-mapping, multimedia processing, and beyond.

Traditional bilateral filters, effective in 2D image processing, often fail to account for the 3D structure of meshes, leading to artifacts in texture filtering. This thesis introduces On-Mesh Bilateral Filtering, a novel method that adapts the bilateral filter to work with non-contiguous texture mappings by incorporating 3D spatial distances and face adjacency information into the filtering process. The On-Mesh Bilateral Filter combines mesh surface sampling techniques with heat geodesic distance calculations to create a geometry-aware kernel that achieves more accurate and context-sensitive smoothing operations, respecting both the mesh topology and texture space properties. This paper hopes to encourage further research in the area of geometry-aware texture filters.

The bilateral filter is a popular filter in image processing and computer vision. This comes from the fact that it is able to blur images while keeping the structure intact. However, the bilateral filter allows for blurring to happen across edges. This can result in halo-like effects around the edges of structures if both sides are made up of different intensities. In this paper, we propose an extension to the bilateral filter that reduces this phenomenon of blurring across edges. By giving the filter knowledge of the edges beforehand, it is possible to prevent the filter from blurring past them. When we filter a pixel, its surrounding area within the kernel is checked for edges. If a pixel within this area lies on or beyond an edge, its weight for blurring is reduced. As a consequence, pixels that lie past an edge have less influence on blurring. We show that this new edge-aware bilateral filter reduces across-edge blurring compared to the standard bilateral filter. Furthermore, when we allow a bigger range of intensities to mix, the new filter is also able to prevent the filtered image from appearing washed out, unlike the bilateral filter.

Dimensionality reduction is an important task in high-dimensional data visualisation. Among the popular algorithms for achieving this is t-SNE, which aims to preserve local neighbourhoods in the lower-dimensional embeddings. While t-SNE traditionally works in Euclidean space, embedding in hyperbolic space offers several advantages, specifically for data of arbitrary size and exponential growth, such as tree-based structures. We propose a new solution to approximate and accelerate the calculation of t-SNE gradients using a uniform grid structure. This new method produces embeddings with better neighbourhood preservation than previous solutions, while also providing better runtime performance.

In this work we aim to implement a variaton of the acceleration of hyperbolic t-SNE done by Skrodzki et. al. [19]. This variation aims to embed the points in the Klein Disk model of hyperbolic space instead of the Poincar ́e Disk model using an altared version of a polar quadtree to speed up the computation in a similar fashion as the Barnes-Hut scheme for the Euclidean versino of t-SNE. We analyze our results to prove our acceleration works for the Klein Disk model and compare the efficiency of our implemen- tation to the one for the Poincar ́e model in terms of quality of results and runtime.

With the rapid growth in data collection, efficient data processing is critical. Dimensionality reduction methods, like t-distributed stochastic neighbour embedding (t-SNE), compress high-dimensional data into embeddings that preserve the key features of the datasets making data less sparse and easier to process further. Recent improvements suggest that using hyperbolic space for data representation can benefit embeddings. As it is a new technique, it remains computationally expensive. Previous work suggests that a Barnes-Hut approximation with a polar quadtree can be applied to the Poincaré disk model to approximate the result of hyperbolic t-SNE and accelerate its calculation. However, the polar quadtree is proposed as a solution to accelerate the calculation without exploring alternative approaches. Aiming to close this gap, we propose an acceleration method using Barnes-Hut approximation with a Cartesian quadtree. We experimentally compare our acceleration method to a polar quadtree and showcase its lower execution time without the loss of quality of the embeddings.
Implementation and scripts for the experiments and plots are available at https://github.com/Sne4kers/hyperbolic-tsne

This paper investigates a method for accelerating hyperbolic t-SNE — a popular high-dimensional data visualization technique. In particular, it focuses on building a hyperbolic t-SNE variant that uses a different model of hyperbolic space (called the Lorentz Hyperboloid model) for representing the low-dimensional embeddings. An acceleration algorithm based on a tree data-structure is then used to achieve a better asymptotic runtime complexity compared to the original version. The paper then compares this implementation with other alternatives — including accelerated variants — and shows that it computes embeddings of better quality at a similar rate.

The objective of this project is to train a model that transforms a tree with its foliage into only its branch structure. This is achieved by employing machine-learning techniques, specifically Generative Adverserial Networks (GANs). By utilizing the proposed method, a predictive model is built that automatically minimizes its own error function through a comparison of a set of input and ground-truth tree images, which are tree images with and without leaves, respectively. The adoption of GANs has shown promising results, both visually and metrically.

The most established and widely used methods for analysing tree images for tasks such as geometry analysis, segmentation and classification often rely on pixels. In this paper, the applicability of analyzing tree geometry based on a graph representation rather than a pixel-based approach is pursued. To do so, 2D renders of different species of trees are converted to spatial graph structures capturing significant points on the tree skeleton. Two independent Graph Convolutional Network algorithms which learn node (coordinate) features are then applied on the obtained dataset to assess the reliability of graph based analysis. The first experiment explores a GCN for assigning correct species labels to the skeleton graph of the original tree image, demonstrating the association between geometry and tree metadata. The second experiment, an unsupervised representation learning, is conducted by using Graph Autoencoders to obtain an embedding for each skeleton graph which can be used to reconstruct partially the same graph, demonstrating the association between GCE latent representation and geometry. Promising results were found in both cases, reinforcing the reliability of the original proposition to rely on geometry as well as pixels for tree analysis tasks.

L-Systems allow for the efficient procedeural generation of trees to be used for rendering in video games and simulations. Currently, however, it is difficult to engineer grammars that mimic the behaviours of real life trees in 3 dimensions. To be able to deduce them, the skeleton of a tree can be used to train a model and generate an L-system for a given tree in particular. The aim of this paper is to provide a pipeline to isolate these skeletons from images of a tree, using Neural Radiance Fields (NeRFs) to reconstruct the tree, and using Laplacian Based Contraction to retrieve the underlying skeleton. We find that this approach leads to 3-dimensional topologies that very closely resemble the given tree.

Trees are essential components of both real and digital environments. Therefore, it is important to have 3D models of trees that are of high quality and computationally efficient. One way to achieve this is by compressing a high-quality model using billboard rendering, which involves partitioning the tree into multiple planes to produce a similar result to the original. Our study explores the compression of 3D models using an optimization loop and adapting billboard rendering techniques. We use computer vision primitives to render basic models, which we then optimize by adjusting the texture to resemble the original tree. The models consist of multiple upright planes that are rotated around the central vertical axis of the original tree. We use different optimization functions, such as L1 and L2 losses, to determine the best approach. We can improve the initial models by bounding the billboards and limiting their heights and widths to that of the trees. Additionally, we can use double-sided textures for the billboards to allow more flexibility for optimizing different species of trees. However, optimizing multiple tree types performs differently for each species, leading to improvements that only benefit certain trees in specific scenarios. Using quantitative metrics, we determined which models perform best and how similar they are to the original after training. We found that our compressed models generally resemble the original while having only a fraction of the original size.

Image inpainting is a problem that has been well studied over the last decades. In contrast, for 3D reconstructions such as neural radiance fields (NeRFs), work in this area is still limited. Most existing 3D inpainting methods follow a similar approach: they perform image inpainting on the training images and use the inpainted images for further training of the 3D model. Due to inconsistencies in the different inpaintings of the images, the 3D inpainting often becomes blurry. With the advent of 3D Gaussian Splatting (3DGS), we identify a new opportunity for 3D inpainting. As 3DGS is more explicit in nature than NeRF, we can manipulate the 3D Gaussians directly rather than relying on image inpainting. Based on that key idea, we propose a method that works similar to the PatchMatch image inpainting algorithm. We first construct a nearest-neighbour field (NNF) by searching for nearest-neighbour patches throughout the scene that look similar to the area we want to inpaint. After constructing the NNF we copy the contents of the nearest-neighbour patches to the inpainting region and blend them together to obtain the inpainting result. In our experiments we found that our method performs well in terms of texture synthesis but struggles with structure synthesis, similar to the original PatchMatch algorithm. In cases where only texture synthesis is required to inpaint the area our method is able to provide good results, although in some cases pre-processing of the scene is necessary, as we found that better quality inputs (e.g. the scene itself, the surface mesh underlying the scene, and precise masks) drastically improve the results of our method. Moreover, some parameters of the algorithm are highly scene-dependent and by tailoring them to the scene we can further enhance the performance of the algorithm. Besides introducing a 3D inpainting method that directly manipulates the scene contents, our work offers valuable new insights into 3DGS editing in general.

The phenomenon of one element moving and progressively overlaying another is common in nature, such as waves swashing and backwashing, or eyelids moving over eyeballs while blinking. Folding Texture, which was proposed by Thorben, can simulate this texture “folding” visual effect in real-time without changing geometry.
However, to date, no tool has been developed to assist in the design and synthesis of folding textures. Applications of the technique so far are achieved through manual creation of the folding texture, which is a tedious process.
This thesis explores the problem of folding-texture design and synthesis. A novel approach is proposed for animating still images based on the folding texture technique. The approach uses a semi-automatic, user-assisted method that combines texture editing, motion profile specification, and folding texture synthesis into one seamless process, reducing the need for extensive manual work. It enables novice users to utilize the technique with a fair level of prior knowledge of folding texture.

Neural radiance fields (NeRF) based solutions for novel view synthesis can achieve state of the art results. Recent work proposes models that take less time to render, need less training data or take up less space. However, few papers explore the use of NeRFs in classic rendering scenarios such as rasterization, which could contribute to wider adoption. Our paper tackles the issue of shadow generation and proposes a deep residual MLP network with fast evaluation times, that generates view-dependent shadow maps. The network distills the knowledge of an existing NeRF model and achieves the speedup through the use of neural light fields, by only doing one network forward per ray.

Neural Radiance Fields (NeRF) and their adaptations are known to be computationally intensive during both the training and the evaluating stages. Despite being the end goal, directly rendering a full-resolution representation of the scene is not necessary and not very practical for scenarios like streamed applications. Our goal is to design a streamable adaptation for a model that can produce fast, rough estimates of 3D scenes, by only using a shallow part of the network. The quality is subsequently improved as more parts of the network are available, such that it can be used in online applications where the model needs to be transferred. Separate models can be trained at different resolutions, but this approach results in a large space overhead and also increases the evaluation time. This can be mitigated by reducing the depth of low-resolution models, but redundancy will still be high as each new model needs to re-evaluate the input data, rendering previous calculations obsolete. Our method combines key concepts from previous approaches to create a progressively trained model that is able to produce intermediate outputs of increasing quality while attempting to optimize the trade-off between overhead and quality. Our model is able to produce a recognizable representation of the scene with as little as one hidden layer from the original model. It also allows for division into streamable chunks which can be sent individually and, upon reconstruction, provide intermediate outputs that bring consistent improvement in quality. The newly streamed data uses the residual output from previous computations in order to reduce redundancy. We show that the final quality of our adaptation is within 2% of the original in terms of previously used quantitative metrics.