Pontus Ebelin defends his thesis Evaluating and Improving Rendered Visual Experiences - Metrics, Compression, Higher Frame Rates & Recoloring

Rendered imagery is presented to us on a daily basis. Special effects in movies, video games, scientific visualizations, and marketing catalogs all often rely on images generated through computer graphics. However, with all the possibilities that rendering offers comes also a plethora of challenges. This thesis proposes novel ways of evaluating the visual errors caused when some of these challenges are not fully solved. It also suggests ways to improve on the perceptual experience observers have when looking at rendered content. Leveraging the capabilities and limitations of the human visual system is a provenly powerful tool for improving rendering and continued research on the combination of perception and rendering is likely to prove beneficial—a claim further strengthened by this thesis.

In the introduction of this thesis, I provide an overview of a subset of the many and fantastic aspects of the human visual system. I also describe how images are rendered using computer graphics, some of the related challenges, and how the final result is displayed to users. Finally, I discuss some of the basics of image and video quality assessment. The scientific publications contained in this thesis focus on image quality metrics, compression, and rendering at high frame rates. In addition, one paper considers the recoloring of images with the goal of giving people with color vision deficiencies an improved visual experience in a process known as daltonization.

Papers I–III provide researchers and developers a new way to evaluate and communicate errors that users may see in the content they render. In these papers, an image’s error is determined by how much it visually differs from a perfect-quality version of the same image. The focus is on the error map, an image that indicates the magnitude and locations of errors. In Paper IV, tools proposed in the first three papers are used to convey how a novel material texture compression algorithm results in lower visual error compared to competing techniques at similar, low bit rates. To achieve good quality at high compression rates, the proposed algorithm exploits that the textures used for materials have similar spatial content.

Starting with Paper V, the thesis puts increased emphasis on temporal effects. That paper estimates the temporal edge detection filters in human vision, while previous research had mainly examined spatial edge detection filters. Paper VI makes a good example of how perceived quality in rendering can be improved by leveraging the human visual system. The paper suggests a method for rendering 4× more frames per second which, paired with content-dependent sampling patterns and reconstruction for different tiles of pixels, improve the overall visual experience of rendered image sequences. This thesis’ final paper, Paper VII, presents a real-time daltonization algorithm that recolors images in a temporally consistent manner, so as to avoid flickering hue changes in image sequences, which is often an issue for competing algorithms that originally target images shown in isolation. The proposed recoloring preserves luminance and thus the important visual ques it provides.