FACULTY OF SCIENCE AND ENGINEERING
Linköping Studies in Science and Technology Dissertation No. 2272
Linköping Studies in Science and Technology, Dissertation No. 2272, 2023
Department of Science and Technology
Linköping University
SE-581 83 Linköping, Sweden
www.liu.se
Tanaboon Tongbuasirilai
Data-Driven Approaches
for Sparse Reflectance
Modeling and Acquisition
Tanaboon Tongbuasirilai
Data-Driven Approaches for
Sparse Reflectance Modeling and Acquisition
2023
Linköping studies in science and technology.
Dissertation, No. 2272
Data-driven approaches for sparse
reflectance modeling and
acquisition
Tanaboon Tongbuasirilai
Division of Media and Information Technology
Department of Science and Technology
Linköping University, SE-601 74, Norrköping, Sweden
February 2023
Data-driven approaches for sparse reflectance modeling and acquisition
Copyright © 2023 Tanaboon Tongbuasirilai (unless otherwise noted)
Division of Media and Information Technology
Department of Science and Technology
Linköping University, Campus Norrköping
SE-601 74 Norrköping, Sweden
ISBN
ISBN
ISSN
DOI
:
:
:
:
978-91-7929-557-8 (Print)
978-91-7929-558-5 (PDF)
0345-7524
10.3384/9789179295585
Printed in Sweden by LiU-Tryck, Linköping, 2023
This work is licensed under a Creative Commons AttributionNonCommercial 4.0 International License.
https://creativecommons.org/licenses/by-nc/4.0/
Description of the cover image
The front cover of this thesis illustrates BRDFs in the form of
multiple rendered spheres along BRDF curves (RGB signals) where
the glossy BRDF curves are placed above the diffuse BRDF curves.
The front illustration portrays the results of the thesis contents.
The blue wave signals in the back cover represent sparse signals
which serve as the BRDF encoding process (thesis’s contributions)
to reconstruct the BRDF signals in the front cover. The orange
abstract waves in the background indicate that the BRDF is a type
of signals around us. The materials used to render all the spheres
are measured isotropic BRDFs from Mitsubishi Electric Research
Laboratories (MERL).
Abstract
Photo-realistic rendering and predictive image synthesis are becoming increasingly
important and utilized in many application areas ranging from production of visual
effects and product visualization to digital design and the generation of synthetic
data for visual machine learning applications. Many essential components of the
realistic image synthesis pipelines have been developed tremendously over the last
decades. One key component is accurate measurement, modeling, and simulation
of how a surface material scatters light. The scattering of light at a point on
a surface (reflectance and color) is described by the Bidirectional Reflectance
Distribution Function (BRDF); which is the main research topic of this thesis.
The BRDF describes how radiance, light, incident at a point on a surface is
scattered towards any view-point from which the surface is observed. Accurate
acquisition and representation of material properties play a fundamental role in
photo-realistic image synthesis, and form a highly interesting research topic with
many applications.
The thesis has explored and studied appearance modeling, sparse representation and
sparse acquisition of BRDFs. The topics of this thesis cover two main areas. Within
the first area, BRDF modeling, we propose several new BRDF models for accurate
representation of material scattering behaviour using simple but efficient methods.
The research challenges in BRDF modeling include tensor decomposition methods
and sparse approximations based on measured BRDF data. The second part of
the contributions focuses on sparse BRDF sampling and novel highly efficient
BRDF acquisition. The sparse BRDF sampling is to tackle tedious and timeconsuming processes for acquiring BRDFs. This challenging problem is addressed
using sparse modeling and compressed sensing techniques and enables a BRDF to
be measured and accurately reconstructed using only a small number of samples.
Additionally, the thesis provides example applications based on the research, as
well as a techniques for BRDF editing and interpolation.
Publicly available BRDF databases are a vital part of the data-driven methods
proposed in this thesis. The measured BRDF data used has revealed insights to
facilitate further development of the proposed methods. The results, algorithms,
and techniques presented in this thesis demonstrate that there is a close connection
between BRDF modeling and BRDF acquisition; efficient and accurate BRDF
modeling is a by-product of sparse BRDF sampling.
v
Populärvetenskaplig Sammanfattning
Fotorealistisk rendering och prediktiv bildsyntes har blivit allt viktigare och används i flera olika tillämpningsområden, allt ifrån produktion av visuella effekter
och produktvisualisering till digital design och generering av syntetiska data för
tillämpningar inom visuell maskininlärning. Utvecklingen har tagit en ordentlig
fart under de senaste decennierna för många av de väsentliga komponenterna i
det fotorealistiska bildsyntes-området. En nyckelkomponent inom området är att
noggrant kunna mäta, modellera och simulera ljusets spridning från ytan hos ett
material. Ljusspridningen från en punkt på en yta med reflektion och färg, beskrivs
av en funktion (eng. BRDF); vilket är det huvudsakliga forskningsområdet i den här
avhandlingen. BRDF beskriver hur strålning, ljus i detta fall, träffar en punkt på
en yta och sprids mot varje observerad synvinkel. En noggrann uppmätning och representation av materialegenskaper är en fundamental del i fotorealistisk bildsyntes
och omger ett väldigt intressant forskningsområde med många tillämpningar.
Den här avhandlingen har utforskat och studerat modellering för materialytors
utseende, glesa representationer och glesa mätningar av ljusspridningsfunktioner. Avhandlingen täcker två huvudområden. Inom det första området, BRDF-modellering,
så presenterar vi ett flertal nya BRDF-modeller för noggrann representation av
ljusspridningens beteende från materialets yta genom att använda simpla men
effektiva metoder. Forskningsutmaningarna inom BRDF-modellering inkluderar
både metoder för tensoruppdelning och glesa approximationer baserat på uppmätt
BRDF-data. Den andra delen fokuserar på gles BRDF-sampling och en ny och effektiv mätningsmetod för att mäta BRDF. Syftet med den glesa BRDF-samplingen
är för att förenkla och snabba upp de enormt tidskrävande processerna som krävs
för att mäta BRDF. Detta utmanande problem löses genom att använda glesa
modeller och tekniker från compressed sensing som möjliggör att BRDF kan be
uppmätt och noggrant rekonstruerad genom att endast använda ett fåtal uppmätta
sampel. Slutligen så visar avhandlingen ett flertal exempel på tillämpningsområden
från forskningen, så väl som tekniker för BRDF-editering och interpolation.
De BRDF-databaser som är öppna och tillgängliga för allmänheten är en vital del
av de datadrivna metoderna som presenteras I denna avhandling. De uppmätta
BRDF-data som använts har öppnat nya insikter för vidare utveckling av de
framtagna metoderna. Resultaten, algoritmerna och teknikerna presenterade i den
här avhandlingen visar på att det finns en nära koppling mellan BRDF modellering
och BRDF-mätning; effektiv och noggrann BRDF-modellering är en biprodukt av
gles BRDF-sampling.
vii
Acknowledgments
Life is a journey and my thesis is analogous to a record of one moment of my life.
It contains loads of wonderful memories during my PhD studies. I set sail and
carried on this expedition with the supports of many amazing and great people. I
could not have done it without their guidance, advice and encouragement. This
part of this thesis is dedicated to the expression of my sincere gratitude towards
my supervisor, co-supervisors, colleagues, friends and family.
First of all, I would like to thank my supervisor, Jonas Unger, for his support,
advice, guidance and patience throughout my PhD expedition. Your guidance
assisted in steering this journey in the right direction. You have had positive
attitudes and confidence to encourage me. All of the manuscripts could not be
perfect without your touching. It has been a privilege to work with such a great
and talented researcher like you. I would like to thank my co-supervisor, Ehsan
Miandji. You introduced me to the sparseland and this inspired me and my future
direction. Your enthusiasm motivated and influenced my research. I am grateful
for all of our discussions and the joyful moments we had outside of work. Your
food preference makes me so happy about Thai food and chilis. Additionally, I
appreciate the research facilities and environment established by Anders Ynnerman.
Your efforts and contributions to the division of Media and Information Technology
(MIT) are invaluable.
Sailing to the sea of academic and research work, I have met fantastic people. They
are amazing and prominent crews on this journey. I would like to thank my friends
and colleagues in the Computer Graphics and Image Processing (CGIP) group.
Apostolia Tsirikoglou is the first crew whom I met before the beginning of this
expedition. Thank you for the enjoyment you brought to our group. Thank you
to Saghi Hajisharif for all suggestions you gave me. Whenever I need information,
you are much better than Wikipedia. When I want to discuss deep-learning topics,
Gabriel Eilertsen is the expert whom I can rely on. Your unintentional humor
makes our chats more fun. Thank you to Gabriel Baravdish for the small talks we
had in the morning at the office. The chilis on your food made me excited. Thank
you to Behnaz Kavoosi for the collaborations, discussions and small chats we had
in our office. Thank you to Fereshteh Abedini for all favors I have asked for. Thank
you to chinese PhD students, Wen Cao and Yifan Ding, for insightful information
about China. Thank you to Igor Fedorov for setting up servers in the lab. I would
like to extend thanks to the old crews, Joel Kronander, Per Larsson and Andrew
Gardner, who strengthened this group with their inventions, ideas and discussions.
Thank to people from Linköping, Joel Hedlund, Karin Stacke, Milda Poceviciute,
Shreyas Shivakumara and Betül Eren Keskin for bringing up the meetings lively.
ix
x
During the transition step of the group, I would like to thank Daniel Jönsson for
taking this responsibility. Moreover, all the paper and administrative works could
not be done smoothly without the help from Eva Skärblom and Carina Enemård.
During my journey in Sweden, I was fortunate to meet a lovely family, who were
incredibly kind and helpful to me. I would like to express my deep gratitude to
Pom Jonsson and her sons, Martin and Robin, for all their assistance and support.
They were like a Swedish version of my own family. I cannot thank you enough
for all Thai friends whom I had a great time with, including Kittichate Visuttijai,
Auttapol Chaisomboonpun, and Pattharaporn Thongnim. Furthermore, my deepest
gratitude is granted to my family in Thailand. You are supportive of whatever I
needed during my studies. I could not have done it without them.
Last but not least, the Buddha’s teachings have been a source of great comfort
and guidance for me, especially during difficult and stressful times. His wisdom
has helped me to prepare my mind and to find peace in the face of uncertainty.
I am grateful for the Buddha’s teachings and for the ways in which they have
helped me to grow and to better understand the world around me. I hope that the
following quote, which has been a source of inspiration for me, will also be helpful
and uplifting for others.
"A disciplined mind brings happiness."
The Gautama Buddha
Tanaboon Tongbuasirilai
Norrköping, February 2023
List of Publications
This thesis includes research that is based the publications listed below:
• A. Tsirikoglou, J. Kronander, P. Larsson, T. Tongbuasirilai, A. Gardner, and
J. Unger, “Differential appearance editing for measured brdfs,” in ACM SIGGRAPH 2016 Talks, ser. SIGGRAPH ’16. New York, NY, USA: Association
for Computing Machinery, 2016
• T. Tongbuasirilai, J. Unger, and M. Kurt, “Efficient brdf sampling using projected
deviation vector parameterization,” in 2017 IEEE International Conference
on Computer Vision Workshop (ICCVW). Los Alamitos, CA, USA: IEEE
Computer Society, oct 2017, pp. 153–158
• T. Tongbuasirilai, J. Unger, J. Kronander, and M. Kurt, “Compact and intuitive
data-driven brdf models,” The Visual Computer, vol. 36, no. 4, pp. 855–872,
2020
• T. Tongbuasirilai, J. Unger, C. Guillemot, and E. Miandji, “A sparse nonparametric brdf model,” ACM Trans. Graph., apr 2022
• E. Miandji, T. Tongbuasirilai, S. Hajisharif, B. Kavoosighafi, and J. Unger, “Frostbrdf : A fast and robust optimal sampling technique for brdf acquisition,” To
be sumitted to ACM Trans. Graph., 2023
Other publication by the author that is not included is a journal paper. The topic
is about person identification using Electroencephalography (EEG) signals.
• T. Wilaiprasitporn, A. Ditthapron, K. Matchaparn, T. Tongbuasirilai, N. Banluesombatkul, and E. Chuangsuwanich, “Affective eeg-based person identification
using the deep learning approach,” IEEE Transactions on Cognitive and Developmental Systems, vol. 12, no. 3, pp. 486–496, 2020
xi
Contributions
This thesis provides two major sets of contributions to Bidirectional Reflectance
Distribution Function (BRDF). The first set of contributions focuses on datadriven BRDF modeling, representations and its applications (BRDF editing and
interpolation) as presented in Paper A, Paper C and Paper D. The second set of
contributions develops algorithms for sparse BRDF measurements as presented in
Paper B and Paper E. In what follows, the publications included in this thesis are
listed with a short description below.
Paper A : Differential appearance editing for measured brdfs
A. Tsirikoglou, J. Kronander, P. Larsson, T. Tongbuasirilai, A. Gardner, and
J. Unger, “Differential appearance editing for measured brdfs,” in ACM SIGGRAPH
2016 Talks, ser. SIGGRAPH ’16. New York, NY, USA: Association for Computing
Machinery, 2016
This paper presents a BRDF editing method as well as a BRDF interpolation
technique on measured BRDF data. Our proposed method utilizes a parametric
model to facilitate users in the editing and interpolation as the first step. The
edited BRDF will be further computed as the differential appearance, a residual
vector. The final step modifies the measured BRDF using the residual vector. Our
method can preserve some effects which cannot be captured by the parametric
model since the final edits are performed on the measured BRDF domain yet users
is able to achieve desirable effects with few parameters. The author of this thesis
was responsible for providing tools and part of implementations for the project.
Paper B : Efficient BRDF Sampling Using Projected Deviation Vector
Parameterization
T. Tongbuasirilai, J. Unger, and M. Kurt, “Efficient brdf sampling using projected
deviation vector parameterization,” in 2017 IEEE International Conference on
Computer Vision Workshop (ICCVW). Los Alamitos, CA, USA: IEEE Computer
Society, oct 2017, pp. 153–158
This work was inspired by a parametric BRDF model. Parameterizing BRDFs under
Projected Deviation Vector (PDV) domain, isotropic BRDFs, specifically glossy
materials, can be represented by only two 1D functions. The two 1D basis functions
can be measured using a simple gonioreflectometer with two degree-of-freedom
configuration as opposed to typical four degree-of-freedom configuration. This
results in an efficient sampling method where a densely-measured isotropic BRDF
can be recovered using two measured bases. The evaluation was performed on
xiii
xiv
simulated measurements to demonstrate small visual errors introduced by our factor
measurement. The project had an international collaboration with a researcher
in the related field. This work was presented in a workshop of ICCV 2017. The
author of this thesis was responsible for the project initiation, implementation of
the algorithm, and evaluations of the method. The author also contributed to the
majority of the written manuscript and the oral presentation.
Paper C : Compact and intuitive data-driven BRDF models
T. Tongbuasirilai, J. Unger, J. Kronander, and M. Kurt, “Compact and intuitive
data-driven brdf models,” The Visual Computer, vol. 36, no. 4, pp. 855–872, 2020
This paper is a continuation of Paper B and was published in a journal. Two
iterative-factorized BRDF models were proposed with importance sampling capability. The BRDF models utilize the characteristics found in the PDV parameterization
and the CP Decomposition method (CPD). The importance sampling strategy is
newly derived for the proposed models. Besides, analyses of parameterizations
are provided to demonstrate some insights of relationship between BRDF values
and several parameterizations. Our models have shown to outperform previous
factorized BRDF models in both quantitative and qualitative evaluations. The
project had a continuation of the international collaboration as in Paper B. The
author of this thesis contributed to the ideas, design and implementations of the
project as well as the written manuscript.
Paper D : A Sparse Non-parametric BRDF Model
T. Tongbuasirilai, J. Unger, C. Guillemot, and E. Miandji, “A sparse non-parametric
brdf model,” ACM Trans. Graph., apr 2022
This paper presents a non-parametric BRDF model with a unique interpolation
technique. Our proposed model represents isotropic BRDFs as multiple dictionaries,
basis functions. A BRDF is then reconstructed using a linear combination of sparse
coefficients and dictionaries. To obtain the dictionaries, a modified Aggregate
Multi Dictionary Ensemble (AMDE) algorithm is developed. The measured BRDF
datasets were used to train and evaluate our proposed model. Additionally, a model
selection method is presented to pick best BRDF representation among resulting
candidates without a visual comparison. Moreover, BRDF interpolations on the
dictionary domain can be achieved through our derived technique. The project
had an international collaboration with a researcher from INRIA. A part of this
work was presented in CLIM 2021 workshop. The final version was published in
ACM Transactions on Graphics 2022 and nominated to present in Siggraph 2023.
The author of this thesis was responsible for the project initiation, design and
implementation of the algorithm as well as the written manuscript.
xv
Paper E : FROST-BRDF : A Fast and Robust Optimal Sampling Technique for BRDF Acquisition
E. Miandji, T. Tongbuasirilai, S. Hajisharif, B. Kavoosighafi, and J. Unger, “Frostbrdf : A fast and robust optimal sampling technique for brdf acquisition,” To be
sumitted to ACM Trans. Graph., 2023
In this paper, a sparse BRDF acquisition framework was proposed. Sparse representation and compressed sensing are foundations of the problem reformulation such
that our method produces an optimal set of sampling regions for a sparse BRDF
acquisition. The technique is simple but efficient as the problem is reformulated
under a system of linear equations. As a result, our method is faster and more
robust as well as outperformed compared to current state-of-the-art which uses a
heuristic search on a probabilistic method. The evaluation was performed through
10-fold cross-validations on publicly available BRDF databases. The author of this
thesis was responsible for the implementation of the method, the experiments as
well as the written manuscript. The manuscript will be submitted to CVPR 2023.
Contents
Abstract
v
Populärvetenskaplig Sammanfattning
vii
Acknowledgments
ix
List of Publications
xi
Contributions
xiii
1 Introduction
1.1 Towards the Virtual World
1.2 Object Representation
1.2.1 Geometric Shape Modeling
1.2.2 Surface Reflectance Modeling
1.3 Surface Reflectance Measurement
1.4 Objectives and Contributions
1.5 Thesis Outline
1
2
4
5
6
8
9
11
2 Fundamentals of Rendering and Material Apperance
2.1 Fundamentals of Light Transport
2.1.1 Light Transport Problem
2.1.1.1 Radiometry
2.1.1.2 Path Integral Equation
2.1.2 Light Transport Algorithm
2.1.2.1 Monte Carlo Integration
2.1.2.2 Importance sampling
2.1.2.3 Transforming between PDFs
2.1.2.4 The cosine-weighted hemisphere sampling
2.2 Surface Reflection
2.2.1 The Reflectance Functions
2.2.1.1 Physically Plausible Properties
2.2.1.2 Hierarchical chart of reflectance functions
2.3 Parameterization
2.3.1 Standard parameterization
2.3.2 Half-Diff parameterization
2.3.3 Half-Out parameterization
2.3.4 Projected Deviation Vector (PDV) parameterization
2.3.5 Other parameterizations
13
14
14
15
17
20
20
21
21
22
22
23
23
26
27
28
28
29
29
30
xvii
Contents
2.4
Data Transformation
2.4.1 Log-based Transformation
2.4.2 Gamma Transformation
Material Databases and Acquisition Systems
2.5.1 MERL
2.5.2 RGL-EPFL
2.5.3 Other databases
2.5.4 The catadioptric measurement
30
31
33
34
34
35
36
36
3 Tensor Decompositions and Sparse Representation
3.1 Tensor Decompositions
3.1.1 Tensor Operations
3.1.2 Tensor Decomposition Methods
3.1.2.1 CP Decomposition (CPD)
3.1.2.2 Higher-Order SVD (HOSVD)
3.2 Sparse Signal Representation
3.2.1 Sparse Representation
3.2.2 Dictionary Learning
37
38
38
40
40
41
42
42
44
4 Surface Reflectance Modeling and its applications
4.1 Outline and Contributions
4.2 BRDF Models
4.2.1 The Empirical BRDF Model
4.2.2 The Analytical BRDF Model
4.2.3 The Data-Driven BRDF Model
4.3 Our Data-driven BRDF Models
4.3.1 The Factorized BRDF Model
4.3.1.1 Motivation
4.3.1.2 Our BRDF model
4.3.1.3 Importance sampling on PDV
4.3.2 The Sparse Non-parametric BRDF Model
4.3.2.1 Motivation
4.3.2.2 An Aggregated Multi Dictionary Ensemble for
BRDFs
4.3.2.3 BRDF Encoding and Decoding
4.3.2.4 BRDF model Selection
4.4 BRDF Applications
4.4.1 Differential apperance editing
4.4.2 BRDF interpolation in sparseland
4.5 Summary and Future Work
47
47
48
48
51
53
55
56
56
58
59
61
61
5 Sparse Sampling for BRDF Acquisition
73
2.5
xviii
62
65
66
68
68
69
71
Contents
5.1 Outline and Contributions
5.2 The sparse BRDF measurement
5.2.1 The BRDF factor measurement using PDV
5.2.1.1 Motivation
5.2.1.2 The separable BRDF models
5.2.1.3 Results
5.2.2 A Fast and Robust Optimal Sampling Technique : FROST
5.2.2.1 Motivation
5.2.2.2 The Dictionary
5.2.2.3 Compressed sensing
5.2.2.4 FROST
5.2.2.5 BRDF Reconstruction
5.2.2.6 Results
5.3 Summary and Future Work
73
74
74
74
74
76
78
78
80
80
82
84
85
88
6 Summary and Outlook
91
Bibliography
95
Publications
121
Paper A
123
Paper B
129
Paper C
139
Paper D
161
Paper E
183
xix
xx
Chapter
1
Introduction
Computer graphics is currently being employed in a wide range of important
domains. Photo-realistic and predictive rendering is a core element and research
area of central importance to many industries as it enables applications including
digital design, visual effects in movies, computer games, product visualization,
computer-aided tools for education, virtual and augmented reality, and even sensor
simulation and generation of input to machine learning and artificial intelligence
systems, to name a few. Figure 1.1 illustrates the layout and steps of a photorealistic rendering pipeline. As can be seen, material appearance is at the heart of
the modeling, representation, and simulation of virtual objects in photo-realistic
image synthesis.
The appearance of different materials is diverse and originates from different
phenomena governed by the physics of light. An object’s appearance is defined
by the underlying scattering properties of the material such as color, reflection,
and transparency. To enhance the accuracy and realism in computer-generated
renderings, it is becoming increasingly common to measure and model real-world
appearance properties, i.e., creating accurate digital copies/twins from physical
material samples. A key challenge, however, of the reflectance measurement process
is that it is very time-consuming. Depending on the measurement device and data
resolution, the measurement process can take hours or even days for obtaining the
reflectance data of a single point on a single material. Moreover, once we have
the measured data, the next challenge is that realistic rendering and simulation
require advanced reflectance models capable of representing the acquired appearance
characteristics accurately while supporting efficient computations. The goal of this
2 Chapter 1 • Introduction
thesis is to contribute to solving these challenges.
In this introductory chapter, we give an overview of techniques for simulating
virtual objects and environments using photo-realistic image synthesis. We also
explain briefly the different components of the representations used to produce
accurate visual results at high fidelity, and finally, we summarize the chapter by
presenting the thesis’s objectives and its outline.
Integrator
Scene
Object
representation
Light
Material
Camera
parameter
Light
parameter
Figure 1.1 — An overview of a photo-realistic rendering pipeline. The main components, highlighted in orange, are a rendering algorithm, and the scene description.
The photo-realistic renderer is the rendering algorithm consisting of Camera and
Integrator modules. Each scene contains object representations, camera parameters
and light parameters. A rendered image at top-right is then reproduced using the
renderer.
1.1
Towards the Virtual World
There are countless benefits of using computer-generated images. It can help
companies save production costs before delivering products to their customers. For
example, automotive manufacturers can develop virtual prototypes for seamless
testing of cars. Movies and advertisements can be produced creatively and uniquely
without the need for the real-world locations. Furniture companies can design their
own catalogs virtually to save costs and efforts in setting up real photo-sets and
environments. Moreover, the advancement of artificial intelligence and autonomous
systems require an abundance of training data. Synthetic data generation [7] is,
thus, a key to develop the next generation autonomous systems. These examples
1.1 • Towards the Virtual World 3
indicate the importance of using tools for generating realistic images and illustrates
some of the motivations as to why we see such growth in tools based on visualization
and image synthesis across industries.
Generating photo-realistic images is a complex and very computationally demanding
problem. Broadly, it requires accurate and detailed representations of all visual
aspects of the objects and light sources making up the virtual scene, as well as
complex rendering algorithms accurately simulating the physics of light and matter
interactions. Indeed, both components have been researched for decades to enhance
the achievable realism and to accelerate the computations.
To simulate the visual aspects of a virtual world realistically, we seek to solve the
light transport problem. Most photo-realistic rendering systems are based on the ray
tracing algorithm [8]. In ray tracing, we cast sampled rays from a virtual camera
into the scene, tracking light paths and their interactions with the virtual objects.
Ray tracing approaches have been developed since 1980s to solve more and more
complex light transport problems, and more and more physically plausible effects
in virtual environments.
(a) dragon
(b) ball
(c) classroom
Figure 1.2 — Three rendered images are generated using PBRT [9]. Each scene
was rendered with 4096 samples per pixel.
As illustrated in Figure 1.1, the rendering engine, or pipeline, contains different
modules working in concert to solve the light transport problem and to generate
the final image. The scene module contains descriptions of the virtual camera, the
objects, lights, and surface materials in the scene and facilitates the simulation of
scattering, i.e., the interaction between light and materials. The rendering module
generates and gathers ray samples, which are traced from the virtual camera into
the scene. When a ray hits an object the light-object interaction is simulated based
on properties such as object surface normals, material properties, and lighting
information. The rays are then processed in the integrator module which is
responsible for the core radiance calculations. The integrator collects information
from the lights (e.g., color and intensity) and from the material properties (e.g.,
color and reflection) according to the scene descriptions, and utilizes a physicallybased solver to compute the radiance information associated to each ray. Finally,
4 Chapter 1 • Introduction
the renderer generates an image in which each pixel is computed by filtering the
radiance contributions from each sampled ray into the final output.
Figures 1.2(a), (b) and (c) are examples of rendered images generated using a
photo-realistic rendering algorithm. The images from left to right have distinct
environments in which different phenomena are demonstrated. The dragon shown
in Figure 1.2(a) was rendered using an infinite light source placed infinitely far away
from the object. Its appearance is painted by a matte material producing a dull look.
The shiny ball, placed on a checkerboard pattern floor, in Figure 1.2(b), appears
to be placed into a real scene as it was illuminated by radiance captured from a
real-world location. Lastly, a Japanese classroom in Figure 1.2(c) portrays shadow
effects, from both direct and indirect lighting, obtained automatically from the
photo-realistic rendering algorithm. We used PBRT [9] as the renderer to reproduce
images from publicly available PBRT scenes provided by Benedikt Bitterli [10].
PBRT is a photo-realistic rendering engine developed using a physically-based
approach, and path tracing. Path tracing is a fundamental concept employed by
physically-based methods for solving the rendering equation proposed by Kajiya [11].
It can be thought of as an enhanced version of Whitted ray tracing [8]. More
details about the light transport problem and the computational solutions will be
further discussed in Chapter 2.
The scene description is another crucial component where artists can take part in
designing the look and feel of rendered images. Each rendered scene, in Figure 1.2,
has a unique setting for light, objects and camera parameters. Scene illumination
can be controlled by light parameters. Camera parameters characterize image
parameters such as view location, image size and resolution. Each virtual object
requires a geometric description, to represent the object’s shape, and reflection
properties, to represent the object’s color. This thesis primarily focuses on surface
reflection modeling for physically-based rendering. In the next section, we will give
a brief description of virtual object representation.
1.2
Object Representation
The human visual system (HVS ) characterizes real-world objects by surface profiles,
reflectance properties, and environmental light. When a light ray hits an object,
some photons are absorbed by the material surface and some are scattered depending
on the material properties. Once the scattered light reaches human eyes, the
perceived light is translated into chemical and electrical signals supporting the
object recognition tasks in the brain. The intrinsic properties of objects are defined
by geometric shape and surface reflectance. The process of creating digital 3D
contents can be divided into two tasks, i.e., geometric shape modeling and surface
reflectance modeling. Modeling of 3D objects assembles primitive surfaces, so called
1.2 • Object Representation 5
polygons, such as rectangles and triangles, to carve an object into a complex shape.
The latter process is to model surface reflectance in which it describes the portion
of light scattering from a surface.
1.2.1
Geometric Shape Modeling
3D geometric shape modeling, in short 3D modeling, is a process in which artists
build up 3D objects from primitive structures. There exist a variety of ComputerAided Design (CAD) software systems and computer graphics tools supporting
these 3D modeling tasks. A 3D model consists of vertices and edges. A vertex is
usually defined by Cartesian coordinates, (x, y, z). Two vertices are interconnected
by an edge. Vertices and edges can be shared among neighboring primitives.
Typically, triangles are used as primary primitives for 3D modeling. One can also
use implicit functions to represent 3D objects such as spheres, cones and cylinders.
(a) an implicit ball
(b) a bunny
Figure 1.3 — Two examples of 3D models defined by (a) an implicit function and
(b) compound triangles of a bunny shape.
Figure 1.3(a) shows an example of an implicit mathematical function representing
a sphere, rendered in an indoor environmental light. The implicit sphere requires
a radius as an input parameter to construct the virtual sphere in a scene. We
can also construct a complex implicit shape using methods such as arithmetic,
blobby and functional methods [12]. One of the advantages for constructing a
complex shape from multiple implicit surfaces is that it guarantees smoothness and
continuity of the object surface [13]. Even though the implicit surfaces in some
cases are computationally expensive, they are still attractive for applications such
as medical data visualization, deformations and collision detection [14, 15].
6 Chapter 1 • Introduction
Instead of representing a 3D object with implicit surfaces, a polygonal mesh can
be used. A polygonal mesh can be formed by connecting tiny triangles making
surfaces smooth. There are many 3D modeling approaches for creating triangular
meshes. One can, for example, create a complex shape from a ball by subdividing it
into small triangles and sculpt it. This task is called digital sculpting. Another 3D
modeling technique is using image-based techniques. For instance, Figure 1.3(b) is
the Standford Bunny containing about 725, 500 triangles. The bunny’s surface was
reconstructed from range images using zippered polygon meshing method [16]. The
scanner was a Cyberware Model3030 MS producing range points for single polygonal
mesh recovery. There is also a well-known technique, so called photogrammetry,
which recovers 3D surface points of a scene from multiple images. The method
recovers not only 3D points of objects in the scene, but also surface reflectance
and/or textures [17, 18]. This was primarily adopted in applications such as 3D
cultural heritage preservation and architectural scene recovery, and later used in
3D forensics recreation [19]. For an overview and more details on algorithms and
representations of 3D modeling, please refer to the following surveys and review
reports [20, 21, 22].
1.2.2
Surface Reflectance Modeling
The characteristics of real-world materials enable various visual phenomena. In
practice, understanding optical effects gives us insight about material surface
modeling. During rendering, optical effects are simulated using light transport
models such as geometric optics, wave optics and quantum optics [23, 24, 25]. The
most simple type of light transport, explaining a wide range of optical phenomena
such as reflection and refraction, can be modeled by geometric optics. When light
reaches a material surface, optical effects, created by scattered light from a material,
are described by the surface reflectance. Scattering of light of different wavelengths
determines the color of an object as perceived by the human visual system.
In general, materials can be classified by light scattering behaviors. For instance,
diffuse materials scatter light almost equally in all directions. The ideal case is
called Lambertian reflection where light scatters equally in all directions. Some
examples of this material class are a matte paint and an acrylic felt. Another
class of material scattering behavior is shiny materials exhibiting light in a lobe, so
called a specular lobe, or perfect reflection generating a mirror-like effect. Mirrors
and some metals, such as gold, are examples of shiny objects. Regarding various
types of materials exhibiting a wide range of phenomenon, we need efficient ways
to model surface reflectance covering a large spectrum of scattering effects.
There is a large number of surface reflectance models representing a range of
material classes; from simple diffuse materials to complex layered materials. In
fact, the models can be broadly classified into three major classes which are
1.2 • Object Representation 7
empirical models, analytical models and data-driven models. In early stages of
development in computer graphics, empirical models were proposed to represent
surface reflectance based on empirical observations. The Lambertian reflection
is an essential component in many empirical reflectance models as it accounts
for approximating diffuse reflection. Phong reflection model [26] is a classical
example of an empirical reflectance model which combines ambient color, a diffuse
reflection and a specular reflection. The model computes a specular term using the
relation between the surface normal, the perfect reflection vector and the viewing
vector. The improved version of the Phong model is Blinn-Phong model [27]
which replaces the perfect reflection vector with the half vector, computed from
the normalized sum of light-view directions. Even though most empirical models
are computationally inexpensive, one major drawback is that they often lack
physical interpretations. Most empirical models lack physical accuracy, thus some
parameters are not intuitive and difficult to configure.
Owing to the limitations of the empirical models, the adoption of physically plausible
theories has been, thus, employed in the analytical modeling. A classical example
of analytical models is Cook-Torrance model [28], which accounts for rough surfaces
describing a broad range of materials such as metals and plastics. Microfacet
theory is the foundation of this class, which includes several terms describing
different microscopic effects, i.e., directional reflected distribution term, shadowing
term, and the Fresnel effect. Considering the physically-based formulation in
analytical models, most of them support fundamental properties of the surface
reflectance, i.e., reciprocity and energy conservation. Analytical models are often
limited when it comes to describing measured real-world materials, and are often
more computationally expensive as they tend to be more complex.
Using measured data of real-world materials has been proven to be a great choice
which enhances realism in 3D digital content. For decades, reflectance acquisition
systems have been developed to obtain real-world reflectance data. The acquired
measured data is normally high-dimensional, hence, for dense sampling, it is a
time-consuming task which requires high memory footprints. CuRet [29], Cornell
reflectance data [30], MERL [31], UTIA [32, 33], Bonn [34] and RGL-EPFL [35]
databases are publicly provided measured reflectance data sets. These databases
offer an extensive range of measured materials in different formats and resolutions.
Indeed, we can fine-tune parameters of an analytical model, or an empirical model,
using the measured data and optimization algorithms. Since the introduction
of measured reflectance data, the parameter fitting approach has, then, been a
typical form of evaluation method for reflectance modeling research. Therefore,
the use of data-driven models to represent the measured reflectance data becomes
important. In this thesis, we focus primarily on data-driven modeling. Moreover,
we will discuss more details about surface reflectance models later in Chapter 2
8 Chapter 1 • Introduction
and Chapter 4.
As mentioned, data acquisition has become an important element for increasing
realism in photo-realistic rendering. Key factors considered when developing a
reflectance acquisition system are measurement speed and data resolution. If it
takes hours or days to measure a single material, it is not practical for generating
databases with a large collection of materials. Moreover, if the measured data is
not dense enough, its representation will be less accurate for recreating realistic
3D digital content. One of the contributions in this thesis is to bridge the gap
between the speed and data density. The developed algorithms can speed up the
measurement process and reconstruct high resolution reflectance data faithfully. In
the next section, the advancement of surface reflectance acquisition systems will be
briefly discussed.
1.3
Surface Reflectance Measurement
The digital transformation trend has sparked increasing interest in material appearance acquisition. To accurately simulate the real world digitally, specialized
acquisition systems are needed to capture the material appearance for rendering
pipelines. Three main components are found in all acquisition systems, i.e., a light
source, a sensor and a material sample. The configuration of these components
varies depending on measurement formats. As shown below in Figure 1, the overall
acquisition process involves data capture, processing, and storage, culminating in
its use for rendering.
Figure 1.4 — A pipeline illustrating the processes from material reflectance
measurement to rendering.
In the past decades, a number of acquisition systems have been proposed [36, 37].
The design choices of the acquisition systems have to consider factors such as type
of measured materials, sample shapes, speed of measurement, setup cost, ease
1.4 • Objectives and Contributions 9
of use, etc. We can classify the acquisition systems broadly into four types, i.e.,
gonioreflectometers, image-based acquisition systems, catadioptric systems, and
texture-based acquisition systems. Each of these acquisition systems has different
pros and cons. The gonioreflectometer, for example, measures a single angle at a
time making it slow for capturing the data but it is reliable and accurate. The
image-based acquisition and catadioptric systems perform faster measurements
but the data is often less reliable. Furthermore, measuring a texture - a type of
material with varying properties across its surface (heterogeneous) - presents a
particularly difficult problem. Accurately capturing this spatial information requires
a sophisticated light setup. This added complexity only serves to underscore the
importance of carefully designing and implementing material appearance acquisition
systems. Regarding the mentioned factors, there is no ultimate solution for the
acquisition system, but it is rather dependent on the application for which it is
intended to be used. In Chapter 2, publicly available material databases and their
setups will be explained in details.
1.4
Objectives and Contributions
The research objectives of this thesis are to leverage measured reflectance data by
developing data-driven reflectance models and develop minimal sampling strategies
enabling accurate reconstruction of measured reflectance data. In particular, we
focus on isotropic materials whose surfaces are homogeneous. Isotropic materials are
invariant to rotation around surface normal. As a result, the isotropic reflectance
data can be defined by three parameters. In addition, we also address the following
problems.
• Applications for editing and interpolating surface reflectance between materials from either measured data or data-driven reflectance models.
• Design of sparse measurement schemes for single-layered isotropic materials.
• A reflectance model capable of a variance reduction procedure reducing the
noise of final rendered images.
• A model selection method providing insight of the model quality without
visual evaluations.
Contributions of the research areas, addressed in this thesis, led by the author are
presented below.
10 Chapter 1 • Introduction
Data-Driven Reflectance Modeling
This thesis presents new data-driven reflectance models which utilize publicly
available databases, especially the MERL database. The first model is based on
empirical observations that single-layered isotropic materials can be factorized into
three factors (Paper B, C ). This observation was found in Projected-DeviationVector parameterization (PDV ), defining how the reflectance data is parameterized.
Another proposed model employs a machine learning technique, called dictionary
learning (Paper D). This approach learns multiple spaces in which reflectance data
can be accurately described, and selects the best candidate model from the learned
spaces. Our technical contributions to this area are as follows:
• Non-parametric models capable of editing and interpolating between materials.
(Paper A, C, D)
• A study of different parameterizations, on hemisphere and projected plane,
including Half-Diff parameterization [38], Half-Out parameterization [39, 40]
and PDV parameterization. (Paper C )
• A variance reduction technique for the factorized model based on the PDV
parameterization. (Paper C )
• A model selection, without the need for visual comparisons, based on a
transformation of approximated reflectance error. (Paper D)
• Mathematical formulations for material interpolations on multiple dictionary
spaces. (Paper D)
Sparse Sampling for Optimal Reflectance Reconstruction
Sampling strategies for optimal reconstruction of reflectance data are presented
in this thesis. We propose sparse sampling methods based on our factorized
model and the compressed sensing technique. The sampling based on the factored
model was derived from an empirical observation that the reflectance data can
be approximated as the linear combinations between two individual components
(Paper B). The characteristics of the PDV parameterization facilitates the design
of factor measurement. The radially circular BRDFs in the PDV enable to model
2D BRDFs, hence, resulting in acquiring BRDFs by moving mechanical arms, a
light source and a sensor, only on a plane. As a result, our factor measurement
can reconstruct single-layered materials accurately, especially shiny and glossy
materials. Additionally, the measured factors can be used for an importance
sampling technique for variance reduction in rendering. Another contribution is the
new formulation of finding a sensing matrix for material capture using compressed
sensing. We formulate a new sampling framework in the form of coefficient recovery
1.5 • Thesis Outline 11
using Multiple Measurement Vector (MMV ) algorithms (Paper E). The formulation
was derived from a dictionary learning perspective. While the state-of-the-art [41]
employs a heuristic-based optimization seeking for sample locations, our sparse
sampling formulation can be solved with deterministic methods that support
theoretical guarantees. This results in faster optimization, robust optimal sampling
locations, and better signal recovery.
1.5
Thesis Outline
This thesis is divided into two major parts. The first part introduces preliminary
concepts, background and modeling of surface reflectance data and efficient acquisition approaches. The second part is the collection of publications providing details
of the research conducted by the author. The selected publications are consolidated
into contents of this thesis.
The first part contains five chapters. Chapter 2 gives overviews of the fundamental
theories for solving the rendering equation as well as more details about surface
reflectance data and acquisition setups. This chapter is suitable for audiences who
are not familiar with rendering theories and the surface reflectance. Additionally,
background and necessary elements of the surface reflectance and publicly available
material databases are provided in the chapter. Chapter 3 is another preliminary
part for tensor decomposition and sparse representation. The fundamental concepts
and theories of tensor decomposition, used in Paper B and Paper C, and sparse
representation, used in Paper D and Paper E, are provided. Then, we describe
the contributions developed during the work on this thesis. When the surface
reflectance data is available, from observations and/or acquisition, modeling comes
into play as the representation method, e.g., for estimating unknown data points
or encoding dense data. Chapter 4 then provides the background of the reflectance
modeling and our data-driven reflectance models (Paper C and Paper D). We
discuss the possibilities of factorized forms of the surface reflectance (Paper B
and Paper C ) which also leads to an efficient sparse sampling for specific types
of materials such as metals or glossy materials. Moreover, a dictionary-learning
method (Paper D) is presented. Additionally, some useful tools (Paper A, C, D)
are presented for working with the surface reflectance data. Chapter 5 introduces
powerful tools for surface reflectance acquisition to efficiently recover a BRDF
from sparsely sampled measurements (Paper B, C, E). We introduce a factor
measurement based on PDV parameterization, and a novel method for the sparse
measurements of reflectance solved by an MMV algorithm. Finally, we conclude
this thesis in Chapter 6.
Chapter
2
Fundamentals of Rendering and
Material Apperance
Throughout this thesis, we review problems and challenges in the area of the surface
reflectance modeling and acquisition for photo-realistic image synthesis. Similar
problems have long been studied as theoretical concepts, practical algorithms,
and computational problems in mathematics, physics, and computer science. In
this chapter, we give an overview of fundamental concepts in physics of light,
radiometric quantities, and the light transport equation (LTE). These areas of
knowledge pave the way for understanding the problems related to material capture
and modeling in more details.
The intersection between physics of light and computer graphics algorithms has led
to the invention of a rendering technology, called Global illumination, which aims
to simulate detailed scattering phenomena. Light can be emitted, scattered, transmitted and absorbed at surfaces. Based on wave optics, light can also bend through
different mediums through light refraction. We distinguish between different global
illumination algorithms, e.g., Path tracing and Radiosity, by the mathematical
frameworks used in their formulations. In this thesis, we focus on path tracing since
it is a general rendering framework and currently the primary technology used for
rendering using Graphics Processing Units (GPUs). An advantage of path tracing
is that it simulates geometric optics in a way such that it enables the simulation of
scattering effects adequate for a wide range of illumination configurations. Thus,
wave optics models and quantum mechanics, contributing to more advanced natural
light phenomena, are not included.
14 Chapter 2 • Fundamentals of Rendering and Material Apperance
Surface reflectance is a key component in realistic image synthesis. Modeling
and representation of light-material interactions are important tools to generate
enticing virtual scenes. Indeed, each material scatters light differently. Categorizing
materials facilitates the complication of material appearance. Material types also
enable an effective way of handling the variety of appearance complexity. Therefore,
understanding material classes, data representation and acquisition systems is an
elementary step to construct the bigger picture of the addressed problems in this
thesis.
This chapter will be divided into two main parts. The first part presents the
fundamentals of light simulation through the rendering equation. The rendering
equation can currently be described by formulating the light transport problem
with radiometric quantities. The fundamentals of light transport are the foundation
of the rendering engine used in the entire thesis. The second part provides the
basic concepts of surface reflectance named Bidirectional Reflectance Distribution
Function (BRDF ). The physically-based rendering employs the BRDF as a key
element for rendering realistic materials in virtual scenes. We also give a brief
overview of material databases and acquisition systems to connect the theories to
real-world use cases.
2.1
2.1.1
Fundamentals of Light Transport
Light Transport Problem
Light is essential for our perception to see and understand the world around us.
When light reaches the eyes, humans see a scene and observe light phenomena.
Light allows us to perceive the colors, shapes, and details of the objects and scenes
in our environment. To create a realistic virtual world, we need to simulate the way
that light behaves and interacts with objects. That means light transport problems
must be solved for radiance accumulation in a scene. This process, known as light
transport simulation, helps us to generate images that look natural. Indeed, one
challenge of the light transport model is to efficiently solve indirect illumination.
This phenomena is inherent to any multiple-scattered light energy in a scene. By
solving this problem, we can create more convincing virtual environments. Thus,
robust light transport algorithms could help simulate indirect lighting correctly.
One necessary condition of light transport algorithms is to find the equilibrium
point of light energy deposit or the steady-state of light at a given time. Here, we
simulate light transport problem using geometric optics. This means that light
travels along straight paths and scatters according to interactions with material
surfaces in the scene. For convenience, participating media, which account for
volume scattering, e.g., fog, are ignored. However, the light transport problem
can be simply extended to include effects of participating media by adding volume
2.1 • Fundamentals of Light Transport 15
scattering and light absorption. The rendering equation is a light transport model
and can be simplified as the following equation
Lo (p, ω) = Le (p, ω) + Lr (p, ω),
(2.1)
where Lo (p, ω) is the radiance leaving point p in direction ω, Le (p, ω) is the emitted
radiance leaving point p in direction ω, and Lr (p, ω) is the reflected radiance leaving
point p in direction ω.
The simplified rendering equation above is at the heart of light transport algorithms.
To further clarify the above radiance terms in Equation (2.1), the basic terminology
must be defined to measure light quantity. We introduce radiometry as a tool for
quantifying optical radiation [42].
2.1.1.1
Radiometry
A photon, emitted from illumination sources, carries energy depending on its
wavelength. To measure the amount of photon energy, radiant energy Qλ with a
wavelength λ is defined by
Qλ =
hc
,
λ
(2.2)
where h ≈ 6.63×10−34 joules·seconds is Planck’s constant, and c = 299, 792, 458 m/s
is the speed of light. As the following quantities are wavelength dependent, we will
omit λ to facilitate the notations below.
In rendering, we assume light under the steady state. Radiant flux Φ defines
the differential amount of energy per differential time and its units are joules per
second (J/s) or watts (W ). The radiant flux is expressed as
Φ=
dQ
.
dt
(2.3)
The quantities measuring density of energy over an area are defined by irradiance
E, flux arriving at a surface, and radiant exitance M , flux leaving a surface.
Their units are W · m−2 . Given an area A at a point p we can define irradiance
and radiant exitance in a similar way as follows
E(p) =
dΦ(p)
.
dA
(2.4)
Note that, although the difference between E and M is the directional power, they
lack a unit expressing its direction. The directional distribution of light can be
quantified by the solid angle representing the angular size and direction of a ray.
Indeed, the solid angle measures the size subtended by an observed object from a
given point. Solid angles are measured in steradians (sr) indicated by a directional
16 Chapter 2 • Fundamentals of Rendering and Material Apperance
variable ω. A hemisphere subtends a solid angle of 2π steradians. Now, we can
define the directional quantities which are intensity I and radiance L. Intensity
is defined as
I=
dΦ
,
dω
(2.5)
which has units W · sr−1 and describes the directional power of light centered at
the unit sphere. Here, the final and most important radiometric quantity, radiance,
is defined as
L(p, ω) =
dEω (p)
,
dω
(2.6)
where Eω (p) denotes irradiance perpendicular to the direction ω at surface point p.
Radiance is defined by the flux density over directional area as follows
L=
d2 Φ
.
dω dA cos(θ)
(2.7)
Radiance has units W · m−2 · sr−1 . The cosine term compensates for the projected
area perpendicular to direction ω. The radiance term here is the most used
radiometric quantity in deriving a light transport framework. Indeed, radiance is
constant and invariant along a ray in vacuum. This property is important enabling
the emerging of ray tracing algorithms which assume invariance of light energy
along a ray. The definition of radiance, then, lets us focus on rays, or light paths,
carrying light energy throughout a given scene.
Figure 2.1 — A geometric illustration of radiometric quantities employed to
describe measured radiance.
A visual explanation of the radiometric quantities introduced above is illustrated in
Figure 2.1. When light is observed at point p with respect to direction ω, radiance
measures the energy Φ over infinitesimal area perpendicular to direction ω which is
2.1 • Fundamentals of Light Transport 17
dA cos(θ). For convenience, we provide a summary table of the above radiometric
quantities in Table 2.1.
Radiometric quantity
Definition
Unit
Radiant energy, Q
h c
λ
Joule (J)
Radiant flux, Φ
dQ
dt
J · s−1 or Watt (W )
Irradiance, E
dΦ
dA
W · m−2
Intensity, I
dΦ
dω
W · sr−1
2
d Φ
dω dA cos(θ)
Radiance, L
W · m−2 · sr−1
Table 2.1 — Summary of radiometric quantities and units of measurement.
2.1.1.2
Path Integral Equation
Equation (2.1) can be expanded as
Lo (p, ωo ) = Le (p, ω) +
Z
ρ(p, ωo , ωi ) Li (p, ωi ) | cos(θi )
{z
} |
{z
Ω|
ref lectance
irradiance
| dωi .
(2.8)
}
Intuitively, the reflected radiance Lr can be thought of as an integral of a surface
reflectance function (BRDF) denoted by ρ(p, ωo , ωi ), and receiving flux at point p
with respect to all possible incoming angles Ω. The BRDF describes how much
light is reflected from the incident direction ωi into the outgoing direction ωo . We
leave the mathematical expressions and properties of BRDF to Section 2.2.1. Here,
we continue on deriving the solution of the integral in Equation (2.8) which is
typically impossible to solve analytically.
The reflected radiance term accounts for simulating both direct and indirect
illuminations. The direct illumination can be found explicitly when irradiance is
contributed directly from light sources in the scene. However, to simulate indirect
lighting, irradiance contributed to the integral will be found through the light
reflected from objects in the scene, which may have undergone multiple scattering
events. Such complexity requires reformulation of Equation (2.8) into the path
integral formulation so that we express the problem on a path space to avoid the
curse of recursiveness. Using the path space makes it more convenient in exploiting
any random path sampling technique as a rendering algorithm.
Let us reformulate Equation (2.8) into the three-point form of area integral. First,
we define the radiance and the BRDF from a point p1 to p0 as follows
18 Chapter 2 • Fundamentals of Rendering and Material Apperance
L(p1 → p0 ) = L(p1 , ω),
(2.9)
ρ(p2 → p1 → p0 ) = ρ(p1 , ωo , ωi ),
(2.10)
where ωo is the direction from point p1 → p0 and ωi is the direction from point
p1 → p2 .
We introduce a geometric term G(x) describing visibility V (x) and surface orientation between two points as follows
G(p0 → p1 ) = V (p0 → p1 )
| cosθ0 | | cosθ1 |
∥p0 − p1 ∥2
.
(2.11)
By substituting the above terms into Equation (2.8), we obtain the area integral
as follows
L(p1 → p0 ) = Le (p1 → p0 ) +
Z
A
ρ(p2 → p1 → p0 )L(p0 → p1 )G(p0 → p1 )dA(p2 ),
(2.12)
where A is all surfaces of the scene.
Figure 2.2 — An illustration of a four-vertex path starting from point p0 where a
sensor, or a camera casts a ray. A sub path contains two vertices, the geometric
term G and the BRDF ρ. Here, the path ends at p3 where a light source is placed
and emits energy.
Now, we repeatedly substitute the integral on the right hand side to obtain a sum
over path integrals. The integrals of the light path will appear continuously so that
the length of the path keeps increasing. An example of four-vertex path is defined
below
L(p1 → p0 ) =Le (p1 → p0 )+
Z
Le (p1 → p0 )ρ(p2 → p1 → p0 )G(p0 → p1 )dA(p2 )+
ZA Z
A A
Le (p3 → p2 )ρ(p3 → p2 → p1 )G(p3 → p2 )
× ρ(p3 → p2 → p1 )G(p2 → p1 )dA(p3 )dA(p2 ).
(2.13)
2.1 • Fundamentals of Light Transport 19
An illustration of the path integral in Equation (2.13) is shown in Figure 2.2. The
points p0 and p1 are predetermined depending on the camera position, however,
the remaining points are randomly chosen over all points in the scene. In fact, a
light path is generated by limitless subpaths. Thus, the integral form is expanded
infinitely. Fortunately, the path integral equation looks similar to a Fredholm
integral equation of the second kind [43]. Such equation can be solved using the
Neumann series expansion. We can then rewrite the path integral equation above
into an infinite sum by reorganizing the BRDF and geometric terms.
L(p1 → p0 ) =
∞
X
PL (p¯n ),
(2.14)
Le (pn → pn−1 )T (p¯n )dA(p2 ) . . . dA(pn ),
(2.15)
n=1
where
PL (p¯n ) =
T (p¯n ) =
Z
···
A
n−1
Y
Z
A
ρ(pi+1 → pi → pi−1 )G(pi+1 → pi ).
(2.16)
i=1
Equation (2.14) expresses the path integral as a sum of radiance contributions over
path p¯n bouncing around in the scene. We also define an operator, T , for handling
visibility and scattering along a path. Finally, PL defines integrals over the chosen
path. These mathematical expressions used in deriving Equation (2.14) construct
a venue for solving the sum of the path by using a Monte Carlo estimator. The
chosen path can be randomly sampled to compute radiance estimates contributing
to point p0 . We can write Equation (2.14) in a more concise way as follows
L(p1 → p0 ) = PL1 (p¯1 ) + PL2 (p¯2 ) +
∞
X
PL (p¯i ),
(2.17)
i=3
where PL1 (p¯1 ) is the contribution from a surface emission at point p1 , PL2 (p¯2 ) is
the direct lighting contribution from a light source.
One interpretation of Equation (2.17) is that we determine each term by the path
length. For example, PL1 (p¯1 ) is a path between the camera and a surface point
in the scene. PL2 (p¯2 ) is a path of length 2 where only one bounce occurs. This
term is usually estimated by sampling all light sources in the scene. PL3 (p¯3 ) is a
path of length 3 where two bounces have contributed and so on. The length of
the path can be predefined with the maximum number of bounces, otherwise we
can estimate by using Russian roulette to terminate the low contribution of the
subpath.
Solving Equation (2.17) requires the estimation of the sum of the integrals from
the last term, where multiple bounces are inherent to indirect illuminations. The
estimator used in evaluating such integral is the Monte Carlo technique. Note
20 Chapter 2 • Fundamentals of Rendering and Material Apperance
that, in practice, we sample the path based on a probability function and we
evaluate radiance contribution from light source to the camera with the Monte
Carlo estimator.
2.1.2
Light Transport Algorithm
In this section, we give details about a light transport algorithm to solve Equation (2.8), or the generalized path integral equation in Equation (2.17). Kajiya [11]
proposed a path tracing algorithm which employs a random path sampling technique to estimate the contributed radiance using a Monte Carlo estimator. The
path tracing suggests only one reflected ray to be generated as a subpath, similar to
an example path illustrated in Figure 2.2, instead of computing multiple reflected
rays at a surface point. The result is computed as the average over the samples
created over the area of each pixel. However, a problem of the Monte Carlo path
tracing is the variance of estimates which appears as noise in the rendered image.
When a sufficient number of rays are traced in the scene, the noise will be reduced.
In a complex environment, even when the rays are traced densely, e.g., 10, 000
samples per pixel, the noise may still be visible. One solution to cope with variance
reduction problem is, thus, reviewed at the end of this section.
2.1.2.1
Monte Carlo Integration
Monte Carlo integration is a point-sampling technique for solving integral equations
by estimating an integrand g(x) from random samples. Suppose that we want to
evaluate a 1D integral as follows
G=
Z b
g(x)dx,
(2.18)
a
where G is the exact solution of the integral in the domain [a, b].
We can estimate G using a sequence of random variables Xi ∼ p(x) by
GM =
M
1 X
g(Xi )
,
M i=1 p(Xi )
(2.19)
where each Xi is drawn from the probability distribution p(x) and M is the number
of samples.
Due to the fact that the expected value of an arbitrary function g(x) is defined as
Ep [g(x)] =
Z
g(x)p(x)dx,
(2.20)
D
where D is the domain of interest. The expected value of GM is the desired integral
of g(x) as shown in Equation (2.18). This confirms the convergence of Monte
Carlo algorithms. One major drawback as mentioned previously is the standard
2.1 • Fundamentals of Light Transport 21
deviation, or variance, of the Monte Carlo estimator, that is inversely proportional
to square root of the number of samples, i.e., σ ∝ √1M . This means that to reduce
the error by half, the number of samples must be quadrupled. To remedy this, we
will introduce a variance reduction technique called importance sampling in the
next subsection.
2.1.2.2
Importance sampling
The idea of using importance sampling is rather straightforward. Suppose that
p(x) = cg(x) where c is a constant. The normalization results in
1
c= R
.
(2.21)
g(x)dx
This indicates that if we could sample from the p(x), the estimate would be
Z
g(Xi )
g(Xi )
=
= g(x)d(x).
p(Xi ) cg(Xi )
(2.22)
The expressions above demonstrate that if samples Xi are drawn proportional to
the integrand g(x), the variance is zero when the integrand is known, because
the estimate in Equation (2.22) is the same constant. However, in rendering,
the integrand is unknown but the probability distribution can be constructed to
approximate the true distribution hence the variance can be decreased. In fact,
one simple way to choose the probability distribution is to construct it from the
BRDF. The reflected radiance in the rendering equation consists of BRDF and
irradiance which identify the shape of the integrand.
For data-driven BRDFs, Lawrance et al. [39] showed that the probability density
function (PDF ) for importance sampling can be constructed by an inversion method
using such BRDFs. In Paper C, we derive an importance sampling strategy based
on our data-driven BRDF model. We adopted the inversion method, see [9], when
two independently uniform samples are drawn to determine the samples based on
an arbitrary PDF.
2.1.2.3
Transforming between PDFs
When working with different parameterizations, the corresponding PDF must be
transformed accordingly. Up until now, the PDF has primarily been defined in
spherical coordinates. However, in many cases, reparameterization reveals many
interesting patterns underlying BRDFs and their distributions. For example, our
BRDF models (Paper B, C, D) have been reparemeterized differently from the
standard spherical coordinates which defines incoming and outgoing directions
(ωi , ωo ). To draw sample according to the corresponding PDFs, it requires a
Jacobian transformation.
22 Chapter 2 • Fundamentals of Rendering and Material Apperance
Let Y = T (X) be a transformation between two distributions, Y and X, where T is
a transformation function. The Jacobian is defined as
 ∂T
1
 ∂x1
 ∂T2

 ∂x1
| JT | = det 
 .
 ..


∂Tn
∂x1
∂T1
∂x2
∂T2
∂x2
..
.
∂Tn
∂x2
...
...
..
.
...
∂T1 
∂xn 
∂T2 

∂xn 
,
.. 
. 


∂Tn
∂xn
(2.23)

The Jacobian is the absolute value of the determinant of the Jacobian matrix of T .
One important property of the transformation function T is bijection which means
that the mapping between distributions is 1-1 (injective) and onto (surjective).
2.1.2.4
The cosine-weighted hemisphere sampling
A common solution when a desired PDF is difficult to draw a sample from is to
perform uniform sampling on hemispherical directions. Malley method [9] uniformly
chooses directions on a disc where polar coordinates are defined by (r, ϕ). The
√
method randomly chooses ( ξ1 , 2πξ2 ) = (r, ϕ) and projects the point on the unit
hemisphere. The resulting directions are cosine-weighted and the corresponding
pdf is
p(θ, ϕ) = | JT | p(r, ϕ) =
cos(θ)
.
π
(2.24)
In summary, the Monte Carlo estimation for solving the light transport problem
requires many tools involving statistical sampling strategies. In typical situations,
the cosine-weighted sampling ensures that the solution is obtained but with a
slow rate of convergence. To speed up the convergence, the importance sampling
strategically chooses paths according to the integrand consisting of both BRDF and
irradiance terms. In case of the specially designed sampling strategy, the inversion
method and the Jacobian term are employed to make sure that the solution is
correctly estimated.
2.2
Surface Reflection
To quantify the surface reflection, reflectance is used to express the amount of
light incident at a surface point to the amount reflected out from the point. This
property is important because it allows us to visually identify and categorize
materials based on how they reflect light. In realistic image synthesis, reflectance
is essential for solving the light transport equation as mentioned in Section 2.1. In
general, hemispherical reflectance is defined as follows
2.2 • Surface Reflection 23
ρΩ =
Φr
,
Φi
(2.25)
where ρΩ is the reflectance over hemisphere, Φr and Φi are reflected and incident
radiant flux, respectively.
Clearly, the reflected energy must not exceed the incident energy at the surface,
hence, ρΩ ∈ [0, 1]. The reflection is an event occurring when light hits a surface but
there are other events such as transmission and subsurface scattering which occur
depending on material properties.
Light transmission can be found in transparent objects such as glasses and lenses.
The transmission occurs when light passes through materials at one point and exits
at the other side of the material. This let us see objects sitting behind transparent
materials. Another type of light behavior is subsurface scattering, which occurs
when light penetrates a material and scatters inside before exiting at a different
point. This is often seen in materials like human skin or jade, where the light
creates a soft and glowing effect. These combinations of light transport mechanisms
can create more complex phenomena in the scene. Each of light events has a
specific function describing the directional and spatial quantities leading to more
accurate and detailed representations of complex surfaces.
2.2.1
The Reflectance Functions
Defining the reflection quantity on a hemisphere as shown in Equation (2.25)
ignores the directional light energy. In rendering, we are interested in directional
quantities in order to solve the rendering equation. The Bidirectional Reflectance
Distribution Function (BRDF) [44] expresses the directionally scattering quantity
for materials. The BRDF is expressed by
ρ(ωo , ωi ) =
dLo (ωo )
dLo (ωo )
=
.
dE(ωi )
Li (ωi )cos(θi )dωi
(2.26)
BRDF is typically a 4D function parameterized by (θo , ϕo , θi , ϕi ) = (ωo , ωi ). Each
material has a unique BRDF describing the visual appearance of an object. For
example, a metal scatters light strongly in some directions where we see highlights
or near mirror reflections while a matte plastic reflects light almost equally in all
directions so its appearance is invariant to viewing angles.
2.2.1.1
Physically Plausible Properties
There are two important BRDF properties to consider in physically plausible
rendering:
24 Chapter 2 • Fundamentals of Rendering and Material Apperance
1. Reciprocity which states that given any pair of directions (ωo , ωi ), the BRDF
is reversible and invariant to exchanging the directions as expressed by
ρ(ωo , ωi ) = ρ(ωi , ωo ).
2. Energy conservation states that the reflected energy should not exceed the
total received energy as denoted by
Z
Ω
ρ(ωo , ωi )cos(θi )dωi ≤ 1.0.
The reciprocity property refers to Helmholtz reciprocity [45] and it is useful in
many situations. For example, when doing measurements or representing BRDFs,
only half of the hemisphere can be measured or stored because the remaining part
can be filled in by the reciprocity property. Another example is in the design
choice of BRDF parameterizations for importance sampling. The path tracing
provides a camera direction for the radiance estimation. The reciprocity enables
us to select the light directions as an auxiliary parameter. We will provide more
details in Chapter 4, where we utilize this property in the design of our BRDF
parameterization.
(a) BRDF
(b) BTDF
(c) BSSRDF
Figure 2.3 — Illustration of different light scattering phenomena. (a) BRDF
scatters light above the surface. (b) BTDF refracts light passing through the surface.
(c) BSSRDF scatters light inside and light leaves at other points of the surface.
BRDFs can be classified into two subcategories which are isotropic and anisotropic
BRDFs. Isotropic BRDFs are invariant to azimuthal rotation of materials. This
means that given an incoming light direction and a viewing direction, rotating the
material around its normal will result in no change for the reflected radiance. On the
other hand, anisotropic BRDFs give different reflected radiance when the rotation
occurs. Anisotropic materials can be found in wood, fabric and polished metals
which often have fibers or threads that cause directionally-dependent reflectance,i.e.,
anisotropy. Nevertheless, most of the BRDFs are isotropic, e.g., glass, plastics and
some metals. More importantly, isotropic BRDFs can be parameterized by only 3D
2.2 • Surface Reflection 25
parameters as ρ(θo , ϕo , θi , ϕi ) = ρ(θo , θi , | ϕo − ϕi |). This is a property we utilize in
this thesis.
The BRDF is used to describe radiance scattering above surfaces. For materials
which have light transmission and subsurface scattering, there are specific functions,
serving such material properties, called Bidirectional Transmission Distribution
Function (BTDF ) and Bidirectional Scattering Surface Reflectance Distribution
Function (BSSRDF ), respectively. Illustrations of each function are shown in
Figure 2.3. Although the BTDF and the BSSRDF are not in the scope of this
thesis, they are worth mentioning as these three functions can cover a wide range
of material properties.
The BTDF does not obey the reciprocity due to Snell’s law, which states that the
index of refraction η between two mediums is proportional to the sine of the angles
as expressed below
sin(θ1 ) η2
= .
sin(θ2 ) η1
(2.27)
Hence, the relationship between incoming radiance and transmitted radiance is
dependent on the index of refraction as follows
Lo (ωo ) = eT r Li (ωi )
ηo2
,
ηi2
(2.28)
where eT r is the transmitted fraction of light, ηo and ηi are indices of refraction of
outgoing and incoming directions, respectively.
The BTDF is a 4D function similar to the BRDF but the transmitted directions are
on the opposite side of the surface. In some context, the BRDF and the BTDF are
blended and represented in a general term as Bidirectional Scattering Distribution
Function (BSDF ). This general function allows us to consider both reflection and
transmission in a single mathematical framework. When using the BSDF, the
domain of integration is expanded to include the entire unit sphere, rather than
just the hemisphere around the surface normal.
The BSSRDF is a more generalized form of material properties where spatial and
directional domains are considered. The function is parameterized by incident
location and direction of light and exitant location and direction of scattered light.
The BSSRDF is a highly-complex 8D function, ρSS (xi , yi , θi , ϕi , xo , yo , θo , ϕo ), which
poses a lot of challenging problems in research. We often find the BSSRDF in
translucent materials. For modeling and acquisition of BSSRDFs, refer [25, 46, 47,
48, 49, 50, 51].
26 Chapter 2 • Fundamentals of Rendering and Material Apperance
Figure 2.4 — A taxonomy of reflectance functions defined by complexity of
parameters, the bar on the right.
2.2.1.2
Hierarchical chart of reflectance functions
In summary, we can classify the reflectance functions using the number of parameters as depicted in Figure 2.4. The right bar in the figure tells the number of
parameters used in each reflectance function, ranging from 3D to 8D. Apparently,
the simplest reflectance function is the isotropic BRDF, which is parameterized
by three parameters, and the most complex reflectance function is the BSSRDF
which has two additional spatial locations. The BSDF is a general form of the 4D
reflectance function which describes both reflection and transmission. Indeed, there
are two important functions which are BTF and SVBRDF where their complexity
is inherent from the BRDF. These functions represent spatial reflection properties
where each point is represented by a BRDF. The difference between the BTF and
the SVBRDF is that SVBRDF surfaces do not have occlusions between each point
while the macro structure of BTF [52] surfaces is non-flat, resulting in non-local
scattering effects such as shadowing and interreflection on each surface point. For
more details about the BTF, please refer to the following review reports [52, 53].
2.3 • Parameterization 27
2.3
Parameterization
Parameterization is a mathematical tool used in expressing data with a coordinate
system. When data is aligned on different coordinate systems, its representation
and modeling can also be changed. The benefit of defining data on different
parameterizations is that the data can be interpreted more easily and the patterns
in the data can be revealed. For example, representing a 2D circle is preferred on
the polar coordinate system while drawing a straight line is easier on the Cartesian
coordinate system. This also applies to the surface reflectance data.
In this section, we describe parameterizations used to represent BRDFs. Each
parameterization has unique properties and structure, which can be used for
different purposes. So one of the main goals for BRDF parameterization is to
efficiently model and represent BRDFs, as well as improving the process of BRDF
acquisition [35].
(a) Standard Parameterization
(b) Half-Diff Parameterization
(c) Half-Out Parameterization
(d) Projected Deviation Vector (PDV)
Figure 2.5 — BRDF parameterizations illustrated on directional vectors on
hemisphere. Normal vectors denoted by N are perpendicular to the plane. Each
vector represents by spherical coordinates using θ and ϕ denoting elevation and
azimuthal angles, respectively.
28 Chapter 2 • Fundamentals of Rendering and Material Apperance
2.3.1
Standard parameterization
Typically, a BRDF is defined by two vectors, the light and the view vectors, on
a unit hemisphere. These vectors are commonly defined in spherical coordinates
(r, θ, ϕ) where r = 1, θ ∈ [0, 90) and ϕ ∈ [0, 360). A pair of light and viewing directions
is denoted by (ωi , ωo ) which is (θi , ϕi , θo , ϕo ). The elevation angle θ is measured
from the normal vector N of the hemisphere. The azimuthal angle is measured on
the surface plane.
We use the standard parameterization in a number of formulations since it is
important in deriving the rendering equation, and intuitive for the design of
BRDF measurement devices. However, a number of disadvantages of using this
parameterization are, for examples, inefficiency of BRDF modeling and BRDF
representation. Kautz and McCool [54] proposed a BRDF decomposition indicating
that the standard parameterization is not a suitable form for a separable BRDF
model. Figure 2.5(a), depicts directions on a hemisphere using the standard
parameterization.
2.3.2
Half-Diff parameterization
In 1998, Rusinkiewicz proposed the Half-Diff parameterization [38]. It is constructed
by changing (ωi , ωo ) to become a half vector ωh and a difference vector ωd . As the
+ωo
name suggests, the half vector, denoted by (θh , ϕh ), is formulated by ωh = ∥ωωi +ω
.
i
o∥
Once the half vector is derived, the surface plane is rotated such that ωh is the
normal of the new coordinate frame. The difference vector is thus obtained by
rotating either ωi or ωo with the same rotation as the half vector. As shown in
Figure 2.5(b), the half vector is perpendicular to the plane defined by the difference
vector. Note that θh is measured relative to the normal vector N while θd is
measured relative to the half vector.
Parameterizing the directions in this way assists in explaining the specular lobe
with fewer number of parameters. Moreover, many properties have been observed,
e.g., the separable BRDFs [3, 38, 54, 55] both in isotropic and anisotropic BRDFs,
reciprocity and retroreflective lobes. Another practical example is the MERL [31]
BRDF database which is widely utilized by the computer graphics community
for data-driven modeling. The BRDF database provided is stored in a modified
Half-Diff parameterization which encodes the specular lobe densely. We will cover
this database in Section 2.5.1. For isotropic BRDFs, this parameterization uses
three parameters (θh , θd , ϕd ).
2.3 • Parameterization 29
2.3.3
Half-Out parameterization
The half vector is widely used in describing specular lobes as shown in many BRDF
models [27, 28, 56, 57]. Moroever, the relationship between the half vector and
the outgoing direction, which indicate a similar contour on the hemisphere, was
presented by Ngan et al. [58]. Reparameterizing with the half vector as a principal
direction has given improved results for BRDF modeling. The other auxiliary
direction is the viewing direction ωo as illustrated in Figure 2.5 (c).
Lawrence et al. [39] proposed the use of Half-Out parameterization for a separable
BRDF. One important feature for this parameterization is that the noise reduction
technique, called importance sampling, can be defined more efficiently as compared
to the Half-Diff parameterization. The Jacobian term and the sampling strategy are
easily derived in connection to the viewing direction. It facilitates the integration
to Monte Carlo ray tracing algorithms. Separable BRDF models [39, 40] utilizing
this parameterization have shown improvements in both compact representation
and fast rendering. Isotropic BRDFs using this parameterization require three
parameters (θo , θh , ϕh ).
2.3.4
Projected Deviation Vector (PDV) parameterization
Parameterization is not limited to be defined on a hemisphere. Reprojection
hemispherical directions on a unit circle plane is an alternative method. The PDV
parameterization begins with the perfect reflection direction ωR from a viewing
direction ωo as a principal vector parameter. The auxiliary parameters describing
ωi is defined by a projected deviation vector Dp and its projected azimuthal angle,
denoted by ϕdp , as depicted in Figure 2.5.
Originally, it was defined by Neumann et al. [59] as a mathematical tool. They
reformulated the directions with orthogonal projections and there were multiple
scenarios for deriving BRDF models. The derived BRDF models covered a variety
of material types such as plastics and metals. The importance sampling strategy
of the BRDF models was also proposed. It was later studied that the measured
BRDF isocontours have a similar shape as the deviation vector Dp . As a result,
ABC BRDF models [60] were empirically observed and developed, inspired by the
Rayleigh-Rice theory.
Our contribution to this parameterization is to introduce a separable capability
for the PDV parameterization found in isotropic BRDFs (Paper B, C ). Our
proposed BRDF models offer separable BRDF representations and sparse BRDF
measurement, which are effective for glossy materials. We also derived a Jacobian
term corresponding directly to the PDV parameterization and our data-driven
BRDF model. Note that the ABC smooth surface model makes use of the polar
coordinates to enable the importance sampling strategy. Our sparse acquisition
30 Chapter 2 • Fundamentals of Rendering and Material Apperance
can be done on a gonioreflectometer with two degrees-of-freedom. Moreover, we
also demonstrate an application of BRDF editing with our separable BRDF model.
In Chapter 4, we will explain in detail our BRDF models and sparse measurements.
We define this parameterization for isotropic BRDFs by (θR , dp , ϕdp ).
2.3.5
Other parameterizations
Redefining BRDF parameters is a way to achieve more efficient BRDF models
enabling separability and lower BRDF dimensionality. McCool et al. [61] proposed
a BRDF decomposition technique on texture-mapping coordinates for a real-time
rendering application. Edwards et al. [62] created a mathematical framework for
the half vector. The framework enforces energy conservation to ensure physically
plausible results. Stark et al. [63] utilized the barycentric coordinates to fit BRDF
models on triangular coordinates. They also explain the geometric interpretations
of BRDF reflections (diffuse, specular, grazing and retro-reflection) over Barycentric
parameterizations. A hybrid parameterization, between the PDV and the Barycentric parameterization, was proposed by Barla et al. [64] showing an improved
distortion appearing on the orthographic projection. Similar to the Barycentric
parameterization, Mallett and Yuksel [65] presented a mathematical framework for
defining 2D parameterization supporting both uniform and nonuniform quantization. An example of the nonuniform quantization is Bezier parameterization derived
from their framework. Using frequency domain as BRDF parameterizations is a
classical approach. Spherical harmonics are simple spherical BRDF representations
on the frequency domain as proposed by a number of works [66, 67, 68, 69]. Fourier
approaches [70, 71] can be used to represent BRDFs. Note that a tabular form of
coefficients is required for storing the encoded BRDFs in the frequency domain.
The examples of parameterizations discussed in this section reveal insights of BRDF
behaviors in many aspects such as reflection characteristics on the half vector,
separability, and irregular quantizations. However, one common key idea found in
many parameterizations is to efficiently represent the specular lobes, as these regions
have strong contributions in rendering. Moreover, when looking at importance
sampling, the specular lobes dominate the corresponding PDFs matched to the
BRDFs. This means that the samples tend to fall on the specular regions. Thus,
the choice of parameterization is an essential part in both modeling and rendering.
2.4
Data Transformation
The utilization of measured BRDFs for modeling has increased due to the development of BRDF acquisition systems. The acquired data has shown that the BRDF
data has a high dynamic range meaning that there is a large range between highest
and lowest values. This creates a difficulty when using BRDF models to obtain
2.4 • Data Transformation 31
parameters by fitting parameters to measured BRDFs. Thus, transformation of
the linear BRDFs are desirable for BRDF modeling. Indeed, the choice of the
transformations is unclear as it has been discussed previously [4, 58, 60]. In this
section, we will briefly explain transformation functions used when working with
BRDFs.
2.4.1
Log-based Transformation
The human visual system perceives light intensity propotional to a logarithmic
function. The nonlinearity of the HVS was characterized by Weber’s law and
Fechner’s scaling [72]. The large dynamic range which can be observed by the HVS
is about 14 log10 units [73]. One simple operator in dealing with the high dynamic
range data is to use logarithm functions as they have been adopted in the High
Dynamic Range Imaging (HDRI) resaerch.
Reference
Sparse measurement
Absolute error
Figure 2.6 — The rendered image (middle) shows gold-metallic-paint3 from
MERL which was reconstructed using our sparse BRDF model from Paper B. The
reference is placed on the left and the normalized absolute error image is on the
right with the range [0, 1].
Due to the high dynamic range of BRDFs, a number of works has observed the
advantages of using the logarithmic functions, e.g., specular lobe interpolations [55],
factorization [61] and BRDF fitting metrics [2, 3, 4, 40, 58, 60, 74]. Since the
logarithmic function behaves poorly when it is near 0, addition of an ϵ < 0.001
or ϵ = 1.0 is preferable to avoid near 0 values. The following functions are two
common logarithmic functions used for this purpose:
ρt1 = log(ρ + ϵ),
(2.29)
ρt2 = log(ρ · cosM ap + ϵ),
(2.30)
32 Chapter 2 • Fundamentals of Rendering and Material Apperance
where ρ is a BRDF value, ρt1 and ρt2 are transformed BRDF values, ϵ is a constant
to avoid near 0 value, and cosM ap is a cosine mapping function. The cosM ap
function can be either cos(θi ) or max(cos(θi ) · cos(θo ), ϵ).
(a) The logarithmic fitting
(b) The linear fitting
red-metallic-paint acrylic-felt-green
Figure 2.7 — BRDF fitting with different error metrics. The logarithmic fitting
metric (a) and the linear fitting metric (b) were performed on the measured alumbronze material from MERL database with the Cook-Torrance BRDF model. The
y-axis indicates BRDF values and the x-axis is scattering angles. The incident
angles are θi = 0◦ and 67◦ . The plots are from Löw et al. [60].
Reference
ρt1
Error (ρt1 )
ρt2
Error (ρt2 )
Figure 2.8 — The rendered images in second and forth columns are reconstructed
BRDFs from ρt1 and ρt2 , respectively. The reference is placed on the left and the corresponding normalized absolute error images are on the right of each reconstruction.
The error scale is [0, 1] shown on the right most column.
In Paper B, C, we employed Equation (2.29) as our mapping function for BRDF
factorization and acquisition, as it performs well with glossy materials or highlyspecular reflections. In Figures 2.6, we present an example of our sparse BRDF
measurement from Paper B. The data transformation plays a key role in aligning
2.4 • Data Transformation 33
the measured BRDFs so that the aligned data facilitates factorization and brings
up the sparse measurement configuration with only two degrees-of-freedom.
Löw et al. [60] demonstrated that the fitting metric in Equation (2.30) can improve
the fitting curve of wide-angle scattering compared to a cosine-weighted function,
using cos(θi ). Figure 2.7 (a) and (b) show the results of using a logarithmic error
metric and a linear error metric. As can be seen, the linear fitting focuses on both
specular lobes but it neglects the wide-angle scattering. Indeed, the shape of BRDF
curves, performing both high specular lobe and wider-angle scattering, appears in
glossy materials. Thus, the BRDF fitting procedure with logarithmic error metrics
serves favorably on glossy materials.
Another example of the logarithmic transformations is presented in Paper D. We
observed that the two transformations, (Equation (2.29) and Equation (2.30)),
have produced efficient reconstructions depending on material classes. We see
that Equation (2.29) performs superior on high-specular reflection and some glossy
materials while Equation (2.30) performs superior on diffuse materials. Figure 2.8
shows two examples of reconstructed BRDFs. The diffuse material on the top row
is acrylic-felt-green, and the glossy material on the bottom row is red-metallic-paint.
Obviously, this reveals an observation from Ngan et al. [58], showing that different
material classes require different fitting error metrics. The cosM ap is, thus, one
important factor for the diffuse BRDFs.
2.4.2
Gamma Transformation
The gamma transformations are often found in imaging techniques [75, 76], e.g.,
CRT display, image quantization, camera response functions, and HDR tone
mapping. One purpose of such functions is to apply nonlinearity to linear data.
This non-linear behavior has a connection found in the HVS, which captures a
certain range of high dynamic range data. The function can be named differently
such as gamma correction, gamma encoding, and gamma compression. The general
form of the gamma transformation is the power-law function as follows
ρt3 = cg ρ1/γ ,
(2.31)
where cg is either a constant or a weight function, and γ ∈ R.
Bieron and Peers [77] proposed the gamma transformation for the BRDF fitting
problem, demonstrating some improvements over the log-based transformation.
Using the power function, controlled by γ variations, has an advantage when there
exists no single optimal solution. The multi-class BRDF’s characteristics thus
enable the strong argument in utilizing adaptive γ values for different BRDF types.
In the study, the range of γ values is fixed at [1, 3]. Low γ values, i.e., close to
γ = 1, are almost similar to the linear mapping, while higher γ values suppress the
34 Chapter 2 • Fundamentals of Rendering and Material Apperance
specular lobe components. This implies that highly specular materials could get
better fits from low γ values. On the other hand, materials exhibiting glossy and
more diffuse characteristics are able to perform well with higher γ values.
Our contribution to the gamma transformation is introduced in Paper D. When
having multiple BRDF parameter sets as mentioned in previous works [58, 77],
one requires rendering of images for selecting the best solution. We thus proposed
a selection method based on the gamma function. Our method operates on the
transformed BRDF domain and can perform suitable selections without rendering
images.
Developing a perceptual error metric for appearance modeling is still an open
problem. In fact, there is still no clear solution for the BRDF fitting procedure
as shown in previous works [77, 78]. The recent perceptual metric study [79]
indicates that the log-based transformations have a small dominance over other
BRDF metrics. Indeed, there are other factors which have impact on the perception
of appearance such as object’s shapes and environment [80]. These studies have
revealed that a BRDF perceptual metric is a complex issue which has not been
solved yet.
2.5
Material Databases and Acquisition Systems
Up to this point, we have provided the theoretical concepts of the material appearance from the physics level to mathematical tools in modeling. In this section, we
give overviews of the publicly available BRDF databases used in BRDF research.
We begin with MERL [31] and RGL-EPFL [35] databases. Both BRDF databases
were used in our work for developing and evaluating the efficiency of our data-driven
BRDF models. The latter part of this section provides details of some additional
databases.
2.5.1
MERL
Matusik et al. [31] proposed an image-based acquisition inspired by the setup
presented by Marschner et al. [81, 82]. The proposed image-based acquisition
method requires a camera, a light source and a spherical material sample as
illustrated below, Figure 2.9(a). Both the camera and the sample are stationary
while the light moves during the capturing process. It takes about 3 hours to
captures 330 HDR images. The acquired BRDFs were then stored in the Half-Diff
parameterization, (θh , θd , ϕd ), with non-linear encoding of the specular lobes. A
measured BRDF in the MERL database has 90 × 90 × 180 = 1, 458, 000 samples
and its storage cost is approximately 33 MB per material.
2.5 • Material Databases and Acquisition Systems 35
The MERL database is the first database of densely sampled BRDFs containing
100 isotropic materials. It contains diverse material classes from diffuse to highly
specular BRDFs. The material samples used in the capturing process are shown
below, Figure 2.9(b). The availability of this database and the BRDF fitting
procedure have accelerated the development of BRDF models and standardized
the evaluation process using real measurements.
(a) The image-based acquisition setup
(b) Material samples
Figure 2.9 — The BRDF acquisition setup (a) proposed by Marschner et al. [81]
was adopted to acquire 100 isotropic BRDFs [31], shown in (b).
2.5.2
RGL-EPFL
Dupuy and Jakob [35] found a relationship between BRDF parameterization and
importance sampling. Assuming that materials follow the microfacet theory, the
microfacet normal distribution (NDF) can be extracted by the retro-reflective
measurements. The extracted NDF is then used to adaptively sample BRDFs
according to retro-reflection behaviors. The device used in the capturing process is
a gonio-photometer with a special lighting setup for retro-reflective measurements.
The measurement time takes about 2-3 hours and the memory footprint is roughly
600 KB for a single material. The RGL-EPFL database provides, at the time of
writing, 62 materials which are 51 isotropic and 11 anisotropic materials. There
exists, in this database, some layered materials which exhibit multi-lobe scattering.
36 Chapter 2 • Fundamentals of Rendering and Material Apperance
2.5.3
Other databases
There are some publicly available databases, however they are limited in terms
of the number of BRDFs or their corresponding resolution. Cornell [30] and
CuRET [29] are the earliest BRDF databases provided for computer graphics
community. CuRET contains 61 materials with 205 sample points. The reflectance
data from Cornell [30] were measured using a custom-built gonioreflectometer.
The database contains 2 categories, i.e., paint material and human skin, however,
they are stored in different formats. Another BRDF database which has many
anisotropic BRDFs is from UTIA. They provided 150 coarsely sampled BRDFs [83]
and several dense anisotropic measurements [33, 84, 85]. A learning-based BRDF
sampling proposed by Nielsen et al. [41] provided 8 reconstructed BRDFs using
20 samples. A learning-based method for sparse BRDF acquisition is proposed
in Paper E, where we show that our deterministic method, entitled FROST, can
produce more robust and accurate results when compared to the state-of-the-art
method.
2.5.4
The catadioptric measurement
The above BRDF databases were acquired using either image-based acquisition or
a gonioreflectometer. There is, however, another type of acquisition setup which
utilizes a catadioptric measurement. The examples are the acquisition devices
proposed by Ward [56], Ghosh et al. [86], and Mukaigawa et al. [87]. They exploited
the parabolic mirrors to reflect light so that less moving parts are needed. This
capturing process was designed to speed up the measurement process at the cost
of accuracy and angular resolution.
Chapter
3
Tensor Decompositions and Sparse
Representation
The foundation of this thesis is built upon tensor decompositions and the sparse
representation. Since we are addressing the BRDF modeling and the sparse BRDF
sampling problems, the two mathematical frameworks, i.e., tensor operation and
sparse representation, are utilized to reformulate and solve the problems.
In this chapter, we present an overview of the tensor and its decomposition methods used in Paper B and Paper C. The tensor decomposition assumes that the
structural data in a high-dimensional tensor, for instance, a 3D matrix, can be
decomposed/factorized into a more compact representation, e.g., three vectorized
forms. The factorization permits the high-dimensional data to be encoded into
lower-dimensional data. Moreover, some fundamental concepts of the tensor decompositions are then incorporated in the latter part to develop richer representation
models.
We provide some preliminaries of the sparse representation where it defines a
mathematical structure to tackle sparse signal estimation problems. The sparse
representation allows us to search for a basis function by learning it from a
collection of signals. The learning process is merely an optimization problem of
the underdetermined system of equations. This sparse representation framework is
used to solve the problems in Paper D and Paper E.
This chapter is organized as follows. In Section 3.1, we provide an overview of tensor
operations which introduces how high-dimensional data is stored and processed.
38 Chapter 3 • Tensor Decompositions and Sparse Representation
We then present tensor decomposition methods which will appear in Chapter 4
and Chapter 5. Lastly, the sparse representation and dictionary learning will be
introduced in Section 3.2.
3.1
Tensor Decompositions
As we are moving forward to our main contributions of this thesis, here we
provide preliminaries of tensor and its operations. In this section, some relevant
tensor operations and methods are introduced. Only partial and necessary tensor
fundamentals are reviewed and will further guide to methods used in later chapters.
For more comprehensive reviews of tensor linear algebra, see the literature [88, 89,
90].
3.1.1
Tensor Operations
The term tensor is defined differently across disciplines [91, 92], but for simplicity,
we define it as a multi-dimensional array. To determine the dimensionality of a
tensor, there are several formal naming conventions, e.g., mode, way, order, which
denotes the number of dimensions. Let X ∈ Rm1 ×m2 ×···×mn denote an n-mode
tensor whose entries are real values. A tensor entry is addressed by a set of
indices. An entry of a 3-mode tensor, for instance, is xijk where i = {1, · · · , m1 },
j = {1, · · · , m2 } and k = {1, · · · , m3 }. Each tensor contains fibers representing values
of a fixed mode. For example, matrix rows are mode-1 fibers and matrix columns
are mode-2 fibers. For a 3D matrix, or a 3-mode tensor, x:jk , xi:k , xij: denotes
column, row and tube, respectively.
Here, we also introduce tensor operations that are frequently used in the literature.
The Frobenius norm of an n-mode tensor X is expressed as follows.
v
um m
mn
1 X
2
uX
X
∥X ∥ = u
···
x2i1 ,i2 ,··· ,in ,
t
i1 =1 i2 =1
(3.1)
in =1
where xi1 ,i2 ,··· ,in is an entry of the tensor X .
The inner product of the same-sized tensors X and Y is the sum of the products
of all entries as follows
⟨X , Y⟩ =
m1 X
m2
X
i1 =1 i2 =1
···
mn
X
xi1 ,i2 ,··· ,in · yi1 ,i2 ,··· ,in ,
in =1
where yi1 ,i2 ,··· ,in is an entry of the tensor Y.
(3.2)
3.1 • Tensor Decompositions 39
An n-mode tensor is rank-one if it can be expressed as the outer product of n
vectors X = a(1) ◦ a(2) ◦ · · · ◦ a(n) , where a(i) is ith vector corresponding to the ith
mode of the tensor. This means that each entry of the tensor can be calculated
by the product of the corresponding vector elements. For example, an element
(1) (2) (3)
xijk of a 3-mode rank-one tensor is xijk = ai aj ak . This tensor property has a
great benefit in dimensionality reduction as the tensor can be encoded as the outer
product of a set of vectors. A tensor decomposition algorithm in Section 3.1.2 will
employ this property for tensor factorization.
The N -mode product of a tensor X with a matrix U ∈ RJ×mN is defined as follows
(X ×N U)i1 ,··· ,iN −1 ,j,iN +1 ,··· ,in =
m
N
X
xi1 ,··· ,in uj,iN ,
(3.3)
iN =1
where (X ×N U) is of size m1 × · · · × mN −1 × J × mN +1 × · · · × mn . This tensor
operation is important for a tensor decomposition which produces a core tensor
and multiple basis matrices.
The process of unfolding or reordering the elements of an n-mode tensor into a
matrix is known as matricization, unfolding or flattening. The unfolding
of a tensor X along mode k is denoted X(k) . This operation maps the tensor
into a matrix of size (mk , m1 × · · · × mk−1 × mk+1 · · · × mn ). An element xi1 ,··· ,in is
mapped into a matrix X(k) (ik , j) where
j = 1+
n
X

(i − 1)
p
p=1,p̸=k
p−1
Y

mq  .
(3.4)
q=1,q̸=n
The Kronecker product is a matrix product between A ∈ RP ×Q and B ∈ RM ×N
which is defined as follows

a1,1 B a1,2 B . . . a1,q B


a2,1 B

A⊗B =  .
 .
 .



a2,2 B . . . a2,q B 


,
..
..
.. 

.
.
. 

(3.5)
ap,1 B ap,2 B . . . ap,q B
where the resulting matrix A ⊗ B is of size P Q × M N. Note that this product is
an extension of the element-wise multiplication of the two matrices.
The Khatri-Rao product is the column-wise Kronecker product. The Khatri-Rao
product between A ∈ RP ×Q and B ∈ RM ×Q is denoted by A ⊙ B. The resulting
matrix is of size (P M × Q) and expressed as follows
40 Chapter 3 • Tensor Decompositions and Sparse Representation

a1,1 b1
a1,2 b2
...
a1,q bq



 a2,1 b1

A⊙B = 
..


.

a2,2 b2
..
.
...
..
.
a2,q bq
..
.




.



(3.6)
apm,1 b1 apm,2 b2 . . . apm,q bq
The rank of a 3-mode tensor X , denoted by rank(X ), is the smallest possible
number of rank-one tensors in which an entry of the tensor xijk can be expressed
as
xijk =
R
X
air bjr ckr ,
(3.7)
r=1
where R is the tensor rank, ai ∈ Rm1 , bi ∈ Rm2 , and ci ∈ Rm3 . Equation (3.7) is a
generalized form of the rank-one tensor defined on a 3-mode tensor where R = 1.
This can also be extended to an n-mode tensor.
The tensor operations and properties introduced here will be used in later sections
where we explain tensor decomposition methods.
3.1.2
Tensor Decomposition Methods
3.1.2.1
CP Decomposition (CPD)
The history of the CP decomposition dates back to 1927 and has been discovered
by several authors [93, 94, 95, 96, 97, 98, 99, 100]. There are several naming
variants, e.g., Canonical Polyadic, PARAFAC, CANDECOMP, since it was discovered by various studies. Later in 2000, Henk A. L. Kiers [100] named it
CP(CANDECOMP/PARAFAC) decomposition, CPD for short.
The CPD seeks for solving a minimization problem such that an n-mode tensor
X is estimated from sum of rank-one tensors. The generalized form of the tensor
optimization is expressed as follows
arg min ∥X − X̂ ∥ s.t. X̂ =
X̂
R
X
(2)
(n)
λr a(1)
r ◦ ar ◦ · · · ◦ ar ,
(3.8)
r=1
where λr is its corresponding weight of the rth rank-one component. To illustrate
the summation expression of the minimization problem above, Figure 3.1 shows
an example of a three-mode tensor approximated by the CPD method with R
components. In this way, a three-mode tensor is modeled as a weighted sum of
rank-one tensors. Each of the elements of X̂ can be accessed and computed by
Equation (3.7).
3.1 • Tensor Decompositions 41
Figure 3.1 — The CP decomposition of a three-mode tensor X with R components.
One useful property of CPD is the uniqueness [101, 102], as its conditions are much
weaker and relaxed than, for instance, Tucker Decomposition. The discussions on
the CPD uniqueness can be found in literature [103, 104, 105, 106]. Here, we also
provide an algorithm to solve Equation (3.8) using an Alternating Least Squares
(ALS) approach, see Algorithm 1.
1
Input: X , R
Result: return λ, A(1) , · · · , A(n)
Initialize A(1) , · · · , A(n)
T
2
3
4
5
6
7
8
9
10
T
Compute Gram matrices G(1) ← A(1) A(1) , · · · , G(n) ← A(n) A(n)
repeat
for i = 1 to n do
S ← G(1) ∗ · · · ∗ G(i−1) ∗ G(i+1) ∗ · · · ∗ G(N )
A(i) ← X(i) (A(N ) ⊙ · · · ⊙ A(i−1) ⊙ A(i+1) ⊙ · · · ⊙ A(1) )S†
Normalize columns of A(i) to store column norms in λ
Update the Gram matrices G(i)
end
until criteria met;
Algorithm 1: An ALS algorithm for computing CPD of an n-mode tensor X
with R components where S† is the pseudo inverse of S and λ is its corresponding
weight.
Note that solving Algorithm 1 Line 6, by default, employs SVD but it can be
replaced by other matrix decompositions, e.g., Cholesky decomposition or QR
decomposition [107]. In Section 4.3.1, CPD is utilized for a BRDF model in which
aligning BRDFs on the PDV parameterization accommodates the use of CPD.
3.1.2.2
Higher-Order SVD (HOSVD)
Higher-Order Singular Value Decomposition (HOSVD) is another class of tensor
decompositions, where matrix SVD is utilized. It was originally introduced by
42 Chapter 3 • Tensor Decompositions and Sparse Representation
Tucker [108, 109] and has appeared in several other pubcliations under different
names [110, 111, 112, 113]. The method decomposes a tensor into a core tensor G
and matrices along each mode. The formulation is expressed as follows
X = G ×1 U(1) ×2 U(2) ×3 · · · ×n U(n) ,
(3.9)
where {U(i) }ni=1 are orthogonal matrices. Note that the size of the core tensor G
is equal to the second mode (column) of each {U(i) }ni=1 . The above expression was
defined in Equation (3.3) but here the orthogonal (typically orthonormal) matrices
are considered. Truncated HOSVD, like truncated SVD of a matrix, can also be
computed; however, the resulting factorization is not necessarily optimal [88].
In Section 4.3.2, we will introduce a variant of HOSVD where we construct an
ensemble of HOSVD to represent multiple BRDF classes. Unlike HOSVD, the
orthogonal matrices U(i) are learned from a given set of BRDFs. Our proposed
method also adopts the concept of sparse representation, presented in a later section,
to encode a BRDF in the form of a linear combination of orthogonal matrices U(i)
and sparse coefficients G.
3.2
3.2.1
Sparse Signal Representation
Sparse Representation
The concept of sparse representation has emerged in the field of signal processing
and adopted by a variety of fields, e.g., image processing, audio, and medical
imaging. It was adopted to visual data by a number of publications [114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124]. Sparse representation, also known as sparse
coding or sparse approximation, formulates the signal recovery problems using
a linear combination of basis functions and coefficients. Generally, an observed
1D signal, for instance, x ∈ Rm can be approximately decomposed into two basic
elements, i.e., a set of bases D ∈ Rm×n and coefficients s ∈ Rn . This is analogous
to solving a linear system of equations x = Ds. The matrix D is referred to as an
overcomplete dictionary, m < n, whose columns are called atoms. The goal of sparse
representation is to find the sparsest coefficient s satisfying the underdetermined
system of linear equations, where there are infinitely many solutions under the ℓ2
norm. This allows any high-dimensional signal to be encoded with a small number
of coefficients. Considering the sparse signal recovery, the signal x is τ sparse if it
can be estimated with the following formulation.
arg min ∥x − Ds∥22 s.t. ∥s∥0 ≤ τ,
s
(3.10)
where ∥s∥0 counts the non-zero elements of the coefficient vector s. The sparsity τ
3.2 • Sparse Signal Representation 43
is strictly small, τ ≪ n, compared to the size of atoms, since we wish the signal x
to be compressible. The above constraint can be rewritten in another form using
an estimated noise or error parameter ϵ as follows
arg min ∥s∥0 s.t. ∥x − Ds∥22 ≤ ϵ.
s
(3.11)
Consequently, solving the equations above gives us a set of sparse coefficients.
This implies that the observed signal x can be reconstructed with fewer non-zero
coefficients. Given the observed signal x and the dictionary D, however, the above
problems are non-convex and NP-hard due to the ℓ0 norm. An
exhaustive search of
the possible solutions is required to solve a combinatorial nτ least square problem.
To address this problem, the condition can be relaxed by replacing the ℓ0 norm
with an ℓ1 norm. This leads to solving a convex problem with Linear Programming
(LP) algorithms. These theoretical works [125, 126, 127, 128, 129, 130] were the
pioneer works which conceptually proved the LP solutions of the above equations.
The geometric interpretations of ℓp norm can be found in [131], demonstrating the
relationship between sparsity and ℓp norm minimization. Hence, one goal of sparse
representation is to seek algorithms to solve the combinatorial and non-convex
optimization with relaxed conditions.
There exist two major categories of sparse recovery algorithms which rely on the
norm used for the regularizer. The first group of algorithms solves Equation (3.11)
with Greedy methods. The popular methods in this category are, for example,
matching pursuit (MP) [132] and orthogonal matching pursuit (OMP) [133, 134].
The greedy algorithms iteratively search for the best atom (support) from the
dictionary for each iteration based on heuristic conditions, e.g., approximated residuals. There are several MP and OMP variants proposed to improve performance,
e.g., Regularized OMP [135, 136] and Compressive Sampling OMP [137]. Here,
we refer to [138, 139, 140, 141, 142, 143, 144, 145] for more extensions. Another
category of sparse approximation solvers exploits convex relaxation [146] with
ℓ1 -norm regularization. The convergence guarantee of ℓ1 minimization makes this
group of algorithms strong but they suffer from the high-computational complexity
compared to the greedy algorithms. The examples of this group are, for instance,
basis pursuit (BP) [147], Gradient Projection Sparse Reconstruction (GPSR) [148],
interior-point based methods [149, 150], alternating direction method (ADM) [151],
proximity-based algorithms [152, 153, 154, 155] and SpaRSA [156]. One can classify the algorithms into more groups, however, here we only mention two major
categories. We direct the reader to literature [131, 157, 158] for a comprehensive
review of sparse recovery algorithms.
Obtaining a sparse signal by solving Equation (3.11) poses a number of challenges.
Firstly, one challenge is the dictionary design for sparse representation. Typically, a
sparse signal recovery algorithm is applied when a dictionary D is given. However,
44 Chapter 3 • Tensor Decompositions and Sparse Representation
in most cases, the dictionary is not given and the choice of the dictionary is
critical to the reconstruction quality. One classical choice for the dictionary
is the pre-specified dictionary based on the transform methods, e.g., the fast
Fourier transform (FFT) [159], Discrete Cosine Transform (DCT) [160], wavelet
transforms [161, 162, 163], curvelets transform [164], bandelets [165], and contourlet
transform [166]. Another promising way to address this is to consider it as a machine
learning task where a dictionary is learned from a training set.
In the next section, we will provide an overview of dictionary learning. One of
the most important applications of sparse representation is efficient sampling and
reconstruction of signals. This opens up the research direction on compressed
sensing in which one would like to sample a compressible signal. In Chapter 5, we
will introduce the concept of compressed sensing utilized in our work. We also refer
the reader to a reference book [167] on compressed sensing.
3.2.2
Dictionary Learning
A motivation of dictionary learning is to find a representative set of bases D by
learning it from collected training data. Several types of dictionary learning methods
have been proposed, e.g., online dictionary learning [168, 169], discriminative
dictionary learning [170, 171] and joint dictionary learning [172, 173]. One can also
categorize the methods into supervised learning and unsupervised learning [131, 174].
Generally speaking, dictionary learning utilizes a method for solving an equation
similar to Equation (3.11) with the ability to update or search for a representative
dictionary D with respect to the sparsity, see Equation (3.12) below.
The common approach for learning the dictionary D is to iteratively update atoms
until certain conditions are met. One of the most commonly used dictionary
learning methods is K-SVD [175]. The algorithm can be seen as a generalized Kmeans clustering algorithm classified as an unsupervised learning method. K-SVD
seeks to solve the following objective function
arg min ∥X − DS∥22 s.t. ∥Si ∥0 ≤ τ, ∀i ∈ 1, . . . , N ,
(3.12)
D,S
where X ∈ Rm×p is a training data matrix whose columns are p known signals,
D ∈ Rm×k is the learned dictionary, and S ∈ Rk×p is the sparse coefficient matrix.
The user-defined parameter τ is the sparsity parameter. When k > m, the learned
dictionary is called overcomplete.
The joint optimization problem, Equation (3.12), can be solved in an iterative
manner by freezing either parameter D or S at a time and updating the other.
K-SVD employs a sparse recovery algorithm, OMP, when the dictionary D is fixed.
When S is fixed, it becomes a regression problem which can be solved using SVD
3.2 • Sparse Signal Representation 45
decomposition. This iterative two-stage approach is commonly used in dictionary
learning algorithms. Notice that the number of atoms k influences K-SVD in the
way that if k is large, it leads to a more representative dictionary but at a high
computational cost.
The unsupervised dictionary learning has been widely adopted in applications
of visual data reconstruction, e.g., image denoising [114, 176, 177] and image
compression [178]. For applications with labeled training data, supervised dictionary
learning algorithms [170, 171, 179, 180, 181, 182] can address the problem.
The dictionary learning algorithms, in the early stage, did not account for data
coherence along different dimensions. The multi-dimensional visual data sets,
e.g., light field data, BRDF and BTF, have shown to have correlations along
different dimensions. Moreover, vectorizing multi-dimensional data into a 1D
signal does not capture the underlying coherent structure and it may significantly
increase the training time. To address this problem, the tensor-based dictionary
learning algorithms [183, 184, 185, 186, 187] are considered as an alternative
approach to this type of training data. Additional algorithms in this category are
CPD-based dictionary learning [188, 189, 190, 191] and Tucker-based dictionary
learning [192, 193, 194, 195] where the algorithms utilize similar formulations of
Equation (3.8) or Equation (3.9) based on multi-dimensional representations with
sparsity constraints.
Therefore, it is reasonable to utilize the tensor-based dictionary learning method
for handling BRDF modeling. We will show the performance of our proposed
BRDF models in Chapter 4 where we employ a tensor decomposition method and
a dictionary learning method to represent high-dimensional reflectance data.
Chapter
4
Surface Reflectance Modeling and its
applications
As discussed in previous chapters, the surface reflectance is a key for the pipeline
of generating realistic images. Solving the light simulation accounts for the degree
of realism obtained from the rendering algorithm and the light-material interaction.
For each material, light interaction is distinctly determined by a special class of
reflectance functions such as the BRDF, the BTDF and the BSSRDF. Indeed,
one research problem that this thesis addresses is to efficiently encode such high
dimensional data obtained from the measurements. We focus on the homogeneous
surface as the spatial location parameters are omitted. More importantly, we
consider the use of data-driven approaches to tackle the problem. The emerging of
the MERL database [31] gave rise not only to evaluation of BRDF models, but
also to leveraging of the measured data for data-driven BRDF models.
4.1
Outline and Contributions
In this chapter, we introduce the background knowledge for BRDF modeling,
see Section 4.2. The modeling approach has been classified into three major
categories. We review each one individually starting from empirical BRDFs to
data-driven BRDFs. In Section 4.3, we present our data-driven BRDF models as
the contributions of the thesis (Paper C and Paper D). Our non-parametric BRDF
models have no assumptions about the underlying physical model, and the number
of model parameters are not fixed. Parametric models, on the other hand, have a
48 Chapter 4 • Surface Reflectance Modeling and its applications
strong assumption about the model structure, and the number of parameters are
predetermined [196]. Our factorized BRDF model, Paper C, is a non-parametric
model whose number of coefficients corresponds to data resolution. The unique
characteristics of the PDV parameterization is adopted to align BRDFs to achieve a
low-rank factorization. In Paper D, a learning-based method is employed with some
additional dictionary properties. The resulting BRDF model is a non-parametric
model whose number of coefficients is flexible according to the required accuracy
and storage. Lastly, in Section 4.4, we present some useful BRDF applications
derived from Paper C and Paper D, as well as a BRDF editing method presented
in Paper A.
4.2
4.2.1
BRDF Models
The Empirical BRDF Model
In the early stage of 3D computer graphics, empirical models were used to approximately imitate specular highlights on rough surfaces. This group of BRDF
models are empirically developed from the observations with the mathematical
frameworks supported. The earliest models of this kind are Phong [26] and BlinnPhong [27] BRDF models which employ the cosine angle between directions. The
Phong model demonstrates that the specular highlight can be modeled by a pair
of the viewing direction, V , and the perfect reflection direction, R, while the
Blinn-Phong model utilizes a pair of the normal vector, N , and the half vector, H.
The energy-conservation constraint was later introduced by Lewis [197], where the
model limits the diffuse and specular coefficients to 1.0. The following equations
are the Phong BRDF model, the Blinn-Phong BRDF model and the Lewis BRDF
model, respectively.
ρs (ωi , ωo ) = ks (R · V )n ,
(4.1)
ρs (ωi , ωo ) = ks (N · H)n ,
(4.2)
ρ(ωi , ωo ) = kd + ks ρs (ωi , ωo ),
(4.3)
where n is the exponent controlling the specular highlight, kd and ks are diffuse
and specular coefficients, respectively, with the constraint that kd + ks = 1.0, and
(ωi , ωo ) is a pair of incoming and outgoing directions.
These models are widely used in simple shading models due to inexpensive computation. Later, another modified Phong BRDF model [198] was derived so that it
follows the physically plausible rules and can be used in a path tracing renderer
with importance sampling. The equation is expressed as follows
4.2 • BRDF Models 49
1
n+2
(R · V )n .
+ ks
π
2π
| {z }
{z
}
|
ρ(ωi , ωo ) = kd
dif f use
(4.4)
specular
Equation (4.4) has both the diffuse and specular components where the diffuse
component is described by the Lambertian model and the specular component
is described by the Phong model. The term n+2
2π was derived to meet the energy
conservation. It is common that many BRDF models separate the diffuse and
specular components because the Lambertian reflectance can efficiently approximate
the diffuse component and it is nearly a constant.
One of the well-known BRDF models was proposed by Gregory J. Ward [56]. It
was designed to represent both isotropic and anisotropic BRDFs by using Gaussian
distributions. Moreover, the proposed models also support the Monte Carlo
sampling. Several variants [199, 200] were proposed to correct the normalization
term for more physically plausible constraints. The Ward BRDF models for
isotropic and anisotropic materials are, respectively, as follows
exp −tan2
1
1
ρ(ωi , ωo ) = kd + ks q
·
π
4πα2
cos(θi )cos(θo )
2
1
1
ρ(ωi , ωo ) = kd + ks q
·
π
cos(θi )cos(θo )
θh
2
α
(4.5)
,
2
2
exp −tan (θh )
cos θh
2
αx
4παx αy
+
sin θh
2
αy
!!
, (4.6)
where α is the Gaussian parameter for the specular highlight in the isotropic BRDF
model, αx and αy parameters control the specular highlight for the anisotropic
BRDF model, and θi , θo and θh are elevation angles of incoming, outgoing and
halfway directions, respectively. All the α parameters can be thought of as the
standard deviation of the surface slope.
The α parameter in Equation (4.5) handles the size of the circular highlight seen in
isotropic materials which is invariant to the rotation while the αx and αy in Equation (4.6) govern the highlight of anisotropy seen in anisotropic materials. Please
note that due to the grazing angle problem in the anisotropic Ward BRDF model,
a similar anisotropic
BRDF model was proposed by Neumann et al. [201] where
q
the cosine term cos(θi )cos(θo ) was replaced by max(cos(θi ), cos(θo )). Neumann
et al. [201] also proposed modifications of physically plausible BRDF models for
the Phong model and the Blinn-Phong model to represent metallic materials.
The empirical approach has opened a research direction for mathematical BRDF
frameworks. A BRDF framework, which was mathematically derived for general
BRDF models with importance sampling support, was proposed by Neumann
et al. [59]. The resulting models are physically plausible for both isotropic and
50 Chapter 4 • Surface Reflectance Modeling and its applications
anisotropic BRDFs. The key idea of the models is their basic BRDF which is
defined as the distance vector on a projected vector domain.
Another BRDF framework was derived by using the half vector, proposed by
Edwards et al. [62]. This framework shares a similar idea from Neumann et al. [59]
in the way that the projected vector on the unit disk was used to derive the shape
of surface reflectance with a pair of the half vector and the normal vector. The
framework puts the energy conservation as a constraint to ensure that the resulting
empirical BRDF model is physically plausible.
At near-grazing angles, light is scattered from the media largely due to the Fresnel
effect which had not been considered until Ashikhmin and Shirley [57] proposed
their BRDF model. This effect can be modeled using the electro-magnetic wave of
light reflection and transmission. Schlick [202] introduced an approximation to the
Fresnel term, Fr (cos θh ), as follow
Fr (cos θh ) = FR0 + (1 − FR0 )(1 − (cos θh ))5 ,
FR0 =
η1 − η2
η1 + η2
(4.7)
!2
(4.8)
,
where Fr is the Fresnel reflection, FR0 is the surface reflectance at the normal
incidence, and η1 and η2 are indices of refraction (IOR) of the two medium.
The approximation in Equation (4.7) was later used in the Ashikhmin and Shirley’s
model to derive the physically plausible normalization term as shown in Equation (4.10) below. The complete form of the model appears as
ρs = ks
(N · H)n
F (cos θh ),
max(cos(θi ), cos(θo )) r
(4.9)
!5  
!5 
cos(θ
28Rd
)
cos(θ
)
i
o
 1 − 1 −
,
ρd =
(1 − Rs ) 1 − 1 −

23π
ρ = ρs + ρd ,
2
2
(4.10)
(4.11)
where ks is an arbitrary specular coefficient, and Rd and Rs are constants for diffuse
and specular reflections, respectively. Its variant [203] was proposed to support the
Monte Carlo sampling and hardware-friendly implementation.
So far, we have reviewed a number of well-known empirical BRDF models. Note
that the empirical BRDF models are derived mathematically to meet the effects
of light scattering instead of using physical theories to support the assumptions.
A benefit of using the mathematical derivation is the freedom of parameters and
constants in the models. In the next subsection, we will go through the BRDF
models whose assumptions have a more physically-accurate meaning.
4.2 • BRDF Models 51
4.2.2
The Analytical BRDF Model
When material surfaces are modeled by using theoretical physics to predict physical
surface phenomena, we categorize the BRDF model as an analytical model or a
physically-based model. The most favoured physically-based model widely used in
computer graphics is the microfacet-based BRDF model. The microfacet theory
suggests that the rough surfaces be modeled as a collection of microfacets defining
the light distributions at any given point. The Torrance-Sparrow model [204] and
Cook-Torrance model [28], as shown in Equation (4.12) appear to be the early
physically-based BRDF models derived by using the microfacet theory, defined as
ρs (ωi , ωo ) =
D(ωh )G(ωi , ωo )Fr (ωo )
,
4 cos(ωi )cos(ωo )
(4.12)
!2
tan(α)
1
,
4 exp − m
m cos α
!
2 (N · H)(N · V ) 2 (N · H)(N · L)
G(ωi , ωo ) = min 1,
,
,
(V · H)
(V · H)
D(ωh ) =
2
(4.13)
(4.14)
where D is the light distribution function, G is the geometrical shadowing and
masking term, and Fr is the Fresnel term.
The Cook-Torrance BRDF model proposed the use of Beckmann distribution
function as shown in Equation (4.13), where m is the Beckmann parameter and α
is the roughness parameter. The geometrical term, G, denotes the approximation
of micro structure of surfaces which blocks and attenuates light when it enters
the microscopic scale of the surface. The G term of the Cook-Torrance model is
presented in Equation (4.14). The Fresnel term, Fr , accounts for the Fresnel effect
appearing near the grazing angle.
The microfacet-based BRDF model is rather well-known in computer graphics and
rendering for decades. Most of the variants tweak and adjust the distribution term
D and the geometrical term G. For example, the microfacet-based model and the
GGX distribution, in Equation (4.15), proposed by Walter et al. [205] was adopted
to model refraction accounting for the BSDF model (BRDF + BTDF). The SGDbased BRDF model [206] proposed more accurate distribution using Shifted-Gamma
Distribution (SGD), in Equation (4.16), for single-layered materials.
D(ωh ) =
χggx (N · H)α2
π cos4 (θh )(α2 + tan2 (θh ))2
χsgd (θh )
D(ωh ) =
P22 (tan2 (θh )),
π cos4 (θh )
,
(4.15)
(4.16)
where χggx (N · H) is equal to 1 if N · H > 0 and 0 otherwise, χsgd (ωh ) is equal to 1
if ωh < π/2 and 0 otherwise, P22 (x) is a positive function of the variable x ∈ [0, ∞).
52 Chapter 4 • Surface Reflectance Modeling and its applications
Note that the GGX distribution, Equation (4.15), looks similar to the SGD distribution, Equation (4.16), as the P22 (x) is a generalization function for the distribution,
which can be the GGX distribution, the Beckmann distribution or the Gaussian distribution. The SGD-based BRDF model [206] proposed the generalization function
as shown below
p−1
P22 (x) =
α
·
Γ(1 − p, α)
2
exp − α α+x
α2 + x
p
(4.17)
,
where Γ is the incomplete gamma function, and α and p are the distribution slope
and the exponent parameters, respectively.
Holzschuch et al. [207] proposed another variant of the microfacet-based BRDF
model by extending the diffraction term, using an approximation of the CookTorrance diffraction. Some variants [208, 209] have proposed the use of different
normalization terms, the denominator term, to steer the near grazing angle lobe.
The Oren-Nayar BRDF model [210] improves the diffuse surfaces by assuming the
V-shaped cavity as micro structures. The resulting approximation of the model is
represented by the following equation
ρ(ωi , ωo ) =
where A = 1 −
2
kd
(A + B max(0, cos(ϕi − ϕo ))sin(α)tan(β)),
π
m
,
2
2(m +0.33)
B=
2
0.45m
,
2
m +0.09
(4.18)
α = max(θi , θo ), β = min(θi , θo ), and m is
the standard deviation of the microfacet orientation angle in radians.
Inspired by Rayleigh-Rice theory, the ABC BRDF models [60] were derived based
on the observations that the measured BRDF isocontours, projected on the unit
disk, are radially symmetrical. The ABC BRDF models are presented in the
following equations.
kd
+ S( Dp )Fr (ωo ),
π
√
S( 1 − N · H)G(ωi , ωo )Fr (ωo )
k
,
ρ(ωi , ωo ) = d +
π
cos(θi )cos(θo )
ρ(ωi , ωo ) =
where S(f ) is the ABC model with arbitrary parameter f and S(f ) =
(4.19)
(4.20)
A
2 C.
(1+Bf )
The
parameters A, B, C are obtained from the fitting process. The projected distance
vector, Dp , is computed from the projected incoming and outgoing directions as
shown in Figure 2.5.
The proposed smooth surface BRDF model in Equation (4.19) represents glossy
materials and metals quite efficiently as these materials exhibits the symmetrical
4.2 • BRDF Models 53
scattering on the projected unit disk. The microfacet-based ABC model in Equation (4.20), on the other hand, works well on dielectric or plastic materials where
rough surfaces are dominant.
The single-lobe scattering models sometimes cannot accurately represent layered
BRDFs which exhibit multiple scattering from multi-layered materials such as car
paint, pearlescence and iridescence. There are several BRDF models proposed
for layered materials. The Beard-Maxwell reflection model [211] proposed the
physically-based reflection model based on the microfacet theory. The reflection
model has an additional term representing the volumetric scattering under the
material surface as follows
ρ = ρf s + ρvol .
|{z}
surf ace
(4.21)
|{z}
volume
Its variant was presented by Westlund and Meyer [212], whose BRDF model is
based on the microfacet model. Weidlich et al. [213] proposed a layered BRDF
model similar to the Beard-Maxwell model but with an absorption term using
the Bouguer-Lambert-Beer law. A hybrid layered BRDF model [214] utilized the
multi-lobe scattering, the microfacet-based BRDF model, and an image-based
fitting technique to represent flakes on metallic car paints.
Clearly, most of the models presented here are microfacet-based BRDF models
and widely used in physically-based rendering and shading. As measured BRDFs
became more available, the evaluation of BRDF models has been shifted toward the
use of measured BRDFs. In the next subsection, we then review the data-driven
BRDF models as they leverage the measured data and mathematical tools for
BRDF modeling.
4.2.3
The Data-Driven BRDF Model
The development of reflectance acquisition methods, presented in Section 2.5, has
brought up the attention to reflectance modeling using measured data. The key
idea of a data-driven model is to define the model as a linear combination of
basis functions using a set of coefficients. This representation does not normally
rely on any physical model, but rather computing the coefficients through data
with the accommodation of the chosen basis functions. In fact, the BRDF models
presented in Section 4.2.1 and Section 4.2.2 have a fixed number of parameters. In
contrast, the number of parameters of a data-driven BRDF model are not fixed
but predetermined by the model accuracy or the memory footprint as needed.
Spherical harmonics are frequency-space basis functions used widely in computer
graphics and vision. The basis function represents angular information on the unit
54 Chapter 4 • Surface Reflectance Modeling and its applications
sphere. The spherical harmonic has been used in BRDF modeling [66, 67, 215] and
BRDF shading [68, 216, 217]. Some example applications of the spherical harmonic,
other than the BRDF modeling, are face recognition [218, 219], an spherical
harmonic framework for illumination [220], environment map representation [221]
and real-time rendering [222, 223]. A comprehensive review of the spherical
harmonic in illumination modeling and computer vision can be found in [224].
Another frequency-space basis function is the Fourier transform [70, 71, 225].
These frequency-space BRDF models require storing the coefficients, depending
on the expansion orders, in a tabular form. The rational function constructed
by the ratio of two polynomials can be used to approximate BRDFs [226]. The
rational BRDF model requires to predetermine the number of coefficients for each
polynomial.
In this thesis, one of our contributions belongs to the category of separable factorization functions. As we focus on isotropic BRDFs, the measured data is stored
in a 3D matrix. Given that the provided data is stored in a 3D regular grid, the
factorized BRDF model can be defined as follows
ρ(x, y, z) =
L
X
F(1,i) (x)F(2,i) (y)F(3,i) (z),
(4.22)
i=1
where F(1,i) , F(2,i) , F(3,i) are 1D factors of the ith term and L is the number of
factorized terms.
The factorized BRDF model has an assumption that any BRDF can be approximated by rank-1 factorization whose factors are 1D non-parametric functions. This
holds true when the measured data is aligned on the parameterization which makes
the BRDF separable as presented by Kautz et al. [54] and McCool [61]. Moreover,
one can further reduce the data dimensionality into 2D for BRDF representations.
Lawrence et al. [39] realigned a measured BRDF into a 2D matrix and applied the
Non-negative Matrix Factorization (NMF) to obtain two 1D factors. It was later
observed by Romeiro et al. [227], who utilized the bivariate representation on the
Half-Diff parameterization by taking the average over ϕd dimension, that MERL
BRDFs [31] can be approximated with the bivariate representation. They then
proposed a BRDF inference framework from a known illumination and a sphere
image. In case of the 2D data, we can drop out the third factor F(3,i) shown in
Equation (4.22) to perform the NMF method. Proposed by Bilgili et al. [40], the
Tucker decompsition [109] was employed to model BRDFs. Their BRDF model
does recursive factorization of the BRDF illumination channel on the Half-Out
parameterization and do BRDF fitting to obtain RGB color reflection. However,
their BRDF model suffers from the color shifting which we believe is inherent to
the BRDF fitting.
Machine learning is another branch of techniques which has been used in BRDF
4.3 • Our Data-driven BRDF Models 55
modeling. The probabilistic methods such as Gaussian process [228, 229] and
directional statistics [230] have been adopted to learning BRDF model parameters.
Another machine learning approach which we adopt for our data-driven BRDF
model is the dictionary-learning approach. The BRDF can be thought of as a signal
observed by light-material interactions. Principal Component Analysis (PCA) is
an example of an unsupervised representation learning. It was first utilized on
measured data by Matusik et al. [31] when they published the MERL database.
Nielsen et al. [41] employ PCA for their BRDF sampling method. However, their
sampling method is sensitive to the training data when low number of samples
is used leading to artifacts often found in reconstructing diffuse materials. They
suggest that their BRDF sampling could improve by performing PCA on different
classes of materials. Later Sun et al. [74] proposed the diffuse-specular separation
with the PCA method for diffuse and specular dictionaries. The basis function of
PCA is the learned dictionary consisting of the orthonormal basis matrix and its
atom variance as
X = U ΣV T ,
(4.23)
where X ∈ RM ×N is the BRDF data, U ∈ RM ×K is the orthonormal basis matrix,
Σ ∈ RK×K is the scaling factor and V T ∈ RK×N is the coefficient matrix whose
columns are orthonormal.
The above equation factorizes the data using Singular Value Decomposition (SVD)
method. We can define the dictionary as Q = U Σ. The size of the dictionary Q
depends on the explained variance parameters (number of columns of U ). The
explained variance of the PCA harnesses the quality of the learned dictionary and
thus the quality of the reconstructed BRDFs. Here, the major problem of using
PCA is the large memory footprint of the dictionary as it grows according to the
number of training data in X.
A data-driven BRDF model proposed by Bagher et al. [231] is considered a hybrid
method which assumes the microfacet-based BRDF as the underlying model. They
proposed a special weighting function to preprocess the measured data. The
weighted BRDFs and the microfacet-based BRDF are then optimized to obtain
coefficients of the distribution term D, the geometrical term G and the Fresnel
term Fr . The number of coefficients are determined by the data resolution used
during the optimization process.
4.3
Our Data-driven BRDF Models
Here, we present the factorized BRDF model and the dictionary-based BRDF model
as our contributions. The factorized BRDF model utilizes the fact that isotropic
56 Chapter 4 • Surface Reflectance Modeling and its applications
BRDFs are separable when the data is aligned on a suitable parameterization. We
propose the use of the Projected Deviation Vector parameterization (PDV) with
regards to observations from previous works [59, 60]. As a result, our factorized
BRDF models can be used in BRDF modeling, BRDF editing and efficient BRDF
acquisition. Secondly, the dictionary-based BRDF model was inspired by Miandji
et al. [232], that presents the compression of light-field data using Aggregated Multi
Dictionary Ensemble (AMDE). With sparsity constraints imposed in the dictionary
learning, our dictionary-based BRDF model is a sparse non-parametric BRDF
model which has much smaller dictionary size than PCA. Additionally, we show
that the encoded BRDFs can be interpolated using our dictionaries and our model
selection method works well on the BRDF domain accounting for variability of
BRDF data sets.
4.3.1
The Factorized BRDF Model
4.3.1.1
Motivation
The BRDF parameterization has played a significant role in BRDF modeling
as shown in a number of works [39, 40, 58, 59, 60]. They showed that a suitable parameterization facilitates the aligned measured data in BRDF modeling.
The observations made by Löw et al. [60] indicate that the measured isotropic
BRDFs [31] have the iso-contour on the unit disk and the hemisphere as shown
below, Figure 4.1(a) and (d). The iso-contour curves have symmetrical circular
shapes preserved along the incident angles on the unit disk. We then further
investigate the visualizations of BRDF parameterizations as shown below in Figure 4.1. Figure 4.1(a), (b) and (c) illustrate the projections of coordinates of each
parameterization on the unit disk by varying the last two parameters as explained
in Section 2.3. The Half-Diff and Half-Out parameterizations do not preserve
the coordinate shapes around the grazing angle on the unit disk. Moreover, the
coordinates on the hemisphere are different from iso-contour curves found in the
measured isotropic BRDFs.
To convert the measured BRDFs from the standard parameterization, we provide
the conversion algorithms presented below in Algorithm 2 and Algorithm 3. One
issue occurs in dp sampling as the BRDF is highly non-linear and that the BRDF
specular lobe is important. We present the quantization based on the inversion
method in Paper B and Paper C. This method normalizes the sum of the non-linear
energy across all BRDFs in the MERL database and the result is non-linear samples
of the dp parameter. The following equation is the normalized BRDF energy along
the dp parameter.
PM P P
Ej =
m
i
k ρm (i, j, k)
,
M
(4.24)
4.3 • Our Data-driven BRDF Models 57
(a) PDV at θR = 30◦
(b) Half-Diff at θh = 30◦
(c) Half-Out at θo = 30◦
(d) PDV at θR = 70◦
(e) Half-Diff at θh = 70◦
(f) Half-Out at θo = 70◦
Figure 4.1 — Illustrations of coordinates of each parameterization on the unit
disk and on the hemisphere. Figures (a), (b) and (c) are the coordinates of the
PDV, the Half-Diff and the Half-Out parameterizations respectively on the unit disk
while figures (d), (e) and (f) are the aligned coordinates of the PDV, the Half-Diff
and the Half-Out parameterizations respectively on the hemisphere. Each curve of
the PDV (a),(d) is dp = 0.044 unit apart. Each curve of the Half-Diff (b),(e) is
θd = 2◦ apart. Each curve of the Half-Out (c),(f) is θh = 2◦ apart.
1
2
3
4
5
6
7
8
Input: (θi , ϕi , θo , ϕo )
Result: return the PDV parameters(θR , dp , ϕdp )
θR = θo
ϕi = ϕi − ϕo
ϕo = 0.0
Rp = (sin(θo )cos(ϕo + π), sin(θo )sin(ϕo + π))
Lp = (sin(θi )cos(ϕi ), sin(θi )sin(ϕi ))
Dp = Lp − Rp
dp = len(Dp )
ϕdp = atan2(Dp .y, Dp .x)
Algorithm 2: Standard-to-PDV parameterization conversion
where Ej is an element of Enormalized = {Ej |j = 1, 2, ..., NE }, NE is the number of
linear dp resolution, ρm is the BRDF value of mth material, and M is the number
of BRDFs in the database. The plot of normalized BRDF energy of the MERL
database is shown in Figure 4.2(a).
58 Chapter 4 • Surface Reflectance Modeling and its applications
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Input: (θR , dp , ϕdp )
Result: return the Standard parameters(θi , ϕi , θo , ϕo )
Rp = (−sin(θR ), 0.0)
Lp = (dp cos(ϕdp ) + Rp .x, dp sin(ϕdp ) + Rp .y)
if len(Rp ) > 1.0 or len(Lp ) > 1.0 then
return null
else
ϕi = atan2(Lp .y, Lp .x)
L .x
p
θi = abs(asin( cos(ϕ
))
i)
π
if θi > 2 then
return null
else
θo = θR
ϕo = 0.0
end
end
Algorithm 3: PDV-to-Standard parameterization conversion
(a) The normalized energy
(b) dp step size
Figure 4.2 — The normalized energy curve (a) is computed on 2000 linear dp
samples. The dp step size (b) from the inversion method demonstrates the non-linear
curve.
The resulting non-linear dp resolution is 90 samples with a non-linear step size,
Figure 4.2(b). Note that our dp sampling can be applied individually on each
BRDF to increase the model accuracy, however, here we take the average of all
BRDFs for the dp step size for general purpose modeling.
4.3.1.2
Our BRDF model
Our 3D separable BRDF model utilizes the 3D PDV representations of the measured
isotropic BRDFs. Our BRDF model iteratively encodes the BRDFs into three 1D
4.3 • Our Data-driven BRDF Models 59
factored functions using the rank-1 CPD method, see Section 3.1.2. We define our
BRDF model as follows
ρt (θR , dp , ϕdp ) =
L
X
F1,i (θR )F2,i (dp )F3,i (ϕdp ),
(4.25)
i=1
ρt = log(ρ + 1.0),
(4.26)
where L is the number of iterative factorized terms, F1,i , F2,i and F3,i are rank-1
factors of θR , dp and ϕdp , respectively.
Here, the high dynamic range values of the BRDF is handled by a logarithmic
transformation, Equation (4.26). One condition for our 1D factors is that the first
term, L = 1, is non-negative to preserve the physical meaning of the BRDF but
the remaining terms L > 1 does not have such constraint. Paper C demonstrates
the increase of our model accuracy as L increases.
4.3.1.3
Importance sampling on PDV
We propose a variance reduction technique as discussed in Section 2.1.2.2 to
integrate our model in a Monte Carlo ray tracing. The inversion method which
requires a cumulative distribution function (CDF) of the BRDF is employed for the
importance sampling setup. The non-negative factors at L = 1 are used to construct
the cumulative distribution function (CDF) as the normalized reconstructed BRDF
of the first term can be viewed as the probability density function (PDF). Therefore,
once a BRDF is represented by our BRDF model, the PDF is computed from the
normalized values of L = 1 reconstruction. This can be done on-the-fly once the
BRDF model is loaded up to the renderer.
p(dpk |θR ) =
ρ(θR , dpk , ϕdp )dpk
PNdp
,
(4.27)
j=1 ρ(θr , dpj , ϕdp )dpj ∆dpj
where Ndp is the dp resolution, ∆dpj is the dp step size, and ρ(θR , dp , ϕdp )dp is the
L = 1 reconstructed BRDF.
To choose the directions, the CDF is constructed as follows
Nd
P (dp |θR ) =
Xp
p(dpk |θR )∆dpk .
(4.28)
i=1
During the rendering, we choose sample directions based on a given probability.
The inversion method randomly draws two uniform random samples (ξ1 , ξ2 ). We
can then generate the ϕdp and dp parameters given (ξ1 , ξ2 ) and θR as follows
60 Chapter 4 • Surface Reflectance Modeling and its applications
ϕdp = 2πξ1 ,
dp = P
−1
(4.29)
(ξ2 |θR ).
(4.30)
As our BRDF model is defined on PDV space, the Jacobian of the mapping is
needed to transform the sampling space. The probability distribution of sampling
an incoming light direction ωi given an outgoing direction ωo is expressed as
p(ωi |ωo ) = p(dp |θR )pϕ (ϕdp )|JT |.
(4.31)
Equation (4.31) can be computed using a uniform PDF of pϕ (ϕdp ) and a PDF from
Equation (4.27). We derive the Jacobian transformation JT to avoid numerical
instabilities as follows
√
2cos(θi )
JT ≈ q
.
(4.32)
2 − cos(2θi ) − cos(2θo ) + 4cos(ϕi − ϕo )sin(θi )sin(θo )
(a) Lawrence et al. [39]
PSNR = 33.40dB
(b) Bilgili et al. [40]
PSNR = 35.27dB
(c) Our BRDF model [3]
PSNR = 37.37dB
Figure 4.3 — BRDF model evaluations on the Princeton scene rendered using
PBRT [9]. The scene contains 3 BRDFs which are blue-metallic-paint, nickel and
yellow-matte-plastic from MERL. The inset represents the absolute error image of
each BRDF model and PSNR of each rendered image is reported under the image.
Figure 4.3 shows the evaluations of our BRDF model with other factorized BRDF
models. Our factorized model in Figure 4.3(c) uses L = 10, resulting in around
87 KB per material. Additionally, our importance sampling indicates our model
efficacy is comparable to other methods. We present the evaluation of importance
sampling below, Figure 4.4. The visual error metric used here are Peak Signalto-Noise Ratio (PSNR) for rendered images and Mean Squared Error (MSE) for
importance sampling efficiency. For more evaluation results, please refer to Paper C.
4.3 • Our Data-driven BRDF Models 61
(b) Lawrence et al.
sampling
(MSE = 4.0662)
(c) Bilgili et al.
sampling
(MSE = 3.6134)
(d) Our model
sampling
(MSE = 3.9474)
Figure 4.4 — The Princeton scene rendered for visual comparisons of importance
sampling strategy. All images were rendered at 256 samples per pixel. The log MSE
values are computed using the rendered images and its reference.
4.3.2
The Sparse Non-parametric BRDF Model
4.3.2.1
Motivation
Regarding the discussion in Section 4.2.3, we show that data-driven BRDF models
have different complexity for the basis functions. Some models have a small storage
requirement for the basis functions such as spherical harmonics and wavelets [221,
233], while the others have greater cost of storage for the basis such as PCA or
decomposition-based BRDFs. The trade-off between the model complexity and
the storage cost opens an opportunity to create a non-parametric BRDF model
to solve this challenge. Moreover, the increasing measured materials allow us to
leverage machine learning for developing a dictionary-based BRDF model.
Indeed, a BRDF is multi-dimensional, typically 3D or 4D. Using one-dimensional
PCA or SVD does not account for multi-way arrays and neglects the correlations
between dimensions. Moreover the resulting dictionary size is large which limits
the applicability. Therefore, our proposed dictionary-based BRDF model is based
on multi-dimensional dictionaries to handle such high-dimensional reflectance data.
The proposed BRDF model learns a BRDF manifold in the form of multiple
collections of dictionaries using a dictionary ensemble. A set of dictionaries are
basis functions which can be interpreted as a BRDF manifold representing a set of
62 Chapter 4 • Surface Reflectance Modeling and its applications
BRDFs. To obtain multiple ensembles, we impose a different sparsity constraint on
each ensemble resulting in covering a wide range of material classes. Additionally,
we observed that each class of BRDFs corresponds to a different sparsity. We thus
employ the sparse representation, see Section 3.2, to put a sparsity constraint in the
learning process. Our learned dictionaries and the sparsity constraints allow us to
have flexible number of coefficients depending on the desired reconstruction quality.
More importantly, the encoded BRDFs can be recovered using only the target
dictionary set leading to fast reconstruction. Lastly, the BRDF parameterization
is still an open research problem. The BRDF model should not be limited by a
parameterization. To summarize, our proposed dictionary BRDF model thus have
the following properties.
• Small dictionary size : Our proposed dictionary size depends on the data
resolution unlike PCA whose size grows according to the number of training
BRDFs.
• Multi-dimensional data : Since the reflectance functions have a range
of model complexity depending on the dimensionality, we can extend our
dictionary-based model to represent high-dimensional data such as BTF,
SVBRDF and BSSRDF.
• Representativeness of dictionary : Once the trained dictionaries are
obtained, any new material which is not included in the training set can be
encoded without retraining the dictionaries.
• Sparsity : The number of coefficients, i.e., sparsity, provides a trade-off
between storage cost and quality.
• Fast reconstruction : Due to the orthogonality of dictionaries, the fast
local reconstruction is achievable.
• Parameterization independence : The learning process is not limited
by the parameterization used for storing BRDFs. The model can be freely
extended to any suitable parameterization.
4.3.2.2
An Aggregated Multi Dictionary Ensemble for BRDFs
The dictionary must handle any arbitrary BRDF dimensionality as N -mode tensors.
An appropriate choice, thus, points to the method proposed by Miandji et al. [232].
They proposed the Aggregated Multi Dictionary Ensemble (AMDE) method which
trains an ensemble of multi-dimensional dictionaries. We thus propose a variant
of the AMDE that addresses the challenges associated with data-driven BRDF
modeling. Indeed, the BRDF has many unique properties such as being highly
non-linear and having a high dynamic range, hence it is not straightforward to
4.3 • Our Data-driven BRDF Models 63
use the AMDE as proposed in [232]. First, the BRDF must be transformed prior
to the learning process. However, the choice of the transformations is a challenge
as studied in the literature [3, 41, 60, 74, 77, 79]. Here, we select the log-based
transformations since they were observed to perform well. Each BRDF is then
transformed as follows
ρt1 (ωh , ωd ) = log(ρ(ωh , ωd ) + 1),
(4.33)
ρt2 (ωh , ωd ) = log(ρ(ωh , ωd ) · cosMap(ωh , ωd ) + 1),
(4.34)
where cosMap(ωh , ωd ) is a weighting function mapping from the input directions
(ωh , ωd ) to (ωi , ωo ) with the weight max(cos(θi ) · cos(θo ), ϵ), where ϵ = 0.001.
cosine-weighted-
(ρt1 )
log (ρt2 )
44.72dB
48.90dB
45.16dB
41.41dB
Reference
glossy-red-paper
cardboard
log-plus
Figure 4.5 — A diffuse material, cardboard (first row), and a glossy material,
glossy-red-paper (second row), were modeled using our BRDF model with both
transformations. The reference images are on the right column. The absolute
error images appear on the right of each reconstructed result. Each error image is
multiplied by 10.0 and normalized for visual comparison. The number presented of
each rendered image is Signal-to-Noise Ratio (SNR).
We refer to (ρt1 ) transformation in Equation (4.33) as log-plus and (ρt2 ) transformation in Equation (4.34) as cosine-weighted-log. Figure 4.5 demonstrates
the visual results and the error introduced by each transformation. The log-plus
transformation performs better on glossy materials, glossy-red-paper, while the
cosine-weighted-log transformation performs better on diffuse materials, cardboard.
More results regarding data transformations are presented in Paper B and Paper C.
64 Chapter 4 • Surface Reflectance Modeling and its applications
The parameterization used here is the Half-Diff parameterization, i.e., the MERL
parmeterization [31] with resolution 90×90×180. Given a RGB BRDF, we separate
the BRDF into three BRDFs, i.e., one for each color channel. The transformations
of each BRDF using ρt1 and ρt2 give two data points to our training set. Hence,
the number of training data points is 6 times as large as the size of the training
BRDFs. For multi-spectral BRDFs, one can get more BRDF data points depending
on the number of provided spectral information, however the available measured
spectral BRDFs are limited. This can be further investigated in the future work.
N
l
We define the training data more concretely as follows. Let {X (i) }i=1
be a set
(i)
m1 ×m2 ×m3
of Nl BRDFs, where X ∈ R
. As an example, for MERL BRDFs, we
have m1 = 90, m2 = 90 and m3 = 180. Our training condition is that the BRDF
resolution must be the same for all data points. Then, each BRDF is represented
by the following decomposition
X (i) = S (i) ×1 U(1,k) ×2 U(2,k) ×3 U(3,k) ,
where
n
U(1,k) ∈ Rm1×m1 , U(2,k) ∈ Rm2×m2 , U(3,k) ∈ Rm3×m3
o
(4.35)
is an orthonormal
dictionary, the coefficient tensor S (i) ∈ Rm1 ×m2 ×m3 , and k ∈ {1, . . . , K} is the
number of dictionaries in the ensemble. The notation ×n denotes the n-mode
product between a tensor and a matrix [88].
Each trained ensemble consists of K multi-dimensional dictionaries denoted by
oK
n
U(1,k) , U(2,k) , U(3,k)
. The parameter K is a user-defined parameter and we
k=1
suggest that K = 8 is suitable for commonly used measured BRDF databases [31, 35].
Another user-defined parameter is τl which harnesses the sparsity by the constraint
S (i) ≤ τl in the training. The following optimization problem is defined for
0
training a BRDF dictionary ensemble
U
(j,k)
min
,S
(i,k)
subject to
Nl X
K
X
Mi,k X (i) − S (i,k) ×1 U(1,k) ×2 U(2,k) ×3 U(3,k)
,Mi,k i=1 k=1
U(j,k)
S
T
(i,k)
K
X
0
U(j,k) = I, ∀k = 1, . . . , K, ∀j = 1, . . . , 3,
≤ τl ,
Mi,k = 1, ∀i = 1, . . . , Nl ,
2
F
(4.36a)
(4.36b)
(4.36c)
(4.36d)
k=1
where the matrix M ∈ RNl ×K is a binary membership matrix representing BRDF
clusters, where each BRDF in the training set is represented by one multidimensional dictionary.
The constraints from Equations (4.36b) - (4.36d) are imposed during the training.
4.3 • Our Data-driven BRDF Models 65
Equation (4.36b) ensures the orthonormality of each dictionary, Equation (4.36c)
enforces the sparsity of the coefficients, and Equation (4.36d) clusters each BRDF
by one dictionary only. This membership clustering improves the representation
error in the way that BRDFs, having high similarity, should belong to only one
class and be represented by the same dictionary.
Our observations in Paper D have shown that the pre-clustering method proposed
in Miandji et al [232] cannot be applied to the BRDFs for two reasons. First,
the number of available measured data is not sufficient for training after preclustering. Second, the size of each BRDF, 1458000 elements, is impractical for the
pre-clustering method. Moreover, the BRDFs exhibit a variable degree of sparsity.
For example, a small number of coefficients is adequate for diffuse materials while
a highly glossy BRDF requires a higher number of coefficients. This corresponds
to our observation in Paper D. A small value of τl results in an efficient model for
low-frequency materials, or diffuse materials. On the other hand, when τl is larger,
the learned dictionaries will capture high-frequency materials while diffuse-like
materials will be degraded due to over-fitting. This observation has also been
witnessed in a number of BRDF models [39, 41, 67, 68, 70, 74, 234].
We then present the training method using multiple sparsity levels. Each ensemble
is trained with one sparsity value that fits a certain frequency of a material class,
e.g., a group of plastics. The results are combined to form one bigger ensemble. The
learned aggregated ensemble, as a result, covers a wider spectrum of BRDF signals.
Although the number of ensembles has an impact to the size of our aggregated
ensemble, we can still achieve low memory capacity with multiple ensembles. The
size of our dictionary is determined by the data resolution (m1 , m2 , m3 ), the number
of dictionary sets K, and the number of coefficients, τl . In Paper D, we trained on
4 ensembles with 8 dictionaries. The storage cost of 32 dictionaries required for
our aggregated ensemble is (4 ∗ 8(90 ∗ 90 + 90 ∗ 90 + 180 ∗ 180) ∗ 4)/10242 = 5.93MB,
assuming 4 bytes for each element. Our dictionary size is much smaller than using
PCA and our model performs much better on both visual quality and BRDF
reconstruction error. For more evaluation details, refer to Paper D.
4.3.2.3
BRDF Encoding and Decoding
Once our dictionary ensemble is obtained from the learning process, each new BRDF
can be encoded and decoded. To encode an unobserved BRDF, we seek for the
sparse set of coefficients such that Mean Squared Error (MSE) of the reconstructed
BRDFs is the lowest among the dictionary candidates. Each candidate is obtained
using Equation (4.37) which projects a transformed BRDF onto each dictionary.
Since we have two transformed BRDFs, the candidates are twice as many as the
number of dictionaries. The projected coefficients are then nullified according to a
user-defined testing sparsity Ŝ (i,k) ≤ τt . The projection step is formulated as
0
66 Chapter 4 • Surface Reflectance Modeling and its applications
Ŝ (i,k) = Y (i) ×1 U(1,k)
T
×2 U(2,k)
T
×3 U(3,k)
T
,
(4.37)
where Ŝ (i,k) is the coefficients of data point i projected onto dictionary k, Y (i) is a
BRDF that was not included in the training set.
The decoder is simply the reconstruction from the given sparse coefficients as
follows
Ŷ (i) = Ŝ (i,k) ×1 U(1,k) ×2 U(2,k) ×3 U(3,k) ,
where Ŷ
(i)
(4.38)
is the reconstructed BRDF.
The obtained candidates Ŝ (i,k) , ∀k ∈ {1, ..., K} from Equation (4.37) are used to
reconstruct the unobserved BRDFs and we select the dictionary corresponding to
the least reconstruction error defined as
Y (i) − Ŝ (i,k) ×1 U(1,k) ×2 U(2,k) ×3 U(3,k)
2
F
.
(4.39)
After we select the best dictionary candidate, here we only store the non-zero
locations
and thecorresponding values of the selected sparse coefficients as tuples
1 2 3
lt , lt , lt , Ŝli1 ,l2 ,l3 , where lt1 , lt2 , lt3 are indices of non-zero elements, and Ŝli1 ,l2 ,l3 are
t t t
t t t
the corresponding values. This candidate selection is done separately for each
transformation. As a result, we have two sets of sparse coefficients after this
encoding step. Up to this point, we have two BRDF models each associating with a
BRDF transformation. This still leaves us the problem of choosing the best BRDF
model. This is a challenging open research problem.
4.3.2.4
BRDF model Selection
Given two BRDF models, the most reliable method to evaluate them is to render
and compare visual results. However, rendering is rather a time-consuming and
computationally expensive task. Using BRDF reconstruction MSE, the resulting
model selection is inefficient and almost completely incorrect regarding to our
observations in Paper D. We observe that this problem is similar to the BRDF
fitting which seeks to find a qualitative BRDF error metric. The perceptual
study of BRDF-space metrics by Lavoué et al. [79] demonstrates the logarithm
function shows high correlation to the perceptual rendering performance. However,
using this logarithm function may push the model selection biased toward our
learned dictionaries and coefficients as our candidates are trained with log-based
transformations. An alternative choice is a Gamma mapping function Γ(ρ, γ) = ρ1/γ
inspired by Bieron and Peers [77]. We propose our BRDF model selection using
the Gamma-mapped-MSE. The default γ parameter is 2.0 but this number can be
fine-tuned per BRDF.
4.3 • Our Data-driven BRDF Models 67
Our BRDF model
Bagher et al. [231]
52.70dB
34.06dB
Reference
Figure 4.6 — Rendering results of red-metallic-paint form the MERL test set.
Our BRDF model was selected from the candidate pool by the Gamma-mapped-MSE.
The absolute error images are multiplied by 10.0 and normalized between two BRDF
models for visual comparison. The number presented for each rendered image is
Signal-to-Noise Ratio (SNR).
Figure 4.7 — Plots of average rendering SNR of test materials from MERL,
EPFL and DTU data sets when reconstructed with increasing number of coefficients.
We performed evaluations against non-parametric BRDFs. Indeed, our number of
coefficients τt is variable depending on the degree of accuracy. For a fair comparison,
the experiments were performed when the coefficients of all evaluated methods
are the same size. We used τt = 262 as our testing sparsity. In Figure 4.6, we
show an example of visual quality of our BRDF model and a result from Bagher
et al. [231]. We additionally experimented with the effect of varied number of
coefficients τt on MERL [31], EPFL [35] and DTU [41] data sets. The plots in
Figure 4.7 clearly demonstrate the increasing performance when the number of
coefficients is increased. Paper D reports additional performance analysis of our
BRDF model along with other non-parametric BRDFs.
68 Chapter 4 • Surface Reflectance Modeling and its applications
4.4
4.4.1
BRDF Applications
Differential apperance editing
Predictive rendering [235] is becoming increasingly popular in many applications
such as product visualization and architectural design. This requires highly-efficient
material representations and flexibility for material editing by artists to be creative
on designing appearance. However, working with measured BRDFs is cumbersome
since it is difficult to tweak and edit the material freely. BRDF models are, thus,
often employed for BRDF editing [236, 237, 238, 239], since a small number of
parameters is involved in the editing step. Moreover, editing on BRDF parameter
space is more feasible to control the BRDF shape according to the interpretable
parameters such as roughness and specular lobe parameters. In addition, when
analytical BRDF models are physically plausible, the edited results also carry the
physically-accurate properties.
Measured BRDF
Edited BRDF
Fitted BRDF parameters
Residual BRDF
Edited BRDF parameters
Figure 4.8 — The differential editing pipeline for BRDF editing and interpolation.
In Paper A, we present a BRDF editing method using differential edits on measured
BRDFs. Analytical BRDF models are main components for our method since most
of them were derived under physically plausible assumptions. Our method begins
with a BRDF fitting process to obtain parameters from a chosen analytical BRDF
model. The user then edits the BRDF shape on the parameter space to produce
the desired material. After this editing process, both fitted parameters and edited
parameters are used to approximate the BRDF basis and coefficients. In this case,
we represent them as BRDF vectors in the Half-Diff parameterization with MERL
encoding. We then obtain the residual vector as follows
f N
e N
{cri }N
i=1 = |{ci }i=1 − {ci }i=1 |,
(4.40)
where N is the number of non-zero elements of the coefficient space, {cri } is the
residual coefficients, {cfi } is the coefficients from the BRDF fitting, and {cei } is the
coefficients from parameter editing. We then update the measured data coefficients
using the following expression
4.4 • BRDF Applications 69
r
cei = max(0, cm
i + ci ),
(4.41)
cm
i
where
is the ith measured BRDF coefficient, and
BRDF coefficient.
cei
is the edited measured
Note that the updated BRDF is computed based on the measured BRDF. This is to
preserve the intrinsic reflection properties of the measured data while the user can
tweak the desired BRDF shapes such as specular lobe and diffuse reflection with
differential edits cri . The pipeline of our proposed differential editing is presented
in Figure 4.8.
BRDF interpolation is applicable with our proposed method. Given two measured
BRDFs, we edit the BRDFs on the parameter space. Our interpolation technique
gradually adjust each measured BRDF individually by differential edits. This step
thus produces two difference vectors, i.e., two residual BRDFs. Each difference
vector generates an edited measured BRDF. The result of this step is two edited
measured BRDFs. The final interpolation result is the weighted average between
the edited materials where the weights are proportional to the distance in the
parameter space.
4.4.2
BRDF interpolation in sparseland
In Paper D, we demonstrate the BRDF interpolation in the coefficient space of our
learned dictionaries. Here, we show the BRDF interpolation between two BRDFs
under two scenarios. The first scenario is to interpolate two BRDFs under the same
dictionary and the other scenario is to interpolate across different dictionaries, i.e.,
assuming that each BRDF is modeled using a distinct dictionary.
Let B1 , B2 , B3 be three BRDFs that are defined under dictionaries U(1,1) , U(2,1) ,
U(3,1) ,
U(1,1) , U(2,1) , U(3,1) , and U(1,2) , U(2,2) , U(3,2) , respectively. The
linear interpolation, denoted by I, between B1 and B2 is then expressed as
I = (1 − α)B1 + αB2
(4.42a)
(1,1)
(2,1)
+ αS2 ×1 U
×2 U
= (1 − α)S1 ×1 U
×2 U
(1,1)
(1,1)
= ((1 − α)S1 + αS2 ) ×1 U
(3,1)
×3 U
(2,1)
×3 U(3,1)
(2,1)
×2 U
(4.42b)
(3,1)
×3 U
,
(4.42c)
where S1 and S2 are the sparse coefficients of B1 and B2 , respectively, and α is
the interpolation weight. Note that Equation (4.42c) holds when the interpolation is under the same dictionary. This can be applied to bilinear and bicubic
interpolations.
70 Chapter 4 • Surface Reflectance Modeling and its applications
To interpolate two BRDFs across two distinct dictionaries, we need an intermediate
dictionary which represents both BRDFs. A solution to the intermediate dictionary
is to choose one dictionary as the reference basis and transform the other dictionary
and its coefficients. Let R(i,k)→(i,j) be an orthonormal transformation matrix
that transforms the orthonormal matrix U(i,k) to U(i,j) , then we have U(i,j) =
U(i,k) R(i,k)→(i,j) . This expression holds due to the following proposition.
Proposition 4.1. Let A and B be two distinct orthonormal matrices. Then the
solution of
min ∥B − AR∥2F subject to RT R = I,
(4.43)
R
T
is R = A B.
The proof of Proposition 4.1 can be found in [240]. We then obtain the linear
interpolation between B1 and B3 , denoted I, by the following expression
I = (1 − α)B1 + αB3
= (1 − α)S1 ×1 U
(4.44a)
(1,1)
(1,2)
+ αS3 ×1 U
= (1 − α)S1 ×1 U
+ αS3 ×1 U
(2,1)
×2 U
×2 U
(2,2)
×2 U
(1,1)
(1,1)
(2,1)
(2,1)
+ α S 3 ×1 R
(4.44b)
(3,1)
×3 U
(1,1)→(1,2)
R(2,1)→(2,2) ×3 U(3,1) R(3,1)→(3,2)
(1,1)
= (1 − α)S1 ×1 U
×3 U
×3 U(3,2)
×2 U
R
(3,1)
(2,1)
×2 U
(1,1)→(1,2)
×3 U
×2 R(2,1)→(2,2) ×3 R(3,1)→(3,2)
{z
|
(1,1)
= (1 − α)S1 + αS̃3 ×1 U
}
S̃3
×1 U(1,1) ×2 U(2,1) ×3 U(3,1) ,
(4.44c)
(3,1)
(2,1)
×2 U
(4.44d)
(3,1)
×3 U
,
(4.44e)
where (4.44d) follows from (4.44c) due to a fundamental property of the N -mode
product [88].
Figure 4.9 — An example of BRDF interpolation on coefficient space of the
dictionaries. The interpolation coefficient α from left to right is in [0.0, 1.0]
incremented by 0.1. The left most BRDF is cc-amber-citrine-rgb from EPFL
and the right most BRDF is silver-metallic-paint2 from MERL
4.5 • Summary and Future Work 71
We observe that S̃3 is the transformation of S3 with the transformation matrices
that transform U(i,1) to U(i,2) , for i = 1, 2, 3. The result from our BRDF interpolation
is not necessary sparse, however, we can achieve the desired sparsity by re-projecting
the interpolated BRDF onto the reference dictionary. Figure 4.9 shows an example
of our BRDF interpolation from a layered material to a glossy material.
Our dictionary ensemble BRDF model is a method of learning the manifold structure
of BRDFs. Each of learned dictionaries represent a chart (a local coordinate frame)
of the BRDF manifold. In our work, the BRDF manifold contains 32 local
coordinate frames represented by the orthogonal dictionaries. Given two BRDFs
that lie within the same local coordinate frame, we can generate a new BRDF
using Equation (4.42a). Doing a BRDF interpolation across two local coordinate
frames can be done using Equation (4.44e).
4.5
Summary and Future Work
This chapter presented BRDF topics in several aspects. We started from an
overview of BRDF models followed by our proposed data-driven BRDF models. By
utilizing special characteristics of the PDV parameterization, an isotropic BRDF
can be decomposed into 1D factors. This results in a simple factorized BRDF
model representing glossy BRDFs accurately. We showed that our factorized
BRDF model performs better than other factorized models on both model accuracy
and importance sampling efficiency. The other proposed BRDF model employs a
dictionary-learning method, leveraging the use of BRDF databases for training. By
expanding into multiple ensembles with different sparsity constraints, our sparse
dictionary BRDF model can efficiently represent BRDFs with a small number of
coefficients. Due to the flexibility of our sparse dictionary BRDFs, we can achieve
higher accuracy with minimal storage cost.
In the last section, we presented BRDF editing and interpolation techniques
under different models. Differential BRDF editing was presented to work with
measured BRDFs. Owing to the physically plausible BRDF model and the BRDF
fitting, the edited measured BRDF still preserves reflection properties from the
measurements, while the user can intuitively edit the desired BRDF shape based
on the model parameters. We also demonstrated the BRDF interpolation on sparse
coefficient space with the learned dictionaries. Even if two BRDFs lie on two
distinct dictionaries, it is possible to linearly interpolate the sparse coefficients of
BRDFs because of the orthonormality constraint.
In the future, we would like to extend the sparse dictionary BRDF to more complex
reflectance data especially BTF and SVBRDF. We believe that the resulting
models could benefit from the dictionary properties as well as the coefficient-space
72 Chapter 4 • Surface Reflectance Modeling and its applications
interpolation in many aspects, e.g. arbitrary dimensions, fast reconstruction, data
augmentation. Moreover, perceptual studies of the BRDF metric is a key to BRDF
fitting and BRDF model selection. To close this gap, BRDF transformations have
to be explored and investigated further. More importantly, BRDF parameterization
for more complex materials, e.g. layered materials and BSSRDF, is an interesting
area of study for efficient representations.
Chapter
5
Sparse Sampling for BRDF
Acquisition
The demand of 3D digital content creation has increased over decades as rendering technology has evolved, facilitating the use of computer graphics in many
applications. Material appearance, primarily measured data, is of importance for
generating realistic images in predictive rendering. However, acquiring BRDF
data has long been a challenging research problem as most capturing techniques
are very time-consuming. This is due to the high dimensionality of the BRDF.
For example, the MERL meaterial acquisition [31] takes approximately 3 hours
to capture reflectance data of a sample material wrapped on a sphere. The time
required prohibits the practicality of the acquisition system. Moreover, the spherical
material sample facilitates the sampling rate per captured image. The downside is
the unreliability of near-grazing measurements, and some materials may be difficult
to be wrapped on the spherical objects. Each BRDF acquisition system, discussed
in Section 2.5, has different benefits and drawbacks.
In this thesis, we address the BRDF data acquisition problem by leveraging the
available BRDF databases. Our goal is to achieve sparse BRDF measurement in
order to speed up the acquisition process with minimized loss of accuracy.
5.1
Outline and Contributions
This chapter presents our contributions towards sparse BRDF acquisition systems
(Paper B and Paper E). We introduce two sparse BRDF sampling procedures
74 Chapter 5 • Sparse Sampling for BRDF Acquisition
under different models in Section 5.2. The section splits into two parts. The first
part, Section 5.2.1, presents a BRDF factor measurement inspired by observations
from Löw et al. [60]. Under a suitable BRDF parameterization, the BRDF factor measurement can efficiently reconstruct isotropic BRDFs, specifically glossy
materials, with a low sampling rate. This requires only two simple mechanical
arms with two degrees-of-freedom. The second part presents the Fast and Robust
Optimal Sampling Technique (FROST) for BRDF acquisition. The problem is
formulated using the compressed sensing framework. To find the sparse BRDF
sampling locations, FROST represents BRDFs in the form of a dictionary and a
set of sparse coefficients, and solves for the optimal sampling pattern under the
Multiple Measurement Vector (MMV) signal model. FROST is a deterministic
method that is fast and more robust compared to previous methods that rely on
heuristics or random sampling.
5.2
The sparse BRDF measurement
5.2.1
The BRDF factor measurement using PDV
5.2.1.1
Motivation
Observations from Löw et al. [60] have shown the radial symmetry of isotropic
BRDFs along the projected vector on the unit disk. This inspired us to further
investigate the PDV parameterization. The study has confirmed that isotropic
BRDFs behave similarly to the observations. This implies that isotropic BRDFs
can be well-approximated with rank-1 factorization in the PDV parameterization.
Therefore, once the BRDF is aligned in the PDV, we can utilize the CP decomposition to obtain our factorized BRDF models on 2D BRDF representations,
see Section 3.1.2. Moreover, our factorized BRDF model is not recursive as in
Paper C. As opposed to the ABC BRDF model [60], we propose a simple and
efficient data-driven BRDF model that is suitable for a BRDF factor acquisition.
The acquisition is sparse due to the fact that only two factors are measured and
the reconstruction error from the factor measurement is minimized.
5.2.1.2
The separable BRDF models
Our 2D separable BRDF model has an assumption that isotropic BRDFs can be
approximated by two factors which are denoted by (θR , dp ). The radial symmetry
has an implication that isotropic BRDFs are invariant to the ϕdp parameter of the
PDV parameterization. Due to the high dynamic range property of a BRDF, as
discussed in Section 2.4 on page 30, the assumption holds under the logarithmic
transformation of BRDFs. The isotropic BRDFs are then transformed using the
following equation
5.2 • The sparse BRDF measurement 75
ρt = log(ρ + 1.0).
(5.1)
In Paper B, we introduce the concept of the BRDF factor measurement using the 2D
BRDF model. Our 2D separable BRDF model for the sparse BRDF measurement
is expressed as follows
ρt (θR , dp , ϕdp ) = F1 (θR )F2 (dp ),
(5.2)
where F1 (θR ) is a basis function associated with θR , and F2 (dp ) is a basis function
associated with dp .
This measurement acquires only two factors of the 2D data matrix. Each factor is
obtained by a configuration, as shown in Figures 5.1. This requires only two degreesof-freedom for the measurement setup on a relatively simple gonioreflectometer.
The first factor is measured by moving two mechanical arms away from the normal
and the other factor is measured by moving a sensor away from a light source.
(a) F1 (θR ) measurement
(c) F2 (dp ) measurement
Figure 5.1 — Two configurations of the factor measurement. Figure (a) demonstrates the perfect reflection measurement for F1 while Figure (b) measures F2 when
the light source is fixed and the sensor moves away from the light source along the
dashed line. ©2017 IEEE.
The obtained factor BRDF values will appear in the 2D BRDF matrix as illustrated
below, Figure 5.2(a), the blue row and column. These values are depicted as two blue
curves shown in Figure 5.2(b). The F1 curve measures BRDF values associated to
θR , i.e., the light scattering at perfect reflection angles while the F2 curve measures
BRDF values associated to dp , i.e., the shape of scattering lobe (specular lobe).
To estimate the missing BRDF values zi,j , we derive a simple relation. The
components we have from our factor measurement are formulated in the following
expressions
BRDF (sr-1)
76 Chapter 5 • Sparse Sampling for BRDF Acquisition
(a) The factor measurement
on the 2D BRDF matrix
(b) The BRDF measurement curves
Figure 5.2 — Illustrations of the proposed factor measurement. Figure (a) shows
the factors obtained on the 2D BRDF matrix corresponding to the measured curves
shown on figure (b). The predicted BRDFs zi,j are on the dashed curve. The
measured BRDF values are on the blue curves.
zi,1 = F1 (xi )F2 (y1 ),
(5.3)
zm,1 = F1 (xi )F2 (y1 ),
(5.4)
zm,j = F1 (xm )F2 (yj ),
(5.5)
zi,j = F1 (xi )F2 (yj ) =
zm,j
zi,1 zm,j
zi,1
=
,
F2 (y1 ) F1 (xm )
zm,1
(5.6)
where zi,j is the value at ith row and jth column, m is the measurement index
corresponding to the fixed light direction, xi is the parameter at ith row, and yj is
the parameter at jth column.
We have provided additional experiments on the impact of the fixed light location
for F2 factor measurement in Paper B. Once all missing values of the 2D BRDF
matrix are estimated, the 2D matrix can be used to approximate the 3D BRDF
with respect to the above assumption.
5.2.1.3
Results
To evaluate our factor measurement, we simulated the measurement system using
materials from the MERL database. As seen in Figure 5.3, our method performs
better compared to other parameterizations both on the absolute error images and
rendering SNR values. For more evaluation results, refer to Paper B
The proposed BRDF factor measurement utilizes the properties of the PDV parameterization. It facilitates the isotropic BRDF acquisition using sparse measurements
(Paper B), and it can be used as a factorization model (Paper C ). In Paper C,
5.2 • The sparse BRDF measurement 77
Our factor measurment
using PDV
2D BRDF model
using Half-Diff
2D BRDF model
using Half-Out
(a) SNR = 31.84dB
(b) SNR = 22.06dB
(c) SNR = 25.59dB
Figure 5.3 — We show the reconstructions of red-metallic-paint from the MERL.
Figures (a), (b) and (c) are rendered images of each reconstruction method with
the error images appeared on the right corner. The rendering SNR of each method
is placed under each image.
(a)
(b)
Figure 5.4 — An example of the BRDF factor measurement of chrome. The
left plot (a) measures the Fresnel effect and the right plot (b) indicates the specular
lobe width from the perfect reflection.
we also show that our factorized BRDFs using L = 1 could be useful in terms
of BRDF editing. Since each factor is independently measured, each BRDF has
two measured factors. Each factor can be replaced by a factor from other BRDFs
generating a new BRDF. Moreover, the user can also edit each factor intuitively. F1
describes the Fresnel effect and the specular reflection magnitude, while F2 controls
the width of the specular lobe. This interpretation is clearly seen in Figure 5.2(b)
and Figure 5.4. The curves in the figures correspond to the BRDF visualization as
observed by Löw et al. [60].
78 Chapter 5 • Sparse Sampling for BRDF Acquisition
Here, we show the BRDF factor measurement of the chrome material from the
MERL database in Figure 5.4. The F1 values are relatively high and steep neargrazing angles, while F2 values control the scattering from the perfect reflection
vector. Our factorized BRDF models, hence, have great benefits in many aspects,
i.e. modeling, measurement and editing, under a simple and intuitive BRDF
parameterization.
5.2.2
A Fast and Robust Optimal Sampling Technique :
FROST
5.2.2.1
Motivation
The sparse BRDF measurement can be casted into several problems, e.g. adaptive
sampling, inverse rendering and efficient sampling. These problems are addressed
in many different aspects while the possible outcomes are accurate reflectance
data from real-world objects. Adaptive Sampling [18, 35, 241, 242] requires an
initial estimate of the signal to determine the initial material properties. This
initial estimation is later used to identify sampling region adaptively. Inverse
rendering [243] is a wide research topic and complex under distinct conditions,
e.g., lighting, geometry and reflectance [227, 230, 244, 245, 246]. The problem
aims to retrieve scene parameters observed from the inputs, e.g., an image with
known geometry of an object and unknown lighting, and reuses the outcomes in a
new fashion such as scene relighting, 3D scene reconstruction and BRDF recovery.
Here, we address FROST as an efficient sampling method [41, 241, 247, 248]. Such
methods learn a basis function, leveraging the available BRDF databases, and
reconstruct unobserved BRDFs from a sampling pattern. The number of samples
is relatively small, around 20 - 100 samples, compared to typically more than a
million of samples from dense BRDF measurements.
The efficient sampling method proposed by Nielsen et al. [41] utilizes a learningbased method, i.e. PCA, resulting in the principal component matrix Q as a basis
function. Recall from Equation (4.23), the principal component used in [41] is
a scaling of the coefficient matrix Q = V Σ. The Q matrix is used to obtain an
optimal set of sampling directions and reconstruct any unobserved BRDF. However,
a major drawback of this method lies in the heuristic optimization scheme with
random initializations. Their optimization tries to minimize the condition number
of Q̂, i.e., sampled rows of Q. Minimizing the condition number reduces the
sensitivity to noise [249] of the matrix leading to finding a well-conditioned Q̂.
This does not necessarily correlate with the reconstruction error. Furthermore,
the algorithm randomly selects rows to construct Q̂. This randomness does not
guarantee consistency in producing sample locations, leading to a distinct result
for each run.
5.2 • The sparse BRDF measurement 79
Figure 5.5 — The FROST pipeline for BRDF sampling and reconstruction. The
dictionary is trained for FROST and reused for BRDF reconstruction. FROST
finds optimal sampling directions. The sampled BRDF values are used in the
reconstruction.
We propose a framework addressing this problem, called FROST. FROST reformulates the optimization process under the compressed sensing framework which
has a user-defined parameter indicating the required number of samples. Indeed,
FROST solves the sensing matrix design problem by finding an optimal sampling
pattern based on the input training data. One important property of FROST
is the shared optimal samples across the BRDFs. This is how we integrate the
Multiple Measurement Vector (MMV) in our sensing model. Due to the compressed
sensing capability, FROST optimizes the sample locations deterministically with
low computational complexity. Therefore, FROST is designed to be a fast and
robust method which resolves the randomness of previous methods. Moreover, to
achieve higher efficient sampling, the training BRDFs may be split into multiple
categories depending on the corresponding material classes, e.g., diffuse and glossy
BRDFs. However, we show here that even without the prior training classification,
our method can achieve accurate BRDF reconstructions. To the best of the author’s
knowledge, FROST is the first efficient BRDF sampling method with theoretical
guarantees.
The BRDF sampling pipeline of FROST begins with a BRDF dictionary trained
on measured BRDFs. Given the trained dictionary, FROST finds an optimal
set of locations by a user-defined parameter for the number of samples. This
optimization process is fast and produces a consistent set of optimal locations for
BRDF sampling. The obtained locations are used to sample BRDF values of an
unobserved BRDF from a BRDF acquisition device. Later, the trained dictionary
is reused together with the sampled BRDF values to reconstruct the full BRDF.
The illustration of the pipeline is shown in Figure 5.5.
Next, we will go through each step of the pipeline in the subsequent parts of this
80 Chapter 5 • Sparse Sampling for BRDF Acquisition
subsection. The BRDF dictionary is a part of this process in which it computes the
sparse representation of the BRDF signal. The trained dictionary is independent
of BRDF parameterization and dimensionality. However, the representation should
be sufficiently accurate for the reconstruction. The chosen dictionary here is PCA
as it is simple and efficient for training and reconstructing. We then explain the
formulation of FROST under the compressed sensing and the MMV model. Lastly,
the BRDF reconstruction from sparse measurements is presented.
5.2.2.2
The Dictionary
The design choice of the trained dictionary used in FROST is not limited by a
single dictionary-learning method. Since the sensing matrix is designed with the
capability of invertible dictionary, hence the only requirement is that the dictionary
should be invertible. We select PCA as it is a simple model for the dictionary
learning. Regarding the high dynamic range property of measured BRDFs, we
perform the log-relative mapping introduced by Nielsen et al. [41] before the PCA
decomposition. The mapping is expressed as follows
!
ρ(ωh , ωd ) · cosMap(ωh , ωd ) + ϵ
ρt (ωh , ωd ) = log
,
ρref (ωh , ωd ) · cosMap(ωh , ωd ) + ϵ
(5.7)
where cosMap is a weighting function similar to Equation (4.34), ϵ = 0.001, and
ρref is the median of the training BRDFs.
Recall Equation (4.23), and let X ∈ Rn×t be a BRDF matrix containing a set of t
vectorized measured BRDFs for training. Each BRDF has n = 90 × 90 × 180 values
on the Half-Diff parameterization. After X is constructed, it is preprocessed by the
log-relative mapping followed by mean subtraction to obtain X − µ = T ∈ Rn×t ,
where µ ∈ Rn×1 is the mean of the transformed X. We then perform a truncated
Singular Value Decomposition (SVD) as T = U ΣV T , where U ∈ Rn×p and V ∈ Rt×p
are orthonormal basis matrices, and Σ ∈ Rp×p is the diagonal matrix containing
the scaling values, where p is the number of singular values. Now let us define the
dictionary D = U Σ and its corresponding coefficients as S = V T . We can express
the training BRDF decomposition using PCA as follows
T = DS + δ,
(5.8)
where δ is the approximation error of the truncated SVD and it is related to the
number of principal components p.
5.2.2.3
Compressed sensing
Compressed sensing is a branch of signal processing field which aims for finding
solutions of sparse sampling and reconstruction. The required sampling rate of
compressed sensing for the exact recovery is far fewer than the Nyquist-Shannon
5.2 • The sparse BRDF measurement 81
sampling theorem [250, 251]. It was originally discovered by Emmanuel Candés [252,
253]. Later, David Donoho [254] published the theoretical framework and used the
term Compressed Sensing for the first time.
Let us define mathematical definitions necessary for the compressed sensing framework. Let x be a 1D signal of length n which we want to reconstruct from its
measurements y. Let Φ ∈ Rm×n be a measurement or sensing matrix mapping from
Rn to Rm , where m is the number of samples and m < n. Applying the sensing
matrix to the signal, we obtain the signal measurements y = Φx + γ, where γ is
the measurement noise. The sensing matrix Φ can be defined as a point sampling
operator, i.e., one that selects a predefined subset of elements from x. We use
a point sampling operator in designing FROST. We would like to integrate the
concept of sparse representations to formulate the underdetertermined system of
linear equations. Let D ∈ Rn×k be an overcomplete dictionary, where each column
defines a basis vector, then the signal can be expressed as x = Ds under the sparsity
constraint ∥s∥0 < τ . Hence the compressed sensing seeks for solving the following
optimization,
ŝ = arg min ∥s∥0 s.t. ∥y − ΦDs∥22 ≤ ϵc ,
s
(5.9)
where ϵc is related to the noise γ.
Once the solution ŝ is obtained, the signal can be reconstructed as x̂ = Dŝ. The
above equation states that we want to find the sparse coefficients of an individual
signal with the known measurements y and the known ΦD. In case we have
p signals, i.e., {x(1) , x(2) , ..., x(p) }, each signal is solved individually. This model
is called the Single Measurement Vector (SMV) where each signal is recovered
independently and without any assumption on the support of s, i.e., the location of
non-zero elements in s.
In the sparse BRDF measurement, we would like to design an optimal sampling
operator so that we can perform the measurement efficiently and effectively. Particularly, having a shared sampling operator for a set of BRDFs, under the compressed
sensing model, can facilitate the acquisition process in terms of designing a device
and accelerating the measurement. Similar to what was described in Section 5.2.1,
the design of the acquisition device can be cost-efficient (two degrees-of-freedom)
and time-effective (fast measurements) with the shared sampling locations. To
address this problem, we assume that the supports of all signals are the same. This
model is known as the Multiple Measurement Vector (MMV) or the Simultaneous
Sparse Approximation [255, 256].
Equation (5.9) can be extended naturally to obtain the MMV model. This can be
done by replacing the 1D signal and its coefficients with matrices. Let X ∈ Rn×p
be a matrix where each column is a vectorized signal and S ∈ Rk×p be a coefficient
82 Chapter 5 • Sparse Sampling for BRDF Acquisition
matrix where each column contains a sparse coefficient vector. Thus, the MMV
formulates the sparse representation as X = DS. The sensing matrix can be applied
explicitly to obtain the measurements of all observed signals Y = ΦX. Here, the
compressed sensing under the MMV solves the following optimization problem,
Ŝ = arg min ∥S∥r0 s.t. ∥Y − ΦDS∥22 ≤ ϵc ,
(5.10)
S
where ∥S∥r0 counts the number of non-zero rows of S. Note that having row
sparsity for S implies that all signals in X have the same support set.
The optimization problem is solved across all signals simultaneously. There are
several algorithms [257, 258, 259, 260, 261] to solve the optimization in Equation (5.10). In our experiments, we employ Simultaneous Orthogonal Matching
Pursuit(SOMP) introduced by Tropp et al. [255]. It is a greedy algorithm which is
a generalization of Orthogonal Matching Pursuit (OMP [133, 134]).
5.2.2.4
FROST
Typically, the BRDF measurement is a point sampling procedure where pairs of
directions (a light source and a sensor) are identified and used in the acquisition
process. FROST is a framework addressing the efficient BRDF sampling problem
for BRDF acquisition. The method determines the optimal sampling directions
such that a small number of samples is sufficient for accurate BRDF reconstruction.
Here, we introduce terms and expressions by connecting the BRDF acquisition and
compressed sensing to formulate FROST.
Let b be a vectorized BRDF signal of length n = 90 × 90 × 180 = 1458000 and
y = Φb be BRDF samples. Sensing matrix design in the context of compressed
sensing is formulated as follows
arg min ∥s∥0 s.t. ∥y − ΦDs∥22 ≤ ϵc , and Φ ∈ Ψ,
(5.11)
Φ
where Ψ is a set of all possible sensing matrices. Note that compared to Equation (5.9), here we are solving for Φ rather than S.
We address the problem of efficient BRDF sampling by seeking the optimal sensing
matrix Φ ∈ Rm×1458000 corresponding to the dictionary D ∈ R1458000×k from the
set of possible solutions Ψ as expressed in Equation (5.11). One solution to this
problem is heuristic gradient descent algorithms used in [41, 241, 248, 262]. The
algorithms draw random samples to initialize the search for a possible solution
depending on the metric used in the optimization scheme. Particularly, the method
proposed by Nielsen et al. [41] minimizes the condition number of the randomly
selected dictionary rows. There are a number of disadvantages for Nielsen et
al.’s method. First, the method suffers from the randomness which produces an
inconsistent sampling pattern over each run, i.e., it converges to a different local
5.2 • The sparse BRDF measurement 83
minima. Second, the condition number of ΦD does not necessary correlate to
the reconstruction error. Third, the heuristic search is a time-consuming and
computationally-expensive task due to the randomness and the gradient descent
method. Moreover, the search is parameterization dependent which requires extra
implementation if another BRDF parameterization is used. FROST overcomes
these issues, while providing theoretical guarantees.
FROST reformulate the sensing matrix design problem as sparse representation
under the MMV model. FROST is deterministic and computationally-inexpensive
due to the algorithms, e.g., SOMP, used to solve the MMV model. Moreover,
solving for the optimal point sampling pattern by FROST is parameterization
independent since the problem is defined on the dictionary space.
As mentioned previously, D must be invertible. We start by inverting Equation (5.8)
to obtain
S = D−1 T − D−1 δ.
(5.12)
In this context, the terms S and D−1 are known, while T is the unknown variable.
Note that rows of T are the locations of measured BRDFs. The term ΦD denotes
choosing rows of D as discussed previously in the context of compressed sensing.
As a result, for D−1 , finding the support set corresponds to finding the row samples
of D. This is the key observation utilized in designing FROST. The objective of
this formulation is to find the sparse solution of T with the known D , leading
to column selection of D−1 . By substituting S and D−1 in Equation (3.11), see
Section 3.2.1, we then can formulate the sensing matrix design problem as follows.
Φ = arg min ∥T ∥r0 s.t. ∥S − D−1 T ∥22 ≤ ϵc .
(5.13)
supp(T )
Notice that the term containing δ is negligible if the reconstruction is not sensitive
to the noise, otherwise it is related to the error ϵc in Equation (5.13). Thus, we
omit the term from the minimization. We employ SOMP to solve the minimization
problem, Equation (5.13). Algorithm 4 below presents SOMP’s pseudo-code.
Indeed, FROST requires only two user-defined parameters, where p denotes the
number of principal components in case of PCA and m denotes the number of
required samples. These two parameters affect FROST’s error, which directly
links to the approximation term δ. Figure 5.6 below illustrates the quality of the
sensing matrix showing the inverse MSE of reconstruction error when m and p are
increased. The higher the inverse MSE is, the more accurate the reconstruction is.
We set p = m for simplicity. In fact, Figure 5.6 verifies the expected behavior of p
and m as an increased p introduces less approximation error and an increased m
provides more information for signal reconstruction.
84 Chapter 5 • Sparse Sampling for BRDF Acquisition
1
2
3
4
5
Input: BRDF coefficient matrix S ∈ Rp×t , BRDF dictionary inverse
D−1 ∈ Rp×n , and user-defined number of samples m ≥ 1
Result: Support set supp(T)
supp(T )0 ← ∅ ;
// Initialize the support set.
(0)
R ←S ;
// Initialize the residual.
l←0;
while l < m do
j ← argmax (D−1 )Ti R(l−1) ;
// Atom selection
1
i
6
7
8
9
supp(T)l ← supp(T)l−1 ∪ {j} ;
R(l) ← S − (D−1
)(D−1
supp(T)
supp(T)
l+1
†
l+1
)S;
// Support set update
// Residual update
l ← l+1 ;
end
140
195
185
175
165
155
145
135
125
115
105
95
85
75
65
55
45
35
25
15
5
120
100
80
60
40
20
5
15
25
35
45
55
65
75
85
95
10
5
11
5
12
5
13
5
14
5
15
5
16
5
17
5
18
5
19
5
number of samples
Algorithm 4: Simultaneous Orthogonal Matching Pursuit (SOMP) for solving
Equation (5.13)
principal components
Figure 5.6 — Inverse MSE of reconstructed BRDFs according to the number of
principal component p and the number of samples m.
The obtained sensing matrix Φ can be used to strategically place sampling locations
for acquiring an unseen BRDF. More importantly, FROST reuses Φ in the BRDF
reconstruction step as the final procedure of the pipeline (Figure 5.5) to recover an
unseen BRDF signal.
5.2.2.5
BRDF Reconstruction
So far, we have described the training details and the optimal sensing matrix
solution with the FROST framework. This section provides the details of the
5.2 • The sparse BRDF measurement 85
BRDF signal recovery in the pipeline. The BRDF reconstruction requires two
steps. First, we acquire an unobserved BRDF according to m paris of directions
identified by Φ. The sampled BRDF is expressed as b̂ = Φb. Second, we apply
Φ on to the trained dictionary to get D̂ = ΦD. Since we set m = p, the BRDF
reconstruction is then defined as a least square problem as follows
arg min ∥b̂ − ΦDc∥22 ,
c
(5.14)
where c is the BRDF coefficients to be obtained, i.e., the corresponding coefficients
for principal components.
The least square problem can be solved by using ridge regression as suggested
in [41, 263]. This results in the coefficient vector c, which is used to reconstruct
the BRDF b̃ = Dc. However, in case we have m < p, the minimization problem
becomes solving an underdetermined system. The solution can be found using the
ℓ0 and ℓ1 regularizers as previously discussed in Chapter 3 and Section 5.2.2.3.
5.2.2.6
Results
We evaluated FROST in comparison to the method proposed by Nielsen et al. [41] as
it is the state-of-the-art of for BRDF point sampling. Due to the limited number of
measured BRDFs, we have combined MERL [31] and EPFL [35] databases to enlarge
the training size. Moreover, we conducted a 10-fold cross-validation by randomly
splitting training and testing materials. For solving the MMV minimization problem,
Equation (5.13), we employed SOMP implemented by Jean-François Determe [264]
in MATLAB. Since the BRDF sampling locations of each run produced by Nielsen
et al.’s method are inconsistent, we ran their method for 5 trials.
The summary of the overall reconstruction quality with respect to the number
of samples is reported below in Figure 5.7(a), starting from m = 5 to 60 with an
incremental step of 5. Each line represents the average reconstruction errors of
the 10-fold cross-validation where the yellow line indicates the errors from Nielsen
et al.’s method. With an equal size for the training set, our method significantly
outperforms Nielsen et a.l [41], especially when the number of samples is increased.
The results indicate that when the training size is reduced from t = 131 materials
(blue) to t = 80 materials (red), our result is comparable to Nielsen et al.’s method,
when it is trained on t = 131 materials. We thus expect the higher reconstruction
accuracy when a larger training set is used compared to Nielsen et a.l [41]. Hence,
this plot motivates the use of FROST over Nielsen et al.’s method even if the
training set is smaller. Figure 5.7(b) demonstrates the robustness of the proposed
method in more details. The high variations of each box from Nielsen et al’s
method (green) clearly present the wide spectrum of its reconstruction quality due
to randomness. While Nielsen et al.’s method produces vastly different results for
86 Chapter 5 • Sparse Sampling for BRDF Acquisition
25
200
FROST-BRDF
Nielsen et al.
FROST-BRDF with t=131
FROST-BRDF with t=80
Nielsen et al. with t=131
150
Inverse MSE
Inverse MSE
20
15
10
50
5
0
100
0
10
20
30
40
Number of samples
50
60
(a) The average reconstruction errors
of each method with respect to the
number of samples.
0
rs
Ou
al
Tri
1
al
Tri
2
al
Tri
3
al
Tri
4
al
Tri
5
ge
era
Av
(b) The reconstruction errors of each
method on an individual test set when
m = 40 samples.
Figure 5.7 — Plots of experimental results between Nielsen et al. [41] and FROST.
The left plot (a) illustrates the reconstruction accuracy (inverse MSE) when the
number of samples is increased. The errors are average of all 10-fold cross-validation
tests. The right plot shows the statistical quantities of the reconstruction accuracy.
Each box represents 25 percentile (bottom of the box), 75 percentile (head of the
box), maximum value (top of the dashed line), minimum value (bottom of the dashed
line) and its mean (star symbol inside the box).
each run leading to unstable reconstruction quality, FROST produces more reliable
results.
m
FROST
Nielsen et al. 2015
5
0.14
76.01
10
0.47
248.67
20
1.86
772.74
30
4.21
1600.46
40
7.49
2772.74
50
12.05
4357.08
60
17.67
6366.88
Table 5.1 — Average optimization time measured in seconds.
Furthermore, FROST solves the minimization problem in seconds which is about
360 times faster than Nielsen et al.’s method at m = 60 samples. This is clearly
seen in Table 5.1 where we report the optimization time in seconds. FROST
demonstrates significant advantages in accelerating the sensing matrix pattern
search compared to the random heuristic search proposed by Nielsen et al.’s method.
Moreover, the heuristic search requires traversing inside the parameterization,
therefore it needs extra implementation if another parameterization is used.
When a small number of samples is used in the sampling step, sometimes ring
artifacts appear on the rendered images of reconstructed BRDFs. This was also
reported in [41] where they suggested to either separate diffuse and glossy materials
5.2 • The sparse BRDF measurement 87
Reference
Nielsen et al. [41]
FROST-BRDF
Figure 5.8 — Rendered images of the reconstructed vch-silk-blue-rgb with
m = 10 samples. The red boxes demonstrate the same regions, magnified on the
right corner, where the ring artifacts noticeably appear on Nielsen et al.’s method.
Each absolute error image is placed on the right column of each method.
in training or increase the number of samples to attenuate these artifacts. However,
the results obtained from FROST show much less severity in terms of ringing
artifacts. We believe that one root cause of such artifacts is related to local minima
which is often found by the random heuristic search. We provided a clear example
of this type of artifacts in Figure 5.8. The figure presents the rendered images of
vch-silk-blue-rgb when the BRDF is acquired with 10 samples. The ring artifacts in
the red regions appear on Nielsen et al.’s method.
Figure 5.9 provides three examples of reconstructed BRDFs. A diffuse material,
green-latex, a glossy material, gold-metallic-paint2, and a layered material, cmwhite-rgb, were reconstructed with m = 20 samples. Most of the errors appear on
highlights and grazing angle regions. This is due to the fact that such regions are
difficult to measure and sensitive to noise both from sensor and the measurement
methods. However, FROST still achieves higher reconstruction quality than the
current state-of-the-art. Particularly for the layered material, which is rather
challenging to represent and acquire, FROST performs quite effectively. Note that
each error image is normalized and multiplied by 10 for visual comparisons.
88 Chapter 5 • Sparse Sampling for BRDF Acquisition
gold-metallic-paint2
cm-white-rgb
Nielsen
green-latex
34.45dB
25.75dB
51.62dB
41.76dB
32.16dB
Reference
FROST-BRDF
44.98dB
Figure 5.9 — Three examples of reconstruced BRDFs, green-latex, goldmetallic-paint2 and cm-white-rgb, from each method with m = 20 samples.
The first row shows reconstructed BRDFs from Nielsen et al.’s method [41] while
the second row shows reconstructed BRDFs from FROST. The last row shows the
reference fro each material. The absolute error image is presented on the right
column of each material where the error scale is on the right most column. We also
report rendering SNR values under each of reconstructed BRDFs.
5.3
Summary and Future Work
This chapter presented sparse BRDF measurement methods using two proposed
models. In Section 5.2.1, we utilized the circular symmetry found in the PDV
characteristics for the BRDF factor measurement. The PDV parameterization
thus enables the 2D BRDF separable model to faciliate the measurements with
two simple acquisition configurations. As a result, the acquisition device which
permits only two degrees-of-freedom can be used to acquire and reconstruct a
BRDF quite faithfully, specifically for glossy BRDFs. In Section 5.2.2, we proposed
the sparse BRDF measurement which adopts compressed sensing as a sensing
matrix design framework, called FROST. FROST reformulates the efficient BRDF
point sampling measurement into a sensing matrix design problem. To this end,
SOMP was employed to find the optimal sampling pattern. This leads to a fast
and robust method compared to the current state-of-the-art while we also achieve
higher BRDF reconstruction quality.
In the future, as the digital trend keeps growing, the sparse reflectance measurement
5.3 • Summary and Future Work 89
will become more important. More compact BRDF representations and intuitive
BRDF parameterizations could be keys and a future venue for BRDF modeling
and acquisition systems, as shown in Section 4.3.1 and Section 5.2.1. The adaptive
PDV parameterization admitting other radial symmetries than circular shape is
also an interesting venue which might be flexible and suitable for more complex
reflectance data. Moreover, we would like to extend and further study FROST
for more complex reflectance data and diverse types of dictionaries. We believe
that FROST initiates a new research direction for not only BRDF acquisition but
also any high-dimensional visual data. Furthermore, we would like to further study
FROST for capturing in the wild.
Chapter
6
Summary and Outlook
This thesis has presented several contributions in two major research areas. i.e.,
appearance modeling and BRDF sampling. The content of this thesis was organized
into three major parts. The first part (Chapter 1) introduced the readers to the
fundamentals of computer graphics, the research questions which are addressed by
this thesis, and the overview of contributions. The second part (Chapter 2 and
Chapter 3) provided preliminaries of BRDFs and relevant methods. The components
of the rendering pipeline, e.g., light transport algorithms, surface reflection and
data acquisition, as well as data-driven methods, e.g., tensor decompositions and
sparse representation, were given to the readers to develop familiarity with the
research content. The third part (Chapter 4 and Chapter 5) has presented the
proposed models for answering the research questions we set out to address. The
data-driven models are the primary key to overcome the issues on both BRDF
representation and acquisition. This final chapter will provide a snapshot of
thesis’s contributions where the contents get consolidated and some future research
directions are discussed.
For BRDF modeling and representation, we identified that non-parametric BRDF
models form a great basis for modeling of captured real-world BRDFs. The first set
of BRDF models was achieved by an iterative CPD method. Our factorized BRDF
models can efficiently and accurately represent isotropic BRDFs with support of
importance sampling. The motivation behind our factorized BRDF models was
based on observations made when analyzing the PDV parameterization. The second
set of BRDF models was motivated by the recent results of multi-dimensional
dictionary learning. We obtained promising results using sparse representation and
92 Chapter 6 • Summary and Outlook
dictionary learning. By learning the dictionaries from material databases, we were
able to encode BRDFs with a flexible number of coefficients. Moreover, BRDF
interpolation and editing techniques were proposed. The differential editing technique facilitated measured BRDF editing regarding to the BRDF parametric fitting.
We also derived the BRDF interpolation technique on our learned dictionaries.
The practicality of BRDF acquisition has long been a problem for acquiring
BRDFs. We addressed this problem with two sparse acquisition models. The
first acquisition setup adopted the factorized BRDF model and the observations
made under PDV parameterization. As a result, we were able to efficiently recover
isotropic BRDFs, especially glossy materials, with only two factor measurements.
The second BRDF acquisition model was formulated using compressed sensing as
the underlying framework. We proposed a BRDF acquisition framework (FROST)
which reformulated the problem into finding the optimal sampling pattern. FROST
employed the dictionary learning method and SOMP algorithm enabling a fast and
robust optimal sampling pattern.
Regarding the proposed models, we find the connections between BRDF modeling
and acquisition. The essence of sparse BRDF sampling and reconstruction has
addressed both BRDF representation and acquisition as a single research goal. In
most cases, BRDF models are not physically interpretable to the real measurements.
In contrast to sparse BRDF acquisition, the addressable methods tend to relate
sparse measurements with measurement models, e.g., signal recovery. The proposed
sparse BRDF acquisition models have shown accurate BRDF reconstruction results,
however, our BRDF models still perform much better in terms of data compression
or representation. We believe that the sparse BRDF sampling is a future trend in
the development of BRDF research. Furthermore, the number of measured BRDFs
is a key for data-driven approaches. We would like to apply FROST to create a
large collection of materials for future research. We also would like to improve and
extend FROST to other reflectance data such as SVBRDF and BTF as our future
contribution.
High-dimensional visual data can not only be seen in surface reflectance data but
also other domains, e.g., medical images, light field images and spectral images.
Data compression, representation and reconstruction are essential for developing
the pipelines related to other domain applications. Here, we believe that tensor
decompositions, sparse representation, and compressed sensing are important tools
with strong mathematical foundations. These frameworks are classical machine
learning approaches as compared to deep learning, modern machine learning.
The integrations between classical frameworks and modern high-computational
learning approaches are complements to each other and their cross-fertilization is
an interesting venue for future work.
Bibliography
[1]
A. Tsirikoglou, J. Kronander, P. Larsson, T. Tongbuasirilai, A. Gardner, and
J. Unger, “Differential appearance editing for measured brdfs,” in ACM SIGGRAPH 2016 Talks, ser. SIGGRAPH ’16. New York, NY, USA: Association
for Computing Machinery, 2016.
[2]
T. Tongbuasirilai, J. Unger, and M. Kurt, “Efficient brdf sampling using
projected deviation vector parameterization,” in 2017 IEEE International
Conference on Computer Vision Workshop (ICCVW). Los Alamitos, CA,
USA: IEEE Computer Society, oct 2017, pp. 153–158. [page 31]
[3]
T. Tongbuasirilai, J. Unger, J. Kronander, and M. Kurt, “Compact and
intuitive data-driven brdf models,” The Visual Computer, vol. 36, no. 4, pp.
855–872, 2020. [pages 28, 31, 60, and 63]
[4]
T. Tongbuasirilai, J. Unger, C. Guillemot, and E. Miandji, “A sparse nonparametric brdf model,” ACM Trans. Graph., apr 2022. [page 31]
[5]
E. Miandji, T. Tongbuasirilai, S. Hajisharif, B. Kavoosighafi, and J. Unger,
“Frost-brdf : A fast and robust optimal sampling technique for brdf acquisition,” To be sumitted to ACM Trans. Graph., 2023.
[6]
T. Wilaiprasitporn, A. Ditthapron, K. Matchaparn, T. Tongbuasirilai, N. Banluesombatkul, and E. Chuangsuwanich, “Affective eeg-based person identification using the deep learning approach,” IEEE Transactions on Cognitive
and Developmental Systems, vol. 12, no. 3, pp. 486–496, 2020.
[7]
A. Tsirikoglou, G. Eilertsen, and J. Unger, “A survey of image synthesis
methods for visual machine learning,” Computer Graphics Forum, vol. 39,
no. 6, pp. 426–451, 2020. [page 2]
[8]
T. Whitted, “An improved illumination model for shaded display,” Commun.
ACM, vol. 23, no. 6, pp. 343–349, jun 1980. [pages 3 and 4]
[9]
M. Pharr, W. Jakob, and G. Humphreys, Physically Based Rendering: From
Theory to Implementation, 3rd ed. San Francisco, CA, USA: Morgan
Kaufmann Publishers Inc., 2016. [pages 3, 4, 21, 22, and 60]
96 Bibliography
[10] B. Bitterli, “Rendering resources,” 2016, https://benedikt-bitterli.me/
resources/. [page 4]
[11] J. T. Kajiya, “The rendering equation,” vol. 20, no. 4, pp. 143–150, aug 1986.
[pages 4 and 20]
[12] J. Menon, B. Wyvill, C. Bajaj, J. Bloomenthal, B. Guo, J. Hart, G. Wyvill,
and C. Bajaj, “Implicit surfaces for geometric modeling and computer graphics,” ACM SIGGRAPH-96 Course Notes, pp. 4–9, 1996. [page 5]
[13] A. Opalach and S. Maddock, “An overview of implicit surfaces,” 05 1995.
[page 5]
[14] J. Flórez, M. Sbert, M. Sainz, and J. Vehí, “Improving the interval ray tracing
of implicit surfaces,” vol. 4035, 01 2006, pp. 655–664. [page 5]
[15] B. R. de Araújo, D. S. Lopes, P. Jepp, J. A. Jorge, and B. Wyvill, “A survey
on implicit surface polygonization,” vol. 47, no. 4, may 2015. [page 5]
[16] G. Turk and M. Levoy, “Zippered polygon meshes from range images,” in
Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’94. New York, NY, USA: Association
for Computing Machinery, 1994, pp. 311–318. [page 6]
[17] P. E. Debevec, C. J. Taylor, and J. Malik, “Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach,” in
Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’96. New York, NY, USA: Association
for Computing Machinery, 1996, pp. 11–20. [page 6]
[18] H. P. A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, “Imagebased reconstruction of spatial appearance and geometric detail,” ACM Trans.
Graph., vol. 22, no. 2, pp. 234–257, apr 2003. [pages 6 and 78]
[19] J. I. Ebert, “Chapter 3 - photogrammetry, photointerpretation, and digital
imaging and mapping in environmental forensics,” in Introduction to Environmental Forensics (Third Edition), third edition ed., B. L. Murphy and
R. D. Morrison, Eds. San Diego: Academic Press, 2015, pp. 39–64, https://
www.sciencedirect.com/science/article/pii/B9780124046962000035. [page 6]
[20] T. Lescoat, M. Ovsjanikov, P. Memari, J.-M. Thiery, and T. Boubekeur, “A
survey on data-driven dictionary-based methods for 3d modeling,” Computer
Graphics Forum, vol. 37, no. 2, pp. 577–601, 2018. [page 6]
[21] L. Xu, R. Wang, J. Zhang, Z. Yang, J. Deng, F. Chen, and L. Liu, “Survey
on sparsity in geometric modeling and processing,” Graph. Models, vol. 82,
no. C, p. 160–180, nov 2015. [page 6]
Bibliography 97
[22] K. Xu, V. G. Kim, Q. Huang, N. Mitra, and E. Kalogerakis, “Data-driven
shape analysis and processing,” in SIGGRAPH ASIA 2016 Courses, ser. SA
’16. New York, NY, USA: Association for Computing Machinery, 2016.
[page 6]
[23] E. Veach, “Robust monte carlo methods for light transport simulation,” Ph.D.
dissertation, Stanford, CA, USA, 1998, aAI9837162. [page 6]
[24] B. E. A. Saleh and M. C. Teich, Fundamentals of photonics; 1st ed. New
York, NY: Wiley, 1991. [page 6]
[25] H. W. Jensen, Realistic Image Synthesis Using Photon Mapping. USA: A.
K. Peters, Ltd., 2001. [pages 6 and 25]
[26] B. T. Phong, “Illumination for computer generated pictures,” Commun. ACM,
vol. 18, no. 6, pp. 311–317, jun 1975. [pages 7 and 48]
[27] J. F. Blinn, “Models of light reflection for computer synthesized pictures,” in
Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’77. New York, NY, USA: Association
for Computing Machinery, 1977, pp. 192–198. [pages 7, 29, and 48]
[28] R. L. Cook and K. E. Torrance, “A reflectance model for computer graphics,”
ACM Trans. Graph., vol. 1, no. 1, pp. 7–24, jan 1982. [pages 7, 29, and 51]
[29] “Curet: Columbia-utrecht reflectance and texture database,” http://www1.
cs.columbia.edu/CAVE/software/curet/. [pages 7 and 36]
[30] “Cornell reflectance data,” http://www.graphics.cornell.edu/online/
measurements/reflectance/index.html/. [pages 7 and 36]
[31] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “A data-driven reflectance model,” ACM Transactions on Graphics, vol. 22, no. 3, pp. 759–769,
jul 2003. [pages 7, 28, 34, 35, 47, 54, 55, 56, 64, 67, 73, and 85]
[32] V. R. Haindl M., Filip J., “Digital material appearance: the curse of terabytes,” ERCIM News, no. 90, pp. 49–50, 2012. [page 7]
[33] J. Filip, M. Kolafová, M. Havlíček, R. Vávra, M. Haindl, and R. H., “Evaluating Physical and Rendered Material Appearance,” The Visual Computer
(Computer Graphics International 2018), 2018. [pages 7 and 36]
[34] M. Weinmann, J. Gall, and R. Klein, “Material classification based on training
data synthesized using a btf database,” in Computer Vision - ECCV 2014,
D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer
International Publishing, 2014, pp. 156–171. [page 7]
98 Bibliography
[35] J. Dupuy and W. Jakob, “An adaptive parameterization for efficient material
acquisition and rendering,” Transactions on Graphics (Proceedings of SIGGRAPH Asia), vol. 37, no. 6, pp. 274:1–274:18, nov 2018. [pages 7, 27, 34,
35, 64, 67, 78, and 85]
[36] D. Guarnera, G. C. Guarnera, A. Ghosh, I. Hall, and M. Glencross, “Capturing and representing brdfs for virtual reality,” in SIGGRAPH ASIA 2016
Courses, ser. SA ’16. New York, NY, USA: Association for Computing
Machinery, 2016. [page 8]
[37] G. C. Guarnera, A. Ghosh, I. Hall, M. Glencross, and D. Guarnera, “Material
capture and representation with applications in virtual reality,” in ACM
SIGGRAPH 2017 Courses, ser. SIGGRAPH ’17. New York, NY, USA:
Association for Computing Machinery, 2017. [page 8]
[38] S. Rusinkiewicz, “A new change of variables for efficient brdf representation.”
in Rendering Techniques, ser. Eurographics, G. Drettakis and N. L. Max,
Eds. Springer, 1998, pp. 11–22. [pages 10 and 28]
[39] J. Lawrence, S. Rusinkiewicz, and R. Ramamoorthi, “Efficient brdf importance sampling using a factored representation,” in ACM SIGGRAPH
2004 Papers, ser. SIGGRAPH ’04. New York, NY, USA: Association for
Computing Machinery, 2004, pp. 496–505. [pages 10, 21, 29, 54, 56, 60,
and 65]
[40] A. Bilgili, A. Öztürk, and M. Kurt, “A general brdf representation based
on tensor decomposition,” Computer Graphics Forum, vol. 30, no. 8, pp.
2427–2439, 2011. [pages 10, 29, 31, 54, 56, and 60]
[41] J. B. Nielsen, H. W. Jensen, and R. Ramamoorthi, “On optimal, minimal
brdf sampling for reflectance acquisition,” ACM Transactions on Graphics
(TOG), vol. 34, no. 6, pp. 186:1–186:11, November 2015. [pages 11, 36, 55,
63, 65, 67, 78, 80, 82, 85, 86, 87, and 88]
[42] G. E. Stillman, “21 - optoelectronics,” in Reference Data for Engineers (Ninth
Edition), ninth edition ed., W. M. Middleton and M. E. Van Valkenburg,
Eds. Woburn: Newnes, 2002, pp. 21–1–21–31. [page 15]
[43] G. B. Arfken and H. J. Weber, Mathematical methods for physicists; 4th ed.
San Diego, CA: Academic Press, 1995. [page 19]
[44] F. Nicodemus, J. Richmond, and J. Hsia, Geometrical Considerations and
Nomenclature for Reflectance, ser. (National Bureau of Standards monograph).
U.S. Government Printing Office, 1977. [page 23]
Bibliography 99
[45] B. Hapke, Theory of Reflectance and Emittance Spectroscopy. Cambridge
University Press, 2012. [page 24]
[46] C. Donner, J. Lawrence, R. Ramamoorthi, T. Hachisuka, H. W. Jensen, and
S. Nayar, “An empirical bssrdf model,” ACM Trans. Graph., vol. 28, no. 3,
jul 2009. [page 25]
[47] A. King, C. Kulla, A. Conty, and M. Fajardo, “Bssrdf importance sampling,”
in ACM SIGGRAPH 2013 Talks, ser. SIGGRAPH ’13. New York, NY, USA:
Association for Computing Machinery, 2013. [page 25]
[48] P. H. Christensen, “An approximate reflectance profile for efficient subsurface
scattering,” in ACM SIGGRAPH 2015 Talks, ser. SIGGRAPH ’15. New
York, NY, USA: Association for Computing Machinery, 2015. [page 25]
[49] L.-Q. Yan, W. Sun, H. W. Jensen, and R. Ramamoorthi, “A bssrdf model
for efficient rendering of fur with global illumination,” ACM Trans. Graph.,
vol. 36, no. 6, nov 2017. [page 25]
[50] C. Castillo, J. López-Moreno, and C. Aliaga, “Recent advances in fabric
appearance reproduction,” Computers and Graphics, vol. 84, pp. 103–121,
2019. [page 25]
[51] Y. Gitlina, G. C. Guarnera, D. S. Dhillon, J. Hansen, A. Lattas, D. Pai,
and A. Ghosh, “Practical measurement and reconstruction of spectral skin
reflectance,” Computer Graphics Forum, vol. 39, no. 4, pp. 75–89, 2020.
[page 25]
[52] M. Haindl and J. Filip, Bidirectional Texture Functions. London: Springer
London, 2013, pp. 147–209. [page 26]
[53] J. Filip and M. Haindl, “Bidirectional texture function modeling: A state
of the art survey,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 31, no. 11, pp. 1921–1940, 2009. [page 26]
[54] J. Kautz and M. D. McCool, “Interactive rendering with arbitrary brdfs
using separable approximations,” in Proceedings of the 10th Eurographics
Conference on Rendering, ser. EGWR’99. Goslar, DEU: Eurographics
Association, 1999, pp. 247–260. [pages 28 and 54]
[55] R. Roland and K. Reinhard, “A compact and editable representation for measured BRDFs,” University of Bonn, Techreport CG-2010-1, 2010. [pages 28
and 31]
[56] G. J. Ward, “Measuring and modeling anisotropic reflection,” in Proceedings of the 19th Annual Conference on Computer Graphics and Interactive
100 Bibliography
Techniques, ser. SIGGRAPH ’92. New York, NY, USA: Association for
Computing Machinery, 1992, pp. 265–272. [pages 29, 36, and 49]
[57] M. Ashikhmin and P. Shirley, “An anisotropic phong brdf model,” Journal
of Graphics Tools, vol. 5, no. 2, pp. 25–32, 2000. [pages 29 and 50]
[58] A. Ngan, F. Durand, and W. Matusik, “Experimental Analysis of BRDF
Models,” in Eurographics Symposium on Rendering (2005), K. Bala and
P. Dutre, Eds. The Eurographics Association, 2005. [pages 29, 31, 33, 34,
and 56]
[59] L. Neumann, A. Neumann, and L. Szirmay-Kalos, “Reflectance Models with
Fast Importance Sampling,” Computer Graphics Forum, 1999. [pages 29, 49,
50, and 56]
[60] J. Löw, J. Kronander, A. Ynnerman, and J. Unger, “Brdf models for accurate
and efficient rendering of glossy surfaces,” ACM Trans. Graph., vol. 31, no. 1,
feb 2012. [pages 29, 31, 32, 33, 52, 56, 63, 74, and 77]
[61] M. D. McCool, J. Ang, and A. Ahmad, “Homomorphic factorization of brdfs
for high-performance rendering,” in Proceedings of the 28th Annual Conference
on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’01. New
York, NY, USA: Association for Computing Machinery, 2001, pp. 171–178.
[pages 30, 31, and 54]
[62] D. Edwards, S. Boulos, J. Johnson, P. Shirley, M. Ashikhmin, M. Stark,
and C. Wyman, “The halfway vector disk for brdf modeling,” ACM Trans.
Graph., vol. 25, no. 1, pp. 1–18, jan 2006. [pages 30 and 50]
[63] M. Stark, J. Arvo, and B. Smits, “Barycentric parameterizations for isotropic
brdfs,” IEEE Transactions on Visualization and Computer Graphics, vol. 11,
no. 2, pp. 126–138, 2005. [page 30]
[64] P. Barla, L. Belcour, and R. Pacanowski, “In praise of an alternative brdf
parametrization,” ser. MAM ’15. Goslar, DEU: Eurographics Association,
2015, pp. 9–13. [page 30]
[65] I. Mallett and C. Yuksel, “Parameterization of tabulated brdfs,” in Proceedings
of the 33rd Computer Graphics International, ser. CGI ’16. New York, NY,
USA: Association for Computing Machinery, 2016, pp. 45–48. [page 30]
[66] B. Cabral, N. Max, and R. Springmeyer, “Bidirectional reflection functions
from surface bump maps,” ser. SIGGRAPH ’87. New York, NY, USA:
Association for Computing Machinery, 1987, pp. 273–281. [pages 30 and 54]
Bibliography 101
[67] S. H. Westin, J. R. Arvo, and K. E. Torrance, “Predicting reflectance functions
from complex surfaces,” SIGGRAPH Comput. Graph., vol. 26, no. 2, pp.
255–264, jul 1992. [pages 30, 54, and 65]
[68] J. Kautz, P.-P. Sloan, and J. Snyder, “Fast, arbitrary brdf shading for
low-frequency lighting using spherical harmonics,” in Proceedings of the
13th Eurographics Workshop on Rendering, ser. EGRW ’02. Goslar, DEU:
Eurographics Association, 2002, pp. 291–296. [pages 30, 54, and 65]
[69] P. Gautron, J. Krivanek, S. Pattanaik, and K. Bouatouch, “A Novel Hemispherical Basis for Accurate and Efficient Rendering,” in Eurographics Workshop on Rendering, A. Keller and H. W. Jensen, Eds. The Eurographics
Association, 2004. [page 30]
[70] H. Xu and Y. Sun, “Compact representation of spectral brdfs using fourier
transform and spherical harmonic expansion,” Computer Graphics Forum,
vol. 25, no. 4, pp. 759–775, 2006. [pages 30, 54, and 65]
[71] W. Jakob, E. d’Eon, O. Jakob, and S. Marschner, “A comprehensive framework for rendering layered materials,” ACM Trans. Graph., vol. 33, no. 4, jul
2014. [pages 30 and 54]
[72] G. Fechner, D. Howes, and E. Boring, Elements of Psychophysics [Elemente
Der Psychophysik], ser. Henry Holt editions in psychology. Holt, 1966, no.
v. 1. [page 31]
[73] J. A. Ferwerda, S. N. Pattanaik, P. Shirley, and D. P. Greenberg, “A model
of visual adaptation for realistic image synthesis,” in Proceedings of the
23rd Annual Conference on Computer Graphics and Interactive Techniques,
ser. SIGGRAPH ’96. New York, NY, USA: Association for Computing
Machinery, 1996, pp. 249–258. [page 31]
[74] T. Sun, H. W. Jensen, and R. Ramamoorthi, “Connecting measured brdfs to
analytic brdfs by data-driven diffuse-specular separation,” ACM Transactions
on Graphics (TOG), vol. 37, no. 6, p. 273, 2018. [pages 31, 55, 63, and 65]
[75] E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, High Dynamic Range
Imaging: Acquisition, Display, and Image-Based Lighting, ser. The Morgan
Kaufmann Series in Computer Graphics. Elsevier Science, 2005. [page 33]
[76] G. Eilertsen, “The high dynamic range imaging pipeline : Tone-mapping, distribution, and single-exposure reconstruction,” Ph.D. dissertation, Linköping
UniversityLinköping University, Media and Information Technology, Faculty
of Science and Engineering, 2018. [page 33]
102 Bibliography
[77] J. Bieron and P. Peers, “An adaptive brdf fitting metric,” Computer Graphics
Forum, vol. 39, no. 4, pp. 59–74, 2020. [pages 33, 34, 63, and 66]
[78] A. Forés, J. Ferwerda, and J. Gu, “Toward a perceptually based metric
for brdf modeling,” Final Program and Proceedings - IS and T/SID Color
Imaging Conference, vol. CIC’12, pp. 142–148, 01 2012. [page 34]
[79] G. Lavoué, N. Bonneel, J.-P. Farrugia, and C. Soler, “Perceptual quality of
BRDF approximations: dataset and metrics,” Computer Graphics Forum,
vol. 40, no. 2, pp. 327–338, May 2021. [pages 34, 63, and 66]
[80] P. Vangorp, J. Laurijssen, and P. Dutré, “The influence of shape on the
perception of material reflectance,” in ACM SIGGRAPH 2007 Papers, ser.
SIGGRAPH ’07. New York, NY, USA: Association for Computing Machinery,
2007, pp. 77–es. [page 34]
[81] S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and
D. P. Greenberg, “Image-based brdf measurement including human skin,” in
Rendering Techniques’ 99, D. Lischinski and G. W. Larson, Eds. Vienna:
Springer Vienna, 1999, pp. 131–144. [pages 34 and 35]
[82] S. R. Marschner, S. H. Westin, E. P. F. Lafortune, and K. E. Torrance,
“Image-based bidirectional reflectance distribution function measurement,”
Appl. Opt., vol. 39, no. 16, pp. 2592–2600, Jun 2000. [page 34]
[83] J. Filip and R. Vávra, “Template-based sampling of anisotropic BRDFs,”
Computer Graphics Forum, vol. 33, no. 7, pp. 91–99, October 2014. [page 36]
[84] J. Filip, R. Vavra, M. Haindl, P. Zid, M. Krupicka, and V. Havran, “Brdf
slices: Accurate adaptive anisotropic appearance acquisition,” in Proceedings
of the 26th IEEE Conference on Computer Vision and Pattern Recognition,
CVPR 2013, June 2013, pp. 4321–4326. [page 36]
[85] R. Vávra and J. Filip, “Adaptive slices for acquisition of anisotropic BRDF,”
Computational Visual Media, no. 4, pp. 55–69, 2018. [page 36]
[86] A. Ghosh, W. Heidrich, S. Achutha, and M. O’Toole, “A basis illumination
approach to brdf measurement,” International journal of computer vision,
vol. 90, no. 2, pp. 183–197, 2010. [page 36]
[87] Y. Mukaigawa, K. Sumino, and Y. Yagi, “Multiplexed illumination for
measuring brdf using an ellipsoidal mirror and a projector,” in Computer
Vision - ACCV 2007, Y. Yagi, S. B. Kang, I. S. Kweon, and H. Zha, Eds.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 246–257. [page 36]
Bibliography 103
[88] T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,”
SIAM Review, vol. 51, no. 3, pp. 455–500, 2009. [pages 38, 42, 64, and 70]
[89] M. Itskov, Tensor Algebra and Tensor Analysis for Engineers: With Applications to Continuum Mechanics, ser. Mathematical Engineering. Springer
Berlin Heidelberg, 2012. [page 38]
[90] T. Needham, Visual Differential Geometry and Forms: A Mathematical
Drama in Five Acts. Princeton University Press, 2021. [page 38]
[91] J. N. Reddy, Principles of Continuum Mechanics: A Study of Conservation
Principles with Applications. Cambridge University Press, 2010. [page 38]
[92] V. de Silva and L.-H. Lim, “Tensor rank and the ill-posedness of the best
low-rank approximation problem,” SIAM Journal on Matrix Analysis and
Applications, vol. 30, no. 3, pp. 1084–1127, 2008. [page 38]
[93] F. L. Hitchcock, “The expression of a tensor or a polyadic as a sum of
products,” Journal of Mathematics and Physics, vol. 6, no. 1-4, pp. 164–189,
1927. [page 40]
[94] F. L. Hitchcock, “Multiple invariants and generalized rank of a p-way matrix
or tensor,” Journal of Mathematics and Physics, vol. 7, no. 1-4, pp. 39–79,
1928. [page 40]
[95] R. B. Cattell, ““parallel proportional profiles” and other principles for determining the choice of factors by rotation,” Psychometrika, pp. 267–283, 1944.
[page 40]
[96] R. B. Cattell, “The three basic factor-analytic research designs—their interrelations and derivatives.” Psychological Bulletin, pp. 499–520, 1952. [page 40]
[97] R. Harshman, Foundations of the Parafac Procedure: Models and Conditions
for an "explanatory" Multimodal Factor Analysis, ser. UCLA working papers
in phonetics. University of California at Los Angeles, 1970. [page 40]
[98] J. D. Carroll, “Analysis of individual differences in multidimensional scaling
via an n-way generalization of “eckart-young” decomposition,” Psychometrika,
pp. 283–319, 1970. [page 40]
[99] J. Mocks, “Topographic components model for event-related potentials and
some biophysical considerations,” IEEE Transactions on Biomedical Engineering, vol. 35, no. 6, pp. 482–484, 1988. [page 40]
[100] H. A. L. Kiers, “Towards a standardized notation and terminology in multiway analysis,” Journal of Chemometrics, vol. 14, no. 3, pp. 105–122, 2000.
[page 40]
104 Bibliography
[101] J. B. Kruskal, “Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics,” Linear
Algebra and its Applications, vol. 18, no. 2, pp. 95–138, 1977. [page 41]
[102] J. B. Kruskal, “Rank, decomposition, and uniqueness for 3-way and n -way
arrays,” 1989. [page 41]
[103] N. Sidiropoulos and R. Bro, “On the uniqueness of multilinear decomposition
of n-way arrays,” Journal of Chemometrics - J CHEMOMETR, vol. 14, pp.
229–239, 05 2000. [page 41]
[104] J. M. F. ten Berge, “The typical rank of tall three-way arrays,” Psychometrika,
vol. 65, pp. 525–532, 2000. [page 41]
[105] X. Liu and N. Sidiropoulos, “Cramer-rao lower bounds for low-rank decomposition of multidimensional arrays,” IEEE Transactions on Signal Processing,
vol. 49, no. 9, pp. 2074–2086, 2001. [page 41]
[106] L. De Lathauwer, “A link between the canonical decomposition in multilinear
algebra and simultaneous matrix diagonalization,” SIAM Journal on Matrix
Analysis and Applications, vol. 28, no. 3, pp. 642–666, 2006. [page 41]
[107] R. Minster, I. Viviano, X. Liu, and G. Ballard, “Cp decomposition for tensors
via alternating least squares with qr decomposition,” 2021. [page 41]
[108] L. R. Tucker, “Implications of factor analysis of three-way matrices for
measurement of change,” in Problems in measuring change., C. W. Harris,
Ed. University of Wisconsin Press, 1963, pp. 122–137. [page 42]
[109] L. Tucker, “Some mathematical notes on three-mode factor analysis,” 1966.
[pages 42 and 54]
[110] P. M. Kroonenberg, “Principal component analysis of three-mode data by
means of alternating least squares algorithms,” Psychometrika, vol. 45, no. 4,
pp. 69–97, 1980. [page 42]
[111] T. Wansbeek, H. Neudecker, and A. Kapteyn, “An approach to n-mode
components analysis,” Psychometrika, vol. 51, pp. 269–275, 1986. [page 42]
[112] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear analysis of image
ensembles: Tensorfaces,” in Computer Vision — ECCV 2002, A. Heyden,
G. Sparr, M. Nielsen, and P. Johansen, Eds. Berlin, Heidelberg: Springer
Berlin Heidelberg, 2002, pp. 447–460. [page 42]
[113] L. D. Lathauwer, B. D. Moor, and J. Vandewalle, “A multilinear singular value
decomposition,” SIAM J. Matrix Anal. Appl., vol. 21, no. 4, pp. 1253–1278,
2000. [page 42]
Bibliography 105
[114] M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Transactions on Image Processing,
vol. 15, no. 12, pp. 3736–3745, 2006. [pages 42 and 45]
[115] M. Aharon, M. Elad, and A. Bruckstein, “K-svd: An algorithm for designing
overcomplete dictionaries for sparse representation,” IEEE Transactions on
Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006. [page 42]
[116] R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, “Self-taught learning:
Transfer learning from unlabeled data,” in Proceedings of the 24th International Conference on Machine Learning, ser. ICML ’07. New York, NY,
USA: Association for Computing Machinery, 2007, p. 759–766. [page 42]
[117] M. Aharon and M. Elad, “Sparse and redundant modeling of image content
using an image-signature-dictionary,” SIAM Journal on Imaging Sciences,
vol. 1, no. 3, pp. 228–247, 2008. [page 42]
[118] J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image
restoration,” IEEE Transactions on Image Processing, vol. 17, no. 1, pp.
53–69, 2008. [page 42]
[119] J. Bagnell and D. Bradley, “Differentiable sparse coding,” in Advances in
Neural Information Processing Systems, D. Koller, D. Schuurmans, Y. Bengio,
and L. Bottou, Eds., vol. 21. Curran Associates, Inc., 2008. [page 42]
[120] J. Yang, K. Yu, Y. Gong, and T. Huang, “Linear spatial pyramid matching
using sparse coding for image classification,” in 2009 IEEE Conference on
Computer Vision and Pattern Recognition, 2009, pp. 1794–1801. [page 42]
[121] G. Peyré, “Sparse modeling of textures,” SIAM Journal on Imaging Sciences,
vol. 34, pp. 17–31, 2009. [page 42]
[122] G. Peyré, J. Fadili, and J.-L. Starck, “Learning the morphological diversity,”
SIAM Journal on Imaging Sciences, vol. 3, no. 3, pp. 646–669, 2010. [page 42]
[123] J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online Learning for Matrix
Factorization and Sparse Coding,” Journal of Machine Learning Research,
vol. 11, no. 1, pp. 19–60, Jan 2010, revised version. [page 42]
[124] E. Elhamifar and R. Vidal, “Robust classification using structured sparse
representation,” in CVPR 2011, 2011, pp. 1873–1879. [page 42]
[125] D. Donoho and X. Huo, “Uncertainty principles and ideal atomic decomposition,” IEEE Transactions on Information Theory, vol. 47, no. 7, pp.
2845–2862, 2001. [page 43]
106 Bibliography
[126] M. Elad and A. Bruckstein, “A generalized uncertainty principle and sparse
representation in pairs of bases,” IEEE Transactions on Information Theory,
vol. 48, no. 9, pp. 2558–2567, 2002. [page 43]
[127] D. L. Donoho and M. Elad, “Optimally sparse representation in general
(non-orthogonal) dictionaries via ℓ1 minimization,” in PROC. NATL ACAD.
SCI. USA 100 2197–202, 2002. [page 43]
[128] R. Gribonval and M. Nielsen, “Sparse representations in unions of bases,”
IEEE Transactions on Information Theory, vol. 49, no. 12, pp. 3320–3325,
2003. [page 43]
[129] D. L. Donoho and J. Tanner, “Sparse nonnegative solution of underdetermined
linear equations by linear programming,” Proceedings of the National Academy
of Sciences, vol. 102, no. 27, pp. 9446–9451, 2005. [page 43]
[130] E. Candes and T. Tao, “Decoding by linear programming,” IEEE Transactions
on Information Theory, vol. 51, no. 12, pp. 4203–4215, 2005. [page 43]
[131] Z. Zhang, Y. Xu, J. Yang, X. Li, and D. Zhang, “A survey of sparse representation: Algorithms and applications,” IEEE Access, vol. 3, pp. 490–530,
2015. [pages 43 and 44]
[132] S. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,”
IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3397–3415, 1993.
[page 43]
[133] Y. Pati, R. Rezaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit:
recursive function approximation with applications to wavelet decomposition,” in Proceedings of 27th Asilomar Conference on Signals, Systems and
Computers, 1993, pp. 40–44 vol.1. [pages 43 and 82]
[134] J. Tropp, “Greed is good: algorithmic results for sparse approximation,”
IEEE Transactions on Information Theory, vol. 50, no. 10, pp. 2231–2242,
2004. [pages 43 and 82]
[135] D. Needell and R. Vershynin, “Uniform uncertainty principle and signal
recovery via regularized orthogonal matching pursuit,” Foundations of Computational Mathematics, vol. 9, pp. 317–334, 2009. [page 43]
[136] D. Needell and R. Vershynin, “Signal recovery from incomplete and inaccurate
measurements via regularized orthogonal matching pursuit,” IEEE Journal
of Selected Topics in Signal Processing, vol. 4, no. 2, pp. 310–316, 2010.
[page 43]
Bibliography 107
[137] D. Needell and J. A. Tropp, “Cosamp: Iterative signal recovery from incomplete and inaccurate samples,” Commun. ACM, vol. 53, no. 12, p. 93–100,
dec 2010. [page 43]
[138] D. Needell, J. Tropp, and R. Vershynin, “Greedy signal recovery review,” in
2008 42nd Asilomar Conference on Signals, Systems and Computers, 2008,
pp. 1048–1050. [page 43]
[139] T. Zhang, “Adaptive forward-backward greedy algorithm for learning sparse
representations,” IEEE Transactions on Information Theory, vol. 57, no. 7,
pp. 4689–4708, 2011. [page 43]
[140] H. Wu and S. Wang, “Adaptive sparsity matching pursuit algorithm for
sparse reconstruction,” IEEE Signal Processing Letters, vol. 19, no. 8, pp.
471–474, 2012. [page 43]
[141] Y. Chi and R. Calderbank, “Knowledge-enhanced matching pursuit,” in 2013
IEEE International Conference on Acoustics, Speech and Signal Processing,
2013, pp. 6576–6580. [page 43]
[142] S. K. Sahoo and A. Makur, “Signal recovery from random measurements
via extended orthogonal matching pursuit,” IEEE Transactions on Signal
Processing, vol. 63, no. 10, pp. 2572–2581, 2015. [page 43]
[143] M. Yaghoobi, D. Wu, and M. E. Davies, “Fast non-negative orthogonal
matching pursuit,” IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1229–
1233, 2015. [page 43]
[144] A. Adler, “Covariance-assisted matching pursuit,” IEEE Signal Processing
Letters, vol. 23, no. 1, pp. 149–153, 2016. [page 43]
[145] N. Lee, “Map support detection for greedy sparse signal recovery algorithms
in compressive sensing,” IEEE Transactions on Signal Processing, vol. 64,
no. 19, pp. 4987–4999, 2016. [page 43]
[146] J. Tropp, “Just relax: convex programming methods for identifying sparse
signals in noise,” IEEE Transactions on Information Theory, vol. 52, no. 3,
pp. 1030–1051, 2006. [page 43]
[147] S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by
basis pursuit,” SIAM J. Sci. Comput., vol. 20, no. 1, p. 33–61, jan 1998.
[page 43]
[148] M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for
sparse reconstruction: Application to compressed sensing and other inverse
problems,” IEEE Journal of Selected Topics in Signal Processing, vol. 1, no. 4,
pp. 586–597, 2007. [page 43]
108 Bibliography
[149] L. F. Portugal, M. G. C. Resende, G. Veiga, and J. J. Júdice, “A truncated
primal-infeasible dual-feasible network interior point method,” Networks,
vol. 35, no. 2, pp. 91–108, 2000. [page 43]
[150] S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point
method for large-scale ℓ1 -regularized least squares,” IEEE Journal of Selected
Topics in Signal Processing, vol. 1, no. 4, pp. 606–617, 2007. [page 43]
[151] J. Yang and Y. Zhang, “Alternating direction algorithms for ℓ1 -problems in
compressive sensing,” SIAM Journal on Scientific Computing, vol. 33, no. 1,
pp. 250–278, 2011. [page 43]
[152] M. Figueiredo and R. Nowak, “A bound optimization approach to waveletbased image deconvolution,” in IEEE International Conference on Image
Processing 2005, vol. 2, 2005, pp. II–782. [page 43]
[153] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm
for linear inverse problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 1,
pp. 183–202, 2009. [page 43]
[154] A. Y. Yang, S. S. Sastry, A. Ganesh, and Y. Ma, “Fast ℓ1 -minimization
algorithms and an application in robust face recognition: A review,” in 2010
IEEE International Conference on Image Processing, 2010, pp. 1849–1852.
[page 43]
[155] P. L. Combettes and J.-C. Pesquet, “Proximal Splitting Methods in Signal
Processing,” in Fixed-Point Algorithms for Inverse Problems in Science
and Engineering, Bauschke, H. Burachik, R. Combettes, P. Elser, V. Luke,
D. Wolkowicz, and H. (Eds.), Eds. Springer, 2011, pp. 185–212. [page 43]
[156] S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, “Sparse reconstruction
by separable approximation,” in 2008 IEEE International Conference on
Acoustics, Speech and Signal Processing, 2008, pp. 3373–3376. [page 43]
[157] J. Adler, B. Rao, and K. Kreutz-Delgado, “Comparison of basis selection
methods,” in Conference Record of The Thirtieth Asilomar Conference on
Signals, Systems and Computers, vol. 1, 1996, pp. 252–257. [page 43]
[158] A. M. Bruckstein, D. L. Donoho, and M. Elad, “From sparse solutions of
systems of equations to sparse modeling of signals and images,” SIAM Review,
vol. 51, no. 1, pp. 34–81, 2009. [page 43]
[159] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation
of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp.
297–301, 1965. [page 44]
Bibliography 109
[160] N. Ahmed, T. Natarajan, and K. Rao, “Discrete cosine transform,” IEEE
Transactions on Computers, vol. C-23, no. 1, pp. 90–93, 1974. [page 44]
[161] E. Simoncelli, W. Freeman, E. Adelson, and D. Heeger, “Shiftable multiscale
transforms,” IEEE Transactions on Information Theory, vol. 38, no. 2, pp.
587–607, 1992. [page 44]
[162] E. Simoncelli and E. Adelson, “Noise removal via bayesian wavelet coring,”
in Proceedings of 3rd IEEE International Conference on Image Processing,
vol. 1, 1996, pp. 379–382 vol.1. [page 44]
[163] W. He, Y. Zi, B. Chen, F. Wu, and Z. He, “Automatic fault feature extraction
of mechanical anomaly on induction motor bearing using ensemble superwavelet transform,” Mechanical Systems and Signal Processing, vol. 54-55,
pp. 457–480, 2015. [page 44]
[164] J.-L. Starck, E. Candes, and D. Donoho, “The curvelet transform for image
denoising,” IEEE Transactions on Image Processing, vol. 11, no. 6, pp.
670–684, 2002. [page 44]
[165] E. Le Pennec and S. Mallat, “Sparse geometric image representations with
bandelets,” IEEE Transactions on Image Processing, vol. 14, no. 4, pp.
423–438, 2005. [page 44]
[166] M. Do and M. Vetterli, “The contourlet transform: An efficient directional
multiresolution image representation,” IEEE transactions on image processing
: a publication of the IEEE Signal Processing Society, vol. 14, pp. 2091–106,
01 2006. [page 44]
[167] S. Foucart and H. Rauhut, A Mathematical Introduction to Compressive
Sensing., ser. Applied and Numerical Harmonic Analysis. Birkhäuser, 2013.
[page 44]
[168] B. Zhao, L. Fei-Fei, and E. P. Xing, “Online detection of unusual events
in videos via dynamic sparse coding,” in CVPR 2011, 2011, pp. 3313–3320.
[page 44]
[169] J. Shi, X. Ren, G. Dai, J. Wang, and Z. Zhang, “A non-convex relaxation
approach to sparse dictionary learning,” in CVPR 2011, 2011, pp. 1809–1816.
[page 44]
[170] M. Yang, L. Zhang, X. Feng, and D. Zhang, “Fisher discrimination dictionary
learning for sparse representation,” in 2011 International Conference on
Computer Vision, 2011, pp. 543–550. [pages 44 and 45]
110 Bibliography
[171] Z. Jiang, Z. Lin, and L. S. Davis, “Learning a discriminative dictionary for
sparse coding via label consistent k-svd,” in CVPR 2011, 2011, pp. 1697–1704.
[pages 44 and 45]
[172] D.-S. Pham and S. Venkatesh, “Joint learning and dictionary construction
for pattern recognition,” in 2008 IEEE Conference on Computer Vision and
Pattern Recognition, 2008, pp. 1–8. [page 44]
[173] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-local sparse
models for image restoration,” in 2009 IEEE 12th International Conference
on Computer Vision, 2009, pp. 2272–2279. [page 44]
[174] H. Cheng, Z. Liu, L. Yang, and X. Chen, “Sparse representation and learning
in visual recognition: Theory and applications,” Signal Processing, vol. 93,
no. 6, pp. 1408–1425, 2013, special issue on Machine Learning in Intelligent
Image Processing. [page 44]
[175] M. Aharon, M. Elad, and A. Bruckstein, “K-svd: An algorithm for designing
overcomplete dictionaries for sparse representation,” IEEE Transactions on
Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006. [page 44]
[176] M. Elad and M. Aharon, “Image denoising via learned dictionaries and
sparse representation,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, 2006, pp. 895–900.
[page 45]
[177] M. L. Fu, Z. M. Wang, and R. X. Cao, “Image denoising based on k-svd with
adaptive dictionary size,” in Modern Technologies in Materials, Mechanics
and Intelligent Systems, ser. Advanced Materials Research, vol. 1049. Trans
Tech Publications Ltd, 11 2014, pp. 1645–1649. [page 45]
[178] O. Bryt and M. Elad, “Compression of facial images using the k-svd algorithm,”
Journal of Visual Communication and Image Representation, vol. 19, no. 4,
pp. 270–282, 2008. [page 45]
[179] J. Mairal, J. Ponce, G. Sapiro, A. Zisserman, and F. Bach, “Supervised
dictionary learning,” in Advances in Neural Information Processing Systems,
D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds., vol. 21. Curran
Associates, Inc., 2008. [page 45]
[180] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 31, no. 2, pp. 210–227, 2009. [page 45]
Bibliography 111
[181] Y. Yang, Y. Yang, Z. Huang, H. T. Shen, and F. Nie, “Tag localization
with spatial correlations and joint group sparsity,” in CVPR 2011, 2011, pp.
881–888. [page 45]
[182] Z. Jiang, G. Zhang, and L. S. Davis, “Submodular dictionary learning for
sparse coding,” in 2012 IEEE Conference on Computer Vision and Pattern
Recognition, 2012, pp. 3418–3425. [page 45]
[183] S. Zubair and W. Wang, “Tensor dictionary learning with sparse tucker
decomposition,” in 2013 18th International Conference on Digital Signal
Processing (DSP), 2013, pp. 1–6. [page 45]
[184] F. Roemer, G. Del Galdo, and M. Haardt, “Tensor-based algorithms for
learning multidimensional separable dictionaries,” in 2014 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp.
3963–3967. [page 45]
[185] C. F. Dantas, M. N. da Costa, and R. d. R. Lopes, “Learning dictionaries as
a sum of kronecker products,” IEEE Signal Processing Letters, vol. 24, no. 5,
pp. 559–563, 2017. [page 45]
[186] Z. Shakeri, W. U. Z. Bajwa, and A. D. Sarwate, “Minimax lower bounds
on dictionary learning for tensor data,” IEEE Transactions on Information
Theory, vol. 64, pp. 2706–2726, 2018. [page 45]
[187] R. Xu, Y. Xu, and Y. Quan, “Factorized tensor dictionary learning for visual
tensor data completion,” IEEE Transactions on Multimedia, vol. 23, pp.
1225–1238, 2021. [page 45]
[188] G. Duan, H. Wang, Z. Liu, J. Deng, and Y.-W. Chen, “K-cpd: Learning
of overcomplete dictionaries for tensor sparse coding,” in Proceedings of the
21st International Conference on Pattern Recognition (ICPR2012), 2012, pp.
493–496. [page 45]
[189] A. Stevens, Y. Pu, Y. Sun, G. Spell, and L. Carin, “Tensor-Dictionary
Learning with Deep Kruskal-Factor Analysis,” in Proceedings of the 20th
International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, A. Singh and J. Zhu, Eds., vol. 54.
PMLR, 20–22 Apr 2017, pp. 121–129. [page 45]
[190] X. Zhang, X. Yuan, and L. Carin, “Nonlocal low-rank tensor factor analysis
for image restoration,” in 2018 IEEE/CVF Conference on Computer Vision
and Pattern Recognition, 2018, pp. 8232–8241. [page 45]
112 Bibliography
[191] C. Fraga Dantas, J. E. Cohen, and R. Gribonval, “Learning fast dictionaries
for sparse representations using low-rank tensor decompositions,” in LVA/ICA
2018 - 14th International Conference on Latent Variable Analysis and Signal
Separation, ser. LNCS, vol. 10891. Guildford, United Kingdom: Springer,
jul 2018, pp. 456–466. [page 45]
[192] S. Zubair and W. Wang, “Signal classification based on block-sparse tensor
representation,” in 2014 19th International Conference on Digital Signal
Processing, 2014, pp. 361–365. [page 45]
[193] Y. Fu, J. Gao, Y. Sun, and X. Hong, “Joint multiple dictionary learning
for tensor sparse coding,” in 2014 International Joint Conference on Neural
Networks (IJCNN), 2014, pp. 2957–2964. [page 45]
[194] N. Qi, Y. Shi, X. Sun, and B. Yin, “Tensr: Multi-dimensional tensor sparse
representation,” in 2016 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2016, pp. 5916–5925. [page 45]
[195] F. Ju, Y. Sun, J. Gao, Y. Hu, and B. Yin, “Nonparametric tensor dictionary
learning with beta process priors,” Neurocomput., vol. 218, no. C, p. 120–130,
dec 2016. [page 45]
[196] J. Salvador, “Chapter 2 - a taxonomy of example-based super resolution,” in
Example-Based Super Resolution, J. Salvador, Ed. Academic Press, 2017,
pp. 15–29. [page 48]
[197] R. R. Lewis, “Making shaders more physically plausible,” Computer Graphics
Forum, vol. 13, no. 2, pp. 109–120, 1994. [page 48]
[198] E. P. Lafortune and Y. D. Willems, “Using the modified phong reflectance
model for physically based rendering,” Tech. Rep., 1994. [page 48]
[199] A. Dür, “An improved normalization for the ward reflectance model,” Journal
of Graphics Tools, vol. 11, pp. 51 – 59, 2006. [page 49]
[200] D. Geisler-Moroder and A. Dür, “A new ward brdf model with bounded
albedo,” in Proceedings of the 21st Eurographics Conference on Rendering,
ser. EGSR’10. Goslar, DEU: Eurographics Association, 2010, pp. 1391–1398.
[page 49]
[201] L. Neumann, A. Neumannn, and L. Szirmay-Kalos, “Compact Metallic
Reflectance Models,” Computer Graphics Forum, 1999. [page 49]
[202] C. Schlick, “An inexpensive brdf model for physically-based rendering,” Computer Graphics Forum, vol. 13, no. 3, pp. 233–246, 1994. [page 50]
Bibliography 113
[203] M. Ashikhmin and S. Premoze, “Distribution-based brdfs,” 2007. [page 50]
[204] K. E. Torrance and E. M. Sparrow, “Theory for off-specular reflection from
roughened surfaces∗,” J. Opt. Soc. Am., vol. 57, no. 9, pp. 1105–1114, Sep
1967. [page 51]
[205] B. Walter, S. R. Marschner, H. Li, and K. E. Torrance, “Microfacet models
for refraction through rough surfaces,” in Proceedings of the 18th Eurographics Conference on Rendering Techniques, ser. EGSR’07. Goslar, DEU:
Eurographics Association, 2007, pp. 195–206. [page 51]
[206] M. M. Bagher, C. Soler, and N. Holzschuch, “Accurate fitting of measured
reflectances using a Shifted Gamma micro-facet distribution,” Computer
Graphics Forum, vol. 31, no. 4, pp. 1509–1518, Jun. 2012. [pages 51 and 52]
[207] N. Holzschuch and R. Pacanowski, “A Two-Scale Microfacet Reflectance
Model Combining Reflection and Diffraction,” ACM Transactions on Graphics, vol. 36, no. 4, p. 12, Jul. 2017, article 66. [page 52]
[208] M. Kurt, L. Szirmay-Kalos, and J. Křivánek, “An anisotropic brdf model for
fitting and monte carlo rendering,” SIGGRAPH Comput. Graph., vol. 44,
no. 1, feb 2010. [page 52]
[209] T. Tongbuasirilai, “Accurate brdf modelling for wide angle scattering,” Master’s thesis, Linköping UniversityLinköping University, Media and Information
Technology, The Institute of Technology, 2013. [page 52]
[210] M. Oren and S. K. Nayar, “Generalization of lambert’s reflectance model,” in
Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’94. New York, NY, USA: Association
for Computing Machinery, 1994, pp. 239–246. [page 52]
[211] J. W. S. L. D. L. S. Maxwell, J. R. Beard, “Bidirectional reflectance model
validation and utilization.” [page 53]
[212] H. Westlund and G. Meyer, “A brdf database employing the beard-maxwell
reflection model,” in Proceedings of the Graphics Interface 2002 Conference,
May 2002, pp. 189–201. [page 53]
[213] A. Weidlich and A. Wilkie, “Arbitrarily layered micro-facet surfaces,” in
Proceedings of the 5th International Conference on Computer Graphics and
Interactive Techniques in Australia and Southeast Asia, ser. GRAPHITE
’07. New York, NY, USA: Association for Computing Machinery, 2007, pp.
171–178. [page 53]
114 Bibliography
[214] M. Rump, G. Müller, R. Sarlette, D. Koch, and R. Klein, “Photo-realistic
rendering of metallic car paint from image-based measurements,” Computer
Graphics Forum, 2008. [page 53]
[215] F. X. Sillion, J. R. Arvo, S. H. Westin, and D. P. Greenberg, “A global
illumination solution for general reflectance distributions,” ser. SIGGRAPH
’91. New York, NY, USA: Association for Computing Machinery, 1991, pp.
187–196. [page 54]
[216] P. Mézières and M. Paulin, “Efficient spherical harmonic shading for separable BRDF,” in 14th ACM SIGGRAPH Conference and Exhibition on
Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA
2021), Special interest group of the Association for Computing Machinery in
computer graphics and interactive techniques. Tokyo, Japan: ACM, Dec.
2021, pp. 1–4. [page 54]
[217] D. Mahajan, R. Ramamoorthi, and B. Curless, “A theory of spherical harmonic identities for brdf/lighting transfer and image consistency,” in Proceedings of the 9th European Conference on Computer Vision - Volume Part IV,
ser. ECCV’06. Berlin, Heidelberg: Springer-Verlag, 2006, p. 41–55. [page 54]
[218] R. Basri and D. Jacobs, “Lambertian reflectance and linear subspaces,” in
Proceedings Eighth IEEE International Conference on Computer Vision.
ICCV 2001, vol. 2, 2001, pp. 383–390 vol.2. [page 54]
[219] L. Zhang and D. Samaras, “Face recognition under variable lighting using
harmonic image exemplars,” in 2003 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 1, 2003,
pp. I–I. [page 54]
[220] R. Ramamoorthi and P. Hanrahan, “A signal-processing framework for
reflection,” ACM Trans. Graph., vol. 23, no. 4, p. 1004–1042, oct 2004.
[page 54]
[221] R. Ramamoorthi and P. Hanrahan, “An efficient representation for irradiance environment maps,” in Proceedings of the 28th Annual Conference on
Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’01. New
York, NY, USA: Association for Computing Machinery, 2001, p. 497–500.
[pages 54 and 61]
[222] P.-P. Sloan, J. Kautz, and J. Snyder, “Precomputed radiance transfer for
real-time rendering in dynamic, low-frequency lighting environments,” in
Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’02. New York, NY, USA: Association
for Computing Machinery, 2002, pp. 527–536. [page 54]
Bibliography 115
[223] Z. Ren, R. Wang, J. Snyder, K. Zhou, X. Liu, B. Sun, P.-P. Sloan, H. Bao,
Q. Peng, and B. Guo, “Real-time soft shadows in dynamic scenes using
spherical harmonic exponentiation,” in ACM SIGGRAPH 2006 Papers, ser.
SIGGRAPH ’06. New York, NY, USA: Association for Computing Machinery,
2006, pp. 977–986. [page 54]
[224] R. Ramamoorthi, “Modeling illumination variation with spherical harmonics,”
in In Face Processing: Advanced Modeling and Methods. Academic Press,
2005. [page 54]
[225] I. G. E. Renhorn and G. D. Boreman, “Developing a generalized brdf model
from experimental data,” Opt. Express, vol. 26, no. 13, pp. 17 099–17 114,
Jun 2018. [page 54]
[226] R. Pacanowski, O. Salazar-Celis, C. Schlick, X. Granier, P. Pierre, and C. Annie, “Rational BRDF,” IEEE Transactions on Visualization and Computer
Graphics, vol. 18, no. 11, pp. 1824–1835, Mar. 2012. [page 54]
[227] F. Romeiro, Y. Vasilyev, and T. Zickler, “Passive reflectometry,” in Computer
Vision – ECCV 2008, D. Forsyth, P. Torr, and A. Zisserman, Eds. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2008, pp. 859–872. [pages 54 and 78]
[228] J. Hao, Y. Liu, and D. Weng, “A brdf representing method based on gaussian
process,” in Computer Vision - ACCV 2014 Workshops, C. Jawahar and
S. Shan, Eds. Cham: Springer International Publishing, 2015, pp. 542–553.
[page 55]
[229] S. Georgoulis, V. Vanweddingen, M. Proesmans, and L. Van Gool, “A gaussian
process latent variable model for brdf inference,” in 2015 IEEE International
Conference on Computer Vision (ICCV), 2015, pp. 3559–3567. [page 55]
[230] K. Nishino and S. Lombardi, “Directional statistics-based reflectance model
for isotropic bidirectional reflectance distribution functions,” J. Opt. Soc.
Am. A, vol. 28, no. 1, pp. 8–18, Jan 2011. [pages 55 and 78]
[231] M. M. Bagher, J. Snyder, and D. Nowrouzezahrai, “A non-parametric factor
microfacet model for isotropic brdfs,” ACM Trans. Graph., vol. 35, no. 5, jul
2016. [pages 55 and 67]
[232] E. Miandji, S. Hajisharif, and J. Unger, “A unified framework for compression
and compressed sensing of light fields and light field videos,” ACM Trans.
Graph., vol. 38, no. 3, may 2019. [pages 56, 62, 63, and 65]
[233] L. Claustres, M. Paulin, and Y. Boucher, “BRDF Measurement Modelling
using Wavelets for Efficient Path Tracing,” Computer Graphics Forum, 2003.
[page 61]
116 Bibliography
[234] B. Tunwattanapong, G. Fyffe, P. Graham, J. Busch, X. Yu, A. Ghosh,
and P. Debevec, “Acquiring reflectance and shape from continuous spherical
harmonic illumination,” ACM Trans. Graph., vol. 32, no. 4, jul 2013. [page 65]
[235] A. Wilkie, A. Weidlich, M. Magnor, and A. Chalmers, “Predictive rendering,”
in ACM SIGGRAPH ASIA 2009 Courses, ser. SIGGRAPH ASIA ’09. New
York, NY, USA: Association for Computing Machinery, 2009. [page 68]
[236] A. Ben-Artzi, R. Overbeck, and R. Ramamoorthi, “Real-time brdf editing in
complex lighting,” ACM Trans. Graph., vol. 25, no. 3, pp. 945–954, jul 2006.
[page 68]
[237] A. Ben-Artzi, K. Egan, F. Durand, and R. Ramamoorthi, “A precomputed
polynomial representation for interactive brdf editing with global illumination,”
ACM Trans. Graph., vol. 27, no. 2, may 2008. [page 68]
[238] X. Sun, K. Zhou, Y. Chen, S. Lin, J. Shi, and B. Guo, “Interactive relighting
with dynamic brdfs,” ACM Trans. Graph., vol. 26, no. 3, pp. 27–es, jul 2007.
[page 68]
[239] T.-W. Schmidt, F. Pellacini, D. Nowrouzezahrai, W. Jarosz, and C. Dachsbacher, “State of the art in artistic editing of appearance, lighting, and
material,” Computer Graphics Forum, vol. 35, no. 1, pp. 216–233, feb 2016.
[page 68]
[240] J. C. Gower and G. B. Dijksterhuis, Procrustes Problems. Oxford University
Press, 01 2004. [page 70]
[241] R. Vávra and J. Filip, “Minimal sampling for effective acquisition of
anisotropic brdfs,” Comput. Graph. Forum, vol. 35, no. 7, pp. 299–309,
oct 2016. [pages 78 and 82]
[242] M. Fuchs, V. Blanz, H. P. Lensch, and H.-P. Seidel, “Adaptive sampling of
reflectance fields,” ACM Trans. Graph., vol. 26, no. 2, pp. 10–es, jun 2007.
[page 78]
[243] G. Patow and X. Pueyo, “A survey of inverse rendering problems,” Computer
Graphics Forum, vol. 22, no. 4, pp. 663–687, 2003. [page 78]
[244] F. Romeiro and T. Zickler, “Blind reflectometry,” in Computer Vision –
ECCV 2010, K. Daniilidis, P. Maragos, and N. Paragios, Eds. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2010, pp. 45–58. [page 78]
[245] S. R. Marschner, “Inverse rendering for computer graphics,” Ph.D. dissertation, USA, 1998, aAI9839924. [page 78]
Bibliography 117
[246] M. Aittala, T. Weyrich, and J. Lehtinen, “Practical svbrdf capture in the
frequency domain,” ACM Trans. Graph., vol. 32, no. 4, jul 2013. [page 78]
[247] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “Efficient Isotropic
BRDF Measurement,” in Eurographics Workshop on Rendering, P. Dutre,
F. Suykens, P. H. Christensen, and D. Cohen-Or, Eds. The Eurographics
Association, 2003. [page 78]
[248] K. Reinhard, H. Matthias B., S. Heinz Christian, and B. Dennis den, “PatchBased Sparse Reconstruction of Material BTFs,” Journal of WSCG, vol. 22,
no. 2, pp. 83–90, 2014. [pages 78 and 82]
[249] I. C. F. Ipsen and T. Wentworth, “The effect of coherence on sampling
from matrices with orthonormal columns, and preconditioned least squares
problems,” SIAM J. Matrix Anal. Appl., vol. 35, no. 4, pp. 1490–1520, 2014.
[page 78]
[250] H. Nyquist, “Certain topics in telegraph transmission theory,” Transactions
of the American Institute of Electrical Engineers, vol. 47, no. 2, pp. 617–644,
1928. [page 81]
[251] C. Shannon, “Communication in the presence of noise,” Proceedings of the
IRE, vol. 37, no. 1, pp. 10–21, 1949. [page 81]
[252] E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact
signal reconstruction from highly incomplete frequency information,” 2004.
[page 81]
[253] E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,”
IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 489–509, 2006.
[page 81]
[254] D. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006. [page 81]
[255] J. A. Tropp, A. C. Gilbert, and M. J. Strauss, “Algorithms for simultaneous sparse approximation. part i: Greedy pursuit,” Signal Processing,
vol. 86, no. 3, pp. 572–588, 2006, sparse Approximations in Signal and Image
Processing. [pages 81 and 82]
[256] J. A. Tropp, “Algorithms for simultaneous sparse approximation. part ii:
Convex relaxation,” Signal Processing, vol. 86, no. 3, pp. 589–602, 2006,
sparse Approximations in Signal and Image Processing. [page 81]
118 Bibliography
[257] M. M. Hyder and K. Mahata, “A robust algorithm for joint-sparse recovery,”
IEEE Signal Processing Letters, vol. 16, no. 12, pp. 1091–1094, 2009. [page 82]
[258] D. L. Donoho, Y. Tsaig, I. Drori, and J. L. Starck, “Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching
pursuit,” IEEE Trans. Inf. Theor., vol. 58, no. 2, pp. 1094–1121, feb 2012.
[page 82]
[259] J. D. Blanchard, M. Cermak, D. Hanle, and Y. Jing, “Greedy algorithms
for joint sparse recovery,” IEEE Transactions on Signal Processing, vol. 62,
no. 7, pp. 1694–1704, 2014. [page 82]
[260] J. Shang, Z. Wang, and Q. Huang, “A robust algorithm for joint sparse
recovery in presence of impulsive noise,” IEEE Signal Processing Letters,
vol. 22, no. 8, pp. 1166–1170, 2015. [page 82]
[261] N. Han, S. Li, and J. Lu, “Orthogonal subspace based fast iterative thresholding algorithms for joint sparsity recovery,” IEEE Signal Processing Letters,
vol. 28, pp. 1320–1324, 2021. [page 82]
[262] Z. Xu, J. B. Nielsen, J. Yu, H. W. Jensen, and R. Ramamoorthi, “Minimal
brdf sampling for two-shot near-field reflectance acquisition,” ACM Trans.
Graph., vol. 35, no. 6, nov 2016. [page 82]
[263] V. Blanz, A. Mehl, T. Vetter, and H.-P. Seidel, “A statistical method for
robust 3d surface reconstruction from sparse data,” in Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission,
2004. 3DPVT 2004., 2004, pp. 293–300. [page 85]
[264] J.-F. Determe, “Somp simulation framework v0.5.3,” 2015, accessed: 2022-01.
[page 85]
Publications
Publications
The publications associated with this thesis have been removed
for copyright reasons. For more details about these see:
https://doi.org/10.3384/9789179295585
FACULTY OF SCIENCE AND ENGINEERING
Linköping Studies in Science and Technology Dissertation No. 2272
Linköping Studies in Science and Technology, Dissertation No. 2272, 2023
Department of Science and Technology
Linköping University
SE-581 83 Linköping, Sweden
www.liu.se
Data-Driven Approaches for
Tanaboon Tongbuasirilai
Sparse Reflectance Modeling and Acquisition
Data-Driven Approaches
for Sparse Reflectance
Modeling and Acquisition
Tanaboon Tongbuasirilai
2023