CNN Training for Material Appearance Capture

• Tel: 04 76 61 54 29
• Mail: cyril.soler -- at -- inria.fr

Master II Thesis -- 2022/2023

This master thesis will last 6 months, and starts in Feb. 2023.

The student will be working at INRIA Rhone-Alpes, near Grenoble, France. Advisor: Cyril Soler.

Work plan

The goal of this master II thesis is to explore the possibility of training a convolutional neural network (a.k.a. CNN) with measured materials reflectance properties, so as to link these data to the effective appearance of a rendered material. Ultimately, the goal is to use the trained CNN to figure out which material are involved in a photograph.

The main difficulty in this work is to generate a sufficient amount of measured material data. Fortunately, the the candidate will benefit from an existing system to interpolate databases of measured materials, and to render them efficiently. One part of the job will therefore consists into understanding how the existing algorithm works and how it can be used to efficiently produced a large enough set of training data.

The main steps of the project will be:

1. Understand the existing method to interpolate/render measured materials

This step involves learning about Gaussian processes, spherical harmonics, and a bit of GPU programming.

2. design a basic CNN implementation to be trained with the input data.

For efficiency reasons, a C++ CNN library should be prefered (whereas students who are already trained with CNNs may better know python implementations).

3. Examine different possible CNN architectures, and choose what's best for the current task
4. Test the CNN on synthetic data with same viewpoint and illumination

Although the ultimate goal of the project is to capture reflectance data from photographs, the validation will be kept simple and limited to known geometry and illumination.

Requirements

Application procedure:

It is essential to contact me and come at INRIA (Montbonnot) to discuss the subject.

• Curriculum vitae
• School report